Harvester Fetching Revised

Tom 132 words

Fyi, I got a complaint about the UvHarvester fetch frequency last week so I’ve revised it (UvHarvester is the program that collects feed data for the aggregator). Previously it visited each feed once an hour, but now it derives how often to check a feed from the frequency of posts on the feed. So, for example, a feed that gets updated on average once a day will now get 4 visits a day. (This has the unfortunate side-effect that new posts will not show up very quickly on feeds that don’t update often, but that’s not a design goal anyway.) I’ve also revised the comment crawling a little to cut out some unnecessary fetches. I will also be trying some other things to cut down on unnecessary fetching in the coming days/weeks.

Originally posted on my personal blog which was active from 2003 to 2020. You may have arrived here by redirection from the original permalink.

Elsewhere: Loading...

Sorry, new comments are disabled on older posts. This helps reduce spam. Active commenting almost always occurs within a day or two of new posts.