Database update on header download

General NewsBin and Usenet related discussions. Make recommendations for additional specific discussion groups.

Moderators: Quade, dexter

Database update on header download

Postby malhusky » Thu Mar 19, 2020 3:17 pm

Is there a way to speed up the process of downloading new headers for an existing group. For some popular groups I find it can take 15 minutes or more to process the headers after only 3 days worth have been downloaded. I suppose lowering the Storage Age might be one option. What about compressing the database? Cache is currently set at 400.

Another issue is that when updating multiple groups it's not always obvious which groups have finished until the cache clears. Is there a way to determine that while the update is in progress?
Thx
malhusky
Occasional Contributor
Occasional Contributor
 
Posts: 47
Joined: Sat Apr 28, 2012 1:01 am

Registered Newsbin User since: 12/13/10

Re: Database update on header download

Postby Quade » Fri Mar 20, 2020 1:58 pm

Some groups simply don't work for headers anymore. Instead of posting files, they're just posting posts with random subjects. These groups, the header database gets 100 times more writes than a group with proper files. These posts are only usable with NZB Files.

One way to tell is to load the group, disable any size filters and see if it's full of ungrouped 300-750K posts.

I'm thinking about putting out a new beta that simply tosses these posts so they don't clutter the header database.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 43750
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 1 guest