Downloading older headers

There is no such thing as a stupid question here as long as it pertains to using NewsBin. Newbies feel free to get help getting started or asking questions that may be obvious. Regular users are asked to be gentle. :-)

Moderators: Quade, dexter

Downloading older headers

Postby bodysurfer » Tue Jan 24, 2017 2:34 pm

This may not be possible, but I wanted to ask. I'd like to download 400K headers from a busy group like alt.binaries.teevee. But the 400K I am interested in are from June 17, 2014. I know headers are numbered by integers. Even if I don't know the exact integers corresponding to 6/17/14, I might be able to iterate through various attempts to eventually find the header numbers corresponding to 6/17/14. Is this a practical approach? Is there any other way?

Quade provided some info on a related question that "may" be relevant to my question:

<Quade wrote>

It always downloads oldest to newest. Whatever oldest happens to be.

1 - In Newsbin's options under network select "Show server commands".

2 - Do a "Download Latest" on boneless. Let is run for 20 seconds them delete it.

3 - Look in the logging tab.

[14:31:00] HIGH AW alt.binaries.boneless GROUP alt.binaries.boneless
|211 12686498095 2256535552 14943033646 alt.binaries.boneless

211 - OK
12686498095 - Count
2256535552 - Min record
14943033646 - Max Record
alt.binaries.boneless - group
<End of Quade quote>

Thanks for any help!
bodysurfer
n00b
n00b
 
Posts: 3
Joined: Sat Sep 15, 2012 11:25 pm

Registered Newsbin User since: 08/31/12

Re: Downloading older headers

Postby dexter » Tue Jan 24, 2017 2:57 pm

Try this:

1) Under Options -> Settings, set "Download Age" to 950 (should get you back to 1/2 through 2014)
2) Under Options -> Settings, make sure "Storage Age" is at least 950
3) Right click the group from the Groups list and select Post Storage then "Use Download Age"
4) Right click the group again and select "Download Latest Headers"
5) Watch the count in the download list and let it go through a couple million headers, then highlight and delete the header download task from the download list.
6) Wait for the backlog to clear by watching the cache display at the bottom of the Newsbin window. Looks something like "Cache 90/100 (xx)". When the numbers in parenthesis goes to 0, the processing is done.
7) Load the group with right-click, "Show All Posts"

If you aren't up to June 2014 yet, then just do a Download Latest again and let it do a few million more depending on how far off you are. If the oldest headers are not old enough, start over at step #1 with an older download age setting.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9203
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: Downloading older headers

Postby bodysurfer » Tue Jan 24, 2017 9:04 pm

Works great!

Thanks very much!!!
bodysurfer
n00b
n00b
 
Posts: 3
Joined: Sat Sep 15, 2012 11:25 pm

Registered Newsbin User since: 08/31/12

Re: Downloading older headers

Postby unyquy » Tue Feb 28, 2017 9:39 pm

Dexter, I have added a new group and only want to go back and download the last 30 days headers. I have set the "download age" to 30 days, but the "storage age" to 300 as I want to keep posts for longer. I have followed the other steps in your message, but when I tell it to "download latest headers" it downloads all the headers in the group most of which have been posted much later than 30 days ago. I am running version 6.62. What am I doing wrong?
unyquy
unyquy
Occasional Contributor
Occasional Contributor
 
Posts: 14
Joined: Sat Apr 25, 2009 1:15 pm

Registered Newsbin User since: 11/17/07

Re: Downloading older headers

Postby Quade » Wed Mar 01, 2017 6:53 am

it downloads all the headers in the group most of which have been posted much later than 30 days ago.


If the group is small it doesn't even bother with the math. It just downloads all the headers. I think the threshold is 100,000 headers which is essentially an empty group.

You can always right click the groups "Post Storage/Use Download Age" to reset the group and make it obey "Download Age". As for the "Storage age", that's perfectly fine. It's only problematic when "Storage" is smaller than "Download" age.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42038
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Downloading older headers

Postby unyquy » Wed Mar 01, 2017 9:10 pm

This worked just fine. Thanks for your help. unyquy
unyquy
Occasional Contributor
Occasional Contributor
 
Posts: 14
Joined: Sat Apr 25, 2009 1:15 pm

Registered Newsbin User since: 11/17/07


Return to Newbie Forum

Who is online

Users browsing this forum: No registered users and 21 guests