How big are the Newsbin database files for binary groups?

This is the place for people switching to Newsbin from other newsreaders, or for people evaluating Newsbin who are already experienced Usenet users. If your old newsreader had a feature you liked that Newsbin doesn't, tell us about it.
Forum rules
- No discussion of specific content found on Usenet.
- Be nice to others. Trolls and Flamers will be banned at our discretion
- No advertising for other newsreaders, products, or services.

How big are the Newsbin database files for binary groups?

Postby KootchieKoo » Tue May 08, 2018 1:43 pm

Hi, I haven't done much downloading from the newsgroups since 2011, back then I was struggling along with Newsleecher 4.x and Forte Agent. Currently I have only a small block account with one provider, I am thinking of getting an account with a high-retention provider with 3500+ days of binary retention. I prefer downloading headers and browsing the groups over doing searches and working with NZB files.

I am wondering how big the resulting Newsbin database file would be if I started from scratch, then downloaded all headers from a group, say a.b.sounds.lossless, from one of the providers with 3500+ days of retention. Does anyone download all headers and browse the groups anymore?

Also, if my PC lost its Internet connection or crashed, would Newsbin, upon re-connect or re-launch, be able to resume downloading headers from where the interruption occurred? How "robust" is the Newsbin mechanism for header downloads when recovering from errors? Thanks
KootchieKoo
n00b
n00b
 
Posts: 4
Joined: Tue May 08, 2018 1:13 pm

Re: How big are the Newsbin database files for binary groups

Postby Quade » Tue May 08, 2018 1:55 pm

Crashing, is a toss up. It might or might not be able to resume. Your PC should never crash. On the other hand, if you need to restart Newsbin, it'll continue the header download where it left off.

I'll download all headers from Lossless and see...
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: How big are the Newsbin database files for binary groups

Postby Quade » Tue May 08, 2018 10:36 pm

a.b.sounds.lossless with 3500 days of headers downloaded is about 2.3 gigs.

Keep in mind it's 140 million headers meaning you have to download

140+E6 * 350 bytes = 50 GB of download to get them all.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: How big are the Newsbin database files for binary groups

Postby KootchieKoo » Wed May 09, 2018 1:52 pm

My PC rarely crashes/bluescreens, the latest one was a few weeks ago and related the the ESET antivirus I use. A couple more happened 1-2 years ago and were driver-related, but just updating the drivers caused them to go away.

OK thanks for running that test, I appreciate it. I am living in a house with a Comcast cable modem, I am capped at 1024 GB per month, so the 50 GB of headers for that group is a bit disappointing, but initially I think I'll grab only the oldest (500 days or so) headers from all the groups I am interested in (lossless music and a.b.dvd.*) then grab the oldest posts, to catch up. Actually I've been using my block account and some free trial accounts to grab some binaries from a.b.dvd.music that are ~3500 days old, I wanted them years ago, but back then I didn't have the disk space to grab them.

My PC may not be up to snuff, it is a Win7 Haswell system I built in 2014. Has several TB of free disk space (spinning HD's as JBOD, no NAS) - currently no SSD but I've just ordered a new Samsung 970 EVO SSD that will arrive tomorrow. It's only the 250 GB model though, I will install a fresh Win7 onto it, then start playing with Newsbin and whichever long-retention provider I choose. The group a.b.sounds.lossless is only 2.3 GB, so hopefully I can keep all the Newsbin group databases on the SSD, and initially download binaries onto the SSD, and later move completed downloads to the slower storage.

The weak link is the low RAM - the PC has only 8 GB of RAM - will that be enough when pulling 140 million headers or more?
KootchieKoo
n00b
n00b
 
Posts: 4
Joined: Tue May 08, 2018 1:13 pm

Re: How big are the Newsbin database files for binary groups

Postby Quade » Wed May 09, 2018 4:15 pm

Header downloads don't consume much RAM. Header display is the RAM killer. If you download headers but the tab to display them isn't open, the memory usage is minimal.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: How big are the Newsbin database files for binary groups

Postby KootchieKoo » Mon Jan 07, 2019 3:22 pm

OK thanks for the info. I am back now, I had some serious health issues in the last half of 2018. I have an account with unlimited downloads at NewsgroupDirect (a Highwinds server with ~3500 days of retention) to play with. It's amazing how I can go now and grab posts from some of the DVD groups that were posted 10 years ago, back then I was always running out of disk space and had to be very selective about what I could download. They Newsbin app is very responsive and intuitive.

I have been using Newsbin to download from various groups that were small enough that I could download all headers of the group for the entire ~3500 days of retention. I have hit a snag though, and don't see how to accomplish something.

From your test and a simple proportionality calculation:

a.b.sounds.lossless has 140 million headers = 2.3 GB Newsbin database size and 50 GB used to download these headers

a.b.dvd has 5560 million headers = 91 GB Newsbin database size and 1986 GB used to download these headers

I would like to pull the OLDEST headers (30 days or so of the absolute oldest headers) of a.b.dvd so that I can grab some binaries I want and know are there, before they fall off the server. Is this possible with Newsbin?? - all I see is

Download Special... --> Download 10,000,000 Older Posts

but that downloads the 10,000,000 NEWEST posts which is not what I want.
KootchieKoo
n00b
n00b
 
Posts: 4
Joined: Tue May 08, 2018 1:13 pm

Re: How big are the Newsbin database files for binary groups

Postby KootchieKoo » Mon Jan 07, 2019 3:38 pm

My bad -- after just a couple minutes more of poking around I see it - Show Posts Special... with the day range option and the calendar control.

That's why I didn't see it, to me Show Posts means display posts for which headers are already downloaded, but I'm glad it's there. Image link not allowed for unregistered users
KootchieKoo
n00b
n00b
 
Posts: 4
Joined: Tue May 08, 2018 1:13 pm

Re: How big are the Newsbin database files for binary groups

Postby max » Tue Sep 03, 2019 11:52 pm

Interesting topic.
max
Seasoned User
Seasoned User
 
Posts: 144
Joined: Thu May 19, 2005 10:45 am


Return to Immigrants from other Newsreaders

Who is online

Users browsing this forum: No registered users and 3 guests