Confusing speed and size Numbers

Technical support and discussion of Newsbin Version 6 series.

Confusing speed and size Numbers

Postby klawdek » Mon Apr 11, 2011 2:51 am

Ok I have been Dling headers for a group for about 30 minutes now.

The size column reads: 28,150,000 / 360,731,686
Data Downloaded this session: 14.35GB
Size of data waiting to download: 316.4MB
Speed: 45.1Mbps

First my ISP is not capable of such speeds it is more than twice what my ISP can do.
I really hope I have not DLed 14.35GB this session as all I have done is DL headers and all headers I have DLed since installation are only about 1.35GB
The downloaded this session number is going up rapidly while the size of data left number is going down very very slowly.
Very roughly, the Data Downloaded this session increased by about 8GBs of data, while the Size of data waiting to download only decreased by about 30MB.

What is going on with these numbers?

Am I really using GBs of bandwidth for MBs of headers?
klawdek
Active Participant
Active Participant
 
Posts: 51
Joined: Sun Mar 27, 2011 9:12 pm

Registered Newsbin User since: 04/09/11

Re: Confusing speed and size Numbers

Postby klawdek » Mon Apr 11, 2011 3:05 am

Here is some more confusion:)

According to the list in the add groups box the goup I am DLing headers for is 54.9MB
When I went to DL all headers it showed up in Size of data waiting to download as about 330MB or so.
Now after Size of data waiting to download indicates about 30MB has been DLed the actual size on the HD is 789MB

I will have to abort this DL as it obvious that it will take many hours and take way more drive space than available.

I just cannot make sense of any of these numbers.
klawdek
Active Participant
Active Participant
 
Posts: 51
Joined: Sun Mar 27, 2011 9:12 pm

Registered Newsbin User since: 04/09/11

Re: Confusing speed and size Numbers

Postby richy99 » Mon Apr 11, 2011 7:01 am

nothing confusinf about it at all, depending on your server the headers will be compressed and will be delivered upto 10x faster than your connection

that shows it has downloaded 28mill out of 360 mill headers
size waiting to download is whats in the que, it should relate to headers as it would have to guesstimate the size
User avatar
richy99
Elite NewsBin User
Elite NewsBin User
 
Posts: 6353
Joined: Fri Nov 21, 2003 8:04 pm
Location: Wales

Registered Newsbin User since: 12/31/03

Re: Confusing speed and size Numbers

Postby itimpi » Mon Apr 11, 2011 8:07 am

When talking about headers, the values are not a size, but the number of headers.

Also, in the Groups tab, the numbers are only indicative and are taken from the server that was in use when that group list was built up. You should never take these as absolute nu,mbers, but just as an indication of the relative sizes of groups. The real number can vary according to the retention of your server and how many days worth of headers youa re downloading. As was mentioned in the group you are trying it looks as though there are 330 million header available.


Also as was mentioned, if the server supports it then headers are downloaded using compression which can give a logical data throughput of up to 10 x the actual download speed (headers compress well). Newsbin is displaying this logical data rate which is why you can see it exceeding your notional line speed. Files unfortunately do not get a benefit from compression as they are already stored compressed.
The Newsbin Online documentation
The Usenettools for tutorials, useful information and links
User avatar
itimpi
Elite NewsBin User
Elite NewsBin User
 
Posts: 12604
Joined: Sat Mar 16, 2002 7:11 am
Location: UK

Registered Newsbin User since: 03/28/03

Re: Confusing speed and size Numbers

Postby klawdek » Mon Apr 11, 2011 7:27 pm

itimpi wrote: You should never take these as absolute nu,mbers, but just as an indication of the relative sizes of groups. The real number can vary according to the retention of your server and how many days worth of headers youa re downloading.



That is one of the points of my confusion. The numbers in the group list are not relative. One group that has 54MB for its size turns out to be substantially bigger than a group that has 100MB for its size.

I have tried re-downloading the group list but I am not sure if it is updating the sizes for groups already listed. It does seem to add new groups but the old groups still have the same date.


I understand the headers are compressed. However the speed of DL is not consistent with the amount of space the headers take in the spool. i.e If I dl headers and they take 300MB of HD space when finished and the amount DLed shows some number of gigabytes. The amount of time it took to DL is more consistent with the gigabyte number.

I am not sure but I think I may have used about 15 or 20 GB since last night (I will have to pay closer attention to my ISPs usage meter when DLing headers next time). All I did was DL about 2GB of headers. The only other downloads were me watching a half hour HULU program (usually about 150-250MB depending on the show) and looking up a few things in wikipedia. And, my roomate looking at something on google earth.
klawdek
Active Participant
Active Participant
 
Posts: 51
Joined: Sun Mar 27, 2011 9:12 pm

Registered Newsbin User since: 04/09/11

Re: Confusing speed and size Numbers

Postby Quade » Mon Apr 11, 2011 8:04 pm

They're compressed in transport and they're super compressed when stored on disk (Logical and regular compression). 6 compresses them on disk tighter than 5.99 but, the overhead of storage means the difference is small. Switching back and forth between versions is asking for trouble.

If you want a clean groups list, exit newsbin delete the groups.db3 file then re-download the groups. The sizes are likely broken in 5.59.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to V6 Technical Support

Who is online

Users browsing this forum: No registered users and 2 guests