Single vs. dual core

While it isn't officially supported, we have many users who successfully run NewsBin under Linux with a Windows emulator (Wine is the most popular). This forum exists for you Linux guys to share your experience getting NewsBin running on Linux as well as report any strangeness that we may or may not be able to fix. DThor is the resident expert.

Moderators: Quade, dexter, newsraider, DThor

Single vs. dual core

Postby jacob733 » Sun Nov 15, 2009 8:43 am

On a single core machine, I see Newsbin taking a very long time decoding RARs while on a dual core machine I see unrar time being up to ten times faster. When comparing the CPUs, I would expect the dual core machine being only slightly faster considering that unrar is single-threaded and that the dual core machines single-threaded performance is not that much greater.

Is there a special reason for it to be like this? Is there maybe a setting I can change?
It seems that at least in some situations, it helps if I change the RAR decode priority. Could that be the solution?
jacob733
Occasional Contributor
Occasional Contributor
 
Posts: 38
Joined: Wed Jan 09, 2002 12:44 pm
Location: Denmark

Registered Newsbin User since: 04/26/03

Postby Quade » Sun Nov 15, 2009 10:48 am

The machines are identical other than CPU? I'd be much more likely to think this is a disk drive issue than CPU.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44865
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Postby jacob733 » Sun Jan 03, 2010 2:35 pm

Actually, the single core machine is a desktop while the dual core is a laptop. The desktop machine has considerably higher performance in the disk drive department than the laptop.

Anyway, it turns out I was way off. Both the single core and dual core machines will exhibit similar behavior if the DL throughput is high enough. I'm not sure if it this a linux/wine issue or if it is a newsbin/config issue. It seems that if the Internet bandwidth is low enough, then decode has no problem on the same machine. Looks like Newsbin gets behind on something in the DL/decode process, which then overloads newsbin in such a way that unrar suffers.

I live quite far from my main server (Easynews), so I run newsbin with 30-50 connections as that will give me a DL rate of about 4-6MB/s. (Don't you just love fiber? :-P)

Any ideas/suggestions?
jacob733
Occasional Contributor
Occasional Contributor
 
Posts: 38
Joined: Wed Jan 09, 2002 12:44 pm
Location: Denmark

Registered Newsbin User since: 04/26/03

Postby Quade » Wed Mar 10, 2010 10:01 am

Some people turn off automatic decode and either do it with an external tool or defer it till after all the files finish downloading. I just think you're running into some inherent performance limits of your setup.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44865
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Postby jacob733 » Thu Mar 11, 2010 12:00 pm

Probably. I just wonder where the limitation comes from, if there might be something in wine doing it. There is definitely a disk I/O aspect of it too, since files that fit in cache are usually ok. Luckily Linux handles disk cache much better than Windows (possibly just pre-7), so I gain something there.
jacob733
Occasional Contributor
Occasional Contributor
 
Posts: 38
Joined: Wed Jan 09, 2002 12:44 pm
Location: Denmark

Registered Newsbin User since: 04/26/03

Postby Quade » Thu Mar 11, 2010 12:11 pm

Typically it's ALL disk IO. SATA doesn't like simultaneous access.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44865
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Postby DThor » Thu Mar 11, 2010 12:28 pm

Yeah, the problem's not wine...I run linux/wine(32 bit of course)/newsbin on a crazy-ass fast system(cpu and IO) at one location and it positively screams. Everything is *blip*, done. At home, on a slightly more humble system running 64 bit Windows, quad core intel with a more typical consumer hard drive, it has all the same IO symptoms you're describing. I'm pretty sure it's all about disk IO. Bring on moar SSD's! :)

DT
V6 Troubleshooting FAQ . V6 docs. Usenet info at Usenet Tools. Thanks!
User avatar
DThor
Elite NewsBin User
Elite NewsBin User
 
Posts: 5943
Joined: Mon Jul 01, 2002 9:50 am

Registered Newsbin User since: 04/01/03

Postby jacob733 » Sat Mar 13, 2010 3:20 am

I see. Recommendation for running Newsbin with 100Mbps: Crazy fast SSD :)
jacob733
Occasional Contributor
Occasional Contributor
 
Posts: 38
Joined: Wed Jan 09, 2002 12:44 pm
Location: Denmark

Registered Newsbin User since: 04/26/03

Postby Quade » Sat Mar 13, 2010 4:15 am

I've run it at 100+ on my machine to a normal disk drive. You just have to be careful what else you're doing with this disk.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44865
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Postby jacob733 » Sat Mar 13, 2010 7:37 am

How many connections were you running then? I have a long distance to my service provider, so I need many connections to get a decent speed. I assume Newsbin does file I/O for each connection independently, which would cause excessive I/O when running 20-30 connections at high speed.
jacob733
Occasional Contributor
Occasional Contributor
 
Posts: 38
Joined: Wed Jan 09, 2002 12:44 pm
Location: Denmark

Registered Newsbin User since: 04/26/03

Postby Quade » Sat Mar 13, 2010 11:28 am

10 connections probably. Disk IO is locked so, only one thread writes at a time.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44865
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Postby jacob733 » Sun Mar 28, 2010 8:33 am

I just got a new PC for running Newsbin (not the reason to buy though), and it looks like it has nothing at all to do with disk I/O or number of cores.
My new PC is a Core i7-860, and when I get it in the slow state, there is almost no disk I/O going on, probably because it is very slow to decode.

When I managed to provoke the state for the decoder, I increased the number of connections to 50, which increased my throughput to about 7-8MB/s. The throughput at this point seems to be limited by CPU, since the CPU load I can measure from Newsbin is approximately 1 core 100% loaded. I'm guessing this means that all download threads run in a single CPU thread?
Furthermore, I can see that the wine service takes a fair amount of CPU during download, and increases a bit during decode. First part of decode is relatively fast, and hits a load of 150% for the Newsbin process. But after going to slow state, this changes radically: The Newsbin process goes down to about 66% CPU load, but the wine process doubles its CPU load. It looks like Newsbin gets into some state where it does excessive system calls, and since Wine might be a bit slower than Windows to service the calls, this can increase the effect.
jacob733
Occasional Contributor
Occasional Contributor
 
Posts: 38
Joined: Wed Jan 09, 2002 12:44 pm
Location: Denmark

Registered Newsbin User since: 04/26/03

Postby jacob733 » Sat Jul 03, 2010 4:32 am

I guess it is time to update this one:
I have been running with 5.55 for a couple of months now, and the first thing to notice is that the slow decode problem is gone. Automatic unrar/repair is very fast in 5.55, even with Newsbin running heavy downloads.
Now a new problem is introduced in 5.55:
When the download runs with a large list, it sometimes stops moving completed files from the chunks folder to the download location, and they just stay in the list as paused/completed. They way I have managed to work around this is to shutdown Newsbin saving the download list, and then start it up again. The files will then stay in the download list as paused, but if I resume them, they will save properly. Only problem is that I have to resume them a small group at a time, otherwise my problem comes back. Another thing is that I see indications that some files have been missed in the process.
jacob733
Occasional Contributor
Occasional Contributor
 
Posts: 38
Joined: Wed Jan 09, 2002 12:44 pm
Location: Denmark

Registered Newsbin User since: 04/26/03

Postby Mastermind » Sun Jul 04, 2010 4:53 am

jacob733 wrote:When the download runs with a large list, it sometimes stops moving completed files from the chunks folder to the download location, and they just stay in the list as paused/completed.


Have you tried to work without a chunk folder, so the files have to rename only.
User avatar
Mastermind
Seasoned User
Seasoned User
 
Posts: 113
Joined: Sun Aug 17, 2003 7:54 am

Registered Newsbin User since: 04/15/03

Postby itimpi » Sun Jul 04, 2010 5:03 am

It should make no difference - even when using a Chunks folder the .nb! files are simply renamed (as long as the post was completel downloaded).
The Newsbin Online documentation
The Usenettools for tutorials, useful information and links
User avatar
itimpi
Elite NewsBin User
Elite NewsBin User
 
Posts: 12604
Joined: Sat Mar 16, 2002 7:11 am
Location: UK

Registered Newsbin User since: 03/28/03

Single vs dual core

Postby Brosss » Wed Jun 22, 2016 2:06 pm

Ubuntu 8.04

I have the OS, but I would like to make dual booting System. I have windows on a 15 GB partition. I have another 80GB on the other partition that i just use for PlayStation related things. Is there a way I can install Ubuntu on my 80GB partition and still be able to boot windows. Or perhaps have some sort of choice to boot an OS when cpu boots from a hard shutdown?
Brosss
n00b
n00b
 
Posts: 2
Joined: Mon Jun 13, 2016 5:47 pm


Return to Linux

Who is online

Users browsing this forum: No registered users and 1 guest

cron