about header compaction in beta 15

This is the place to help test and discuss Version 6 Beta releases.

about header compaction in beta 15

Postby syshog » Sat Jun 06, 2015 10:04 pm

It's running faster than 6.56 which is very nice. I was wondering: Does any of the performance options affect NB's db access speed?

The raid array I have the NB db on can write at 200MB/s under heavy load from two arrays. NB while accessing the db files and actively compressing with no other programs accessing the db is only writing at 8-20MB/s and the disk load is not even reaching 50%, the cpu is around 8% with spikes to 20%, on only one thread. The array while being accessed by other 2 other arrays can easily read at 500MB/s simultaneously. Any way I can speed it up?

Also the logger reports being stuck on compressing boneless, but resource monitor states NB is accessing and writing warez, x, and worms.

All three groups were accessed after boneless but the verbose log did not update to reflect that. I'll keep an eye and see if it does that again.

I will say this compressing 225 groups and only taking 40 minutes vs hours on the old versions is mighty nice!
system:
xeon system with 128GB+ of ram and a lot of storage. I don't update my sig much.
User avatar
syshog
Seasoned User
Seasoned User
 
Posts: 117
Joined: Sun Jun 18, 2006 7:26 pm

Registered Newsbin User since: 06/17/06

Re: about header compaction in beta 15

Postby Quade » Sat Jun 06, 2015 10:31 pm

1 - Newsbin doesn't compress header downloads anymore.

2 - If you're talking about "Compact Group", the larger the group, the longer it takes. Boneless will clearly be the slowest.

You might want to tell Windows to use the raid array for the "temp" folders. You do that in the control panel environment settings. I have all my temp files going to a shallow folder on an SSD drive so I can manually see what's there. The default is in some windows folders.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: about header compaction in beta 15

Postby syshog » Tue Jun 09, 2015 6:42 pm

The os runs off of a SSD (samsung 840 pro 512GB).

the NB runs off the raid array and interfaces with the db only using a single thread on the cpu.
system:
xeon system with 128GB+ of ram and a lot of storage. I don't update my sig much.
User avatar
syshog
Seasoned User
Seasoned User
 
Posts: 117
Joined: Sun Jun 18, 2006 7:26 pm

Registered Newsbin User since: 06/17/06

Re: about header compaction in beta 15

Postby Quade » Tue Jun 09, 2015 6:58 pm

Only one writer at a time into the DB. That's how it works.

Since compaction should be relatively rare, I don't see much point to multi-threading it.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: about header compaction in beta 15

Postby syshog » Wed Jun 10, 2015 1:08 am

I have headers for 225 groups the spool folder is 1.35TB, so any space and time I can save is important to me. I can use winirar5 to cut the db down to 600GB but that takes hours.
system:
xeon system with 128GB+ of ram and a lot of storage. I don't update my sig much.
User avatar
syshog
Seasoned User
Seasoned User
 
Posts: 117
Joined: Sun Jun 18, 2006 7:26 pm

Registered Newsbin User since: 06/17/06


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 3 guests