Cache Not Flushing

This is the place to help test and discuss Version 6 Beta releases.

Cache Not Flushing

Postby TBlack » Thu Oct 29, 2015 8:46 am

6.62B All Versions Win7 SP1 32-Bit 4Gb ChunkCacheSize=750

There's been three instances where during normal operations of downloading posts, assembling, etc that NB froze at Cache: 0/750 (0). All downloading stopped, even Header Updates. Had to do a Help/About which did work instead of an Exit and Restart. There is no Windows Event Log.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby Quade » Thu Oct 29, 2015 11:12 am

If you're running 32 bits, you need to make sure Newsbin doesn't go over 2 gigs of RAM use. Not saying that's the problem but with 750 cache blocks, you're already at 750 megs of RAM. If you "load all" of any large groups with long retention, you can easily hit 2 gigs.

Other than that, I don't know at this point. Sounds almost like it's stuck in "autopar" mode which prevents flushing.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Thu Oct 29, 2015 11:51 am

Have gone back to the Default Cache Size. Will watch and advise.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby TBlack » Thu Oct 29, 2015 8:41 pm

Am running ChunkCacheSize=500 and it just happened again. NB froze. Cache: 0/500 (0)

NB was downloading a single 1.47 Gb file and started a Header update on 15 Groups.

https://onedrive.live.com/redir?resid=5 ... hoto%2cjpg

The Resource Monitor Cap says there is available memory if I am reading it correctly.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby Quade » Thu Oct 29, 2015 10:52 pm

It's not how much RAM you system has, It's how much RAM Newsbin is using. 32 bit apps in 32 bit windows are capped at 2 Gigs no matter how much RAM you have. It was just something to think about.

What else has changed on this system? What's the download and data folder free disk space look like?

It sounds like flushing to disk isn't working. If an UnRAR or repair is currently running, it's normal to stall until they finish.

Thinking about it, I have to wonder why it's using 500 blocks in the first place. Normally it just uses enough blocks for the one currently downloading file and maybe a couple for the look-ahead. It'll use more if there are incomplete files.

It's pointing more toward something to do with the unrar/repair process because that's the only time all 500 should be getting used.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Thu Oct 29, 2015 11:17 pm

D: Drive for Download/UnRar has 255 Gb free. Running 5 Gb Page File on C: Drive. This has been the same and not changed for a very long time.

Have not changed anything on my system. NB Versions of course and have been playing with ChunkCacheSize to get concurrent downloads during Rar Repair and UnRars. Returned the ChunkCacheSize back to default.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby TBlack » Thu Oct 29, 2015 11:23 pm

Just froze again while completing my last message. Cache: 0/200 (0)

5 Files in the Download List, some with Black Bars on the right of Purple Bars
Header download update did not complete.

Help/About unfroze NB.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby TBlack » Fri Oct 30, 2015 9:38 am

Just had Win7 do a CheckDisk on both my C & D Drives.....no issues. Both drives are NTSF.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby Quade » Fri Oct 30, 2015 10:01 am

If you look at the status bar, 6th segment from the left, what are the 2 numbers?
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Fri Oct 30, 2015 10:10 am

1.75 TB | 250.12 GB
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby aln » Thu Nov 05, 2015 2:24 am

More data points to help pinpoint the problem.

Still happening in 6.62B4 - Special: 00 F1 50 D3 1E AD .

I can't always be monitoring the program to catch it
out of cache "0/200 (0)". Yes, the Help|About trick flushes
the cache ... but it's still happening.

I'm using Win7 SP1 64 bit with 16GB of memory and lots
of free HD space. The OS shows free mem of >10GB.

Should I increase the cache size? How? Where?

Thanks.
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby Quade » Thu Nov 05, 2015 10:38 am

What build number is that? I punched up another one last night.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Thu Nov 05, 2015 10:51 am

6.62B4 - Special: 00 FB ED AD 9C E4
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby aln » Thu Nov 05, 2015 12:37 pm

6.62B4 Build 4337
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby Quade » Thu Nov 05, 2015 12:41 pm

http://www.newsbin.com/downloads/nb662B4.exe

This is build 4348. I can't make it fail like you guys so, I'm just trying some stuff and hoping I catch it.

The build number is always on top of the main screen. You don't need to look in help/about.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Thu Nov 05, 2015 12:52 pm

Ok. Yes am running 4348 since late Last evening.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby aln » Fri Nov 06, 2015 11:43 am

Using build 4348 - still happening. Same symptoms. How can I increase the cache size?
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby Quade » Fri Nov 06, 2015 11:54 am

Using build 4348 - still happening. Same symptoms. How can I increase the cache size?


You can but it won't help. The problem is you're running out of chunks for some reason. More cache might delay but won't fix it. Lets drill into how you download more.

How many servers are you using?
Are they all good servers?
How many connections to the servers are you using?

I think we're going to find that there's some connection to what or how you're downloading that will lead to a solution.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby aln » Fri Nov 06, 2015 12:04 pm

1 server (newshosting)
18 connections (should I lower this?)

Quality of connection to Newshosting is generally excellent, but the past few
weeks have seen some propagation delays. Seems to have resolved itself.

One thought - I check for new posts every 15 minutes - and noticed that
90% of the time when there is a stall (cache goes to zero) seems to be during
a refresh of headers while an extraction is in progress.
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby aln » Fri Nov 06, 2015 12:06 pm

Correction: group updates are set at 20 minutes (not 15)
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby kirm » Fri Nov 06, 2015 1:09 pm

1 server (newshosting)
18 connections (should I lower this?)


Using newshosting with win10 and have no issues like you are reporting. I am using 8 connections to this server. Not for sure positive, but you may want to try to lower your number of connections.

18 connections might be overkill.
kirm
Seasoned User
Seasoned User
 
Posts: 249
Joined: Sun Aug 17, 2003 4:17 pm

Registered Newsbin User since: 05/15/03

Re: Cache Not Flushing

Postby TBlack » Fri Nov 06, 2015 5:01 pm

Here's a Tip from the Advanced Settings Section of the Board:

Connections: You should not use all the connections available to you. This might seem a bit counter-intuitive, but it is because news servers tend to be a bit slow in making connections available for re-use if Newsbin closes and then re-opens a connection. This can result in errors being reported on such connections and this causes Newsbin to have to do re-tries which slows things down. You only need the amount of connections required to use your full bandwidth. Using more can cause to other issues and wastes system resources.

I can get the max bandwidth out of UseNetServer with 8 Connections. You don't need any more connections than what give you the max d/l speed.

Hope that helps
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby aln » Sat Nov 07, 2015 11:48 am

Using latest build - 6.62RC1 4351 - still happens. I had already reduced my
server connections from 18 down to 9. Any suggestions?
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby Quade » Sat Nov 07, 2015 12:27 pm

Do you have another PC to try it on? I can't make it break here which makes it pretty hard to trouble-shoot.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby aln » Sat Nov 07, 2015 1:00 pm

Just another observation: When the program "stalls" - as I call it - doing the
Help|About trick will get it going again - but the frequency of stalling
starts to increase. If I exit the program and start anew - the stalling
goes away and doesn't return for many hours.

I suspect a memory leak.

FYI: With the exception of the mp3 groups - the general display age is 14
and storage age is 21. (My mp3 groups are set to 180 and 180 because
I never know when I will want to browse the group.)

FYI: My Watch_Topic list numbers 208 - but most are disabled.

FYI: As mentioned before, I dropped the number of server connections
to 9 from a previous value of 18.

This configuration had been rock steady until the last few weeks. Newsbin
is the only program that runs continuously.

As for using another PC to try it on - I have an old 32bit laptop that is
painfully slow with little HD space. I'd rather not try and duplicate the
problem on it - since it will be a different OS version and, did I mention,
it is painfully slow? :-)

I am trying to think of how my configuration or usage might be unusual.
Any suggestions at what to look at?
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby aln » Wed Nov 11, 2015 6:46 pm

Still failing (stalling) in 6.62 RC Build 4356
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby Quade » Wed Nov 11, 2015 8:29 pm

How fast is your download speed?
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby aln » Thu Nov 12, 2015 3:50 am

50Mbps down, 11-12Mbps upload.
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby aln » Thu Nov 12, 2015 7:18 pm

Kind of tired of it stalling 5-6 times per day. Any suggestions before I start walking back the releases
for a stable branch?
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby Quade » Thu Nov 12, 2015 8:17 pm

I'll be honest, I have no idea why you're having this issue. I queued up about 50 gigs last night. I changed Newsbin to auto-update headers every 15 minutes. Most of the files were 430's so I have a bunch of bad files. When it was done I still had 200 chunks in the cache. You're not alone. At least one other person seems to have the same issue. It's just not clear why you're having it and I'm not.

It would be interesting to see what happens if you roll back to 6.56. I intend to keep looking into it but I'm guessing I'm going to find the answer through some accident.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Mon Nov 16, 2015 1:13 pm

RC2

Downloading new Headers and did a Ctrl-R to view file contents. Froze. Cache: 0/400 (0) with two Groups still needing Header Updates.

Nothing in the Log File. Nothing in the Event Log.

Help/About unfroze.

[12:24:41] ERROR UseNet alt.binaries.teevee Header Download Failed: ReadFailure

Could a Read Failure cause a freeze?

Thanks
Last edited by TBlack on Mon Nov 16, 2015 2:36 pm, edited 1 time in total.
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby Quade » Mon Nov 16, 2015 2:36 pm

Do you have typical retention set on any of your servers?
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Mon Nov 16, 2015 2:41 pm

Have one Server...Usenetserver

Typical Retention is not set (unchecked)

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby Quade » Mon Nov 16, 2015 3:15 pm

How many connections do you have set for them?
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Mon Nov 16, 2015 4:27 pm

8 on 90 MB/s line. Eight usually maxes my bandwidth.

Also have my Router QOS set with NewsBin on Port 563 set to Normal Priority with all of my Lan Ports set to Highest Priority.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby Quade » Mon Nov 16, 2015 4:49 pm

Ok thanks.

Just digging through the code.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Mon Nov 16, 2015 5:09 pm

No problem :)
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby Quade » Mon Nov 16, 2015 7:10 pm

We're going to release with the current 6.62. That doesn't mean I'm ignoring this though. Just that it's much better than 6.61.

I spent the day looking at the code. I suspect some race condition that simply doesn't happen to me. I might give you some test versions to try while I trouble-shoot this.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby TBlack » Tue Nov 17, 2015 4:21 pm

Just had another freeze at Cache: 0/400 (0)

Was downloading a file that was partially complete and updated Headers for one Group and everything froze.

After Help/About, Cache went to 237/400 (0). NB continued Header Download and continued the file that was incomplete.

Just did Help/About again, same result: 237/400 (0).

Had to close and reopen NB to get 400/400.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Not Flushing

Postby aln » Fri Dec 04, 2015 2:09 am

Just an FYI - still happening with latest release.
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby Quade » Fri Dec 04, 2015 2:25 am

Just an FYI - still happening with latest release.


Which latest release is that?

http://www.newsbin.com/downloads/nb663B1.exe

I've asked several people to try this one and gotten no feedback on this issue.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby aln » Fri Dec 04, 2015 2:52 am

I was using the RC release from the beta page.

I'll try this new version and report back.

(Thanks for not giving up.)
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby aln » Sun Dec 06, 2015 4:28 am

No stalling so far using the 6.63 B1 release ... it's been 24+ hours ...
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby Quade » Sun Dec 06, 2015 11:13 am

Thanks for the feedback. I hope it continues to work properly.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Not Flushing

Postby aln » Tue Dec 08, 2015 1:56 am

The 6.63B1 Special has been working flawlessly for over 3 days. I think you've finally nailed it!
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04

Re: Cache Not Flushing

Postby aln » Fri Dec 11, 2015 1:52 am

More info to report:

Still rock solid with no restarts required in 6+ days. The cache line
returns to 200/200 when there is no active downloads in progress.

Good job!
aln
Occasional Contributor
Occasional Contributor
 
Posts: 22
Joined: Sat Mar 06, 2004 5:45 pm

Registered Newsbin User since: 03/06/04


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: Google [Bot] and 2 guests