Page 1 of 1

Cache Not Flushing

PostPosted: Thu Oct 29, 2015 8:46 am
by TBlack
6.62B All Versions Win7 SP1 32-Bit 4Gb ChunkCacheSize=750

There's been three instances where during normal operations of downloading posts, assembling, etc that NB froze at Cache: 0/750 (0). All downloading stopped, even Header Updates. Had to do a Help/About which did work instead of an Exit and Restart. There is no Windows Event Log.

Thanks

Re: Cache Not Flushing

PostPosted: Thu Oct 29, 2015 11:12 am
by Quade
If you're running 32 bits, you need to make sure Newsbin doesn't go over 2 gigs of RAM use. Not saying that's the problem but with 750 cache blocks, you're already at 750 megs of RAM. If you "load all" of any large groups with long retention, you can easily hit 2 gigs.

Other than that, I don't know at this point. Sounds almost like it's stuck in "autopar" mode which prevents flushing.

Re: Cache Not Flushing

PostPosted: Thu Oct 29, 2015 11:51 am
by TBlack
Have gone back to the Default Cache Size. Will watch and advise.

Thanks

Re: Cache Not Flushing

PostPosted: Thu Oct 29, 2015 8:41 pm
by TBlack
Am running ChunkCacheSize=500 and it just happened again. NB froze. Cache: 0/500 (0)

NB was downloading a single 1.47 Gb file and started a Header update on 15 Groups.

https://onedrive.live.com/redir?resid=5 ... hoto%2cjpg

The Resource Monitor Cap says there is available memory if I am reading it correctly.

Thanks

Re: Cache Not Flushing

PostPosted: Thu Oct 29, 2015 10:52 pm
by Quade
It's not how much RAM you system has, It's how much RAM Newsbin is using. 32 bit apps in 32 bit windows are capped at 2 Gigs no matter how much RAM you have. It was just something to think about.

What else has changed on this system? What's the download and data folder free disk space look like?

It sounds like flushing to disk isn't working. If an UnRAR or repair is currently running, it's normal to stall until they finish.

Thinking about it, I have to wonder why it's using 500 blocks in the first place. Normally it just uses enough blocks for the one currently downloading file and maybe a couple for the look-ahead. It'll use more if there are incomplete files.

It's pointing more toward something to do with the unrar/repair process because that's the only time all 500 should be getting used.

Re: Cache Not Flushing

PostPosted: Thu Oct 29, 2015 11:17 pm
by TBlack
D: Drive for Download/UnRar has 255 Gb free. Running 5 Gb Page File on C: Drive. This has been the same and not changed for a very long time.

Have not changed anything on my system. NB Versions of course and have been playing with ChunkCacheSize to get concurrent downloads during Rar Repair and UnRars. Returned the ChunkCacheSize back to default.

Thanks

Re: Cache Not Flushing

PostPosted: Thu Oct 29, 2015 11:23 pm
by TBlack
Just froze again while completing my last message. Cache: 0/200 (0)

5 Files in the Download List, some with Black Bars on the right of Purple Bars
Header download update did not complete.

Help/About unfroze NB.

Thanks

Re: Cache Not Flushing

PostPosted: Fri Oct 30, 2015 9:38 am
by TBlack
Just had Win7 do a CheckDisk on both my C & D Drives.....no issues. Both drives are NTSF.

Thanks

Re: Cache Not Flushing

PostPosted: Fri Oct 30, 2015 10:01 am
by Quade
If you look at the status bar, 6th segment from the left, what are the 2 numbers?

Re: Cache Not Flushing

PostPosted: Fri Oct 30, 2015 10:10 am
by TBlack
1.75 TB | 250.12 GB

Re: Cache Not Flushing

PostPosted: Thu Nov 05, 2015 2:24 am
by aln
More data points to help pinpoint the problem.

Still happening in 6.62B4 - Special: 00 F1 50 D3 1E AD .

I can't always be monitoring the program to catch it
out of cache "0/200 (0)". Yes, the Help|About trick flushes
the cache ... but it's still happening.

I'm using Win7 SP1 64 bit with 16GB of memory and lots
of free HD space. The OS shows free mem of >10GB.

Should I increase the cache size? How? Where?

Thanks.

Re: Cache Not Flushing

PostPosted: Thu Nov 05, 2015 10:38 am
by Quade
What build number is that? I punched up another one last night.

Re: Cache Not Flushing

PostPosted: Thu Nov 05, 2015 10:51 am
by TBlack
6.62B4 - Special: 00 FB ED AD 9C E4

Re: Cache Not Flushing

PostPosted: Thu Nov 05, 2015 12:37 pm
by aln
6.62B4 Build 4337

Re: Cache Not Flushing

PostPosted: Thu Nov 05, 2015 12:41 pm
by Quade
http://www.newsbin.com/downloads/nb662B4.exe

This is build 4348. I can't make it fail like you guys so, I'm just trying some stuff and hoping I catch it.

The build number is always on top of the main screen. You don't need to look in help/about.

Re: Cache Not Flushing

PostPosted: Thu Nov 05, 2015 12:52 pm
by TBlack
Ok. Yes am running 4348 since late Last evening.

Thanks

Re: Cache Not Flushing

PostPosted: Fri Nov 06, 2015 11:43 am
by aln
Using build 4348 - still happening. Same symptoms. How can I increase the cache size?

Re: Cache Not Flushing

PostPosted: Fri Nov 06, 2015 11:54 am
by Quade
Using build 4348 - still happening. Same symptoms. How can I increase the cache size?


You can but it won't help. The problem is you're running out of chunks for some reason. More cache might delay but won't fix it. Lets drill into how you download more.

How many servers are you using?
Are they all good servers?
How many connections to the servers are you using?

I think we're going to find that there's some connection to what or how you're downloading that will lead to a solution.

Re: Cache Not Flushing

PostPosted: Fri Nov 06, 2015 12:04 pm
by aln
1 server (newshosting)
18 connections (should I lower this?)

Quality of connection to Newshosting is generally excellent, but the past few
weeks have seen some propagation delays. Seems to have resolved itself.

One thought - I check for new posts every 15 minutes - and noticed that
90% of the time when there is a stall (cache goes to zero) seems to be during
a refresh of headers while an extraction is in progress.

Re: Cache Not Flushing

PostPosted: Fri Nov 06, 2015 12:06 pm
by aln
Correction: group updates are set at 20 minutes (not 15)

Re: Cache Not Flushing

PostPosted: Fri Nov 06, 2015 1:09 pm
by kirm
1 server (newshosting)
18 connections (should I lower this?)


Using newshosting with win10 and have no issues like you are reporting. I am using 8 connections to this server. Not for sure positive, but you may want to try to lower your number of connections.

18 connections might be overkill.

Re: Cache Not Flushing

PostPosted: Fri Nov 06, 2015 5:01 pm
by TBlack
Here's a Tip from the Advanced Settings Section of the Board:

Connections: You should not use all the connections available to you. This might seem a bit counter-intuitive, but it is because news servers tend to be a bit slow in making connections available for re-use if Newsbin closes and then re-opens a connection. This can result in errors being reported on such connections and this causes Newsbin to have to do re-tries which slows things down. You only need the amount of connections required to use your full bandwidth. Using more can cause to other issues and wastes system resources.

I can get the max bandwidth out of UseNetServer with 8 Connections. You don't need any more connections than what give you the max d/l speed.

Hope that helps

Re: Cache Not Flushing

PostPosted: Sat Nov 07, 2015 11:48 am
by aln
Using latest build - 6.62RC1 4351 - still happens. I had already reduced my
server connections from 18 down to 9. Any suggestions?

Re: Cache Not Flushing

PostPosted: Sat Nov 07, 2015 12:27 pm
by Quade
Do you have another PC to try it on? I can't make it break here which makes it pretty hard to trouble-shoot.

Re: Cache Not Flushing

PostPosted: Sat Nov 07, 2015 1:00 pm
by aln
Just another observation: When the program "stalls" - as I call it - doing the
Help|About trick will get it going again - but the frequency of stalling
starts to increase. If I exit the program and start anew - the stalling
goes away and doesn't return for many hours.

I suspect a memory leak.

FYI: With the exception of the mp3 groups - the general display age is 14
and storage age is 21. (My mp3 groups are set to 180 and 180 because
I never know when I will want to browse the group.)

FYI: My Watch_Topic list numbers 208 - but most are disabled.

FYI: As mentioned before, I dropped the number of server connections
to 9 from a previous value of 18.

This configuration had been rock steady until the last few weeks. Newsbin
is the only program that runs continuously.

As for using another PC to try it on - I have an old 32bit laptop that is
painfully slow with little HD space. I'd rather not try and duplicate the
problem on it - since it will be a different OS version and, did I mention,
it is painfully slow? :-)

I am trying to think of how my configuration or usage might be unusual.
Any suggestions at what to look at?

Re: Cache Not Flushing

PostPosted: Wed Nov 11, 2015 6:46 pm
by aln
Still failing (stalling) in 6.62 RC Build 4356

Re: Cache Not Flushing

PostPosted: Wed Nov 11, 2015 8:29 pm
by Quade
How fast is your download speed?

Re: Cache Not Flushing

PostPosted: Thu Nov 12, 2015 3:50 am
by aln
50Mbps down, 11-12Mbps upload.

Re: Cache Not Flushing

PostPosted: Thu Nov 12, 2015 7:18 pm
by aln
Kind of tired of it stalling 5-6 times per day. Any suggestions before I start walking back the releases
for a stable branch?

Re: Cache Not Flushing

PostPosted: Thu Nov 12, 2015 8:17 pm
by Quade
I'll be honest, I have no idea why you're having this issue. I queued up about 50 gigs last night. I changed Newsbin to auto-update headers every 15 minutes. Most of the files were 430's so I have a bunch of bad files. When it was done I still had 200 chunks in the cache. You're not alone. At least one other person seems to have the same issue. It's just not clear why you're having it and I'm not.

It would be interesting to see what happens if you roll back to 6.56. I intend to keep looking into it but I'm guessing I'm going to find the answer through some accident.

Re: Cache Not Flushing

PostPosted: Mon Nov 16, 2015 1:13 pm
by TBlack
RC2

Downloading new Headers and did a Ctrl-R to view file contents. Froze. Cache: 0/400 (0) with two Groups still needing Header Updates.

Nothing in the Log File. Nothing in the Event Log.

Help/About unfroze.

[12:24:41] ERROR UseNet alt.binaries.teevee Header Download Failed: ReadFailure

Could a Read Failure cause a freeze?

Thanks

Re: Cache Not Flushing

PostPosted: Mon Nov 16, 2015 2:36 pm
by Quade
Do you have typical retention set on any of your servers?

Re: Cache Not Flushing

PostPosted: Mon Nov 16, 2015 2:41 pm
by TBlack
Have one Server...Usenetserver

Typical Retention is not set (unchecked)

Thanks

Re: Cache Not Flushing

PostPosted: Mon Nov 16, 2015 3:15 pm
by Quade
How many connections do you have set for them?

Re: Cache Not Flushing

PostPosted: Mon Nov 16, 2015 4:27 pm
by TBlack
8 on 90 MB/s line. Eight usually maxes my bandwidth.

Also have my Router QOS set with NewsBin on Port 563 set to Normal Priority with all of my Lan Ports set to Highest Priority.

Thanks

Re: Cache Not Flushing

PostPosted: Mon Nov 16, 2015 4:49 pm
by Quade
Ok thanks.

Just digging through the code.

Re: Cache Not Flushing

PostPosted: Mon Nov 16, 2015 5:09 pm
by TBlack
No problem :)

Re: Cache Not Flushing

PostPosted: Mon Nov 16, 2015 7:10 pm
by Quade
We're going to release with the current 6.62. That doesn't mean I'm ignoring this though. Just that it's much better than 6.61.

I spent the day looking at the code. I suspect some race condition that simply doesn't happen to me. I might give you some test versions to try while I trouble-shoot this.

Re: Cache Not Flushing

PostPosted: Tue Nov 17, 2015 4:21 pm
by TBlack
Just had another freeze at Cache: 0/400 (0)

Was downloading a file that was partially complete and updated Headers for one Group and everything froze.

After Help/About, Cache went to 237/400 (0). NB continued Header Download and continued the file that was incomplete.

Just did Help/About again, same result: 237/400 (0).

Had to close and reopen NB to get 400/400.

Thanks

Re: Cache Not Flushing

PostPosted: Fri Dec 04, 2015 2:09 am
by aln
Just an FYI - still happening with latest release.

Re: Cache Not Flushing

PostPosted: Fri Dec 04, 2015 2:25 am
by Quade
Just an FYI - still happening with latest release.


Which latest release is that?

http://www.newsbin.com/downloads/nb663B1.exe

I've asked several people to try this one and gotten no feedback on this issue.

Re: Cache Not Flushing

PostPosted: Fri Dec 04, 2015 2:52 am
by aln
I was using the RC release from the beta page.

I'll try this new version and report back.

(Thanks for not giving up.)

Re: Cache Not Flushing

PostPosted: Sun Dec 06, 2015 4:28 am
by aln
No stalling so far using the 6.63 B1 release ... it's been 24+ hours ...

Re: Cache Not Flushing

PostPosted: Sun Dec 06, 2015 11:13 am
by Quade
Thanks for the feedback. I hope it continues to work properly.

Re: Cache Not Flushing

PostPosted: Tue Dec 08, 2015 1:56 am
by aln
The 6.63B1 Special has been working flawlessly for over 3 days. I think you've finally nailed it!

Re: Cache Not Flushing

PostPosted: Fri Dec 11, 2015 1:52 am
by aln
More info to report:

Still rock solid with no restarts required in 6+ days. The cache line
returns to 200/200 when there is no active downloads in progress.

Good job!