Cache Flush?

This is the place to help test and discuss Version 6 Beta releases.

Cache Flush?

Postby TBlack » Wed Apr 29, 2015 10:37 am

Beta 12

NB is not doing anything after a file and header download....it is totally idle and the Cache entry on the Status Bar says:

Cache: 143/200 (0)

Is something possibly not be flushed? Could this be contributing to the hang that some are reporting?

If I close NB and then reopen, after the Header Download is parsed, I see Cache: 200/200 (0)

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Flush?

Postby Quade » Wed Apr 29, 2015 10:48 am

Just some chunks hanging out. Might be partial files or signature matches. You can flush it by hitting "Help/About".

Partial or filtered files can leave stragglers, they get flushed out when the space is needed by something else.

Thanks.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby TBlack » Wed Apr 29, 2015 11:00 am

Quade wrote:You can flush it by hitting "Help/About"


A hidden feature of Help/About? Don't see anything that says Flush the Cache.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Flush?

Postby Quade » Wed Apr 29, 2015 11:15 am

I just use it for testing. All it does is force stragglers to disk.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby RayMark » Wed May 06, 2015 9:47 am

I continue having problems with downloads stalling @ 0/200 even with beta 13. Only beta 3 is working without stalling.
I have to do Help/About many many times to download anything - and that is with books only. I don't even try anything above beta 3 with large files.

Can you please hook up help/about so that it is done continuously without manual interventions? As it is now, 6.60 certainly cannot be released as final until that issue is fixed.


P.S.

While I was typing this message, NewsBin stalled again―obviously―I was not doing Help/About every 3 seconds while I was typing this...
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: Cache Flush?

Postby Quade » Wed May 06, 2015 9:57 am

It is automatic for me. You're just having another weird problem.

I accept that there's a problem in Newsbin because this shouldn't happen. The question is why is it happening to you? Something is different about how you download. Something is happening that prevents the cache flush during normal operation.

I'd suggest removing your V scanner and firewall (except the MS supplies ones, they're OK). Then reboot and try again running naked. Then try some test downloads and see if this still happens.

I'd like to get to the bottom of this but it's hard because it doesn't happen to me.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby Quade » Wed May 06, 2015 10:22 am

I have an idea. Something to try anyway.

In the performance options if you have "Pause Download during unrar" checked, uncheck it and then restart Newsbin.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby RayMark » Wed May 06, 2015 2:01 pm

Yes, pause during unrar was switched on. I will try without it.
However, when it happened today, autopar was disabled and rar files were not unrarred at all, so not sure if it can help.
I was mostly downloading lots of very small rar files and not extracting. Many less than 1 MB, but some 30 MB size files began causing the problems.


Here is a hint: the problems began after beta 3. With beta 3 there is no stalling. I missed beta 4, but the problems began in beta 5 and are continuing up to and including beta 13.
Think what you changed in beta 4 or beta 5
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: Cache Flush?

Postby Quade » Wed May 06, 2015 2:15 pm

Turn autopar back in. Disable unrar in the group properties. See what happens.

I never run with autopar disabled.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby RayMark » Mon May 18, 2015 4:53 am

I did. Autopar is on, pause during unrarring is off.

It is worse with beta 13 now if anything. Downloads stopped both with 700 small files (ebooks) (with unrarring switched off locally) and also with regular large files added from the internet search (with unrarring on). Not only Help/About was needed to move forward (but only very little forward, soon again stalls) but also now I noticed that while doing nothing with 0/200, NewsBin was still using 75-92% CPU (quad-core). I never noticed such CPU usage before. Not sure if it was related to the 0/200 stall or maybe just coincidence.
Generally, it seems that with autopar on and pausing during unrarring off it takes longer for that 0/200 stall to occur. But sometimes it occurs pretty quickly. Maybe it is caused by some specific files being downloaded.
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: Cache Flush?

Postby RayMark » Mon May 18, 2015 5:04 am

Unfortunately, it occurred again while I was typing the previous post. In minutes. With regular large files. Forgot to check the CPU usage....

And again - in a couple of minutes. So it probably has to be because of some specific combination of files. I basically loaded a bunch of web-dl posts from the internet search.
I am watching the CPU usage now: Initially after stalling it was below 4% but gradually grew up to 15% but then stabilized at 11%. Still a lot, considering that NewsBin has stalled and does nothing (nothing useful to me, at least).

And again - this time the CPU usage stays @ 1-2%. Of course, each time before NewsBin stalls, it downloads some files so the situation is different each time.
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: Cache Flush?

Postby RayMark » Tue May 19, 2015 11:55 pm

Quade, it might be important?

Now I had the stall with 6.60 beta 3 - I always thought that the problem began AFTER beta3.
I really don't remember downloading stalling with beta 3 ever before. I am using beta 3 specifically for stall-free downloads.
And Help/About helped very little. The download speed got up from 0, but almost immediately went back to 0 again.

I have another thought about the reason and why it might be difficult to reproduce for you:
I am using 4 servers simultaneously, and after re-starting NewsBin now I got lots and lots of "430 No such article" messages from two of them.
Although the posts were complete on the other servers and downloaded successfully.
Perhaps multiple servers + multiple "430 No such article" messages can cause this stalling?
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: Cache Flush?

Postby RayMark » Wed May 20, 2015 12:11 am

UPDATE:

I take it back about beta 3 stalling - turns out, it encountered par2 file and began a long process of creating almost 5 GB worth of rar files that were next in the download list and could have been downloaded much faster. CPU usage: 17%
It also does not show any progress so I don't even know how long to wait. But I counted the downloaded rar files and they all are present even though 17 of them are not yet downloaded.
I moved those par2 files to the end of the list but that does not make the rar files to be downloaded. I guess, I will restart again, thus interrupting that par2 process.
The newer betas perhaps handle this better. I just forgot this side of beta 3...

However, the reason for stalling of newer betas perhaps might be related to multiple servers + multiple 430 "No such article" messages?
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: Cache Flush?

Postby RayMark » Wed Jul 15, 2015 1:22 am

NewsBin 6.60 RC4 stalled @ 0/200. After that, a single Help->About downloads 3-4 MB only. I have to press repeatedly (continuously) Help->About in order to finish downloads.
Why can't it be done automatically? If I can click and click and click Help->About, the same thing can be done automatically when downloading stalls @ 0/200. Probably not a solution but a work-around at least.

I had big hopes that the fix in RC2 against stalling @ 0/200 was general enough. But no, no such luck.
This time the problem is certainly not with duplicate spam. Or is it?
It seems to be rather with my headers that sometimes have duplicate entries: when downloading headers from 4 different servers somehow it happened. Although possibly sometimes even with different message IDs.
So perhaps duplicate headers can be considered as spam....
Anyway. There is something very wrong with 6.60 if it cannot be fixed once and for all not to stall @ 0/200 whatever the garbage input.

I am very afraid that also future versions such as 6.70, 6.80, 7.00, 8.00, 10.00, etc. will continue with 6.60 stalling problems and the last good NewsBin will forever remain 6.55 Final... :(
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: Cache Flush?

Postby Quade » Wed Jul 15, 2015 9:17 am

It seems to be rather with my headers that sometimes have duplicate entries: when downloading headers from 4 different servers somehow it happened. Although possibly sometimes even with different message IDs.


Ok, download headers from a single server then and tell me if the problem goes away. I have no problem working on this, just no way to reproduce it. I download headers from 2 servers. I only really need to download from one but I do 2 simply to test it.

Since you feel it's something about how you download that causes this. I suggest changing how you download till it goes away. For example if you download headers from a single server and it goes away, that's a good clue.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby TBlack » Wed Jul 15, 2015 10:03 am

Quade,

Any chance you can put the 6.56 Cache Code in 6.60 with a Option Selection in Options for Testing and/or Use? 6.56 seems to be stable. Was there a reason to change it?

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Flush?

Postby Quade » Wed Jul 15, 2015 10:36 am

No, no chance. The problem probably has nothing to do with the actual cache code. More than likely something else is crashing internally and losing a cache block. Just patching code in and hoping it'll fix this is kinda pointless. The only way to fix this is to characterize the problem and then go from there. It's not happening to me so, clearly my PC or something I'm doing is different than Raymark.

Are you having download stalling issue too Tblack? With Raymark, it never recovers. Going to zero is normal. Not recovering from going to zero isn't.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby Quade » Wed Jul 15, 2015 11:04 am

Just for shits and grins. Disable "Use Recycle Bin" in Newsbin and disable it for the download drive too and see if the symptoms change. I just had a long stall that I thought was Raymarks problem but in fact I had an unrar what was doing a shitload of post-unrar deletes and this process stalled the rest of the download.

The filter from a single unrar tossed 6000 files in the recycle bin. Took forever and the download stalled while it was happening.

6.56 didn't recycle this stuff.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby TBlack » Wed Jul 15, 2015 11:32 am

Quade wrote:Are you having download stalling issue too Tblack? With Raymark, it never recovers. Going to zero is normal. Not recovering from going to zero isn't.


No, I am not. Think I only ever had one stall and reported it, but have not for a long time. No recent issues.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Flush?

Postby TBlack » Wed Jul 15, 2015 12:19 pm

Just had this happen three times just today, same group:

[10:58:57] ERROR UseNet alt.binaries.boneless Header Download Failed: ReadFailure

Cache is: 200/200 (1)

There currently is a file stuck is a Boneless file stuck in the Import Folder.

Untouched file is here:

https://onedrive.live.com/redir?resid=5 ... =folder%2c
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Cache Flush?

Postby Quade » Wed Jul 15, 2015 12:30 pm

[10:58:57] ERROR UseNet alt.binaries.boneless Header Download Failed: ReadFailure


1 - Is a network read failure. Not a problem reading from the Import file.

2 - Does the import file end in .txt? If not, you can just delete it. Newsbin only imports GZ and TXT files. Header download failures like this can leave stragglers that won't ever import. The periodic data folder purge should eventually kill them.

Yeah looked at the file. No txt extension. It's basically junk at this point. Thanks.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Cache Flush?

Postby TBlack » Wed Jul 15, 2015 12:39 pm

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 2 guests