Stuck downloading headers

This is the place to help test and discuss Version 6 Beta releases.

Stuck downloading headers

Postby oferlaor » Wed Nov 11, 2015 5:28 pm

I had beta4 stuck for a while not downloading headers. Restarting the app started it working again.

Is this a known issue? I updated to rc2
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: Stuck downloading headers

Postby Quade » Wed Nov 11, 2015 6:51 pm

What does the cache count look like down at the status bar? Do you have it set to automatically update headers.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Stuck downloading headers

Postby oferlaor » Thu Nov 12, 2015 2:24 am

I will do more research if it happens again. Yes, update headers is used and the download list had the headers on top.
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: Stuck downloading headers

Postby oferlaor » Thu Nov 12, 2015 5:30 pm

It occurred again. Cache says 0/200
Logs have a lot of 430 errors
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: Stuck downloading headers

Postby Quade » Thu Nov 12, 2015 5:37 pm

What happens if you hit "help/about" when it gets stuck?

430 errors happen during file downloads.

Long unrars will block the header downloads until the unrar completes because download runs out of packets.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Stuck downloading headers

Postby oferlaor » Fri Nov 13, 2015 3:22 am

When this happens no unrars or repairs are happening.

Restarting the application kicks outback into action


It looks like this happens after enough 430s are encountered.

The list of connections is empty on the right and all the server connections are disconnected.

What is help/about supposed to do?
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: Stuck downloading headers

Postby Quade » Fri Nov 13, 2015 12:38 pm

How many servers do you have?
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Stuck downloading headers

Postby oferlaor » Fri Nov 13, 2015 1:38 pm

One active.

Is it possible it is doing some rar/par work that I am not seeing?

Is there a way to determine what exactly the software is doing?
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: Stuck downloading headers

Postby syshog » Sat Nov 14, 2015 4:56 pm

Quade wrote:What happens if you hit "help/about" when it gets stuck?

430 errors happen during file downloads.

Long unrars will block the header downloads until the unrar completes because download runs out of packets.


How does the download run out of packets when the unrar process should not be connected to the internet? Would that not free up connections to allow the download process to continue?
system:
xeon system with 128GB+ of ram and a lot of storage. I don't update my sig much.
User avatar
syshog
Seasoned User
Seasoned User
 
Posts: 117
Joined: Sun Jun 18, 2006 7:26 pm

Registered Newsbin User since: 06/17/06

Re: Stuck downloading headers

Postby Quade » Sat Nov 14, 2015 7:22 pm

Unrars don't block downloading but it does block writing chunks of partial files to disk so when you run out of cache blocks, Newsbin will wait till more free up. The cache block writing to disk continues when the unrar/repair complete. That's how 6.62 works.

You can download headers and unrar at the same time as long as your file downloads haven't already run the system out of blocks during the unrar. The reason it blocks writes is that unrars can take as much as 3 times longer when the chunk writes happen at the same time.

People complained about this so a future version will permit them to run concurrently again. It's already working that way in the new engine. The problem with the old way was that file downloads could get way ahead of the unrar/repair and restarts became problematic. The new concurrent download doesn't permit that to happen because if writes chunks to disk. It's assembly of the files which is now deferred till after unrar/repair completes. So you still get full download speed and the unrar/repair can't get behind.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Stuck downloading headers

Postby oferlaor » Fri Nov 20, 2015 7:25 am

syshog wrote:
Quade wrote:What happens if you hit "help/about" when it gets stuck?

430 errors happen during file downloads.

Long unrars will block the header downloads until the unrar completes because download runs out of packets.


How does the download run out of packets when the unrar process should not be connected to the internet? Would that not free up connections to allow the download process to continue?


So, it happened again (6.62 RC2). cache 0/200. No Urar or PARs going on. Looks like it was stuck like that for about 24hrs.

All connections were disconnected.

There were a few strange errors in the logs (ERROR <name of server> Error talking to Server).
Also, ERROR NW_DirectDownloadChunkWorker: <name of file>. Then, ERROR NW_CloseChunk - closing a dirty connection.
Finally, a ERROR NW_DirectDownloadChunkWorker:Connection to News server failed <name of server> ARTICLE <name of article> |400 session time limit reached (please reconnect)

Is it possible that there's a server side session time limit and that NBP simply does not try to disconnect and reconnect when it gets it?

pressing help | About caused it to be immediately unstuck.

What does help | About do?
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: Stuck downloading headers

Postby Quade » Fri Nov 20, 2015 9:33 am

So, it happened again (6.62 RC2). cache 0/200. No Urar or PARs going on. Looks like it was stuck like that for about 24hrs.


Do you use periodic header downloads? I mean the setting where Newsbin automatically downloads headers for you at set intervals? I have 3-4 people now who have a problem with the cache running out and the one common thing seems to be that they all use automatic header downloads.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Stuck downloading headers

Postby oferlaor » Fri Nov 20, 2015 10:53 am

Yes. I have automatic header downloads with regex download filters.
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: Stuck downloading headers

Postby Quade » Fri Nov 20, 2015 1:06 pm

TBlack thinks he's found an issue with appending log files together.


[LOGGING]
Append=0



You might search your "NBI" file for "Append" and if it's set to 1, set it to zero and see if it changes the symptoms.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Stuck downloading headers

Postby oferlaor » Fri Nov 20, 2015 8:13 pm

There was no append value at all. What is the default for this entry?

I added it to the nbi file.
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: Stuck downloading headers

Postby Quade » Fri Nov 20, 2015 9:40 pm

No entry means it's disabled. So basically this isn't the fix you're looking for.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 2 guests