failed to clear socket should never happen

Technical support and discussion of Newsbin Version 6 series.

failed to clear socket should never happen

Postby pjcamp » Thu Feb 09, 2017 1:46 pm

That's the error message. What does it mean? It happens in the middle of a header download and all the headers downloaded up to that point disappear.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby Quade » Thu Feb 09, 2017 11:56 pm

It means something is probably killing the connection out from under Newsbin. If you use SSL, you might try turning it off and seeing if that changes anything. If it does, it would suggest that maybe your security software is stepping on the connection.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42578
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Fri Feb 10, 2017 8:49 pm

Quade wrote:It means something is probably killing the connection out from under Newsbin. If you use SSL, you might try turning it off and seeing if that changes anything. If it does, it would suggest that maybe your security software is stepping on the connection.


I tried two things. First I turned off the security software. Same problem. Then I turned off SSL as well. Same problem.

I'm wondering if it has to do with the number of headers. This is a biggish group, probably hit by the recent spammers (which started this whole cycle in the first place). I nuked the whole installation and started over. So it is trying to grab a fairly large number of headers all at once.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby Quade » Sat Feb 11, 2017 12:28 am

I'm wondering if it has to do with the number of headers. This is a biggish group, probably hit by the recent spammers (which started this whole cycle in the first place). I nuked the whole installation and started over. So it is trying to grab a fairly large number of headers all at once.


I suppose anything is possible. Newsbin doesn't really do anything with the downloaded headers except write them to disk. The actual header processing happens at a different time.

What server are you using? You might want to try disabling "Xfeatures" in the server advanced options.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42578
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Sat Feb 11, 2017 1:54 am

Quade wrote:What server are you using? You might want to try disabling "Xfeatures" in the server advanced options.


Newshosting.

I cut the time back from 180 days to 30 days. That's more than enough to get me caught back up. If that doesn't work, I'll try xfeatures.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby pjcamp » Sun Feb 12, 2017 2:27 pm

Well, turning off xfeatures caused a different error. Now I get this:

[02/11 11:18:57] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded
[02/11 20:41:34] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded
[02/12 06:00:31] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby dexter » Sun Feb 12, 2017 3:58 pm

I've seen a couple of reports of this. I wonder if NewsHosting implemented some new connection timeout or something. You should be able to just do a "Download Latest Headers" on the group and it will pick up again from where the server errored out.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9297
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Sun Feb 12, 2017 4:44 pm

dexter wrote:I've seen a couple of reports of this. I wonder if NewsHosting implemented some new connection timeout or something. You should be able to just do a "Download Latest Headers" on the group and it will pick up again from where the server errored out.


Thanks! That's what I'm doing. It comes in stages. If they have done such a thing, I'll change providers.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby dexter » Sun Feb 12, 2017 5:55 pm

I'm curious which plan you have with them. I wonder if only their metered plan does this or if it is across their tiers of service.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9297
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Mon Feb 13, 2017 10:13 pm

dexter wrote:I'm curious which plan you have with them. I wonder if only their metered plan does this or if it is across their tiers of service.


I have the unlimited one.

The error message appears to be on Newsbin's end. The log shows that it sends a bunch of requests for data that go unanswered and then times out. So other than the spam problem slowing it to a crawl, this seems to be on Newshosting's side.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby Quade » Wed Feb 15, 2017 1:33 pm

[02/12 06:00:31] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded


This error is from the server. It's considered a fatal error in usenet. It suggests they're thrown some max connection time onto each connection.

How long does the connection go before this kicks in?

If you're already caught up with your header downloads, I'm surprised you'd see this.

Might ask Newshosting what the whole point of this error is.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42578
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Wed Feb 15, 2017 7:38 pm

Quade wrote:
[02/12 06:00:31] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded


This error is from the server. It's considered a fatal error in usenet. It suggests they're thrown some max connection time onto each connection.

How long does the connection go before this kicks in?

If you're already caught up with your header downloads, I'm surprised you'd see this.

Might ask Newshosting what the whole point of this error is.


I did ask Newshosting. Their first claim was that it was a problematic interaction between Newsbin and my security software. I agree with you that it is more likely to be on their end. It went for several hours before bailing.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby dexter » Wed Feb 15, 2017 8:10 pm

pjcamp wrote:Their first claim was that it was a problematic interaction between Newsbin and my security software.


I suppose this is possible if your security software is holding a persistent connection or something. Try configuring your security software to ignore Newsbin and see if that makes a difference. Or if you can live without it for the duration of a test download, disable your security software altogether, reboot your machine, then start Newsbin and see if you can get this failure again.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9297
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Sat Feb 18, 2017 1:12 pm

The log file is a little more explicit now in that there is a new line. It goes like this.

Code: Select all
[02/17 15:54:45]  DEBUG XOVER Numbers: <newsgroup> Min: 272082411 Max: 273082411 Newshosting
[02/17 15:54:45]  DEBUG Newshosting <newsgroup> XOVER: <newsgroup> 272082411:273082411
[02/17 15:55:00]  DEBUG DbWorker: Db Gz to Db  Failed: <newsgroup>


After trying this a buncha buncha times, it also says:

Code: Select all
[02/18 04:51:35]  ERROR Server:Newshosting NNTPServer: Failed to read text line:


once.

So I'm assuming DbWorker is local and is failing to load information into a database, perhaps because that information is corrupt?
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby Quade » Sat Feb 18, 2017 3:23 pm

I count at least 3 issues which look semi-unrelated.

1 - GZ to DB error. Has nothing to do with downloading headers. It happens after a successful download when headers are imported.

2 - 500 error from the server for old connection

3 - Read timeouts while reading headers indicating the connection dropped out.

Might be worth disabling xfeatures in the server advanced options. Then restart Newsbin. I'm not seeing any obvious security software symptoms but since security software does tend to F up downloading, it wouldn't shock me that it's related to that.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42578
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Sat Feb 18, 2017 3:55 pm

Quade wrote:I count at least 3 issues which look semi-unrelated.

1 - GZ to DB error. Has nothing to do with downloading headers. It happens after a successful download when headers are imported.

2 - 500 error from the server for old connection

3 - Read timeouts while reading headers indicating the connection dropped out.

Might be worth disabling xfeatures in the server advanced options. Then restart Newsbin. I'm not seeing any obvious security software symptoms but since security software does tend to F up downloading, it wouldn't shock me that it's related to that.


xfeatures is disabled already. Security software also disabled, though that doesn't make it go completely away.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby pjcamp » Sun Feb 19, 2017 12:56 am

OK now it timed out while paused. That doesn't seem like it would be server related.

Just to be sure, I'm going to get a free 3 day trial from Supernews. That's a different Tier 1 provider entirely.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby Quade » Sun Feb 19, 2017 3:39 am

OK now it timed out while paused. That doesn't seem like it would be server related.


Pause doesn't pause header downloads. Headers are different then file downloads. There's no good way to stop the download on the fly. The news server protocol doesn't support it. If you want to stop a header download you're better off just deleting it, then updating the groups again when you're ready for more. Newsbin will continue downloading from where it left off as long as you don't "Download all Headers" next time you download headers.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42578
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Sun Feb 19, 2017 4:45 am

Quade wrote:
OK now it timed out while paused. That doesn't seem like it would be server related.


Pause doesn't pause header downloads. Headers are different then file downloads. There's no good way to stop the download on the fly. The news server protocol doesn't support it. If you want to stop a header download you're better off just deleting it, then updating the groups again when you're ready for more. Newsbin will continue downloading from where it left off as long as you don't "Download all Headers" next time you download headers.


Ah. Then Supernews will be a good test. So far so good.

But if you can resume a download by deleting and restarting, why doesn't that happen on a timeout as well?
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby pjcamp » Mon Feb 20, 2017 10:25 pm

Well, it is interesting so far. Still haven't got a full set of headers, but there are no timeouts from Supernews. It has stopped downloading three times but each time was the same error message: Failed to read text line. That's progress of a sort.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby Quade » Tue Feb 21, 2017 1:05 am

But if you can resume a download by deleting and restarting, why doesn't that happen on a timeout as well?


A 500 error is considered to be a permanent error. Any lesser error, Newsbin just retries the download. Other news servers don't do this. It's worth thinking about trying to mitigate this but since it's a new issue, it would have to examined first.

. It has stopped downloading three times but each time was the same error message: Failed to read text line. That's progress of a sort.


This is likely a simple read timeout without the 500 error.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42578
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Mon Feb 27, 2017 7:13 pm

So this was interesting.

I got an account on Supernews. That improved the situation a little bit. I could at least see headers long enough to see who the offending poster was and put him in the filter accordingly.

So does that mean he's now invisible, banished from Newsbin? Well, no. There are a couple and they still cause the headers to load painfully slowly, and they still appear in the list alongside a red icon and a status message "Poster Filtered." That seems somewhat less than helpful. If he's filtered, should he not be gone altogether? Totally absent? I strongly suspect I'm missing a configuration somewhere but damned if I can find it.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07

Re: failed to clear socket should never happen

Postby dexter » Mon Feb 27, 2017 7:54 pm

Depends on where you are applying the filter. Sounds like you are just using a display filter which means the header info is still in the database. If you turn filters on for the post list by clicking the icon that looks like a power button, it will actively hide any headers matching your filter. Otherwise, the filtered posts are still displayed and are just tagged as filtered as you are seeing now.

If you want to keep these posts from getting in to the database at all, you need to apply the filter as a header download filter.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9297
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: failed to clear socket should never happen

Postby pjcamp » Tue Feb 28, 2017 8:10 pm

dexter wrote:Depends on where you are applying the filter. Sounds like you are just using a display filter which means the header info is still in the database. If you turn filters on for the post list by clicking the icon that looks like a power button, it will actively hide any headers matching your filter. Otherwise, the filtered posts are still displayed and are just tagged as filtered as you are seeing now.

If you want to keep these posts from getting in to the database at all, you need to apply the filter as a header download filter.


Thank you! That did the trick. I knew I was not finding something.
pjcamp
Active Participant
Active Participant
 
Posts: 63
Joined: Sat Sep 19, 2009 3:58 pm

Registered Newsbin User since: 08/25/07


Return to V6 Technical Support

Who is online

Users browsing this forum: Bing [Bot] and 6 guests