Headers not being shown

[JayPea]

Heya Quade,

Got a weird one for you. I've been suffering from missing headers for a while on a subscribed group. I thought that it was probably my news server not picking them up so hand-waved it away until I had time to troubleshoot it, which is what I've been doing for the last hour and I would like your input. I'm using 6.91B10 (5783).

I subscribe to a.b.android and when I refresh the group 85% of headers just don't appear when using NBP. I installed Grabit to check that it wasn't my news host and they show up fine using that application. This made me think it was something to do with my filters or another setting I had set and forgot about. Wiped out everything from my data location and repulled the headers. Still nothing. I then completely uninstalled NBP and wiped the data location again and still nothing. Pull the headers and they just don't appear - but only in NBP. Which leaves me very confused. I've been using NBP daily for almost 20 years. Is it time for me to migrate to an alternative or is there a simple answer that I'm not seeing or haven't considered? It certainly makes me wonder what other headers I've been missing....

Thanks for your time, as always.

[Quade]

I'll check it out. Thanks.

I'm going to guess it's extensions Newsbin doesn't know.

[Quade]

Do you know what kinds of files are missing? Or are we talking text posts?

[JayPea]

Do you know what kinds of files are missing? Or are we talking text posts?

So in NBP I can see the .par files collapsed into single posts and they expand as usual when clicked. It's the actual .apk files that are not shown. For example, there was a bunch of stuff posted on 17th August beginning with the Subject 'APP MISC'. None of those are visible on either my historical install or a newly installed one. However, the related .par files that were posted the same day with the subject 'Android - Only Paid - Week 33' show up fine and collapse/expand as normal. Both of the above examples are listed in Grabit using the same news server, creds, etc. Text files show up every time as I can see & read people responding to posts made there. I can also see my own posts asking where TF the files have all gone

Your theory about extensions appears correct as I can see stuff posted in the same group when compressed as .rar files but precisely zero .apk files throughout the entire group after a full update.

Cheers

[Quade]

Ok thanks. I'll get it working.

[JayPea]

Appreciate it. Thanks very much indeed.

[JayPea]

Thought a bit more about this. Is there any scope for the end user blacklisting extensions they don't want to see instead of you whitelisting what is presented? it kinda makes me wonder what else was being hidden...

At the very least it saves this type of admin work you need to undertake every time some arbitrary new file type comes along. I also like the idea of getting a untouched 'warts 'n all' view of my favourite 20y obsessive compulsive behaviour disorder

[Quade]

I'm looking in my android group and seeing apk files. What my plan is, is to do a survey on the number of APK files in the group and compare it to what Newsbin shows. I think APK is already in the white list.

I already re-downloaded all the headers for this group so I can look into the raw data and count what's there.

The white list is for grouping files. It shouldn't prevent files from showing up.

[JayPea]

Then I'm stumped. When I tried a new install of NBP I deliberately didn't install to a second location to stop me 'crossing the streams' of results. I uninstalled then wiped the data drive (I have a different SSD specifically for NBP data/downloads/cache/etc). The main application files are installed to my primary system/boot NVME drive in the regular program files location. Unless NBP is storing stuff in other locations (\programdata or the registry for example) then I don't see what more I could've done as a litmus test for a fresh install but, ofc, I'm happy to take direction from yourself if you have any suggestions?

I think I'll try a non-beta version next. Just to see what happens. Might not be until the weekend, though, as I'm sat on a jury this week and need to decompress after spending the day in court

Cheers.

[Quade]

I'm just reporting my observations. It's not a plan for action. Once I get some real numbers from the group we can start comparing.

I'm using what will be RC2 once it's ready.

I've been writing some tools to make it easier to analyze the header dumps.

[JayPea]

Noted. I'll be patient and won't post anything further until you either ask me to do something or I discover something significant. Cheers.

[Quade]

Got my tools in order and then surveyed the group. I can see the missing files. What's peculiar is that the slightly older "18+" ones show up but newer week 34 ones are missing. I have a feeling it's filter but I'm not clear yet what filter it is. That's the next step.

[Quade]

Ok it's solved.

I'll try to get you a version to test.

[JayPea]

Excellent. Many thanks for the effort and the update on progress. No need to rush it over to me. Perhaps roll it into your regular release cadence? Happy to wait until whenver that is. Unless you explicitly want it tested before that happens, ofc. Will let you decide on best way forward

Cheers.

[Stan]

I have seen this happen now and tried both 6.82 and 6.90.
So far, only one listing of one post in one group.

The one thing I noticed by loading the group in another newsreader is that the particular post was cross posted to a large number of other groups.

Does newsbin for some reason then ignore this post because of the large cross posting.
I don't see any setting about cross posting limit in newsbin.
I looked in filters there are none on and the count when looking in 6.90 says 1926283/1926283.
The problems with the other newsreaders being used is they are all stuck at 32bit and no longer being updated.

If fact now I found one other newsreader that shows total count, and it states 1926364 and that's not fully updated from group.
I am going to delete all from the group and download headers one more time with 6.82.
Ok, I did get other program to now fully update group and final count is now up to 1927089.
So not sure if both that program and newbin count the same way, but it's big discrepancy.

Unfortunately, due to the reporting of the SIZES being wiped out since the 6.91 betas or RCs I can't use them. REFERENCE > https://forums.newsbin.com/viewtopic.ph ... 60#p233460
I did learn and have the sizes backed up to a separate file now.

I will try to do a final update once 6.82 gets done, grabbing all headers from the group. But will have to go to 6.90 to load the 6.82 updated info into 6.90 to get a final count, as 6.82 don't have a count like 6.90.
OK, I am totally confused now, so will have to use both the other program and newsbin, as 6.82 now shows 1927131, yet the one post I know is there via other program with multiple cross post groups is still missing.
6.90 shows 1926283, and the one post I was trying to get still missing, so really confused.
But even if we overlook that, why such a difference in count between 6.82 and 6.90?

[Quade]

The filters have a cross post limits. It defaults to zero which is no limit. It's possible there's some mechanical limit that I wasn't expecting. Like the size of the area I use to parse the crosspost groups might be too small.

Newsbin sets practical limit to field sizes because Usenet is a wild place and people do wild things there still. I've recently had to make significant changes because people are moving from posting RARs to posting single really large files.

PM me the group and the subject and I can take a look to see why it might be getting filtered out.

[Stan]

The filters have a cross post limits. It defaults to zero which is no limit. It's possible there's some mechanical limit that I wasn't expecting. Like the size of the area I use to parse the crosspost groups might be too small.

Newsbin sets practical limit to field sizes because Usenet is a wild place and people do wild things there still. I've recently had to make significant changes because people are moving from posting RARs to posting single really large files.

PM me the group and the subject and I can take a look to see why it might be getting filtered out.

Sent via PM I hope what info I could give you helps in someway.

Been running into this some now, on forum?
You cannot make another post so soon after your last.