newsbin.com

Hi,
Using build 6.90 - 5524 and I'm doing a LOCAL search across my spool.
Spool is about 400 groups, about 3TB of data.

It is taking quite a while, how can I monitor the progress of the LOCAL search?

All I am seeing is a small blue bar shuttleing across the bottom of the search window, with no results found.
I can't see any other feedback as to what is going on, which GoG/group is being currently searched
or anything else useful in the DEBUG log.

I'm not really worried about the time taken, but I would like to know more about what is actually happening

regards
Stavros.

I use Sysinternals' Process Monitor {ProcMon}. Under "Filter" set "Display entries matching these" to 'Process Name' 'is' "NewsbinPro64.exe" then 'include' and click Add and OK.
Again under "Filter" click "Save Filter..." and enter a name - I use "Newsbin"
The 3rd icon {magnifier} starts/stops monitoring and the 5th {paper being erased} clears the list of event logged

Thanks BZee, does that show which GoG and / or group is currently being searched?

Later:
I see that it does, I switched on 'autoscroll' (4th Icon) and I can follow the paths changing as it works it's way
through the spool.

Thanks BZee - a good workaround

The problem is Newsbin doesn't know the total counts in each group until it counts them which is in itself fairly time consuming.

So to get a real progress bar, I'd have to count the contents of each group, note it, then during the search count up the number of records as it goes. It's probably doable.

Hi Quade,

I'd be happy with some text in the status bar (where the scrolling blue bar is) at the bottom of the search window saying something like:

currently searching <group name>, # of total # groups

getting updated when it starts searching a new group
(EDIT* and maybe dumped to the debug log as it gets updated?).

It would be nice if there were more details, but anything showing progress would be better that the total lack of feedback at the moment.
I just did a search that bought back about a hundred rows from my spool, but it took over 90 minutes before the first row was returned,
and another 60 before the search completed.

Maybe something in the debug log at the end of the search saying what the search string was, how many groups searched, how many hits found, any other easily acquired stats?

regards
Stavros

I tried a local search for one of the bigger groups, I downloaded headers for.
At the moment I downloaded about 24 of 32 billion headers and the database for the group is nearly 500 GB in size.
(At the moment the database contains headers from August 2008 to January 2022.)
I use a SSD to speed up header importing so I thought I make a test of the search speed.
The other reason was, that I had many aborted header import files (without the .txt extension) in the import folder.
This happens when the connection is closed or sometimes when I delete the header download job.

So I looked in the first of the import files without extension and did a search for a subject that was posted at that that time.
The header with that subject was posted in July 2012 so I used the "Search in Groups" feature with setting to local search.
I entered the subject in the search field and pressed enter. Then I was waiting for any results.
The blue search animation bar/block was running from left to right. About 12 minutes later the animation disappeared.
So I thought nothing was found and closed the search tab.

To make sure the post existed, I loaded the time period for the headers, filtered by subject and the post showed up in full.

I then did another try with the "Search in Groups", this time I monitored the Newsbin process CPU tab with the Task Manager.
Newsbin was using about 3% of CPU time while searching. After the blue search animation disappeared, Newsbin was still using
about 3% of CPU time. So I thought, that the search was still running. I didn't close the search tab and was waiting for any results.
After another 9 minutes, 3 lines of results found were shown and the CPU time was going down.

So the disappearing animation bar gives the impression, that the search finished, which it doesn't.
The animation should end, when the search is done.

I was also surprised that no further search was apparently made after the result was displayed.
The Subject was posted in 2012, so I think the database should be searched through the end (Janary 2022 at the moment).
But perhaps the result will not be displayed until the entire database has been searched?

Edit:
Just did another local search while downloading and importing headers.
This time the animation bar disappeared after about 17 minutes.
After 31 minutes the correct post was found (from 2012) and displayed.

Newsbin Internet search found 2 results after a few seconds.
(One post of 2011 had identical words but in different order.)

Using Newsbin Pro 6.90 - Build: 5524.

All the best,
Calahan

I just started up a new beta so I can try to reproduce it and see if there's any way to make it clearer.