Continuous updating of Queue status so crashes don't screw up download history.

If you found a bug, please report it here.
Forum rules
When reporting a bug, please be as specific as possible.

To access the logfile in Finder:

If the programs crashed, please attach or email us the crash report AND logfile.

In Finder press SHIFT-CMD-G (or in menu Go->Go to Folder)
Then in the textfield enter "~/Library/Application Support/Usenetic" (including the tilde ~)
There you'll find the logfile called: Usenetic.log

Other issues:

You can always attach an image/screenshot with the problem for better understanding.
jculp
Posts: 47
Joined: Thu Jun 08, 2017 3:26 am

Continuous updating of Queue status so crashes don't screw up download history.

Post by jculp » Thu Oct 05, 2017 11:00 am

When crashes (spinning ball and Unresponsive App) occur, the Queue history is not accurately saved.
When app is restarted, the Queue is rebuilt to last saved state even if lots of files have been downloaded already, necessitating manual review and deletion of duplicate Queue entries to reflect updated download status. Then quit of app to force the Queue to save normally. Then restart to continue downloading

A fail-safe to preserve Queue status and Download history in Megasearch results would be a great Database feature.

User avatar
Support
Posts: 143
Joined: Tue Jun 06, 2017 12:38 pm

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by Support » Thu Oct 05, 2017 11:58 am

Good idea, except that a large queued object can have ten thousands or more articles. Constantly updating the database for each downloaded articles will have a huge impact on the resources and app. A better choice would be to update the database for each finished or newly added queue object. I'll put this on the todo list.

BTW. At which point do the crashes occur? Or is the resident app you mentioned before still interfering?
jculp wrote:
Thu Oct 05, 2017 11:00 am
When crashes (spinning ball and Unresponsive App) occur, the Queue history is not accurately saved.
When app is restarted, the Queue is rebuilt to last saved state even if lots of files have been downloaded already, necessitating manual review and deletion of duplicate Queue entries to reflect updated download status. Then quit of app to force the Queue to save normally. Then restart to continue downloading

A fail-safe to preserve Queue status and Download history in Megasearch results would be a great Database feature.
If you like Usenetic, please give it 5 stars at MacUpdate: https://www.macupdate.com/app/mac/59624/usenetic

jculp
Posts: 47
Joined: Thu Jun 08, 2017 3:26 am

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by jculp » Fri Oct 06, 2017 12:02 pm

A better choice would be to update the database for each finished or newly added queue object. I'll put this on the todo list.

I agree and that was what I was intending to suggest. Just didn't know how to word it.

BTW. At which point do the crashes occur? Or is the resident app you mentioned before still interfering?

Usenet hangs when various changes occur to Queue.
So far I've noted it does it sporadically when
1. moving queue entries up or down multiple time. (Perhaps it dislikes the abrupt interruption to the file parts downloads changing)
2. When Entries are no longer available from the time it was added to the queue till the time it actually comes up for download (this occurs with large download Queue that may take 15-24 hours to get through)
3. deleting queue entries repeatedly after checking content of rars and terminating/deleting the downloads of redundant or incomplete file sets.
(have noted a lot of posted files are missing enough rar parts that even the par recovery set cannot revive the download - perhaps a pre-check for complete rar sequential sets can be built in to MegaSearch to eliminate these results as they are just wasted download bandwidth and time. Similar to the option to eliminate password protected files)

To work around this I have manually checked the original search results and noted that certain posters are guilty of this practice and I try to avoid them.
I also check the Queue files to verify the contents of the file sets are all sequentially present and complete.
I routinely quit and restart the App to keep the database updated in case of a crash.


Overall, the app is getting more stable and useful. I have been very impressed with your updates and responsiveness.

Thanks for the great work.

User avatar
Support
Posts: 143
Joined: Tue Jun 06, 2017 12:38 pm

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by Support » Fri Oct 06, 2017 1:53 pm

Thank you for the follow up.
Your detailed information is very useful.

However, I have one more question.
What settings are you using to cleanup the queue.
Is it "Auto cleanup", "Manual cleanup" or "Auto reorder"?
And did you ever tried a different setting with better results?

Thanks.
If you like Usenetic, please give it 5 stars at MacUpdate: https://www.macupdate.com/app/mac/59624/usenetic

jculp
Posts: 47
Joined: Thu Jun 08, 2017 3:26 am

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by jculp » Sat Oct 07, 2017 2:54 am

Clean up has always been set to Auto CleanUp.

The Queue is auto cleaning the list as it completes downloads and completes extraction into the designated extraction folder.
When the spinning ball occurs, a force quit is required and no db saved state is updated..
When restarting the app, it reloads the last properly saved state, regardless of what was previously downloaded and Auto Cleaned since that proper saved state db.

Is their a functional difference among the cleanup methods?

jculp
Posts: 47
Joined: Thu Jun 08, 2017 3:26 am

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by jculp » Sat Oct 07, 2017 3:03 am

I may be using the term "crash" incorrectly.

The Usenetic app becomes unresponsive and I get the spinning ball, so technically not a crash of the app but it hangs and it requires a force quit and then restart.

I'll forward the logs the next time it happens.

Hope this helps.

User avatar
Support
Posts: 143
Joined: Tue Jun 06, 2017 12:38 pm

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by Support » Sat Oct 07, 2017 10:19 am

jculp wrote:
Sat Oct 07, 2017 3:03 am
I may be using the term "crash" incorrectly.

The Usenetic app becomes unresponsive and I get the spinning ball, so technically not a crash of the app but it hangs and it requires a force quit and then restart.
Ah, that is something totally different and more explicable :D

When a large queue object is finished, Usenetic safely cleans the used memory. This is done in several steps to avoid crashing. This can take a while for very large objects and will freeze the interface because this is done on the main thread. That gives you the idea that the app hangs, but if you wait long enough it should become responsively again and will continue with the remaining tasks.

To fix this I'll change the cleaning routines and let them work in a different thread than the main thread.

Thanks.
If you like Usenetic, please give it 5 stars at MacUpdate: https://www.macupdate.com/app/mac/59624/usenetic

jculp
Posts: 47
Joined: Thu Jun 08, 2017 3:26 am

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by jculp » Mon Oct 09, 2017 7:09 am

Thanks for the clarification.

It happens often enough that I get impatient after 5-10 seconds.

It happened again (hung for over a hour), so I took a screen shot of Queue and copied the force quit report and have enclosed for your review.

Hope it helps.
Attachments
2017.10.07 .txt.zip
(91.74 KiB) Downloaded 178 times
Screen Shot 2017-10-07 at 11.54.22 PM (2).png
Screen Shot 2017-10-07 at 11.54.22 PM (2).png (687.83 KiB) Viewed 4287 times

User avatar
Support
Posts: 143
Joined: Tue Jun 06, 2017 12:38 pm

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by Support » Mon Oct 09, 2017 9:43 am

According to the log you attached, I was wrong with my previous theory.
This problem is difficult to track and unfortunately I can't reproduce it here.

Can you please answer the following questions that might help to tackle the issue?

1) I see you are using the bandwidth limiter. Does the same issue happens when the bandwidth limiter is off?
2) Did this issue started since a few releases back or has it always been there?
3) Does the hang also occurs by lets say 10 or less queued items or just with 100+?
4) Does the hang just occurs while adding more items to the queue?

Thanks so far.
If you like Usenetic, please give it 5 stars at MacUpdate: https://www.macupdate.com/app/mac/59624/usenetic

jculp
Posts: 47
Joined: Thu Jun 08, 2017 3:26 am

Re: Continuous updating of Queue status so crashes don't screw up download history.

Post by jculp » Tue Oct 10, 2017 2:13 pm

1) I see you are using the bandwidth limiter. Does the same issue happens when the bandwidth limiter is off?

I have always had the limiter check as on. I will try with it off to see if it recurs.

2) Did this issue started since a few releases back or has it always been there?

Now that you mention it it does seem that this particular hang is occurring more frequently since the last update.

3) Does the hang also occurs by lets say 10 or less queued items or just with 100+?

The hang usually occurs with larger queues with larger file sets.

4) Does the hang just occurs while adding more items to the queue?

Have not noted that occurring. Adds to Queue have been fine.
Removing too quickly multiple file sets (especially ones that are in downloading process) however can hang the Queue and app.
If I pause the downloads and wait for the progress bar to stop moving, deleting multiple file sets are more stable.




Perhaps related to #2 the following has occurred several times since the last update:
It also seems that it "loses server connection" when more than two file sets are no longer available for download, no dialogue box pops up stating this, it does not reclassify it with the red badge, just no download and everything in Queue is paused with the explanation of "lost server connection". I have included a screenshot.
I test the internet and no problems show up.
When I recycle a file set farther down the Queue list it starts downloading it fine.
I then quit and restart the app to refresh the Queue db.
When I try to download the two or three file sets that were not downloading nothing happens, so I pause them.
I go farther down the list repeating it till I get to one that starts downloading. Usually the third or fourth.

I'll try to do a screen recording and edit it to the pertinent section where it occurs.

Post Reply