Updated BiglyBT from 3.1.0.0 to 3.6.0.0, started crashing due to heap size too small about biglybt

[parg]

Amusing, you come here wanting help and feel it necessary to say that you would ditch BiglyBT if you could. Good luck

[rlesac]

Amusing, you come here wanting help and feel it necessary to say that you would ditch BiglyBT if you could. Good luck

So, instead of helping or asking for more info about the problem, you closed the issue so it goes unfixed and more users can experience it?

I know that it can be hard to understand for some people, but when a piece of software that they've been using for almost 20 years starts crashing multiple times a day after an update can be frustrating. Even more so when people have other stuff to deal with IRL, all the while trying to serve communities.

[parg]

Don't use the I2P Helper plugin; Don't run too many active downloads; Monitor JVM memory via View->Log Views->JVM Info; Update more frequently so that potential issues can more easily be narrowed down; Join the beta program to help find potential issues prior to release

[2peer]

@rlesac
Do you have the logging enabled? See #3332 .

[parg]

Aas you can see from the other thread, you could install VisualVM and see if there is some class type dominating memory usage

[rlesac]

Before updating I used the backup function in BiglyBT, when 3.6 started crashing, I installed 3.1 again, and when it started crashing I restored the backup, it made no difference. My max heap size was set to 2gb and it was working fine that way for years. I stuck with 3.1 because the next update was crashing.

I put max heap to 6gb, and after say 10 minutes it will use up the memory and start hogging the cpu.
Logging was not enabled and most plugins were disabled.

I set max heap to 12gb as I don't have the time to debug at the moment.

[2peer]

@rlesac
Well, the thing to note is that with BiglyBT (as with any other even moderately complex software) the memory usage pattern will be largely affected by state (be it configuration, plugins, torrents you have ...). It's not like it's leaking memory for everyone right now, so unless you put in a little bit of work, nobody will be able to help you (we don't have a crystal ball).

The log thing probably wouldn't affect you (it only demonstrated if you used the telnet/console UI), and might have been only affecting the current Beta channel releases (not sure).

Looking at the memory profile really isn't that difficult (certainly much easier that in many non garbage collected languages). You can use jconsole to monitor the heap use / garbage collection, jmap (or VisualVM directly) to take a snapshot (dump) of the heap & VisualVM (or other tools) to view it.

These links could help you get started:
https://www.baeldung.com/java-heap-dump-capture
https://www.baeldung.com/java-profilers

Side note:
You kind of did yourself a disservice by not reporting the issue the first time it happened. Could have been easier to bisect the exact point where it was introduced. Depending on others to report and fix it will only get you so far. It's open-source - we are all better off if we contribute a little.

[rlesac]

You kind of did yourself a disservice by not reporting the issue the first time it happened. Could have been easier to bisect the exact point where it was introduced. Depending on others to report and fix it will only get you so far. It's open-source - we are all better off if we contribute a little.

I reported this as soon as I could, I updated 4 days ago.
The 3.2-3.4 updates?, well it was a long time ago, but that produced instant crashes and rolling back worked.
I can debug, but it takes time which I can't spare currently due to personal circumstances, which is why I'm very frustrated, as rolling back should have worked. Yeah, you could also say I should have imaged my system before updating etc.

I'll look into memory profiling, when I can.
If you have any suggestion for at least a partial fix until I can devote more time to this I'd be grateful. I just need the torrent state to persist as I have a lot of them scattered across multiple drives and directories.

[2peer]

If you have any suggestion for at least a partial fix until I can devote more time to this I'd be grateful. I just need the torrent state to persist as I have a lot of them scattered across multiple drives and directories.

I'm not sure when the "Modern UI" came along but maybe you could try switching back to Classic for now? Using the older backup of the Bigly's config directory to revert (if you have one)? Check/update your JVM version (if it's really ancient)?

[rlesac]

If you have any suggestion for at least a partial fix until I can devote more time to this I'd be grateful. I just need the torrent state to persist as I have a lot of them scattered across multiple drives and directories.

I'm not sure when the "Modern UI" came along but maybe you could try switching back to Classic for now? Using the older backup of the Bigly's config directory to revert (if you have one)? Check/update your JVM version (if it's really ancient)?

I'm using the classic UI. After the crashes started a few days ago I updated JVM.
Currently it takes a bit longer, a few hours to fill up 12gb but it still does and with it comes lagging and so on.

[2peer]

Currently it takes a bit longer, a few hours to fill up 12gb but it still does and with it comes lagging and so on.

Well at this point I would see running the VisualVM as your best option. It takes like 10min max. (including install), attaches directly to the running instance, it's free/opensource (GPL) & build by (mostly) people from Oracle.

Just open com.bigly.ui.Main in the Local section in the left panel. Switch to Monitor Tab and click Heap Dump. Then you can open it, look what instances take the most memory (it will sort it for you) and where are they referenced from. Post your findings here. That's all.

[rlesac]

Currently it takes a bit longer, a few hours to fill up 12gb but it still does and with it comes lagging and so on.

Well at this point I would see running the VisualVM as your best option. It takes like 10min max. (including install), attaches directly to the running instance, it's free/opensource (GPL) & build by (mostly) people from Oracle.

Just open com.bigly.ui.Main in the Local section in the left panel. Switch to Monitor Tab and click Heap Dump. Then you can open it, look what instances take the most memory (it will sort it for you) and where are they referenced from. Post your findings here. That's all.
Well, this is the heap dump, I'm not sure how to interpret it.

[parg]

You seem to have 20,000 connected peers and 1000000 assigned file, is this what you would expect?

[2peer]

which was fine for some 3500 torrents

Some data for comparison:
The numbers in the screenshot looks on average about 10x inflated to what what I have in my dump (6-15 depending on which part). I'm running about 3800 torrents right now and the GC bottoms at around 1.2GB.

View->All peers currently shows numbers around 700 | 2800

Statistics->Transfer->Current->Routing - around 700 active connections total, with another 2000 pending

[rlesac]

You seem to have 20,000 connected peers and 1000000 assigned file, is this what you would expect?

Seems excessive, I did not change any settings when updating, and I'm not sure what would I need to do to even allow so many peers.

which was fine for some 3500 torrents

Some data for comparison: The numbers in the screenshot looks on average about 10x inflated to what what I have in my dump (6-15 depending on which part). I'm running about 3800 torrents right now and the GC bottoms at around 1.2GB.

View->All peers currently shows numbers around 700 | 2800

Well, before I tried the update, it was working fine here with 2gb heap size.

[2peer]

Did you check the peer count / connection count in the UI? Might be possible that your settings were always this "unbounded" but the older client just wasn't able to handle them / had some other internal limits...

[rlesac]

Did you check the peer count / connection count in the UI? Might be possible that your settings were always this "unbounded" but the older client just wasn't able to handle them / had some other internal limits...

In the UI I get 50|1200 peers, 4.5gb mem used (after restart). I rolled back to 3.1, so it's the same client.