Unable to download beyond 900 chapters - OutOfMemoryError about novel-grabber

[OxySempra]

I seem to have fixed it by downloading and using the Java 64-bit. Downloaded from here: https://www.java.com/en/download/manual.jsp

Maybe you can let Java 64-bit to be one of the prerequisite of using the program? Anyways, thanks again for your work. It makes my military life so much better.

[Flameish]

Thanks for the info! I'm not sure what the default heap size on 32-bit is but it seems to be pretty low. You can manually increase it with the parameters -xms / -xmx (java -Xms1g -Xmx2g -jar NovelGrabber...) so I don't think 64-bit is a requirement just yet.
If a novel take more than 4gb of memory (32-bit limit) I'll seriously need to reevaluate my memory management.

[OxySempra]

For reference, Library of Heaven's Path and Gate of God are the two novels I tried that failed to go over the 930 chapters mark.

…

On Sun, 6 Sep 2020, 02:57 Flameish, ***@***.***> wrote: Thanks for the info! I'm not sure what the default heap size on 32-bit is but it seems to be pretty low. You can manually increase it with the parameters -xms / -xmx (java -Xms1g -Xmx2g -jar NovelGrabber...) so I don't think 64-bit is a requirement just yet. If a novel take more than 4gb of memory (32-bit limit) I'll seriously need to reevaluate my memory management. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#58 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQ4JJULCIJYSKMU3MDIYHDDSEKCYTANCNFSM4Q3FTR4Q> .

[Flameish]

Tested it with 50 MB. Ran out of memory after 160 chapters / 800 pages w/o images. I'm going to take a look where I can optimize it.

[OxySempra]

On my system, it seems to be using about half a GB at first while grabbing and slowly increasing to just over a GB after it is complete.

…

On Sun, 6 Sep 2020, 03:09 Flameish, ***@***.***> wrote: Tested it with 50 MB. Ran out of memory after 160 chapters / 800 pages w/o images. I'm going to take a look where I can optimize it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#58 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQ4JJUKHHN4LGRNW665ZQ4TSEKEFDANCNFSM4Q3FTR4Q> .

[Flameish]

Cleared some unused chapter fields which improved garbage collection. Managed ~750 chapters on 50 MB

This is after 1300 chapters on default java settings:

[xx255q]

What changed from before where i could download 4000+ chapters without issue

[Flameish]

Sometimes it could look like this for some novels: (~200 chapters)