program crashes somewhere when the browser is launched and chromium starts to consume very high cpu about codezinger-due-date-scraper

[RoguedBear]

update: when this happens, the bot doesn't post anything on discord.
possible source of error could be these lines since:

codezinger-due-date-scraper/scrap_data.py

Lines 173 to 176 in 7401be2

	browser = await launch(executablePath=chrome_path,
	headless=True,
	options={'args': ['--no-sandbox']})
	page = await browser.newPage()

since they're not under try/except

---

relevant log files:

red is the last log and since when the chromium high cpu usage is started
green is when i manually restarted the container

[RoguedBear]

processed running when cpu usage is high:

Last log message:

[RoguedBear]

possible workarounds:

• if opening chrome or that process takes more than 90seconds, kill it and restart
  (and/or set a cpu limit in compose)
• add the --disable-gpu param
• downgrade to python 3.9
• install all puppeteer's dependencies

[RoguedBear]

exact LOC where the freezing happens (confirmed via logs):

codezinger-due-date-scraper/scrap_data.py

Line 176 in 7401be2

page = await browser.newPage()

Logs when the program freezes:

crond: USER root pid   9 cmd /entrypoint.sh
codezinger-due-date-scraper  Copyright (C) 2022  RoguedBear, Ya-s-h
    This program comes with ABSOLUTELY NO WARRANTY; see COPYING
    This is free software, and you are welcome to redistribute it
    under certain conditions; see COPYING
loaded short names
Starting browser...
browser started

Logs when the program runs normally

crond: USER root pid   9 cmd /entrypoint.sh
codezinger-due-date-scraper  Copyright (C) 2022  RoguedBear, Ya-s-h
    This program comes with ABSOLUTELY NO WARRANTY; see COPYING
    This is free software, and you are welcome to redistribute it
    under certain conditions; see COPYING
loaded short names
Starting browser...
browser started
logging in.
Logged in. allegedly
sorting
loading more
Processing stuff
[<pyppeteer.element_handle.ElementHandle object at 0x7f1d26000640>, <pyppeteer.element_handle.ElementHandle object at 0x7f1d26001030>]
2
Scraped 2 questions(50%)
awaiting browser close
Browser closed
Processing new events...
Sent 0 new webhook messages
Finished, exiting...

[RoguedBear]

update:
unable to reproduce the problem when running current version and previous version repeatedly, but --disable-gpu does help in drastically reducing CPU usage.

graph:

[RoguedBear]

running the program for 2 weeks, commit a64681c does seem to have fixed the issue