Telegram Bot when upload file finished not destroy crawler instance about lightnovel-crawler

[dipu-bd]

This is definitely a ghost! I am discarding the object when creating new session. There is no way this should happen. Wait a bit, I am fetching my talismans... there is gonna be a ghost hunt.

[dipu-bd]

It should be fixed in latest push

[dipu-bd]

I have tested it and it is working fine now

[yudilee]

Yes. It works 👍. Great works. You fixed it real fast. I check it for many hours and don't have any clue. Awesome. You may closed this issue and thanks a lot

[yudilee]

Hi Sudipto,

i'm testing latest push, but the issue is not fixed. step to regenerate problem is

1. When i call /start and generate book, for example my house of horror, it has 3 volume 255 chapter
2. After file uploaded and bot says session closed, then i call /start again but bot not responded until i call /cancel and bot responded with session closed then i call /start again, the bot says session started
3. If i then choose lord of mysteries in boxnovel source then bot will says it has 4 volume 329 chapter which i presume, is summary from novel in step 1 + novel from this step ( 255 chapter + 74 chapter = 329 chapter = 4 volume).
4. If i continue asking bot to generate epub, it stopped and generate error says thread has been shutdown. raise RuntimeError('cannot schedule new futures after shutdown')

I think after session closed, even though program call command to destroy app (should destroy crawler and then should shutdown thread executor too), it can't return to initial state when we call the program via python3 main.py. It also happened in discord bot

[yudilee]

Hi Sudipto,

I just download latest update and try it. The error is still reproduceable. Here the conversation done with bot :

Yudi Lee, [1/27/19, 3:38 PM]
/start

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
A new session is created.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
I recognize input of these two categories:

• Profile page url of a lightnovel.
• A query to search your lightnovel.
  Enter whatever you want or send /cancel to stop.

Yudi Lee, [1/27/19, 3:38 PM]
Lord of mysteries

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
Got your query text

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
Choose where to search for your novel,
or send /skip to search everywhere.

Yudi Lee, [1/27/19, 3:38 PM]
2 - boxnovel.com

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
Searching for your novel in 1 sites.
Please wait patiently or send /cancel to stop.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
https://boxnovel.com/novel/lord-of-the-mysteries/

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
Reading novel info...

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
1 volumes and 79 chapters found. - - > showing the right volume and chapter

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
Send /all to download everything.
Send /last to download last 50 chapters.
Send /first to download first 50 chapters.
Send /volume to choose specific volumes to download
To tereminate this session, send /cancel.

Yudi Lee, [1/27/19, 3:38 PM]
/all

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
You have selected 79 chapters to download

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
Do you want to generate a single file or split the books into volumes?

Yudi Lee, [1/27/19, 3:38 PM]
Single file

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
I will generate single output files whenever possible

🔥 Book Smelting Bot 🔥, [1/27/19, 3:38 PM]
Your request has been received.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:39 PM]
Download finished.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:39 PM]
Generating output files

🔥 Book Smelting Bot 🔥, [1/27/19, 3:41 PM]
Output file generated.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:41 PM]
Compressing output folder.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
[Lord of the Mysteries.zip]

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
This file will be available for 24 hours to download

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
Session closed - - > at this point session closed but below when I call command start. Nothing happened until I call cancel

Yudi Lee, [1/27/19, 3:42 PM]
/start

Yudi Lee, [1/27/19, 3:42 PM]
/cancel

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
Session closed

Yudi Lee, [1/27/19, 3:42 PM]
/start

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
A new session is created.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
I recognize input of these two categories:

• Profile page url of a lightnovel.
• A query to search your lightnovel.
  Enter whatever you want or send /cancel to stop.

Yudi Lee, [1/27/19, 3:42 PM]
My house of horror

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
Got your query text

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
Choose where to search for your novel,
or send /skip to search everywhere.

Yudi Lee, [1/27/19, 3:42 PM]
2 - boxnovel.com

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
Searching for your novel in 1 sites.
Please wait patiently or send /cancel to stop.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
https://boxnovel.com/novel/my-house-of-horrors/

🔥 Book Smelting Bot 🔥, [1/27/19, 3:42 PM]
Reading novel info...

🔥 Book Smelting Bot 🔥, [1/27/19, 3:43 PM]
4 volumes and 338 chapters found. - - > info shown here is wrong because 79 in lord of mysteries above is added.

🔥 Book Smelting Bot 🔥, [1/27/19, 3:43 PM]
Send /all to download everything.
Send /last to download last 50 chapters.
Send /first to download first 50 chapters.
Send /volume to choose specific volumes to download
To tereminate this session, send /cancel.

So I think the problem still there.

Best Regard

Yudi Lee

[dipu-bd]

Actually, a new session has already started as soon as the previous was terminated. But I guess this behavior is not expected. I will fix this ASAP

[dipu-bd]

The issue is at handle_pack_by_volume method. At the end I return 'handle_downloader' state. So the bot wait for an input before terminating by itself.

[dipu-bd]

I tweaked the code a bit. Now I am returning ConversationHandler.END from handle_pack_by_volume. And moved handle_downloader method to global scope. This should fix this issue.

[yudilee]

Just try latest update. After bot send file session is closed(this problem is solved). But the second problem still reproduceable, when I start new session, number of chapter from get novel info still being added with number of chapter from previous session.

[dipu-bd]

Check now. The ghost should be purified to the heaven