Daemon problem? Might be me. about pychrome

[fate0]

I can reproduce this problem for now, I am thinking how to fix it ...

[fate0]

I can not reproduce this problem. actually, this is quite weird, your queue module seems to be a NoneType object.

[matthewlilley]

You have to use the gist provided, and a URL that will timeout to replicate the problem. Has to be run a few times to show up once.

I'm using this to run the file: python app.py --urls=https://google.com,https://ebay.com,https://facebook.com,https://sasadacxzcsadsa.com --force-kill=True --timeout=5

It doesn't always show that error, for example, this time it's showing this:

Exception in thread Thread-10:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/local/lib/python2.7/dist-packages/pychrome/tab.py", line 158, in _handle_event_loop
logger.error("[-] callback %s error: %s" % (event['method'], str(e)))
AttributeError: 'NoneType' object has no attribute 'error'

[matthewlilley]

It seems to work every time in 3.5, but in 2.7 i'm seeing errors like these.

[fate0]

I still failed to reproduce this problem by using your gist, this time logger seems to be a NoneType object, I am pretty sure logger been initialized tab.py#L25, Could you reproduce the problem on other machines？

[matthewlilley]

I'm running it on a fresh Ubuntu 16.04.2 LTS virtual box with Python 2.7.12 & 3.5.2.

I've noticed on 3.5, although it seems to be working, it's not performing the write in the EventHandler detached method. It seems to close while it's inside of that method. I've updated the gist with comments. Look at the detached function, and specifically line 150.

https://gist.github.com/matthewlilley/a5782b204a976484f8fe04443f509018

[matthewlilley]

I was unsure if you needed to enable inspector, because it gets into the detached method and has the correct "reason" returned with it enabled and without. On 3.5 it actually gets into the detached method, but doesn't make it to the end of the function, and in 2.7 it never enters the function.

[fate0]

when you call tab.stop(), It will stop handle all events, so you need to delete this line of code
https://gist.github.com/matthewlilley/a5782b204a976484f8fe04443f509018#file-gistfile1-txt-L117

[matthewlilley]

I mistakenly thought I could use the Inspector.detached event to capture a timeout, but that seems wrong now as every page will fire it if tab.stop() isn't called there. It was okay in the situation though, because Page.frameStoppedLoading event doesn't get fired when a request has timed out.

All I want to do is capture a failed page load and write a log.

Also, that still doesn't solve: 'NoneType' object has no attribute 'Empty' on the Queue object in tab.py. It makes it worse :D

[matthewlilley]

I've update the gist. https://pastebin.com/i6JneJ5b - I'm not worried about using inspector.

I'll see if I can see what's going on in 2.7 and causing Exceptions like this:

Exception in thread Thread-8 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 754, in run
File "/usr/local/lib/python2.7/dist-packages/pychrome/tab.py", line 151, in _handle_event_loop
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'Empty'

[matthewlilley]

Seems like the daemon isn't being shut down correctly, at least on 2.7: http://prntscr.com/gaux4f

[matthewlilley]

@fate0

I solved this by calling close_all_tabs at the end of the script, instead of closing them one by one. I just ran the same code with that change on 2.7, and no exceptions.

I should have been ensuring the offending tab was stopped at the end of the script. This isn't a bug.