marty90 / pychromedevtools Goto Github PK

PyChromeDevTools is a python module that allows one to interact with Google Chrome using Chrome DevTools Protocol within a Python script.

License: Apache License 2.0

Python 100.00%

pychromedevtools's People

Contributors

Stargazers

Watchers

pychromedevtools's Issues

ModuleNotFoundError: No module named 'PyChromeDevTools'

After installing the packages as described, tried executing your code in a local py file and my terminal keeps returning "ModuleNotFoundError: No module named 'PyChromeDevTools'". Is there something I'm missing to import the lib?

Websocket times out in wait_event

This happens before the timeout, and leads to the except/break.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "project.py", line 200, in some_method
    loadEvent, _ = self._chrome.wait_event('Page.loadEventFired', timeout=60)
  File "/path/to/project/PyChromeDevTools/__init__.py", line 84, in wait_event
    message = self.ws.recv()
  File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 293, in recv
    opcode, data = self.recv_data()
  File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 310, in recv_data
    opcode, frame = self.recv_data_frame(control_frame)
  File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 323, in recv_data_frame
    frame = self.recv_frame()
  File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 357, in recv_frame
    return self.frame_buffer.recv_frame()
  File "/path/to/project/PyChromeDevTools/websocket/_abnf.py", line 340, in recv_frame
    self.recv_header()
  File "/path/to/project/PyChromeDevTools/websocket/_abnf.py", line 288, in recv_header
    header = self.recv_strict(2)
  File "/path/to/project/PyChromeDevTools/websocket/_abnf.py", line 375, in recv_strict
    bytes_ = self.recv(min(16384, shortage))
  File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 427, in _recv
    return recv(self.sock, bufsize)
  File "/path/to/project/PyChromeDevTools/websocket/_socket.py", line 83, in recv
    raise WebSocketTimeoutException(message)
PyChromeDevTools.websocket._exceptions.WebSocketTimeoutException: timed out

Error Connection refused

Traceback (most recent call last):
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 356, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10b4652b0>: Failed to establish a new connection: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 649, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=1234): Max retries exceeded with url: /json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10b4652b0>: Failed to establish a new connection: [Errno 61] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/aldoec/PycharmProjects/untitled/index.py", line 3, in <module>
    chrome = PyChromeDevTools.ChromeInterface(host="127.0.0.1", port=1234)
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/PyChromeDevTools-0.1-py3.6.egg/PyChromeDevTools/__init__.py", line 43, in __init__
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/PyChromeDevTools-0.1-py3.6.egg/PyChromeDevTools/__init__.py", line 51, in connect
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/PyChromeDevTools-0.1-py3.6.egg/PyChromeDevTools/__init__.py", line 46, in get_tabs
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/sessions.py", line 502, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/sessions.py", line 612, in send
    r = adapter.send(request, **kwargs)
  File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/adapters.py", line 504, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=1234): Max retries exceeded with url: /json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10b4652b0>: Failed to establish a new connection: [Errno 61] Connection refused',))

websocket._exceptions.WebSocketConnectionClosedException: socket is already closed.

Hi, I open the chrome browser and try to do anything after connecting to the dev tools, but anything I try results in 'socket is already closed'...

Any suggestions?? (I have tried all of those lines and all of them end up throwing the socket closed error.. chrome = PyChromeDevTools.ChromeInterface() - seems to be working fine.

subprocess.call(['open', '-a', "/Applications/Google Chrome.app", '--args', '--remote-debugging-port=9222'])
chrome = PyChromeDevTools.ChromeInterface()
chrome.Network.enable()
# chrome.Page.enable()
# chrome.Page.navigate(url="https://www.google.com")
# chrome.wait_event("Page.loadEventFired", timeout=60000)```

targetID?

You can connect directly to a targetID using the method connect_targetID. You must pass the targetID as parameter to this function. In this case, you avoid querying the json and finding the tab id.

the "tab id" is webSocketDebuggerUrl?

Is it possible to get websocket data?

With Chrome in general, I can see the websocket data from Chrome dev tools - network tab - websocket request - Frame tab.

Is it possible to get these websocket data via PyChromeDevTools If i go to the url where using websocket?

How to do Page.javascriptDialogOpening?

chrome.Runtime.evaluate(expression="alert(1111)")
I want to callback do it

ep:

chrome.Page.handleJavaScriptDialog(accept=True)

will be close the alert

PyChromeDevTools.ChromeInterface() return decoder error

C:/work/harChrome/harCapturer.py:50: DeprecationWarning: use options instead of chrome_options
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),
Traceback (most recent call last):
File "C:\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
python-BaseException
Process finished with exit code -1073741510 (0xC000013A: interrupted by Ctrl+C)

Puplish to PyPi

Hey,

I just wanted to ask, if it is possible for you to publish this package to PyPi? This would ease the installation process. As you have already a setup.py file, this should not be to complicated.

Cannot Make PDF Landscape or Change Size of Screenshots

Please see title. I know these features are experimental, I was wondering if more information could be given, such as whether these will be included in the final version and if so, what would the estimated time of their completion be?

Thank you, I appreciate your work.

WebSocketBadStatusException: Handshake status 403 Forbidden when connecting to >= WebView 111.0.5563.58

I'm getting following error for recent versions of WebView.

Call: PyChromeDevTools.ChromeInterface(host='127.0.0.1', port=37461, timeout=5)

Stack trace:

    websock.connect(url, **options)
  File "/home/.../.pyenv/versions/venv/lib/python3.10/site-packages/websocket/_core.py", line 253, in connect
    self.handshake_response = handshake(self.sock, url, *addrs, **options)
  File "/home/.../.pyenv/versions/venv/lib/python3.10/site-packages/websocket/_handshake.py", line 57, in handshake
    status, resp = _get_resp_headers(sock)
  File "/home/.../.pyenv/versions/venv/lib/python3.10/site-packages/websocket/_handshake.py", line 147, in _get_resp_headers
    raise WebSocketBadStatusException("Handshake status %d %s", status, status_message, resp_headers)
websocket._exceptions.WebSocketBadStatusException: Handshake status 403 Forbidden

It sems server is not accepting connection from the localhost. When I've added suppress_origin=True to create_connection call it worked.

I think the example to measure time to load is wrong

Used the exact same example from notes. (python 2.7.3) Every website I go to takes 2 seconds to load.
Tested with www.httpbin.org/delay/8 that should take a 8s delay and it also loaded in 2 seconds. how can I troubleshoot to understand what is going on?

setGeolocationOverride

Hi, trying to geolocate the browser doesn't seem to work.

print(chrome.Emulation.canEmulate())
returns {'id': 3, 'result': {'result': True}} - OK

chrome.Emulation.setDeviceMetricsOverride(width=2436, height=1125, fitWindow=True, deviceScaleFactor=1, mobile=True)
returns a landscaped bigscreen - OK

but...
chrome.Emulation.setGeolocationOverride(latitude=27, longitude=78, accuracy=100)
then
chrome.Page.navigate(url="https://www.google.com/maps")
returns Google maps at @40.4992353,-3.8787972,16z, my IP coordinates.

Is it possible to override geolocation with the protocol viewer?
Thank you

Catch Ajax Request

How can i catch ajax request?
Example: Google Maps

If I want to fetch real-time http or https traffic

If I want to fetch real-time http or https traffic, but create a new tag, I can't get the traffic of the new tag using chrome.wait_message()

chrome = ChromeInterface()
chrome.Network.enable()
chrome.Page.enable()

count = 0
while True:
messages = chrome.wait_message()

if messages != None:

    try:
        print(messages["params"]["response"]["url"])
    except:
        pass

How to get the HTML source code? Thanks

Some questions about --remote-allow-origins and how I solved it

I often use PyChromeDevTools to debug Android Webview pages, which is an excellent open source library.
Recently, I discovered that due to the upgrade of the Android Webview kernel, PyChromeDevTools cannot connect to Android Webview normally because of the security policy --remote-allow-origins.
However, I couldn't specify Android to use this command line parameter to start Webview, and encountered a similar error: #28.

After a long period of research and attempts, I think PyChromeDevTools needs to be optimized:
In the PyChromeDevTools source code _init_.py, lines 60, 67, and 71,
websocket.create_connection(wsurl)
should be changed to
websocket.create_connection(wsurl, suppress_origin=True).

Now let me explain, the modified code blocks the Origin header in the first http connection for WebSocket connection. Chromium does not seem to be restricted by the parameter --remote-allow-origins when the Origin header does not exist.

After this modification, my Android Webview can use PyChromeDevTools to restore normal connection.
The above is my experience sharing, I hope it can help people in need:)

Can it drive multiple Chrome at the same time?

Sometimes, I want to visit various pages at the same time.

Errors are hidden

Errors in the protocol responses should be turned into Python exceptions, so that devs can see details and fix their code.

Get logs of AdBlock Plugins

Hi Marty90,

When an Adblock Block Plus extension is added in chrome, the Adblock console can be viewed using inpspect. I would like to get logs of Adblock Plus using your tool. Is it possible to integrate that?

Basically, the code will person:
inpsect > Adblock > select (show blocked, items of all) type and logs the requests to a file.

Thanks!

How to use it to interact with Chrome console using .py file?

Thank you for the code. I would like to ask that how to use it to interact with Chrome console using .py file?

How does one start a new context in chrome via Target.createBrowserContext ?

Question is title. Thanks

In the case of network.enable, when the webpage has a looped video, pop_messages will enter an endless loop

1、Network.enable().
2、open a webpage , the webpage has a looped video.
3、try Page.getFrameTree()
The code is stuck in an endless loop at self.parent.pop_messages () of generic_function。In pop_messages, it will always receive Network.dataReceived Events

Can it work with chromedriver and selenium?

The tool work well when I use it to scrapy the AJAX web. However, I have to click links in the ajax page to load the data I needed. So, I wonder whether there is a way to automatically navigate the page， like what chromedriver and selenium do.

Not able to open an instance of google chrome with DevTools

I am working on a macbook pro Retina, 13-inch, Late 2012. High Sierra OS verision 10.13.15 (17F77)

My problem is that I am attempting to open Chrome with the remote debugging port option and the program stalls in terminal. I have to press ^C, killing the process, because otherwise it just sits there. Here is what I see in my terminal:

(3rdPartyInstagramApp) iphone:debugger-protocol-viewer forestdewberry$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 --user-data-dir=remote-profile

DevTools listening on ws://127.0.0.1:9222/devtools/browser/25b29bd9-3b71-4f5a-b2f8-bc97d0ba2ed5
2018-07-15 16:26:42.302 Google Chrome[48614:4393942] *** Owner supplied to -[NSTrackingArea initWithRect:options:owner:userInfo:] referenced a deallocating object. Tracking area behavior is undefined. Break on NSTrackingAreaDeallocatingOwnerError to debug.
2018-07-15 16:26:56.076 Google Chrome[48614:4393942] *** Owner supplied to -[NSTrackingArea initWithRect:options:owner:userInfo:] referenced a deallocating object. Tracking area behavior is undefined. Break on NSTrackingAreaDeallocatingOwnerError to debug.
^C(3rdPartyInstagramApp) iphone:debugger-protocol-viewer forestdewberry

What is causing this error and how do I fix it?

Browser.close() does not work

I read Chrome DevTools protocol docs.
After v1.3 it support gracefully close method in Browser domain.
but it doesn't work with PyChromeDevTools.

My chrome version is 71.0.3578.98

Thanks.

marty90 / pychromedevtools Goto Github PK

pychromedevtools's People

Contributors

Stargazers

Watchers

Forkers

pychromedevtools's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs