webfp / tor-browser-selenium Goto Github PK

Would help to be able to easily checkout the version of the repository each wheel is built from.

Hello receive this error upon launching tor through this.

The torrc file is missing and could not be created.

When I manually run the torbrowser from command line it works fine Before I provide the massive amounts of logs I have , is this a known issue? Is there anything I can do?

Let me know if you want the runtime logs,

Thanks
I am just using the basic example on the site.

from tbselenium.tbdriver import TorBrowserDriver
with TorBrowserDriver("/path/to/myBundle/") as driver:
    driver.get('https://check.torproject.org')

We get the following import error for Stem on CI when try to import from stem.control import Controller :

No module named stem.control

At the same time logs say Successfully installed stem-1.4.0
Travis log: https://travis-ci.org/webfp/tor-browser-selenium/builds/122708529

$ sudo -H pip install stem
Collecting stem
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading stem-1.4.0.tar.bz2 (1.6MB)
    100% |████████████���███████████████████| 1.6MB 365kB/s 
Building wheels for collected packages: stem
  Running setup.py bdist_wheel for stem
  Stored in directory: /root/.cache/pip/wheels/80/6d/23/2db8210a00ee425efa4f31f4d374214e5325a26901ae57f64a
Successfully built stem
Installing collected packages: stem
Successfully installed stem-1.4.0
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
You are using pip version 7.1.2, however version 8.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

We should make it easy to print the contents of FirefoxBinary's log_file (or the tbb_log_file) to debug the test failures and other errors.

I'm having some issues with the basic usage path if someone can assist.

from tbselenium.tbdriver import TorBrowserDriver
with TorBrowserDriver("/path/to/TorBrowserBundle/") as driver:
driver.get('https://check.torproject.org')

/path/to/TorBrowserBundle/ << what should it be exactly.

My TorBrowser is installed in C:\Test software\Tor Browser, I've tried that directory and in addition I've tried "C:\Test software\Tor Browser\Browser" and "C:\Test software\Tor Browser\Browser\firefox.exe". Please clarify this part.

Hi,

I have installed tbselenium, geckodriver and TBB as required, but when I try first basic usage:

from tbselenium.tbdriver import TorBrowserDriver
with TorBrowserDriver("C:/Users/msagovac/Desktop/Tor Browser") as driver:
    driver.get('https://check.torproject.org')

I got an error:
Environment variable TBB_PATH can't be found.
I am not sure what should be TorBrowserBundle path. Try many alternatives.

I ma using Python 3 on Windows

Hello!

I'm working on some stuff for SecureDrop, and it looks like maybe tbselenium can't read cookies, using driver.get_cookies().

I can set cookies using driver.add_cookies(), and read them back - but when I see a Cookie header in tcpdump, I can't read the cookie in question.

This could well be a problem with the test setup for SecureDrop - I don't have a minimal example, but I'll work on one.

Thanks!

Currently, if a user wants to take screenshots of the pages visited with the driver, he will need to add all the URLs as exceptions using the class method TorBrowserDriver.add_exception() before instantiating the TorBrowserDriver.

Adding an exception means granting permissions to a domain to access the Canvas, otherwise restricted by the TorBrowser to protect against browser fingerprinting. Selenium requires access to the Canvas to take the screenshot. However, the changes we commit to permissions.sqlite do not have any effect when the browser is running (probably the database is loaded into memory by Firefox).

The aim of this ticket is to improve the interface for taking screenshots so that users do not need to add an exception for each future visit beforehand. We need to investigate if there is a way to refresh the permissions loaded in the browser on runtime.

I'm using these parameters for TorBrowserDriver():

tbb_fx_binary_path='/Applications/TorBrowser.app/Contents/MacOS/firefox',
tbb_profile_path='/Users/tommaso/Library/Application Support/TorBrowser-Data/Browser/52e8j7wk.default',
tor_cfg=cm.USE_RUNNING_TOR,
socks_port=7000,
control_port=cm.STEM_CONTROL_PORT

I get the following error:

Traceback (most recent call last):

  File "/Users/tommaso/repos/anon_visits/anon_visits.py", line 6, in <module>
    main()
  File "/Users/tommaso/repos/anon_visits/com/reloadcom/anonvisits/main.py", line 20, in main
    raise exc
  File "/Users/tommaso/repos/anon_visits/com/reloadcom/anonvisits/main.py", line 14, in main
    browser = TorBrowser()
  File "/Users/tommaso/repos/anon_visits/com/reloadcom/anonvisits/tor_browser.py", line 19, in __init__
    control_port=cm.STEM_CONTROL_PORT
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tbselenium/tbdriver.py", line 44, in __init__
    tbb_profile_path, tor_data_dir)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tbselenium/tbdriver.py", line 139, in setup_tbb_paths
    chdir(self.tbb_browser_dir)
FileNotFoundError: [Errno 2] No such file or directory: 'Browser'

I thought the first parameter (tbb_path) was optional, but then you join() it with 'Browser' and try to change the directory inside it (chdir()) which is raising the error because it tries to cd into './Browser/' I think.

How can I fix this, and start TBB on macOS? I'm not too sure what each path parameter shoud be

We should try running all the tests with different tor configs a la #4

When I install tbselenium using pip I can't import the fixtures.py module or any other module in test/. It would be useful for tbselenium users to include them.

I'm able to get the Tor browser to launch but sending the get command does nothing. I have checked that I am able to use Tor outside of calling it via Selenium and it works fine. If I wait long enough I get a connection refused message

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/user/tor-browser-selenium/tbselenium/tbdriver.py", line 57, in __init__ log_path=tbb_logfile_path) File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/firefox/webdriver.py", line 154, in __init__ keep_alive=True) File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 151, in __init__ self.start_session(desired_capabilities, browser_profile) File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 240, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: connection refused

Python 3.5.2, Selenium 3.8, Gecko 0.17.0
Tor 7.0.10, Linux Mint 18.1 (Ubuntu 16.04)

Once in a while we get [Errno 111] Connection refused error during the initialization:
https://travis-ci.org/gunesacar/tor-browser-selenium/jobs/153531217#L380

We should handle this error within the test fixture to retry the initialization.

When using TorBrowserDriver(tor_cfg=cm.USE_RUNNING_TOR) can it connect to the tor process, when a password is set for the controller connection?
If yes, how should i pass the password for the authentication to the TorBrowserDriver()?

We can't start the Tor process with the new alpha (6.0a5)
https://lists.torproject.org/pipermail/tor-qa/2016-April/000803.html

Running with the existing Tor process works ok, but starting a new Tor always fail.

We should investigate the issue @fowlslegs mentions in #10 (comment):

so now it's failing because certain libraries libxul.so (the main Firefox library) is wanting to load are not present on this minimal image.

We should make sure that TB prefers the Firefox libraries shipped with TBB, not the ones found on the system.

test_example.py module provides some examples as tests, which we don't run.
We should turn the tests into examples, such as: http://phantomjs.org/examples/

If we use the tor process in the TBB folder, we don't start with a new data directory and pollute the existing tor state files.

It should be possible to clone tor data dir into a fresh one, as we already do for Firefox profile.

Question

Is there any possibility of running Tor using remote web driver? like:

var remoteWebDriver = new RemoteWebDriver(DesidedCapabilities.Tor());

We could replace it with a py.test single liner.
We just need be able to accept command line arguments in conftest.py.

When using TBB 5.5.3 for the tests, Selenium cannot connect to the TorBrowser (Firefox) driver. The message error from Firefox logs points to a problem with glibc:

tor-browser_en-US/Browser/firefox: relocation error: ../tor-browser_en-US/Browser/firefox: symbol _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERmm, version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

I am able to reproduce it running the tests on an Ubuntu 14.04. The problem goes away when using newer versions of Ubuntu, as they have a newer version of glibc that allows running the firefox binary in TBB. However, Travis only supports instances of Ubuntu up to 14.04.

I am using TBB 4.5.1 for now. Any suggestion on how to solve this problem?

When using the USE_TBB_TOR_LAUNCH_NEW mode we don't modify the control port of TB (9150) and run into port conflicts when another instance of TB is running on the system.

In addition to setting Firefox prefs, we need to modify the torrc file.

If TBB_PATH is set to a relative path I get a popup error message from FF saying it can't find the profile. The issue is fixed by getting the absolute path of Browser directory. I'm going to send a PR in a minute.

In majority of the tests, we load web pages over Tor network, which is not the most reliable transport.

We should look for ways to make tests more resilient, such as rerunning tests when the failure is due to the network.

We should allow users to pick which tor process they want to use:
1. Use tor running on a certain port: e.g. system tor daemon, non-daemon process started by the user
2. Start tor in the TBB directory: this is the default behavior of TB (torlauncher).
3. Start tor through Stem: allows custom torrc configs and more elaborate control over tor the process

Traceback (most recent call last):
  File "examples.py", line 3, in <module>
    with TorBrowserDriver("/home/me/Downloads/tor-browser_en-US/") as driver:
  File "/home/me/Downloads/tor-browser_en-US/env/lib/python3.6/site-packages/tbselenium/tbdriver.py", line 59, in __init__
    timeout=cm.TB_INIT_TIMEOUT)
  File "/home/me/Downloads/tor-browser_en-US/env/lib/python3.6/site-packages/selenium/webdriver/firefox/webdriver.py", line 80, in __init__
    self.binary, timeout)
  File "/home/me/Downloads/tor-browser_en-US/env/lib/python3.6/site-packages/selenium/webdriver/firefox/extension_connection.py", line 52, in __init__
    self.binary.launch_browser(self.profile, timeout=timeout)
  File "/home/me/Downloads/tor-browser_en-US/env/lib/python3.6/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 68, in launch_browser
    self._wait_until_connectable(timeout=timeout)
  File "/home/me/Downloads/tor-browser_en-US/env/lib/python3.6/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 108, in _wait_until_connectable
    % (self.profile.path))
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Profile Dir: /tmp/tmpiu_7hssx/webdriver-py-profilecopy If you specified a log_file in the FirefoxBinary constructor, check it for details.

I can now see that the error was mentioned here, under troubleshooting: https://github.com/webfp/tor-browser-selenium/blob/master/README.md
If there was any solution linked there, it's very unclear, at least to me. It looks like the Selenium version I'm using is 3.4.2, Python version I use is 3.6.

It would be better to use subprocess module which has been written to replace it, and even better to use packages like psutil in place of shelling out wherever possible.

In fixtures.py, the variable last_err may be accessed without being initialized. It is safer to initialize it outside the try-except. I actually found the bug because the call inside on of the fixture's for loop raised an exception that it was not catched in the except. I'll be sending a PR soon.

If we use any port other that the defaults (9050-9051, 9150-9151) we should add it to banned ports using network.security.ports.banned preference of Firefox.

We must export those variables to enable bundled fonts and related configuration.

I tried the basic usage
with TorBrowserDriver("/home/blackturtle/Downloads/tor-browser-linux64-7.5.3_en-US/tor-browser_en-US/") as driver: driver.get('http://www.google.com')

It opens the webbrowser which is firefox but dont go to adresss?
Any suggestion
Thank you in Advance

In CI tests we pass TBB_PATH as a a relative path which was a major source of mismatch between the local tests and CI tests. We should convert paths to absolute paths as early as possible.

We should check if the library needs any change to be compatible with Selenium 3.
At least we'll have to add 'geckodriver' executable to the PATH for the CI tests:

We need to wait for the ESR52-based Tor Browsers to land this support.

https://travis-ci.org/gunesacar/tor-browser-selenium/jobs/149722181
Message: 'geckodriver' executable needs to be in PATH.

When tor-browser-selenium is asked to start a Tor process and Tor cannot bootstrap, we get a cryptic error from Selenium:

Can't load the profile. Profile Dir: /tmp/tmpxyz/webdriver-py-profilecopy If you specified a log_file in the FirefoxBinary constructor, check it for details

@mjuarezm proposed that it could be useful to expose Tor bootstrap errors and/or status to callers.

WebDriverException: Can't load the profile. Profile Dir: /tmp/tmp_rzsi126/webdriver-py-profilecopy If you specified a log_file in the FirefoxBinary constructor, check it for details.

This occurs after i do:

from tbselenium.tbdriver import TorBrowserDriver
with TorBrowserDriver("/storage/programfiles/tor-browser_en-US") as driver:
    driver.get('www.reddit.com')

Any idea on how to solve this? :)

Using this example, I found a problem in def main()

def main():
    desc = "Visit check.torproject.org website"
    parser = ArgumentParser(description=desc)
    parser.add_argument('TBB_PATH')
    args = parser.parse_args()
    visit(args.tbb_path)

In line args = parser.parse_args() I got an error:

the following arguments are required: TBB_PATH

in py3.4, and

error: too few arguments

in py2.7

If I try debugging step by step it raise:

args = parser.parse_args()
        ^
SyntaxError: invalid syntax

and

SystemExit: 2

TBB_PATH writed to environment variables
OS: Win10

I have tried running my scripts with Tor Browser version 8.0 on Lubuntu 16.04:
https://www.torproject.org/dist/torbrowser/8.0/tor-browser-linux64-8.0_en-US.tar.xz

The browser is working fine when I start it from command line.
When I start it via tbselenium, it does not connect to the Tor process and issues the following error message:
"The proxy server is refusing connections".

The script works fine when I downgrade Tor Browser to version 7.5.6.

Traceback (most recent call last):
File "./run_tests.py", line 25, in
"term-missing", "--durations=10", test_dir])
File "/usr/lib/python3.6/subprocess.py", line 267, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.6/subprocess.py", line 709, in init
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'py.test': 'py.test'

Hi,
could you please give me an example of correct "tbb_fx_binary_path" for windows?
I have this error:
Traceback (most recent call last):
File "C:/Python36/Moje/webTest/Tor2.py", line 10, in
with TorBrowserDriver("I:\Tor\Tor Browser\") as driver:
File "C:\Python36\lib\site-packages\tbselenium\tbdriver.py", line 44, in init
tbb_profile_path, tor_data_dir)
File "C:\Python36\lib\site-packages\tbselenium\tbdriver.py", line 124, in setup_tbb_paths
% tbb_fx_binary_path)
tbselenium.exceptions.TBDriverPathError: Invalid Firefox binary I:\Tor\Tor Browser\Browser\firefox

You should consider shipping this as wheel via PyPi. It would be easier to use as a dependency for people and there are no real alternatives for those wishing to drive Tor Browser with a Python module.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/tbselenium/tbdriver.py", line 57, in __init__
    log_path=tbb_logfile_path)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/firefox/webdriver.py", line 158, in __init__
    keep_alive=True)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 154, in __init__
    self.start_session(desired_capabilities, browser_profile)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 243, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 311, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 237, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: connection refused

Also, the init function of the TorBrowserDriver class seems to be blocking, why is that so ?

raise TBTestEnvVarError("Environment variable `TBB_PATH` can't be found.")

tbselenium.exceptions.TBTestEnvVarError: Environment variable TBB_PATH can't be found.
using windows btw
I try:
C:\Users\D1\Desktop\Tor Browser\Start Tor Browser.lnk

C:\Users\D1\Desktop\Tor Browser\Browser\firefox.exe

C:\Users\D1\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe

What am I missing here?

from tbselenium.tbdriver import TorBrowserDriver
with TorBrowserDriver("C:/Users/D1/Desktop/Tor Browser/Browser/orBrowser/Tor/tor.exe") as driver:
driver.get('https://check.torproject.org')

The browser always initializes with obfs4 bridge when opened through the library. This happens even after I change it to any other bridge like meek-amazon and obfs4 takes long time to open.

Directly opening the browser takes the saved settings but when opened with tbselenium, the torrc file /home/olam/tor-browser_en-US/Browser/TorBrowser/Data/Tor/torrc is being overwritten.

Tor Bundle Version: 6.5.1

Starting and stopping a virtual display is shouldn't be done within TorBrowserDriver. It's rather caller's responsibility to manage xvfb.

Yet, we'd want to run tests using xvfb.

We recently had intermittent test failures on Travis due to an initialization error:

https://travis-ci.org/webfp/tor-browser-selenium/builds/152743802
https://travis-ci.org/webfp/tor-browser-selenium/builds/152723579

The error message from Selenium:
Can't load the profile. Profile Dir: /tmp/tmpO7i1lL/webdriver-py-profilecopy If you specified a log_file in the FirefoxBinary constructor, check it for details