adibalcan / pytor

Python module for http requests by tor network

PyTor

PyTor is a wrapper for requests python module that injects and controls a proxy obtained from Tor network or from a proxy list.

PyTor change automatically the IP address and retry when a string marker is detected in response to a request.

Requirements

Tor,

Privoxy

Below you will find how to configure each of them

Installation

ATENTION: First install pip and git

pip install git+git://github.com/adibalcan/PyTor.git@master

Usage

The basic configuration

from pytor import pytor

# Use here, password from tor configuration
pytor.behaviour.password='YOURPASSWORD'

# Automatically change the IP address if one of those strings are met in http response
pytor.invalidStringList = ['Sorry, you\'re not allowed to access this page.', 'One more step']

# Get source :)
print(pytor.get('https://google.com').text)

The advanced configuration

You can add multiple options like in the following sample:

from pytor import pytor

# Use here, password from tor configuration
pytor.behaviour.password='YOURPASSWORD' 

# Automatically change the IP address if one of those strings are met in http response
pytor.invalidStringList = ['Sorry, you\'re not allowed to access this page.', 'One more step']

# Change the IP address if response is shorter then 200 chars
pytor.minSourceLength = 200 

# Show IP address at every request 
pytor.showIp = True

# Show output messages 
pytor.silent = False

# Get source :)
print(pytor.get('https://google.com').text)

Privoxy configuration

Privoxy's main configuration file is already prepared for Tor, if you are using a default Tor configuration and run it on the same system as Privoxy, you just have to edit the forwarding section and uncomment the next line from /etc/privoxy/config:

#        forward-socks5t             /     127.0.0.1:9050 .

Tor configuration

Generate a hash password

tor --hash-password YOURPASSWORD

Uncomment this line from /etc/tor/torrc

ControlPort 9051

Update hash generated above in /etc/tor/torrc

HashedControlPassword 16:3C78EB9AB441234760C108BCC7F8CF3138FA14378C116ECD3E9C942E51

pytor's Issues

What is purpose of pytor.password

Is this to get the password from the torrc file or to set the password?

error: could not create '/usr/local/lib/python2.7/dist-packages/stem': Permission denied

Hi Adrian,

Thanks for making this package available. I'm trying to install it on an ec2 node using ubuntu 14.04 LTS. I'm getting the error in the title. Any idea how to fix it. My commands and output are:

ubuntu@ip-172-31-28-196:$ pip install git+git://github.com/adibalcan/PyTor.git@master
Downloading/unpacking git+git://github.com/adibalcan/PyTor.git@master
Cloning git://github.com/adibalcan/PyTor.git (to master) to /tmp/pip-_SWCZw-build
Cleaning up...
Cannot find command 'git'
Storing debug log for failure in /home/ubuntu/.pip/pip.log
ubuntu@ip-172-31-28-196:$ sudo apt-get -y install git-core
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
git git-man liberror-perl
Suggested packages:
git-daemon-run git-daemon-sysvinit git-doc git-el git-email git-gui gitk
gitweb git-arch git-bzr git-cvs git-mediawiki git-svn
The following NEW packages will be installed:
git git-core git-man liberror-perl
0 upgraded, 4 newly installed, 0 to remove and 57 not upgraded.
Need to get 3,347 kB of archives.
After this operation, 21.6 MB of additional disk space will be used.
Get:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ trusty/main liberror-perl all 0.17-1.1 [21.1 k
B]
Get:2 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ trusty-updates/main git-man all 1:1.9.1-1ubunt
u0.1 [698 kB]
Get:3 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ trusty-updates/main git amd64 1:1.9.1-1ubuntu0
.1 [2,627 kB]
Get:4 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ trusty-updates/main git-core all 1:1.9.1-1ubun
tu0.1 [1,458 B]
Fetched 3,347 kB in 1s (2,650 kB/s)
Selecting previously unselected package liberror-perl.
(Reading database ... 57260 files and directories currently installed.)
Preparing to unpack .../liberror-perl_0.17-1.1_all.deb ...
Unpacking liberror-perl (0.17-1.1) ...
Selecting previously unselected package git-man.
Preparing to unpack .../git-man_1%3a1.9.1-1ubuntu0.1_all.deb ...
Unpacking git-man (1:1.9.1-1ubuntu0.1) ...
Selecting previously unselected package git.
Preparing to unpack .../git_1%3a1.9.1-1ubuntu0.1_amd64.deb ...
Unpacking git (1:1.9.1-1ubuntu0.1) ...
Selecting previously unselected package git-core.
Preparing to unpack .../git-core_1%3a1.9.1-1ubuntu0.1_all.deb ...
Unpacking git-core (1:1.9.1-1ubuntu0.1) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up liberror-perl (0.17-1.1) ...
Setting up git-man (1:1.9.1-1ubuntu0.1) ...
Setting up git (1:1.9.1-1ubuntu0.1) ...
Setting up git-core (1:1.9.1-1ubuntu0.1) ...
ubuntu@ip-172-31-28-196:~$ pip install git+git://github.com/adibalcan/PyTor.git@master
Downloading/unpacking git+git://github.com/adibalcan/PyTor.git@master
Cloning git://github.com/adibalcan/PyTor.git (to master) to /tmp/pip-JKoRkP-build
Running setup.py (path:/tmp/pip-JKoRkP-build/setup.py) egg_info for package from git+git://github.
com/adibalcan/PyTor.git@master

Downloading/unpacking stem (from pytor==1.5)
Downloading stem-1.3.0.tar.bz2 (1.5MB): 1.5MB downloaded
Running setup.py (path:/tmp/pip_build_ubuntu/stem/setup.py) egg_info for package stem

Requirement already satisfied (use --upgrade to upgrade): requests in /usr/lib/python2.7/dist-packag
es (from pytor==1.5)
Installing collected packages: stem, pytor
Running setup.py install for stem
changing mode of build/scripts-2.7/tor-prompt from 664 to 775
error: could not create '/usr/local/lib/python2.7/dist-packages/stem': Permission denied
Complete output from command /usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip_
build_ubuntu/stem/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('
r\n', '\n'), file, 'exec'))" install --record /tmp/pip-MoV3kV-record/install-record.txt --single
-version-externally-managed --compile:
running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/stem

copying stem/process.py -> build/lib.linux-x86_64-2.7/stem

copying stem/socket.py -> build/lib.linux-x86_64-2.7/stem

copying stem/prereq.py -> build/lib.linux-x86_64-2.7/stem

copying stem/version.py -> build/lib.linux-x86_64-2.7/stem

copying stem/init.py -> build/lib.linux-x86_64-2.7/stem

copying stem/connection.py -> build/lib.linux-x86_64-2.7/stem

copying stem/exit_policy.py -> build/lib.linux-x86_64-2.7/stem

copying stem/control.py -> build/lib.linux-x86_64-2.7/stem

creating build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/microdescriptor.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/router_status_entry.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/init.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/server_descriptor.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/reader.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/export.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/networkstatus.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/extrainfo_descriptor.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/tordnsel.py -> build/lib.linux-x86_64-2.7/stem/descriptor

copying stem/descriptor/remote.py -> build/lib.linux-x86_64-2.7/stem/descriptor

creating build/lib.linux-x86_64-2.7/stem/interpreter

copying stem/interpreter/init.py -> build/lib.linux-x86_64-2.7/stem/interpreter

copying stem/interpreter/help.py -> build/lib.linux-x86_64-2.7/stem/interpreter

copying stem/interpreter/arguments.py -> build/lib.linux-x86_64-2.7/stem/interpreter

copying stem/interpreter/commands.py -> build/lib.linux-x86_64-2.7/stem/interpreter

copying stem/interpreter/autocomplete.py -> build/lib.linux-x86_64-2.7/stem/interpreter

creating build/lib.linux-x86_64-2.7/stem/response

copying stem/response/authchallenge.py -> build/lib.linux-x86_64-2.7/stem/response

copying stem/response/init.py -> build/lib.linux-x86_64-2.7/stem/response

copying stem/response/mapaddress.py -> build/lib.linux-x86_64-2.7/stem/response

copying stem/response/events.py -> build/lib.linux-x86_64-2.7/stem/response

copying stem/response/getinfo.py -> build/lib.linux-x86_64-2.7/stem/response

copying stem/response/getconf.py -> build/lib.linux-x86_64-2.7/stem/response

copying stem/response/protocolinfo.py -> build/lib.linux-x86_64-2.7/stem/response

creating build/lib.linux-x86_64-2.7/stem/util

copying stem/util/str_tools.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/conf.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/proc.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/test_tools.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/ordereddict.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/init.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/system.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/lru_cache.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/connection.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/term.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/enum.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/tor_tools.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/util/log.py -> build/lib.linux-x86_64-2.7/stem/util

copying stem/interpreter/settings.cfg -> build/lib.linux-x86_64-2.7/stem/interpreter

running build_scripts

creating build/scripts-2.7

copying and adjusting tor-prompt -> build/scripts-2.7

changing mode of build/scripts-2.7/tor-prompt from 664 to 775

running install_lib

creating /usr/local/lib/python2.7/dist-packages/stem

error: could not create '/usr/local/lib/python2.7/dist-packages/stem': Permission denied

---

Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip_build_ubuntu/stem/setup.p
y';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'e
xec'))" install --record /tmp/pip-MoV3kV-record/install-record.txt --single-version-externally-manag
ed --compile failed with error code 1 in /tmp/pip_build_ubuntu/stem
Storing debug log for failure in /home/ubuntu/.pip/pip.log

Why dependency on privoxy?

Why does it rely on Privoxy? With SocksiPy and urllib should be all set right?
Like in: https://stem.torproject.org/tutorials/to_russia_with_love.html