vikstrous / pirate-get

A command line interface for The Pirate Bay

pirate-get

pirate-get is a convenient command line tool (inspired by APT) to speed up your trip to the Pirate Bay and get your completely legal torrents more quickly.

Installation

Make sure you have python 3.4 and pip installed. On Ubuntu 14.04 you may also need to install the libxslt1-dev and libxml2-dev packages.

Run pip3 install pirate-get

Usage

To search use pirate-get [search term].

See pirate-get -h for more options.

Watch this for an example usage.

Configuration file

You can use a file to override pirate-get's default settings. Default is $XDG_CONFIG_HOME/pirate-get. If it does not exist then $HOME/.config/pirate-get.

Default config file

Here the available options and their behaviors are when unset:

[Save]
; directory where to save files
directory = $PWD

; save each selected magnet link in a .magnet file
magnets = false

; save each selected torrent in a .torrent file
torrents = false

[LocalDB]
; use a local copy of the csv formatted pirate bay database
enabled = false

; path of the database
path = ~/downloads/pirate-get/db

[Search]
; maximum number of results to show
total-results = 50

[Misc]
; specify a custom command for opening the magnet
; ex. myprogram --open %s
; %s represent the magnet uri
openCommand =

; open magnets with transmission-remote client
transmission = false
; set to username:password if needed
transmission-auth =
; set to the port number if needed
transmission-port =

; use colored output
colors = true

; the pirate bay mirror(s) to use:
; one or more space separated URLs
mirror = http://thepiratebay.org

Note: Any command line option will override its respective setting in the config file.

Local Database

If you want to use a local copy of the Pirate Bay database download a copy here (or wherever the latest version is currently):

https://thepiratebay.org/static/dump/csv/

License

pirate-get is licensed under the GNU Affero General Public License version 3 or later. See the accompanying file LICENSE or http://www.gnu.org/licenses/agpl.html.

pirate-get's Issues

Error parsing config

Traceback (most recent call last):
│ File "/usr/bin/pirate-get", line 696, in
│ main()
│ File "/usr/bin/pirate-get", line 685, in main
│ elif args.command or config.get('Misc', 'openCommand'):
│ File "/usr/lib/python3.4/configparser.py", line 773, in get
│ d)
│ File "/usr/lib/python3.4/configparser.py", line 374, in before_get
│ self._interpolate_some(parser, option, L, value, section, defaults, 1)
│ File "/usr/lib/python3.4/configparser.py", line 423, in _interpolate_some
│ "found: %r" % (rest,))
│configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%
│s ; specify a custom command for opening the magnet'

Add gzip support

Looks like the pirate bay uses gzip, so we need to handle that. Here's an example: http://stackoverflow.com/questions/3947120/does-python-urllib2-automatically-uncompress-gzip-data-fetched-from-webpage

sslv3 alert handshake failure

In OSX 10.9.5, will throw an urllib.error.URLError

<urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:600)>

This seems to be a problem of urllib.request.

The following code will throw the same error

def main():
    req = request.Request('https://proxybay.co/list.txt', headers=default_headers)
    f = request.urlopen(req, timeout=default_timeout)

Could not fetch additional mirrors

As in the issue title i can't fetch anymore the list from proxybay but I can get the list from the given url with my browser.

"NoneType" exception on search - API change required?

This looks like a sweet tool, but I'm having a few problems trying to connect to TPB. I've tried various proxies in the config file, all to no avail. I get the following error upon connecting.

λ ~/ pirate-get -b 
Trying https://piratebaymirror.eu/... Traceback (most recent call last):
  File "/home/chris/.local/bin/pirate-get", line 11, in <module>
    sys.exit(main())
  File "/home/chris/.local/lib/python3.5/site-packages/pirate/pirate.py", line 416, in main
    pirate_main(args)
  File "/home/chris/.local/lib/python3.5/site-packages/pirate/pirate.py", line 319, in pirate_main
    results, site = search_mirrors(printer, args)
  File "/home/chris/.local/lib/python3.5/site-packages/pirate/pirate.py", line 256, in search_mirrors
    result = connect_mirror(mirror, printer, args)
  File "/home/chris/.local/lib/python3.5/site-packages/pirate/pirate.py", line 244, in connect_mirror
    mirror=mirror)
  File "/home/chris/.local/lib/python3.5/site-packages/pirate/torrent.py", line 147, in remote
    res_l += parse_page(res)
  File "/home/chris/.local/lib/python3.5/site-packages/pirate/torrent.py", line 106, in parse_page
    tag['href'].startswith('magnet'))['href']
TypeError: 'NoneType' object is not subscriptable

λ ~/ pirate-get stuff
same result!

Running Elementary 0.4 (Loki) and Python 3.5.

Update: appears to be fetching and parsing the HTML for a given page correctly. Problem is encountered somewhere in the row.find(...) command. Adding a simple print(row) at torrent.py:106 yields:

<center>
<a href="/browse/200" title="More from this category">Video</a><br>
                (<a href="/browse/207" title="More from this category">HD - Movies</a>)
            </br></center>
</td>
<td>
<div class="detName"> <a class="detLink" href="/torrent/15842962/War_Dogs_2016_720p_BrRip_x264_-_FoRM" title="Details for War Dogs 2016 720p BrRip x264 - FoRM">War Dogs 2016 720p BrRip x264 - FoRM</a>
</div>
<a href="/torrent/15842962/War_Dogs_2016_720p_BrRip_x264_-_FoRM" title="Download this torrent using magnet"><img alt="Magnet link" src="/static/img/icon-magnet.gif"/></a><img alt="This torrent has 2 comments." src="/static/img/icon_comment.gif" title="This torrent has 2 comments."><img src="/static/img/11x11p.png"><font class="detDesc">Uploaded Y-day 22:32, Size 760.99 MiB, ULed by <a class="detDesc" href="/user/MegaBanana/" title="Browse MegaBanana">MegaBanana</a></font>
</img></img></td>
<td align="right">10312</td>
<td align="right">1235</td>
</tr>

In the example above, the magnet link begins with "/torrent", rather than "magnet". Perhaps this line needs to be changed to reflect a change in the HTML output?

Update 2: after modifying torrent.py:106 to reflect this change...

tag['title'] == 'Download this torrent using magnet')['href']

we successfully connect to the proxy, but fail at print.py:65

λ ~/git/ pirate-get cool                 
Trying https://piratebaymirror.eu/... Ok 
Traceback (most recent call last):
  File "/usr/local/bin/pirate-get", line 9, in <module>
    load_entry_point('pirate-get==0.2.9', 'console_scripts', 'pirate-get')()
  File "/usr/local/lib/python3.5/dist-packages/pirate_get-0.2.9-py3.5.egg/pirate/pirate.py", line 416, in main
    pirate_main(args)
  File "/usr/local/lib/python3.5/dist-packages/pirate_get-0.2.9-py3.5.egg/pirate/pirate.py", line 330, in pirate_main
    printer.search_results(results, local=args.source == 'local_tpb')
  File "/usr/local/lib/python3.5/dist-packages/pirate_get-0.2.9-py3.5.egg/pirate/print.py", line 65, in search_results
    torrent_name = parse.unquote_plus(name.group(1))
AttributeError: 'NoneType' object has no attribute 'group'

Update 3:
The bug is still at torrent.py:106, and stems from a change in the reporting of magnet links. Now the link takes you to the HTML page for that particular torrent, rather than giving you a nice magnet link. You could add another step to go and fetch the magnet link from that URL (but that's adding another HTTP request per entry, which isn't going to be nice).

Dysfunctional mirror returns HTTP/200

In Denmark (and probably a lot of other places as well) some mirrow like http://thepiratebay.se/ is blocked by ISP, so the GET-request will return HTTP/200, even though it won't work.

Exception when entering 'q' or 'h'

In the main input loop entering a letter 'q' or 'h' results in this exception:

Exception: invalid literal for int() with base 10: '' 

document that this tool connects to transmission

Hi !

It's when reading the sources that I saw that this tool connects to Transmission. That's a killer feature ! It would make sense to have it documented in the readme.

Thanks and keep up the good work :)

Allow more options to be set in the config file

One useful example would be -t because transmission users want to always use transmission and they shouldn't need to specify that every time.

Browse categories/top100

How about adding the ability to browse the top 100 torrents? I personally use that a lot

Can't fetch mirrors

The mirror_list ('https://proxybay.co/list.txt') seems to be down as I keep receiving the follwoing error upon attempting to connect:

Trying https://thepiratebay.org/... Failed 
Could not fetch mirrors :(  
[Errno 111] Connection refused

I've tried to find other existing lists and this site seems to be decent http://www.piratebayproxylist.co/. I have stored all the mirrors links from this webpage in a txt file named new_mirror_list.txt. I have gotten around this issue by doing the following command:

pirate-get ubuntu -m `cat new_mirror_list.txt`

I was wondering if this can somehow be implemented in the code itself? Essentially replacing the mirror_list variable maybe?

when using -t check if transmission is running before doing a search

I've had times when I would search and pick a torrent and only then realize that transmission is not on. It would be nice if we could fail early in that case.

Only one magnet link file is created when downloading multiple magnet links to files

Hi,

I'm trying to download several torrents from pirate-get results, but it only accepts the first one and I have to restart program. Could you please tell me how the command is for, for example download "1", "5" and "15" respective torrents?

fix potential command execution vulnerability with transmission mode (-t)

A malicious pirate bay mirror can attack a pirate-get user who uses transmission mode (-t) by serving malicious results https://github.com/vikstrous/pirate-get/blob/master/pirate-get.py#L671

Write tests

This project is getting big enough that it's non-trivial to verify that all of its options work. We need write tests that mock out the network and test all the different command line options.

manage proxies better

We have to address a few issues:

• For users who can access the pirate bay directly we need to prioritize the real pirate bay
• For users who can't we need to make sure they can configure their preferred proxy or proxy list to be first
• We need an extensible way of parsing proxy lists
• We should consider adding actual http proxies as an option in addition to reverse proxies
• We already have a way to blacklist proxies, but this is something that we might want to have more control over. I propose that we run a github hosted website with json lists that the client can periodically sync with. I imagine the client caching the data for about a day.
• We should prefer https over http whenever possible

We'll need some refactoring to make this happen and we really need proper testing once the codebase is testable.

I'd say this is blocked on #44 to avoid breaking anything in the process of adding these features.

Index links starting from 1

I find it a bit weird that the first item in the search result list is accessed with 0.

Thoughts?

Torrent rather than magnet

Thanks for getting this working with oldpiratebay, it works well.

Is there an option to download the torrent file rather than the magnet link? Otherwise i have to convert the magnet link to get it to work with rtorrent.

sudo pirate-get update

Create an update mechanism that just fetches and installs the latest version of the code from the github repo.

Finding an unblocked pirate bay?

In the UK (and elsewhere) the Pirate Bay is blocked but there are loads of mirrors that let us in. It would be good if this ran through a list of mirrors located on the web until it found one that worked!

add a version command that prints the version

This is useful for verifying that a release was successful.

exception while searching

$ pirate-get game of thrones
Trying https://www.piratefly.com... Ok 
LINK   SEED  LEECH  RATIO  SIZE       UPLOAD       NAME                                       
Traceback (most recent call last):
  File "/usr/bin/pirate-get", line 691, in <module>
    main()
  File "/usr/bin/pirate-get", line 586, in main
    print_search_results(mags, sizes, uploaded, local=args.database)
  File "/usr/bin/pirate-get", line 362, in print_search_results
    size = float(sizes[m][0])
IndexError: list index out of range

install issue

hey, when i just press pip3 install pirate-get, i got this error:

hope for your reply, thx~

Consider .btsearch support

BEP18 describes a .btsearch file format to standardize searching trackers. AFAIK this supported only in uTorrent. We should look into supporting this because it might make it possible to add many other search backends really easily. Unfortunately the only reference to actual .btsearch files I found is this abandoned torrent: http://www.btsdl.cc/buscadores-esenciales-para-utorrent-bittorrent-tf3091664.html If anyone who has Windows is reading this, I'd appreciate it if you could help me get my hands on uTorrent's .btsearch files.

http://www.bittorrent.org/beps/bep_0018.html

http://www.opensearch.org/Specifications/OpenSearch/1.1/Draft_5

Getting errors when running on Raspberry Pi

I have installed python3 and pip on my raspberry pi and on ubuntu on a different computer. In ubuntu pirate-get works fine. Pirate-get used to work fine on my raspberry pi too until recently, so I tried installing this new version that relies on python3.

$pirate-get -h
Traceback (most recent call last):
File "/usr/local/bin/pirate-get", line 9, in
load_entry_point('pirate-get==0.2.7', 'console_scripts', 'pirate-get')()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 337, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2280, in load_entry_point
return ep.load()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 1990, in load
entry = import(self.module_name, globals(),globals(), ['name'])
File "/usr/local/lib/python3.2/dist-packages/pirate/pirate.py", line 11, in
import pirate.data
File "/usr/local/lib/python3.2/dist-packages/pirate/data.py", line 8, in
categories = json.loads(get_resource('categories.json').decode())
File "/usr/local/lib/python3.2/dist-packages/pirate/data.py", line 6, in get_resource
return pkgutil.get_data(package, 'data/' + filename)
File "/usr/lib/python3.2/pkgutil.py", line 573, in get_data
loader = get_loader(package)
File "/usr/lib/python3.2/pkgutil.py", line 461, in get_loader
return find_loader(fullname)
File "/usr/lib/python3.2/pkgutil.py", line 471, in find_loader
for importer in iter_importers(fullname):
File "/usr/lib/python3.2/pkgutil.py", line 421, in iter_importers
if fullname.startswith('.'):
AttributeError: 'NoneType' object has no attribute 'startswith'

It's broken again

They removed the sorting feature from oldpiratebay.org. I don't know why or how long it will be like this, but it might make sense to go back to using thepiratebay.se. The long term solution is to use more than one source, but that will take more effort.

Front slash in torrent name treated as file path

Attempting to download a torrent file with a name including a front slash into a local directory throws a "FileNotFoundError"

Document the config file

SyntaxError: invalid syntax on Mac

After fresh sudo pip install pirate-get, I'm unable to run the program:

~ » pirate-get -h
Traceback (most recent call last):
  File "/usr/local/bin/pirate-get", line 9, in <module>
    load_entry_point('pirate-get==0.2.8', 'console_scripts', 'pirate-get')()
  File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources/__init__.py", line 519, in load_entry_point
  File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources/__init__.py", line 2630, in load_entry_point
  File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources/__init__.py", line 2310, in load
  File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources/__init__.py", line 2316, in resolve
  File "/usr/local/lib/python2.7/site-packages/pirate/pirate.py", line 20
    from pirate.print import Printer
                    ^
SyntaxError: invalid syntax

Problem in latest update

I was really happy to see how quick you fixed the seeders and leechers but there is something broken in the new version when you try to fetch the magnet. Anyway thanks for this once again. If there was some way to see the top 100 list as well I would never need to visit that page again =).

Other than that these versions has been tested on crunchbang as well now.

Error code:
Traceback (most recent call last):
File "/usr/bin/pirate-get", line 100, in
main()
File "/usr/bin/pirate-get", line 93, in main
webbrowser.open(mags[choice])
File "/usr/lib/python2.7/webbrowser.py", line 61, in open
if browser.open(url, new, autoraise):
File "/usr/lib/python2.7/webbrowser.py", line 190, in open
for arg in self.args]
TypeError: expected a character buffer object

use prettytable for the output formatting

https://code.google.com/p/prettytable/wiki/Tutorial

This would make it much more testable. Right now the test is a huge hack.

system can't find pirate-get

hi! python's noob question.. I installed pirate-get successfully:
pip3 list | grep pirate-get
pirate-get (0.2.8)
but if I try to execute it, it says "pirate-get: command not found"
in what other way can I launch the program? maybe some argument to python executable?
thanks! (ubuntu 16, phython 3.4.3+)

Using pipes in openCommand

I'm trying to get pirate-get to save the magnet URI to the clipboard by using openCommand without success.

This is what I have in my config file:

openCommand = echo '%s' | pbcopy

For reasons unknown to me, the shell is considering everything after echo as an argument to it, so it just outputs the magnet URI, the pipe and the pbcopy command. Is there some escaping or quoting I need to do? I tried fiddling around a bit with backslashes and quotes without luck.

Any help is appreciated.

remove dependency on torcache

It looks dead.

[Wishlist] Add and list amount of seeders

Would it not be good and pretty easy to list amount of seeders and maybe change so that the torrent with most seeders gets top rank? Just a thought. Anyway really clever project!

UnicodeEncodeError: 'latin-1' codec can't encode characters

Hi, here you can find the full error log:

Traceback (most recent call last):
File "/usr/local/bin/pirate-get", line 9, in
load_entry_point('pirate-get==0.2.10', 'console_scripts', 'pirate-get')()
File "/usr/local/lib/python3.4/dist-packages/pirate/pirate.py", line 424, in main
pirate_main(args)
File "/usr/local/lib/python3.4/dist-packages/pirate/pirate.py", line 338, in pirate_main
printer.search_results(results, local=args.source == 'local_tpb')
File "/usr/local/lib/python3.4/dist-packages/pirate/print.py", line 98, in search_results
self.print(table)
File "/usr/local/lib/python3.4/dist-packages/pirate/print.py", line 37, in print
return builtins.print(*args, **kwargs)
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 1869-1870: ordinal not in range(256)

Can you help me? Thank you.

SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)

$ pirate-get -0 -S /TV Shows/Downloads/ search the.blacklist.S03e19 -m https://pirateproxy.red
Trying https://pirateproxy.red... Failed
Could not fetch mirrors :(
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)

Hi there. Great tool but keeps generating the error above when trying to use it on El Capitan (Mac OS X 10.11). I have read it might be an OpenSSL issue but, after having spent hours on forum, I am (I think) up to date with:

openssl version -a
OpenSSL 1.0.2j 26 Sep 2016
built on: reproducible build, date unspecified
platform: darwin64-x86_64-cc
options: bn(64,64) rc4(ptr,int) des(idx,cisc,16,int) idea(int) blowfish(idx)
compiler: /usr/bin/clang -I. -I.. -I../include -fPIC -fno-common -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
OPENSSLDIR: "/opt/local/etc/openssl"

It's the same in Python:

$ python -c "import ssl; print ssl.OPENSSL_VERSION"
OpenSSL 1.0.2j 26 Sep 2016

but for some reason it doesn't seem to be working in Python3, is that why?

python3 -c "import ssl; print ssl.OPENSSL_VERSION"
File "", line 1
import ssl; print ssl.OPENSSL_VERSION
^
SyntaxError: invalid syntax

I would appreciate some help here, I really like the tool!

Many thanks.

UnicodeEncodeError

Good morning,
I've this error when I launch command 'pirate-get' (without any options):

UnicodeEncodeError: 'ascii' codec can't encode character '\xa0' in position 164: ordinal not in range(128)

This is the output of command:

Trying https://thepiratebay.mn...
Failed
Trying https://pirateproxy.pl...
Ok
Traceback (most recent call last):
File "/usr/local/bin/pirate-get", line 9, in
load_entry_point('pirate-get==0.2.8', 'console_scripts', 'pirate-get')()
File "/usr/local/lib/python3.4/dist-packages/pirate/pirate.py", line 396, in main
pirate_main(args)
File "/usr/local/lib/python3.4/dist-packages/pirate/pirate.py", line 314, in pirate_main
printer.search_results(results, local=args.source == 'local_tpb')
File "/usr/local/lib/python3.4/dist-packages/pirate/print.py", line 95, in search_results
self.print(table)
File "/usr/local/lib/python3.4/dist-packages/pirate/print.py", line 38, in print
return builtins.print(_args, *_kwargs)

UnicodeEncodeError: 'ascii' codec can't encode character '\xa0' in position 164: ordinal not in range(128)

I use pirate-get in raspberry pi running OSMC (variant of raspbian with kodi) and Python3.4.

I do not know how to solve this problem, also researched streams returns "no results".

Make proxy order and priority configurable; always try the real pirate bay first

is there any way to get this working with proxies from the proxybay? bt blocks the default piratebay.se for me

Libxml2 and Libxslt1 required

On Ubuntu 14.04.3 LTS, I need to install this 2 packages:

libxslt1-dev
libxml2-dev

Maybe you can includes this 2 packages as requirement in README.md ? Thanks.

Publish to package managers

it would awesome if we could

# use apt-get
apt-get install pirate-get

# apt-get's osx cousin
brew install pirate-get

related links
https://github.com/Homebrew/homebrew/wiki/Formula-Cookbook
https://help.launchpad.net/Packaging/PPA
https://wiki.archlinux.org/index.php/creating_packages

default flag -T not clearly documented

When using the option to download the .torrent file as so

pirate-get -T 'ubuntu-1206'

Here's a dump of the trace

Saving selected torrents...
Traceback (most recent call last):
  File "/usr/local/bin/pirate-get", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/site-packages/pirate/pirate.py", line 396, in main
    pirate_main(args)
  File "/usr/local/lib/python3.5/site-packages/pirate/pirate.py", line 377, in pirate_main
    pirate.torrent.save_torrents(choices, results, args.save_directory)
TypeError: save_torrents() missing 1 required positional argument: 'folder'

Also, where is the log file on the mac?

Pirate Bay Gone

As you may have noticed the pirate bay is no more. None of the mirrors are working.

There is an isohunt version that still works. How can pirate-get be modified to work with this version?

https://oldpiratebay.org

I tried adding it as a mirror but got the following error:

Could not fetch additional mirrors
Trying https://oldpiratebay.org
HTTP Error 404: Not Found
Could not contact https://oldpiratebay.org
Traceback (most recent call last):
File "/usr/bin/pirate-get", line 393, in
main()
File "/usr/bin/pirate-get", line 232, in main
if not mags or len(mags) == 0:
UnboundLocalError: local variable 'mags' referenced before assignment

local database errors

$ pirate-get --local "poor3.xml" archlinux

Traceback (most recent call last):
File "/usr/bin/pirate-get", line 310, in
main()
File "/usr/bin/pirate-get", line 208, in main
mags = local(args)
File "/usr/bin/pirate-get", line 193, in local
htmlparser = MyHTMLParser(args.q)
AttributeError: 'Namespace' object has no attribute 'q'

Add an option for fetching more than one page of results

Support for other trackers

Since we've had issues with TPB going down many times, we should make pirate-get a bit more modular and add support for more trackers. Here's the list of popular trackers that we should support:

http://torrentfreak.com/top-popular-torrent-sites-2015-150104/

command execution vuln when using openCommand config or -C

Refactor the interface

Here's my plan for a new interface:

pirate-get <action> <options>

where <action> can be one of browse, recent, search, top, categories, sorts

and <options> is everything else we already have as options.

One of the options will be -o / --output specifying a directory to output the results. This replaces --save-directory

One more new option will be -O / --output-type which can be browser, transmission, magnet, torrent, or command. If using command -o specifies the PWD and -c / --command specifies the command to execute.

---custom fails when `%s` is quoted

follow up from #48

From #48:

I'm currently travelling, so it might take me some time before I get to fixing it. If anyone wants to give it a shot, I recommend just stripping the quotation marks around any of the items in the ret array.

support routing through tor or any other custom proxy

We should add a config and/or command line parameter to set a proxy. This can probably already be done through an environment variable (I haven't checked), but this would be easier to use.