rndusr / torf-cli Goto Github PK
View Code? Open in Web Editor NEWCLI tool for creating, reading and editing torrent files
License: GNU General Public License v3.0
CLI tool for creating, reading and editing torrent files
License: GNU General Public License v3.0
Currently I'm creating very large torrents (in excess of 300GB) and I need a bigger piece size than 16MiB.
Normally this wouldn't be an issue, but due to rtorrent's poor programming it crashes when trying to force check piece hashes for a large amount of piece sizes (almost 20,000 in my case at 16 MiB)
Would it be possible to override this when explicitly giving an argument of a higher piece size than 16MiB? I understand that the piece size logic is done in the base library, but it also has the capability of being overriden by any tool that uses it (in this case torf-cli)
After doing some research, it seems that a lot of recent clients support piece sizes larger than 16MiB, and the 16MiB cap is just a result of a limitation with an old version of uTorrent.
I understand that this may lead to undesirable effects like more wasted data and much less control over selected files to download, but I don't really have a choice here.
something like --piece-size, -ps 16MiB
torf 5.1.0
python 3.12.2
Windows 10 Pro
Doing a simple torf -x -p DIRECTORY
will include Thumbs.db
in that directory, which is undesirable and unexpected. Many users have hidden files showing in Explorer, but very few have "protected operating system files" set to show. From a quick test, I observed that qBittorrent's torrent creator excludes hidden files, both "protected system files" like Thumbs.db
and just plain hidden files.
Thank you! This tool is too powerful, but there is a small regret that it is impossible to view and modify the "source" field of the private torrent file. Can you add this feature? Because the "source" field is important for private trackers.
Ps: You can view and modify the ‘’source’’ field of the torrent file via https://github.com/drizt/torrent-file-editor this, but this tool cannot be operated by commander.
torfcli/_vars_.py was renamed to torfcli/_vars.py but setup.py is still referencing torfcli/_vars_.py:
paulo:~/src/torf-cli$ python3 setup.py install --user
Traceback (most recent call last):
File "setup.py", line 3, in <module>
with open('torfcli/__vars__.py') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'torfcli/__vars__.py'
Running 435d328.
Please disregard.
Wouldn't it make more sense to set the default creation date to now
, instead of today
? I know I can set it in the config file, but it has just occurred to me that now
seems a more natural, expected default. I only noticed when I was testing stig filtering rules against locally created torrents.
On a tracker one of the requirements is set entropy field to a random number.
Is this possible with torf-cli?
First time i ever heard of entropy field and i cant find anything online about it with any torrent creator tool
Right now it's only possible to add multiple files to a torrent by specifying a directory, but this includes the top level directory in the torrent as the root.
For example, running torf /my/files
creates this:
files/file1.txt
files/directory/file2.txt
when for my use case, I want this instead:
file1.txt
directory/file2.txt
Is it possible to allow for torf-cli to have multiple file arguments specified? Such that this would work:
torf /my/files/* ...
This would solve my issue as above.
Alternatively, a switch for excluding the top level directory and having the contents as the root(s) would be acceptable.
It would be useful to have a command-line option to force a formatted output even when piping the output.
Here's why: I'm replacing transmission-show
with dorf
to show torrent files attached to mail messages and also on mc
. In both cases, the output is piped to an internal pager, so the result is, for instance, with mc
:
It's perfect for parsing, I can see there's a single TAB character between each header and its value, but not for viewing. mc
pipes the output to its interval viewer. With neomutt
, of course, the result will be the same, since it also pipes the output to its internal viewer:
An explicit option to force formatted output would be ideal for these cases.
Traceback (most recent call last):
File "/usr/bin/torf", line 33, in <module>
sys.exit(load_entry_point('torf-cli==3.2.0', 'console_scripts', 'torf')())
File "/usr/bin/torf", line 22, in importlib_load_entry_point
for entry_point in distribution(dist_name).entry_points
File "/usr/lib/python3.9/importlib/metadata.py", line 524, in distribution
return Distribution.from_name(distribution_name)
File "/usr/lib/python3.9/importlib/metadata.py", line 187, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: torf-cli
Running Arch and using the torf-cli AUR package.
Could it be possible to add an option like --threads or --workers to control the number of hashing threads?
As part of my own tool I am creating a benchmark of multiple bittorrent metafile utilities.
I want to include this tool but I need to control the number of hashing threads for a fair comparison.
It would be a beneficial feature to be able to remove files from a torrent file as it would mean we do not have to create a new torrent.
Use Case:
You have a torrent that contains a large video file and some small text files.
You want to upload the torrent on another site but do not want to include the small test files only the video file.
Currently we would have to recreate the torrent with just the large video file and this can be time consuming.
As we already have the large file pieces in the exisiting torrent it would be great if we could remove the small files from the exsiting torrent.
This tool appears to have this feature - https://torrent-file-editor.github.io/
I was trying to create a torrent of a directory which contained a soft symlink to another file also in the target directory:
$ ls
OVMF_CODE.fd
OVMF_VARS.fd
snapshot.qcow2
base.qcow2
latest.qcow2 -> snapshot.qcow2
Attempting to create a torrent with torf
resulted in the following error:
Name stuff
Size 28.48 GiB
Created 2022-01-05 14:29:00
Created By torf 3.4.0
Private yes
Piece Size 8 MiB
Piece Count 3646
File Count 5
Files stuff
├─OVMF_CODE.fd [3.48 MiB]
├─OVMF_VARS.fd [528 KiB]
├─latest.qcow2 [2.55 GiB]
├─base.qcow2 [23.38 GiB]
└─snapshot.qcow2 [2.55 GiB]
Progress 90.04 % ▕snapshot.qcow2 ▏ 5.69 GiB/s
torf: Invalid metainfo: ['info']['pieces'] is empty
On a hunch, I removed the symlink latest.qcow2
and was able to successfully create the torrent.
I have no suggestion for how to resolve this, I just wanted to document that a symlink will cause torf
to fail with an error message which does not clearly indicate the root cause of the failure.
If you edit the source and a tracker at the same time the resulting torrent will be NEW and not an edited version of the original.
Steps to reproduce
1). Create new torrent if you do not have an existing one to edit.
torf somefile -t http://bar:123/announce -t http://baz:321/announce --private
2). add torrent to your torrent client
3). edit the torrent by changing the source and adding a new tracker
torf -i somefile.torrent -t http://test:123/announce -s "test" -o new-somefile.torrent
4). add torrent to your torrent client
you will see that the edited torrent did not update the original torrent in your client and instead created a new torrent.
Torrent client tested with: transmission 2.94
Hi, I have a folder with 50 .torrent files. How can I run torf-cli to change the annouce and the source of these 50 .torrent files in batch mode? And, is there any way to run torf-cli in Windows terminal? Thanks in advance.
PS: I installed pipx in my window and then I installed torf. But I got this error below when run torf. Any idea how to solve this problem?
C:\>torf -i Mirc.torrent Traceback (most recent call last): File "c:\python38\lib\runpy.py", line 193, in _run_module_as_main return _run_code(code, main_globals, None, File "c:\python38\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\torf.exe\__main__.py", line 7, in <module> File "C:\Users\xandynho\AppData\Roaming\Python\Python38\site-packages\torfcli\__init__.py", line 16, in run from . import _config, _errors, _main, _ui File "C:\Users\xandynho\AppData\Roaming\Python\Python38\site-packages\torfcli\_ui.py", line 24, in <module> from . import _term, _utils, _vars File "C:\Users\xandynho\AppData\Roaming\Python\Python38\site-packages\torfcli\_term.py", line 14, in <module> import termios ModuleNotFoundError: No module named 'termios'
Can you please ad support for creating hybrid torrents as a default option but, with the option to create a v1 one torrent if required.
hybrid torrent test at the bottom:
https://blog.libtorrent.org/2020/09/bittorrent-v2/
Using the following command
torf 'sample' \
-t http://nyaa.tracker.wf:7777/announce \
-t http://open.nyaatorrents.info:6544/announce \
-t http://anidex.moe:6969/announce \
-t http://tracker.anirena.com:80/announce \
-t udp://open.stealth.si:80/announce \
-t udp://tracker.opentrackr.org:1337/announce \
-t udp://tracker.uw0.xyz:6969 \
-t udp://tracker.coppersurfer.tk:6969/announce \
-t udp://tracker.openbittorrent.com:80/announce \
-P -o sample.torrent
shows the following error
Traceback (most recent call last):
File "/usr/local/bin/torf", line 8, in <module>
sys.exit(run())
File "/usr/local/lib/python3.6/dist-packages/torfcli/__init__.py", line 21, in run
run()
File "/usr/local/lib/python3.6/dist-packages/torfcli/_main.py", line 43, in run
_create_mode(cfg)
File "/usr/local/lib/python3.6/dist-packages/torfcli/_main.py", line 71, in _create_mode
created_by=None if cfg['nocreator'] else _config.DEFAULT_CREATOR
TypeError: __init__() got an unexpected keyword argument 'exclude'
I'm getting a test failure when building version 3.2.0:
============================= test session starts ==============================
platform linux -- Python 3.8.3, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
rootdir: /build/torf-cli/src/torf-cli-3.2.0
collected 189 items
tests/test_basics.py ... [ 1%]
tests/test_configfile.py ............. [ 8%]
tests/test_configformat.py ....... [ 12%]
tests/test_create.py ................................................. [ 38%]
tests/test_edit.py ................................. [ 55%]
tests/test_errors.py ..... [ 58%]
tests/test_info.py .................... [ 68%]
tests/test_json.py ........ [ 73%]
tests/test_metainfo.py .......F.. [ 78%]
tests/test_profiles.py ..... [ 80%]
tests/test_progress.py ........ [ 85%]
tests/test_stdin.py .... [ 87%]
tests/test_utils.py .. [ 88%]
tests/test_verify.py ...................... [100%]
=================================== FAILURES ===================================
______________________ test_metainfo_when_editing_torrent ______________________
capsys = <_pytest.capture.CaptureFixture object at 0x7fdea15c6fa0>
create_torrent = functools.partial(<function _create_torrent at 0x7fdea1875940>, local('/tmp/pytest-of-builduser/pytest-0/test_metainfo_when_editing_tor0'), local('/tmp/pytest-of-builduser/pytest-0/test_metainfo_when_editing_tor0/My Torrent'))
def test_metainfo_when_editing_torrent(capsys, create_torrent):
with create_torrent(trackers=['http://foo', 'http://bar']) as orig_torrent:
run(['-i', str(orig_torrent),
'--comment', 'This comment was not here before.',
'--date', '1999-07-23 14:00',
'--nowebseed', '--webseed', 'https://new.webseeds',
'-o', 'new.torrent', '--metainfo', '-v'])
cap = capsys.readouterr()
assert cap.err == ''
j = json.loads(cap.out)
assert 'info' in j
assert 'name' in j['info']
assert 'pieces' not in j['info']
assert j['comment'] == 'This comment was not here before.'
> assert j['creation date'] == 932731200
E assert 932749200 == 932731200
/build/torf-cli/src/torf-cli-3.2.0/tests/test_metainfo.py:135: AssertionError
=========================== short test summary info ============================
FAILED tests/test_metainfo.py::test_metainfo_when_editing_torrent - assert 93...
======================== 1 failed, 188 passed in 1.80s =========================
System information:
I've just installed torf on Xubuntu 18.04 with Python 3.65:
paulo:~/src$ pip3 install --user git+https://github.com/rndusr/torf-cli.git
Collecting git+https://github.com/rndusr/torf-cli.git
Cloning https://github.com/rndusr/torf-cli.git to /tmp/pip-kssluewc-build
Collecting pyxdg (from torf-cli==1.2)
Using cached https://files.pythonhosted.org/packages/39/03/12eb9062f43adb94e30f366743cb5c83fd15fef026500cd4de42c7c12280/pyxdg-0.26-py2.py3-none-any.whl
Collecting torf>=1.3 (from torf-cli==1.2)
Using cached https://files.pythonhosted.org/packages/dc/9b/8b82bb81dc06753921c02205c3eadf600b1ceec002761baab6188f115675/torf-1.3.tar.gz
Collecting bencoder.pyx<2.0.0,>=1.1.1 (from torf>=1.3->torf-cli==1.2)
Using cached https://files.pythonhosted.org/packages/90/96/aed139c3b893ce09e8d1033303e5451c2799a9741d29028d3c80c5c66853/bencoder.pyx-1.2.1-cp36-cp36m-manylinux1_x86_64.whl
Building wheels for collected packages: torf
Running setup.py bdist_wheel for torf ... done
Stored in directory: /home/paulo/.cache/pip/wheels/55/19/37/eeaf4874c7718d0d3b29c85d980b946d5d29706863853e2400
Successfully built torf
Installing collected packages: pyxdg, bencoder.pyx, torf, torf-cli
Running setup.py install for torf-cli ... done
Successfully installed bencoder.pyx-1.2.1 pyxdg-0.26 torf-1.3 torf-cli-1.2
and if I run torf with any option/arg, I always get the same error, for instance, running torf -h
:
paulo:~/src$ torf -h
Traceback (most recent call last):
File "/home/paulo/.local/bin/torf", line 11, in <module>
load_entry_point('torf-cli==1.2', 'console_scripts', 'torf')()
File "/home/paulo/.local/lib/python3.6/site-packages/torfcli/__init__.py", line 13, in run
from ._cli import run, CLIError
File "/home/paulo/.local/lib/python3.6/site-packages/torfcli/_cli.py", line 39, in <module>
_DEFAULT_PROFILE_FILE = os.path.join(BaseDirectory.load_first_config(_vars.__appname__), 'config')
File "/usr/lib/python3.6/posixpath.py", line 80, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
I'm getting the following result when running the tests on torf-cli 3.1.0:
=================================== FAILURES ===================================
______________________________ test_creation_date ______________________________
capsys = <_pytest.capture.CaptureFixture object at 0x7f2545432100>
create_torrent = functools.partial(<function _create_torrent at 0x7f25456668b0>, local('/tmp/pytest-of-builduser/pytest-0/test_creation_date0'), local('/tmp/pytest-of-builduser/pytest-0/test_creation_date0/My Torrent'))
human_readable = functools.partial(<function human_readable.<locals>._human_readable at 0x7f254531d790>, <_pytest.monkeypatch.MonkeyPatch object at 0x7f25453bb250>)
clear_ansi = <function clear_ansi.<locals>._clear_ansi at 0x7f254533aee0>
regex = <class 'conftest.regex.<locals>._regex'>
def test_creation_date(capsys, create_torrent, human_readable, clear_ansi, regex):
date = datetime(2000, 5, 10, 0, 30, 45)
with create_torrent(creation_date=date) as torrent_file:
with human_readable(True):
run(['-i', torrent_file])
cap = capsys.readouterr()
assert clear_ansi(cap.out) == regex(r'^\s*Created 2000-05-10 00:30:45$', flags=re.MULTILINE)
with human_readable(False):
run(['-i', torrent_file])
cap = capsys.readouterr()
> assert cap.out == regex(r'^Created\t957911445$', flags=re.MULTILINE)
E AssertionError: assert 'Name\tMy Torrent\nInfo Hash\t5351b9e1a1da3e752a39ff22af929a7c47a85d90\nSize\t27\nComment\tOriginal Comment\nCreated\t...51b9e1a1da3e752a39ff22af929a7c47a85d90&dn=My+Torrent&xl=27&tr=http%3A%2F%2Fsome.tracker&ws=http%3A%2F%2Fsome.webseed\n' == ^Created\t957911445$
E + where 'Name\tMy Torrent\nInfo Hash\t5351b9e1a1da3e752a39ff22af929a7c47a85d90\nSize\t27\nComment\tOriginal Comment\nCreated\t...51b9e1a1da3e752a39ff22af929a7c47a85d90&dn=My+Torrent&xl=27&tr=http%3A%2F%2Fsome.tracker&ws=http%3A%2F%2Fsome.webseed\n' = CaptureResult(out='Name\tMy Torrent\nInfo Hash\t5351b9e1a1da3e752a39ff22af929a7c47a85d90\nSize\t27\nComment\tOriginal ...a3e752a39ff22af929a7c47a85d90&dn=My+Torrent&xl=27&tr=http%3A%2F%2Fsome.tracker&ws=http%3A%2F%2Fsome.webseed\n', err='').out
E + and ^Created\t957911445$ = <class 'conftest.regex.<locals>._regex'>('^Created\\t957911445$', flags=re.MULTILINE)
E + where re.MULTILINE = re.MULTILINE
/build/torf-cli/src/torf-cli-3.1.0/tests/test_info.py:147: AssertionError
=========================== short test summary info ============================
FAILED tests/test_info.py::test_creation_date - AssertionError: assert 'Name\...
======================== 1 failed, 173 passed in 1.61s =========================
System Information:
When creating a magnet link (torf -m
), it would be more useful to output only the magnet link if stdout is piped, to avoid unnecessary parsing, so that this (on bash):
magnet=$(torf -m foo)
wouldn't need any extra parsing. Of course, it's trivial to parse the output, but it feels more natural/intuitive if torf -m
piped only the magnet link.
I've just created a biggsh torrent with many files and noticed that the progress/ETA status line keeps moving across the screen and ends up garbled at the far right. For instance, here's what ended up on the screen after creating this big torrent:
When piping the output, the progress/ETA status line also moves across the screen, which results in garbled output when piped into a text file.
Running torf 6537c03 on Xubuntu 18.04.
It might be useful to be able to expand environment values in the config file. For instance, I set the comment for sef-created torrents in a bash script:
COMMENT=$USER@$HOSTNAME
It would be nice to be able to set it in torf's config file, like:
comment = $USER@$HOSTNAME
It would be useful to generate a magnet link when creating a new torrent, I guess by default, or perhaps actioned by a command line option.
Here's the situation: I want to send someone a file, so I create a torrent, start seeding it, and would like to send the corresponding magnet link to this someone.
Is this possible with CLI? Only seeing example to make torrent/magnet from existing files.
When adding trackers to a .torrent via -t t1,t2,t3, it adds them all under one block.
When adding trackers to a .torrent via -t t1 -t t2 -t t3, it does the same.
Adding via separate -t parameters, it should add that -t set + and CSV into one block, and the next -t into a new block.
some torrent clients only use the first tracker in a block.
separating each tracker into separate blocks uses all trackers at once.
I know it may be possible to script it in, but if there is a flag that can get it automatically and also use the tracker defined via -t
flag.
Like:
torf 'folder name' \
--tracker-url https://ngosang.github.io/trackerslist/trackers_best.txt \
-t udp://tracker.internetwarriors.net:1337/announce \
-t udp://tracker.opentrackr.org:1337/announce \
-t udp://tracker.torrent.eu.org:451/announce \
-o "test.torrent"
Hi @rndusr, is it possible include the options:
--nopublisher
--nopublisher-url
Some trackers include these informations, but is not necessary.
i have a torrent that i wanted to reuse but change source and trackers but am getting the following error.
torf: Invalid metainfo: ['info']['piece length'] is invalid: 827392
torrent seems to add fine and download fine in qbittorrent.
question: any way to reuse a torrent that gives this error ['info']['piece length'] is invalid
For example,
If the input is: torf -z public demo-folder
then the resulting .torrent file should be named something like [public] demo-folder.torrent
or PUBLIC.demo-folder.torrent
and if the input is: torf -z private demo-folder
then the resulting .torrent file should be named something like [private] demo-folder.torrent
or PRIVATE.demo-folder.torrent
Basically appending the profile name at the start of the torrent file to prevent overwriting and help recognizing what .torrent
file is made with what profile when making multiple torrents using multiple profiles for the same folder/file.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.