niedakh / pqdm

Comfortable parallel TQDM using concurrent.futures

Parallel TQDM

PQDM is a TQDM and concurrent futures wrapper to allow enjoyable paralellization of iterating through an Iterable with a progress bar.

• Free software: MIT license
• Documentation: https://pqdm.readthedocs.io.

Install & Use

To install :

pip install pqdm

and use :

from pqdm.processes import pqdm
# If you want threads instead:
# from pqdm.threads import pqdm

args = [1, 2, 3, 4, 5]
# args = range(1,6) would also work

def square(a):
    return a*a

result = pqdm(args, square, n_jobs=2)

For more examples variants check the Usage section of the docs.

Features

• parellize your tqdm runs using processes or threads thanks to concurrent.futures,
• just import pqdm from pqdm.threads or pqdm.processes to start,
• automatic usage of tqdm.notebook when iPython/Jupyter notebook environment detected, custom tqdm class accepted
• automatic parsing of pqdm kwargs and separating between concurrent.Executor args and tqdm args,
• support for any iterable and passing items as kwargs, args or directly to function which is being applied
• support bounded exectutors via https://github.com/mowshon/bounded_pool_executor

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.