rayondemiel / iiif_collector Goto Github PK

changer prompt : https://rich.readthedocs.io/en/stable/prompt.html (pour faire des choix)

progress bar : https://rich.readthedocs.io/en/stable/progress.html

Utilize concurrent processing: Use concurrency libraries like concurrent.futures or asyncio to parallelize the requests and processing. This can significantly speed up the execution time, especially if there are many URLs to process.

add test app with github action

Work pathlib option to execute it in windows/linux

And add execute file for windows

exemple : # test2 https://api.digitale-sammlungen.de/iiif/presentation/v2/bsb10402127/manifest -> voir pour telecharger sortie ocr (seealso)

retourne lien html (https://github.com/kba/hocr-spec)

number is an option, if NONE -> get all images

cli to get all (or determinate) link image API in manifest IIIF

Use a connection pool for requests: Rather than creating a new requests.Session object for each request, utilize a connection pool by creating a session outside the loop and reusing it. This helps improve performance by reusing connections and reducing overhead.

Use generator functions: Instead of creating a large list to hold all the URLs, consider using generator functions to yield URLs one at a time. This avoids loading all URLs into memory at once, especially if there is a large number of URLs.

get_next -> yield