Corrupted about weibopicdownloader

[nondanee]

You can delete those broken files manually and rerun the tool.
Don't worry for redoing 802 times downloading. By default, tool will skip existing files.

Failed-and-remove function has already been added to the tool
commit f186381e07e216cb02c407d80a435b287f966554 ------ fix bugs: check before remove #22
weiboPicDownloader.py #L204

If you don't do keyboardinterrupt (Ctrl + C) before #21, this error will not occur.
May be the resource itself is broken 😓

[wankio]

ty
so we dont have archive option to store downloaded urls ?

[nondanee]

yes.
but easy for you to do it yourself,
just add output function after weiboPicDownloader.py#L255 like

with open(your_path, "w") as f:
    f.write('\n'.join(urls))

[wankio]

thank

1. some weibo user have 1k+ pics, is there have anyway to detect corrupt file fastest and automatic ?
2. so, i just rerun the tool after delete corrupt files, files still corrupted. I think it would great if we have ability to export only jpg,gif,video directlink into output txt file, so i can use wget or other tool to download it and prevent corrupted files

[nondanee]

for detection, refer to this How to check if a file is a valid image file? on stackoverflow

[wankio]

Thank

so it can't export urls ? because when i'm trying to open corrupted picture url, it load very slow and timeout when loading, but if i run it again in 4 or 6 hours after that, it will load pretty fast.

I think it should limit simultaneous download at 1 or 2 to prevent overload, it will slower but prevent corrupted picture

[nondanee]

yes, do it youself, just two line

i concern if i add the export option, someone will complain, why it not support to resume from break point, it has already generated a list, then i have to pay more to maintain a download queue ...

option -s for concurrency limitation, just set -s 2 as you want

[wankio]

do we have option to download from specific date ?
eg : 01/Jan/2018 to 01/Feb/2018

thank

[nondanee]

#25