[Feature Request] Validate file size before writing to archive about podcast-dl

[lightpohl]

Hey @wangxu94!

That sounds a bit annoying. Definitely will try to add a better check/cleanup of those failed downloads. A couple questions that will help me track it down:

1. Would you mind outputting the version number? Using --version should return it.
2. Does podcast-dl log a warning and/or delete the zero-byte file when this happens?
3. If able, mind sharing one of the podcast feeds where you've seen this happen? Does it seem random?

[wangxu94]

Thanks for replying!

1. This happened on 4.1.1. I have just upgraded to 4.1.2 this morning so can't say for sure whether it happens on that version.
2. It does not delete the zero-byte file. The meta.json file is written correctly. Unfortunately I don't know whether it logs a warning because I was running it on a cron job piped to /dev/null. I will write the output to a logfile now and get back to you if I see this happen again.
3. I've seen this behaviour on multiple feeds, including a private patreon feed and a public podbean feed.

[lightpohl]

Thanks a bunch for the info!

The script currently does a very simple check using fs.statSync to determine the size and log an error if it's empty. My current guess is that it's behaving differently when running as a background task, but it's definitely something I would like to support.

Let me do some testing and see what I can find. 🧪

[wangxu94]

Ah yes, I see now that you do have a file size check. Thanks for looking into this, I'll report back with logs if I notice this behaviour again.

[lightpohl]

Oh! Sorry, one more quick question: are you running it with the built binary or Node? If you're using Node 12 (either the binary or locally), I could package the script with Node 14. If it doesn't appear again it might be a bug in the OS-level hooks in Node 12.

Just another avenue to check! Let me know if you're interested.

[wangxu94]

Hey sorry for the late reply! I am running the built binary.

EDIT: Also, looking more closely at the archive, it seems that when a download fails resulting in a zero-byte file, the meta.json file (which is written correctly) is written to the archive, but not the mp3 file. However this apparently still results in the file being skipped on subsequent runs.

[lightpohl]

Ah! That's really interesting. It makes me think that the permissions around how the script is being run are preventing the empty files from being deleted on your system by the script, but it's good that they're not being written out to the archive when it's empty.

At the moment I don't think it is an issue with the archive check. The script by default will skip any local collisions, so if the next run of podcast-dl encounters one of the zero-byte files that haven't been cleaned up, it'll skip that download just because it sees it locally.

Since you're using --archive, enabling the --override flag as well might be a good solution if you're not able to figure out the permissions issue. The script will continue to skip items found in the archive but will write over those zero-byte files.

[wangxu94]

I see. The folders are all owned by the user running the script, so I'm not sure why there would be a permissions issue. Thanks for suggesting the --override flag though — that should solve the problem for me.

Thanks for your help, and feel free to close this issue if there is nothing else on your end :)

[lightpohl]

Cool! Glad that works out for now. Definitely let me know if you run into anything in the logs that doesn't look right.

Always happy to see someone using the script. 🚀