endkb / soundsdownloadscript Goto Github PK

Download programs from BBC Sounds and publish them to a podcast feed

PowerShell 100.00%

bbc ffmpeg kid3-cli podcast powershell rclone rss yt-dlp openvpn task-scheduler ffprobe xmlwriter bbc-sounds json

soundsdownloadscript's Introduction

ABOUT

SoundsDownloadScript.ps1 and genRSS.ps1 are Powershell scripts that can work together to download episodes from the BBC Sounds website and then publish them to a podcast feed. SoundsDownloadScript.ps1 can work without genRSS.ps1 if you just want to download the audio files, but genRSS.ps1 won't really work with audio files tagged with other tools because they won't be tagged properly to build a podcast feed.

GETTING STARTED

Package: (Latest release)

genRSS.ps1
ProfileTemplate (for use with genRSS.ps1)
README.htm (includes installation instructions and how to run)
SoundsDownloadScript.ps1

Prerequisites:

ffmpeg (Make sure the package you choose comes with ffprobe.)
kid3
Powershell (The scripts were developed and tested on v7.0.3 for Windows.)
yt-dlp

Optional:

OpenVPN Community (If you want to download higher quality audio from outside the UK. You must have a VPN provider with UK servers.)
rclone (If you want to upload files somewhere like S3 buckets, FTP, or archive.org.)

I believe there are Linux versions for all of these packages, but I've only ever used this on Windows. The script may work on Powershell for Linux, but it will likely take a lot of tweaking. There's probably another language that's more appropriate. If you're up for the challenge, feel free to use the logic as a guide and go for it!

INSTALLATION & HOW TO RUN

Step by step instructions and other notes are in the README.htm file in the package. If anyone would like to translate the html instructons to markdown for this document, be my guest!

SUPPORT

To report an bug or request a feature in either SoundsDownloadService.ps1 or genRSS.ps1, open an issue on github.

LICENSE

soundsdownloadscript's People

Contributors

Watchers

soundsdownloadscript's Issues

SoundsDownloadScript not pulling the correct release date sometimes

SDS is showing these two episodes were both released on 5-jul-23:
https://www.bbc.co.uk/programmes/m001m5p5
https://www.bbc.co.uk/programmes/m001mcfy

One was released 24-may and the other was released 31-may.

Check to make sure SDS is pulling the right date field.

ConvertFrom-Json errors out on this page: https://www.bbc.co.uk/sounds/play/m001z3hv

Errors from the log:

PS>TerminatingError(ConvertFrom-Json): "Conversion from JSON failed with error: After parsing a value an unexpected character was encountered: F. Path 'modules.data[0].data[0].synopses.long', line 1, position 1314."

ConvertFrom-Json: C:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:271
Line |
 271 |      $jsonData = $jsonResult | ConvertFrom-Json
     |                                ~~~~~~~~~~~~~~~~
     | Conversion from JSON failed with error: After parsing a value an unexpected character was encountered: F. Path 'modules.data[0].data[0].synopses.long', line 1, position 1314.

�[91mConvertFrom-Json: �[0mC:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:271
�[96mLine |
�[96m 271 | �[0m     $jsonData = $jsonResult | �[96mConvertFrom-Json�[0m
�[96m     | �[91m                               ~~~~~~~~~~~~~~~~
�[91m�[96m     | �[91mConversion from JSON failed with error: After parsing a value an unexpected character was encountered:
�[96m     | �[91mF. Path 'modules.data[0].data[0].synopses.long', line 1, position 1314.
�[0m

InvalidOperation: C:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:274
Line |
 274 |      $TitleTable = $($jsonData.modules.data[0].data.titles)
     |                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot index into a null array.

�[91mInvalidOperation: �[0mC:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:274
�[96mLine |
�[96m 274 | �[0m     $TitleTable = $(�[96m$jsonData.modules.data[0].data.titles�[0m)
�[96m     | �[91m                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
�[91m�[96m     | �[91mCannot index into a null array.
�[0m

InvalidOperation: C:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:284
Line |
 284 |      $SynopsesTable = $($jsonData.modules.data[0].data.synopses)
     |                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot index into a null array.

�[91mInvalidOperation: �[0mC:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:284
�[96mLine |
�[96m 284 | �[0m     $SynopsesTable = $(�[96m$jsonData.modules.data[0].data.synopses�[0m)
�[96m     | �[91m                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
�[91m�[96m     | �[91mCannot index into a null array.
�[0m

InvalidOperation: C:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:298
Line |
 298 |      $Station = $($jsonData.modules.data[0].data.network.short_title)
     |                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot index into a null array.

�[91mInvalidOperation: �[0mC:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:298
�[96mLine |
�[96m 298 | �[0m     $Station = $(�[96m$jsonData.modules.data[0].data.network.short_title�[0m)
�[96m     | �[91m                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
�[91m�[96m     | �[91mCannot index into a null array.
�[0m

InvalidOperation: C:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:301
Line |
 301 |  … eDate = [datetime]$($jsonData.modules.data[0].data.availability.from)
     |                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot index into a null array.

�[91mInvalidOperation: �[0mC:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:301
�[96mLine |
�[96m 301 | �[0m … eDate = [datetime]$(�[96m$jsonData.modules.data[0].data.availability.from�[0m)
�[96m     | �[91m                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
�[91m�[96m     | �[91mCannot index into a null array.
�[0m

InvalidOperation: C:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:318
Line |
 318 |      $CoverResult = $($jsonData.modules.data[0].data.image_url).replac …
     |                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot index into a null array.

�[91mInvalidOperation: �[0mC:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:318
�[96mLine |
�[96m 318 | �[0m     $CoverResult = $(�[96m$jsonData.modules.data[0].data.image_url�[0m).replac …
�[96m     | �[91m                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
�[91m�[96m     | �[91mCannot index into a null array.
�[0m

InvalidOperation: C:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:318
Line |
 318 |      $CoverResult = $($jsonData.modules.data[0].data.image_url).replac …
     |                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | You cannot call a method on a null-valued expression.

�[91mInvalidOperation: �[0mC:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:318
�[96mLine |
�[96m 318 | �[0m     $CoverResult = $(�[96m$jsonData.modules.data[0].data.image_url�[0m).replac …
�[96m     | �[91m                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
�[91m�[96m     | �[91mYou cannot call a method on a null-valued expression.
�[0m

None of the metadata gets set correctly, of course.

Would like a better way to control which audio quality yt-dlp (via SoundDownloadScript) downloads.

Right now SoundDownloadScript tells yt-dlp to download the highest level quality that's available. If it's connected to the VPN, that's 320, if it's not connected to the VPN, it's 96.

I don't have an active need for these changes, but for completeness, I would like to be able to tell it to download 96 when still on VPN.

I don't know if it will be possible, but I'll mess around with yt-dlp.

Issues deleting the LockFile if an orphaned one was found earlier.

I noticed one time that the script had trouble deleting its LockFile:

Remove-Item: C:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:203
Line |
 203 |          Remove-Item -Path $Script:LockFile -Force
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot find path 'C:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\4848.lock' because it does not exist.

�[91mRemove-Item: �[0mC:\Program Files (x86)\VideoLAN\VLC\SoundsDownloadScript.ps1:203
�[96mLine |
�[96m 203 | �[0m         �[96mRemove-Item -Path $Script:LockFile -Force�[0m
�[96m     | �[91m         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
�[91m�[96m     | �[91mCannot find path 'C:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\4848.lock' because it
�[96m     | �[91mdoes not exist.
�[0m

4848 was not the script's PID. It was the orphaned LockFile that it deleted earlier:

**Released C:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\4848.lock (possibly orphaned)

I think the issue comes from the routine that checks for orphaned LockFiles using the same variable ($LockFile) as the one that creates and removes the current LockFile:

$GetLockFiles = Get-ChildItem -Path $LockFileDirectory -Filter "*.lock" -Force | Where-Object {$_.CreationTime -lt (Get-Date).AddSeconds($LockFileMaxDuration*-1)}
		# Delete each one
		ForEach ($LockFile in $GetLockFiles) {
			Remove-Item $LockFile.FullName
			Write-Host "**Released $LockFile (possibly orphaned)"
			}

Implement a true ini file for genRSS profiles?

Could implement a single config file with script variables at the top and profiles right in the same file.

Command would look like:
genRSS.ps1 -Config "E:\Folder\genRSS.conf" -Profile "PodcastName"

An ini file should also allow multiline values which could be useful in fields like the podcast description.

Need to figure out what file extension to use, if any:
.ini
.conf
.cfg
nothing?

Store variables in external file

Right now, "script-wide" variables are stored at the top of the .ps1. If the script version gets updated, the values need to be transcribed.

Ini file might be a good option, there's already a function in the script to do that. It might need to be moved to a different section.

I don't know how an ini file would work with the remote script blocks.

Use ConvertFrom-Json to parse the program metadata instead of ConvertFrom-StringData

SoundsDownloadScript.ps1 uses regex and ConvertFrom-StringData to parse the program metadata from the BBC Sounds page. This isn't always reliable when there are certain characters, especially quotes and commas.

The metadata is already in JSON format.

Recommend using regex to pull the whole JSON string, and then ConvertFrom-Json to parse the needed data.