Enhancements to the API about audioserve

[izderadicka]

Thanks for great ideas,
Some comments:

Aggregated Info:
I'd see this as par of folder endpoint. If we will store/cache audio metadata to db (right now thinking rather about embedded KV store] we can include some info from tags. As as we will have duration and bitrate of audiofiles (this is primary motivation for creating this store, as extracting metadata is costly). So just sum durations up and store there total folder/audiobook time is natural.

Group details:
I'd rather consider different approach:
for folder endpoint to have parameter group, if used it'll extend folder data with last position of group in that folder and if folder was finished for that group.
It will not provide as much info as you've described. But it'll be more in line with current design.

** Folder playback status **
I do agree that providing last position in folder via websocket is bit artificial. It might deserve separate API endpoint. It was part PoC design - all goes through WS.
But, on the other hand, in order to have actual position you have to update server in regular (10 secs interval] where are you now in playback. And for this indeed web socket is the best solution. We just tick very short messages (10-15 bytes) over websocket as playback progresses.
This will also limit your idea for history of positions - I think it just make sense to have last position per folder. Previous history will be almost always -10 secs from last.
But maybe I did not understand your intent.

[KodeStar]

Yeah, sorry, I might not have been clear, both the Aggregated info and Folder playback status were part of the folder endpoint. The API endpoints for start and stop that log to a history were more so when someone clicks play or pause, we hit the server, rather than every 10 seconds (you could still hit the server every 10 seconds if you really wanted) because the only current position that matters is when you hit pause? In my examples Kode and Bob are groups, because that's how i'd use them, I'd have different groups for my kids so that it tracked where they were.

The group details section would allow someone to put in their "Group" and then we could have a screen that showed books they were currently listening to. If they went to a book we could see if the folder hash was in the finished_books list and display it as finished, and we could store settings that were specific to the "group" (In reality the user), such as their preferred playback speed.

I'm trying to work within the limits a KV store sets, obviously a lot of this data world work better in a relational database, but it's also possible to work in a KV.

[KodeStar]

https://imgur.com/2qY4PFI heres what the history looks like on prologue - it means if you accidentally play something you can just easily go back to a previous time stamp.

[KodeStar]

I probably could do it all on the client side, but then people would have to sign up for a firebase account or something to host it locally which wouldn't be ideal

[izderadicka]

Ohh, I see - we have two notions of "last position"

1. Based on user action - he pushed pause/stop or sleep timer had fired (BTW sleep timer for web client would be another cool functionality]
2. Based on playback just going on - this was my initial design intention, assure we have last played position, considering unreliable network, we can get disconnected any time - mostly focused on mobile client (as I'm now using mobile client mostly]

These all are great ideas, but again warning - I think I'll not be able to implement in near future, so I'd recommend to stick to current API.

[KodeStar]

https://youtu.be/-fd32v6gBeY current state of it

[KodeStar]

Is there a way to get a download url that works for all files? using download/ works fine for an mp3, but doesn't work for an m4b, for example download/David%20Walliams/Code%20Name%20Bananas/Code%20Name%20Bananas.m4b$$000%20-%20Chapter%201$$0-41819$$.m4b however, if I strip off $$000%20-%20Chapter%201$$0-41819$$.m4b it does work (I think) so is there a way to determine what the file should be?

Here's the use case. I'm adding a download/cache button to the overview page, when that is pressed it downloads the file to cache storage, then when you bring the player up, if it exists in cache storage it plays it directly from there, otherwise it loads the file in the standard way http://192.168.0.20:3333/audio/Eoin%20Colfer/Artemis%20Fowl/AF03%20-%20The%20Eternity%20Code/The%20Eternity%20Code%2001.mp3?trans=0

[izderadicka]

@KodeStar
/download endpoint serves to download whole folder. m4b is something like "virtual" folder, so it should be referred as path to file without .m4b - download/David%20Walliams/Code%20Name%20Bananas/Code%20Name%20Bananas should work then. Similar exception is for other chapterised files - but they are not so often as .m4b.

[izderadicka]

@KodeStar
actually it is bit more complex for chapterized file like .m4b - if there are many of them in one directory, then each is virtual directory and download link will be download/file.m4b and it will just download this file as it is ( no compression, just this file).
But if .m4b is the only audiofile in directory, we flatten the hierarchy - so download link is download/directory and it contains zipped directory (so .m4b and description and image files).

[KodeStar]

I've put a post here https://www.reddit.com/r/audiobooks/comments/pmekaz/a_new_interface_for_audioserve_and_a_request_for/ and have asked if there are any Rust devs who can help or point me in the right direction to take the burden off you, so hopefully someone will step up, though it's not looking promising at the moment.

[izderadicka]

@KodeStar - I've started to work on some changes - see branch collection-db.
I refactor out the logic for keeping and querying collection information - now it is in crates/collection crate.
It's backed up by hi performance rust KV store Sled. Now it caches the info for folders, so loading huge folders - with hundreds of audiofiles is fast.
It also open a way to add easily tags info into API (however folder structure will always remain primary navigation).
Couple of new things added to API endpoint /folder - modified - last modified time as timestamp and total_time - total play time per folder.

Next couple of things are:

• watch collections directory via inotify and update the KV store if files/foldes change - watching of files is there, but update logic will be more complicated.
• rework search to use this KV store
• store here last position in folder per group, and finished mark per group - and probably add some end point around it.

[KodeStar]

That's awesome, look forward to :)

[KodeStar]

Meta information I'd like pulled out if possible

/folder

• Title
• Author
• Narrator (Usually stored in the Composer tag)
• Book series (Usually stored in the Album tag)
• Genre (if it exists)
• Description (if no description file in the folder pull from the files)
• Cover (if there isn't a cover in the directory pull it out of the files)
• Release date

File level metadata

• duration (already available)
• bitrate (already available)
• Chapter number/name - This is already sort of there for m4b files / single files, though I don't know if it gets the actual chapter names (I believe there are 2 kinds of chapter information, Nero and Quicktime), but it would be useful to have a standardised location to get the chapter name or number (so metadata makes sense).

[izderadicka]

There are some challenges with metadata

• first m4bs or single file audiobooks are still handled as special case in audioserve. Majority of audiobooks are split to files by chapter/ fixed duration files. In this case question is - what are folder metadata - as each file has it's own - if meta tag is same for all files in folder - ok that's clear then. But what if it is different in each file?
• Extracting cover image (and also to some extent description) directly from tag would be challenging (apart of decision in case of multiple files in folder - where to take if from). Definitely not something to tackle in near future.
• For file meta: chapter name number depends again on structure of the folder - if it consists from multiple files, then it is title and track no. If it is m4b that it is chapters metadata as per MPEG4 standard - actually here we depend on libavformat - whatever it parses from the audio file container (and it's usually driven by standard - Nero/Quicktime are just encoding programs, they have to stick to standard like MPEG4).

Plus one more thing - folder structure still will be key for audioserve - metadata from tags could be used for additional information - for presentation only, but should not change any main logic. Do not plan to use them for "rich" relational model - to have list of all authors, narrators, series etc. as they are very unreliable. And server storage is just simple key value store, not relational DB.

[KodeStar]

I agree that folder structure should always be key, especially for main navigation, but if the data is there it's a shame we can't use it.

[KodeStar]

Is there any chance you could output an id that identifies a book? In the same place you currently have total_time, something like book_id: whatever

I'm currently doing hash = sha256(route), but it's really flaky and I seem to be ending up with multiple copies of a book (noteably in active books after a book has started playing).

It doesn't need to be a hash, but preferably an ID that will stay the same even if someone updates their folder structure (though if that isn't feasible at the moment then I'd be happy with anything)

[izderadicka]

All is in filesystem, so what you doing is more or less all one can do, server url and collection number should be included too to hash to distinguish between servers and collections.

On server side hash of all directory content, would take too much time to compute and would be unstable (for instance if you edit description, or edit tags in audiofile).

Maybe inode number, but it'll have limited compatibility and will need so consideration if it'll be useful.

If you are doing hash of folder path with collection number snd server url it has to be unique (if folder structure is not modified). Maybe duplication problem is somewhere else?

[KodeStar]

Fair enough. I think the duplication was because I was using the url rather than the path, so it was possible to end up at the same path but from a different url (though not sure how I specifically did it) for example ?collection=1&folder=path would have a different hash to ?folder=path&collection=1. I've now changed it to use the path instead, so hopefully that will fix that. I added collection to the book entry earlier so I'll look at including that in the hash, though I'm not sure it matters, if you have the same book in 2 different collections at the same path, sharing the listening position is probably what you want.

I do have a todo on the projects list to tie items to a server, though I'm not sure what's going to be best for this yet.

[izderadicka]

I added API enhancements to master and they are also available in pre-release image izderadicka/audioserve:unstable, just need now to document them. They include:

• total duration in folder
• last playback position in folder
• optional tags metadata in folder and audiofiles
• subfolder finished - meaning played to end is available
• REST API for playback position

[izderadicka]

Planned API enhancement are now in released version.
Also documented API with OAS3/Swagger