extract audio via url about just_waveform

[ahetawal-p]

One use case which we have is, we have multiple audio clips on a given page. And user can click and play any one of them. Would ideally not want to download the file for each clip and play as it adds latency on our UI plus maintaining the file system sizing if we always download a file for playing the waveform, while the actual player can just work from a url instead of downloading the file every time.
From flutter_sound or just_audio compatibility perspective this plugin supporting a url would be super helpful.

[ryanheise]

I do not see how that use case won't work, and how extracting audio via a URL will help your use case at all. Decoding directly from a URL doesn't mean you avoid downloading, because you must in all cases download something to read the data and decode it. All decoding directly from a URL really means is that you are downloading "simultaneously" while decoding. The downloading is still there. The only difference here is whether you do those two steps in sequence, or simultaneously. You can and should of course delete downloaded data once you've extracted whatever you needed to extract from it, and you should of course cache the extracted data since it was very expensive to extract in the first place.

But by far the biggest latency is the actual decoding, not the downloading, and so with the feature you're asking for, you still must wait for the decoding to finish before you can display the waveform, and the decoding takes at least 10x longer than the downloading. You are not going to help your latency problem this way. In all cases, you must download and decode.

If you want something that eliminates latency, then what you should be asking for is streaming the decoded waveform data while it's processing, rather than having to wait until the entire job is done.

[ryanheise]

@zzterrozz

would it be possible to extract the waveform as String of Json in order to store it and then use it to display the waveform instead of downloading + decoding the file?

It should be clear from the plugin's API because there are only two methods in the API. The first:

final progressStream = JustWaveform.extract(
  audioInFile: '/path/to/audio.mp3',
  waveOutFile: '/path/to/waveform.wave',
  zoom: const WaveformZoom.pixelsPerSecond(100),
);

The waveform is extracted from the given input file (e.g. MP3) and placed into the output file waveOutFile. In other words, the extracted waveform is "stored" in the given output file. You are basically asking if the waveOutFile parameter exists. Yes it exists. You are also asking me to use a JSON format instead, but the current format is much more efficient than the JSON format you are proposing because it's binary and compact. The format I'm currently using is called "audiowaveform" which is a binary format invented by the BBC for this purpose. I have mentioned this format in the plugin's documentation, please follow the link in the documentation to read more about this format.

Once you have outputted a waveform file in this format, you can read it into your program using the only other method in that class:

final waveform = await JustWaveform.parse(File('/path/to/waveform.wave');

So you use the first method only the first time when you haven't previously decoded the file. Once you've done that, however, you can keep the output file and next time just load this existing file using the second method.

[ryanheise]

@ahetawal-p I'm sorry if there was any confusion in my question, but it was this:

If anyone has a use case for which this won't work, please describe it below, otherwise I advise downloading the file first.

It sounds like you'd be satisfied with a convenience method so that you don't have to write an extra line of Dart code in your app. As such, it's not that the current plugin does not work, and the priority of this issue remains as before. For now, I advise adding 1 extra line of code to your app. I am not in a rush to add this to the API because API decisions should be carefully thought out so as to minimise breaking changes in the future.

[pasrot943]

I am looking for the url function as well ?

[ryanheise]

For now, you can download the URL to a file and then use the file with just_waveform.

[loic-hamdi]

For now, you can download the URL to a file and then use the file with just_waveform.

is this still the way to do today?

[ryanheise]

Yes it is.

[ryanheise]

Note you can also check the changelog to see if there have been updates.

[andynvt]

Any updates?

[ryanheise]

Just to update everyone, the current way is still do download the file first and then pass the file into the API.

There is no urgency to do it another way at the moment, and usually it is better to download the file first because if you lose the connection half way through the download, it won't cause problems for the decoder.

If anyone has a use case for which this won't work, please describe it below, otherwise I advise downloading the file first.

[Faaatman]

One use case which we have is, we have multiple audio clips on a given page. And user can click and play any one of them. Would ideally not want to download the file for each clip and play as it adds latency on our UI plus maintaining the file system sizing if we always download a file for playing the waveform, while the actual player can just work from a url instead of downloading the file every time. From flutter_sound or just_audio compatibility perspective this plugin supporting a url would be super helpful.

I second this and would love an integration at least with just_audio.

[loic-hamdi]

Hi @ryanheise,
would it be possible to extract the waveform as String of Json in order to store it and then use it to display the waveform instead of downloading + decoding the file?
It would allow to go through the download + decoding process only once.

[ahetawal-p]

@ryanheise

All decoding directly from a URL really means is that you are downloading "simultaneously" while decoding. The downloading is still there. The only difference here is whether you do those two steps in sequence, or simultaneously

Yes that's what I am looking for the simulatneous part. This no different than how a image widget works, as it downloads and then decodes each image when it's shown.

[ryanheise]

@ahetawal-p maybe you can make a separate feature request for that because they are distinct features that can be implemented separately.

[ryanheise]

@ahetawal-p actually on rereading your last comment, you are indeed requesting the feature described in this issue, so not a separate issue.

But my question remains as to why this actually solves your problem. You do realise that this will do almost nothing to address latency? Can you please provide me with a benchmark in your app measuring how much time is spent downloading the file vs how much time is spent extracting the waveform?

[loic-hamdi]

@zzterrozz

would it be possible to extract the waveform as String of Json in order to store it and then use it to display the waveform instead of downloading + decoding the file?

It should be clear from the plugin's API because there are only two methods in the API. The first:

final progressStream = JustWaveform.extract(
  audioInFile: '/path/to/audio.mp3',
  waveOutFile: '/path/to/waveform.wave',
  zoom: const WaveformZoom.pixelsPerSecond(100),
);

The waveform is extracted from the given input file (e.g. MP3) and placed into the output file waveOutFile. In other words, the extracted waveform is "stored" in the given output file. You are basically asking if the waveOutFile parameter exists. Yes it exists. You are also asking me to use a JSON format instead, but the current format is much more efficient than the JSON format you are proposing because it's binary and compact. The format I'm currently using is called "audiowaveform" which is a binary format invented by the BBC for this purpose. I have mentioned this format in the plugin's documentation, please follow the link in the documentation to read more about this format.

Once you have outputted a waveform file in this format, you can read it into your program using the only other method in that class:

final waveform = await JustWaveform.parse(File('/path/to/waveform.wave');

So you use the first method only the first time when you haven't previously decoded the file. Once you've done that, however, you can keep the output file and next time just load this existing file using the second method.

Nice, I didn't realize that!
Is there a async future to extract the waveform instead of the stream in order to await the extraction and then upload the file?

[ryanheise]

@zzterrozz I can't keep answering questions of the form: Does the API have X or Y? These types of questions can be found by studying the API. There are literally only two API methods, and I was trying to make it clear in my previous response that you really should have been able to fully see everything that is int his plugin by looking at the API reference, in almost no time. It takes me much longer to write these responses to you than it would take for you to actually read the documentation, and so it can't be that difficult to take a look and see what is there and what isn't there.

Since a stream does more than a future, it is not a concern of the plugin. You are advised to ask someone on StackOverflow if there is an easy way to get a future from a stream, or else, read the Dart documentation to find the answer yourself.

[ahetawal-p]

Hey @ryanheise I don't think its about latency as much as it's about the usage of this package. All other sound related packages support urls, so it was just less work as a consumer to not worry about downloading the file first and then passing it down to the plugin. Instead let the plugin do it's work with the given url.
I do understand the technical limitation with the decoding latency while streaming ...as you mentioned

[Faaatman]

@ahetawal-p
The workaround that I will be using is as follows: since I'm using just_audio I will get the remote files using LockCachingAudioSource and specify the cached file location, wait for the audio player to download the file, and then use just_waveform to get the waveform.

I will get back to you once I implement it and test it.

[jpolstre]

Is there a way to get the data from the service and not from the files. The data is already there it is the sound that is being emitted, why get it again.

I need it for live broadcasts.

[ryanheise]

@jpolstre the only way to prevent a double download is to tap into the decoder of whatever player you're using. just_waveform will never be able to do that since it is a standalone plugin, not integrated with any player. For your specific use case, the closest you can get currently would be to use the visualizer branch of just_audio. It is not the goal of just_waveform.

[jpolstre]

@jpolstre the only way to prevent a double download is to tap into the decoder of whatever player you're using. just_waveform will never be able to do that since it is a standalone plugin, not integrated with any player. For your specific use case, the closest you can get currently would be to use the visualizer branch of just_audio. It is not the goal of just_waveform.

Thank you for your reply, do you plan to implement it also for just_audio_background?

[ryanheise]

Eventually, but not in any reasonable timespan since the visualizer branch is experimental. If you need it urgently, you would need to use audio_service directly. In any case, this is not an issue that pertains to just_waveform.