Playing a remote file and cache the file at the same time about just_audio

[ryanheise]

This update is now published in release 0.6.13 (still marked as "experimental" but at least now it is more usable.)

And in other news, just_audio was just yesterday accepted as a Flutter Favorite! It's been a long process from start to end going through the various stages, but thanks @afkcodes and anyone else who expressed your support for the project as I'm certain it helped to make a difference.

[ryanheise]

Just FYI, how to try out the above caching implementation:

player.setAudioSource(LockCachingAudioSource(
  uri: Uri.parse('https://............mp3'),
));

In addition to uri, there are parameters for headers and cacheFile so you can specify your own location for the cache file. By default, the filename is a hash of the URL stored in just_audio's internal cache directory. Note that the implementation will append .part to the cache file name while downloading and then rename it to the originally requested name after the download completes.

The API is annotated as @experimental so if I go ahead and release this, it will still be possible for me to change/fix the API without a major version bump. Once the API stabilises and I'm happy with it, I'll remove the @experimental annotation.

Of course before I do any release it would be good to get some feedback on both the API and whether functionally it works.

[esiqveland]

I have noticed some issues with LockCachingAudioSource on iOS. After having the app in the background for a few minutes, opening the app and setting a new AudioSource as a LockCachingAudioSource will often fail to load. After it has failed, the player remains unresponsive, failing any new LockCachingAudioSource. I have not been able to find the cause of this. Have anyone else seen this on ios?

[afkcodes]

@ryanheise You are awesome man. are you on steroids ? Just kidding Kudos Brother. This package is certainly going to be flutter favorite and help lot of other trying to build audio based app.

[nuc134r]

Maybe using https://pub.dev/packages/flutter_cache_manager

@zxl777 That would make you have to either wait for file to download fully before playing or download file twice. It is discussed above.

Was thinking a lot about extending existing proxy to allow realtime caching (and even tried) and then stumbled upon #172. Considering that I want to provide data from disk on repeating requests, get notified of caching progress and have an index of cached data I figured that it would be better to set up my own proxy between API and just_audio.

Also I see it as a better approach in terms of plugin ecosystem architecture. I think I'm going to try to implement it and if I reach any success I'll update this issue.

I want to report that I got it working and currently finishing features described above to cover my own needs. When it's ready I'm going publish a package and link it here. Not sure if Ryan would like to integrate it into just_audio but will be easy to set it up yourself anyway:

final url = await _proxy.addUrl(track.url, track.key, meta: track);
_audioSource.add(ProgressiveAudioSource(url, tag: track));

And there are still performance issues in some cases that need to be addressed.

[ryanheise]

I am not sure since the downloading actually happens on the platform side of the plugin.

However, it just occurred to me that after recently implementing an HTTP proxy in the plugin, this may actually help, since now the Flutter (Dart) side of the plugin can intercept the bytes being downloaded from the server and I could potentially implement caching in the proxy.

[afkcodes]

I am surely going to do that, also i appreciate how you think about other plugins. I do understand that it should not be the case that only plugin being used more than the others, but the contradiction is also there that not all the plugins are well maintained as just_audio. In my personal opinion why to have 10 plugins with less features rather than having 2 plugins will more features. Also i think that those amazing devs instead of creating standalone plugins should contribute to one plugin and make them them more feature rich and robust. It's just an opinion, the flutter community is awesome and thats what has made flutter so popular.
Also @ryanheise i would love to see casting support in just_audio if its possible.

[ryanheise]

I am surely going to do that, also i appreciate how you think about other plugins. I do understand that it should not be the case that only plugin being used more than the others, but the contradiction is also there that not all the plugins are well maintained as just_audio. In my personal opinion why to have 10 plugins with less features rather than having 2 plugins will more features.

Open source is wonderfully counterintuitive :-)

Also @ryanheise i would love to see casting support in just_audio if its possible.

It's definitely something I am planning to do! If you're interested, you can track #211 for progress.

Anyway, my apologies for derailing the discussion. Let's keep further comments in this thread on the caching feature, and in particular I would greatly appreciate if people can try it out and let me know how it goes. Thanks everyone!

[BruceZhang1993]

This feature seems great. The flutter_cache_manager plugin's getFile method will return a Stream of FileInfo. I wonder whether it's possible to achieve this feature.

[ryanheise]

@nuc134r hehe :-)

Unfortunately, it may be a while for me to get around to this, since there are some higher priority things that need to be implemented in audio_service first.

(That is an open invitation for anyone who wishes to submit a PR! It seems reasonable to add an option to the relevant AudioSource subclasses to cache the downloaded audio to a given file.)

[ryanheise]

Hi @ashishfeels yes it has both custom headers and caching support, while iOS doesn't. Because I needed this to work on iOS, I ended up writing a proxy that lets me specify headers, and it would also allow me to do my own caching, and this is done in a way that would work consistently on both iOS and Android. If you take a look at the Dart code, you'll see that it is not actually too difficult to implement this solution, so either way, it is a matter of getting around to it.

[ryanheise]

@ashishfeels , The caching side isn't implemented yet, but the proxy is, so it would be a short step from there to add a cache.

(Note that since iOS doesn't have these features, and "implementing your own proxy" is the usual approach iOS developers have to resort to to have them, so that becomes our lowest common denominator.)

[nuc134r]

@ashishfeels Ryan means that we need to implement caching using the proxy instead of ExoPlayer's mechanisms because on iOS (and web) there is no ExoPlayer. And that the course of this ship is to have features supported by all platforms so we better find solutions in common code which is Dart. And it already has a proxy implemented.

[ryanheise]

@nuc134r excellent work! As mentioned in my previous comment, I very much would like to support this feature directly in just_audio. The proxy exists within just_audio, and I can think of 3 features that this would enable directly within the plugin:

1. Caching audio while downloading/playing.
2. Serving audio directly from the asset bundle rather than writing the asset to file first (the current approach)
3. Playing audio directly from a supplied stream of bytes (e.g. may be used for custom DRM solutions.)

[ryanheise]

Thanks @defsub , that is very informative.

Out of curiosity, how would you prefer to handle the situation when the user seeks to somewhere near the end of the file while the file is still downloading? Typically the player will stop downloading data at the current point and then make an HTTP range request for data starting at the new seek point.

I think ideally the cache store should have a data structure that consists of a list of downloaded ranges, and these ranges can be disjoint, but adjacent ranges can be merged. It could be useful to have an option to specify whether you want the whole file downloaded even if the user seeks in the middle of downloading.

Perhaps the way to go could be to create some sample code and publish that in an FAQ for how to subclass StreamAudioSource to perform some sort of caching, and this will give me some time to see what different use cases different people have. If things stabilise, a default caching implementation could be added into just_audio.

[ryanheise]

Looking at the Nginx documentation, it seems to offer two different approaches for byte-range caching:

https://www.nginx.com/blog/smart-efficient-byte-range-caching-nginx/

1. Cache lock: Downloads the entire file on the initial request. During the download, any subsequent byte-range requests are forwarded to the original server. After the download, byte-range requests are serviced from the cache.
2. Cache slicing: Divides the file into segments and downloads+caches them on demand to service byte-range requests (which can span multiple segments).

That's a lot better than what I was trying to come up with, and both seem straightforward to implement. A variation on 2 would be to have an option to automatically download all segments but to bump the priority of a segment the user has just done a seek to.

@nuc134r using an isolate sounds like a great idea. The whole proxy could be made to run in a normal isolate if it restricts itself to dart:io. As for whether the new StreamAudioSource makes things easier, let's see how that goes because there's a good chance that through this I'll discover some further API changes that might be appropriate. Currently the read() method only concerns itself with the bytes, but I did contemplate passing the headers along. I just don't have a use case for that yet. Another thing is that the proxy is currently hard coded to always advertise its support for byte range requests. Though, I don't think that's a bad thing since all modern web services should support these. What happens is that the range parameters of read() will be null if the client makes an ordinary request (for the whole file).

[gtu-myowin]

The main purpose of this thread is to reduce data consumption and/or server load. So, the best workaround solution is

1. do the play and download simultaneously
2. once the download is finished, get the position of streaming player and stop the player
3. play again from downloaded file and seek to the streamed position

If you would need the code, here it is

import 'package:dio/dio.dart';
import 'package:just_audio/just_audio.dart';

final Dio dio = Dio();
final _player = AudioPlayer();
final String url = 'your audio url';

try {
  // Play live while downloading
  _player.setUrl(url);
  _player.play();

  var dir = await getApplicationDocumentsDirectory();
  var fullPath = dir.path + '/Audios/audio.mp3';

  // Downloading file
  await dio.download(url, fullPath);

  // Download is finished at here
  // So, get the streamed position
  // Set it to zero since it can't be null when seeking
  Duration streamedPosition = Duration.zero;

  // Reduce 1 second to make sure user won't miss any part of the audio
  // and thus he/she won't notice the file change
  if (_player.position > Duration(seconds: 1)) {
    streamedPosition = _player.position - Duration(seconds: 1);
  }

  // stop the live player
  _player.stop();

  // Switch the file with locally downloaded file
  _player.setFilePath(fullPath);

  // Play it
  _player.play();

  // and seek it to the streamed position
  _player.seek(streamedPosition);

} catch (e) {
  print(e);
}

Hope it helps!

[ryanheise]

The source code for LockCachingAudioSource is not too mysterious, hopefully, so feel free to improve its behaviour as desired. One of the planned improvements for this caching strategy was to open a new HTTP request on a seek, while the complete download of the entire file continues to happen asynchronously. That way, a seek request can be serviced immediately even before the complete download has completed.

If you happen to implement this, you are welcome to make a pull request. If nobody else puts up their hand, I'll add this when I get a chance.

[ryanheise]

The latest commit on master now supports seeking during the download. If you seek to a region that has already been downloaded, it will be fulfilled from the cache. If you seek to a region that hasn't yet been downloaded, a separate HTTP request will be made while the whole-file caching continues to happen in parallel.

I would be interested to hear how it goes, and whether you run into any problems.

[suragch]

Congratulations on the Flutter Favorite!

I'm still planning on testing out the caching but haven't gotten around to it.

[afkcodes]

Awesome @ryanheise, this was eventually going to happen, i knew that. Also if we can update the example app also for a caching example if not done.

[ryanheise]

Also if we can update the example app also for a caching example if not done.

That's a good idea, although I'd like to have this working for the web first so that the demo works on all platforms (which is waiting on a way to implement service workers in Flutter.)

But I think a good compromise could be to add a caching example to the README.

[ryanheise]

Those can't all be easily avoided since if you write:

if (!await file.exists()) {
   ...
}

By the time it gets into the body of the IF statement other code could have executed asynchronously so we couldn't guarantee that the file doesn't exist anymore. That's not to say we couldn't work around that, but the code would be more complicated.

So before doing that, I'd probably want to know that existsSync causes a problem.

[ryanheise]

I have a new branch with some bug fixes for LockCachingAudioSource for those who would like to test it out before the release (I wanted to share it here to a wider audience in case someone is able to find any serious problems with it before I release it):

fix/not_range_request

Previously it would always try to fulfill range requests even when the origin server doesn't. Now I've modified the API a little to allow subclasses of StreamAudioSource the option to switch off support for range requests. For context the bug fix is for issue #570 .

[ryanheise]

I'm not sure of any easy way to do this. Some podcast players make it a binary choice: You either stream it, or you download it in advance.

[lkho]

The flutter_cache_manager plugin's getFile method will return a Stream of FileInfo

The FileInfo stream only provides the progress of the download (while the file is still downloading), but not a data stream. You need to wait for the whole file to be downloaded to get a real file path.

BTW, if you really want to play a URL while it is still being downloaded, there is a CacheDataSourceFactory in ExoPlayer, which might serve this purpose. However, this is platform side implementation and not controlled by Flutter. If it was to be implemented in this library, it would be a huge feature and requires a lot of platform communications.

Another suggestion regarding ExoPlayer, is to implement a custom DataSource, which can then be intercepted by Flutter, and we can send binary buffers via platform channels to populate the DataSource. In this way you can handle all caching/downloading on Flutter's side and do whatever you want to fetch the buffers. But this sounds a bit over complicated, and also requires a huge API change.

[ryanheise]

I would eventually like to replace the current setUrl/setXYZ API by a setDataSource API which would make it a lot easier to integrate with custom and other ExoPlayer data sources. There are various other motivations for wanting to do this including gapless playback. So I would say this idea is certainly on the table, but not in the short term.

[ryanheise]

I'll leave this link here for later as a guide for how this could be done on the iOS side:

https://vombat.tumblr.com/post/86294492874/caching-audio-streamed-using-avplayer

[sachaarbonel]

I was reading the rxdart doc when I came across this feature that could become handy for this feature request

[nuc134r]

As audio_service and just_audio gets more and more functional I gradually let go any thoughts of forking and implementing needed features myself as they eventually appear implemented by Ryan.

So one really important feature I plan for my app is realtime caching. Since http proxy appeared, could you please expose some interception API so that data can be written to disk while being downloaded?

Or since you're the mighty Ryan Heise, maybe you could make an API similar to album art cache, using flutter_cache_manager? I would be very happy.

Thank you for your work, you're the MVP.

[afkcodes]

@ryanheise exoplayer has some option to cache i guess its SimpleCache. will look into this.

[afkcodes]

@ryanheise sorry i was not using the plugin so didn't get the chance to go through, and now im moving to just_audio as i feel this plugin is home for audio related apps. thanks man, do you mean that the feature is already implemented or it still need some work. i will go through the code anyway

[afkcodes]

@ryanheise although your words are bit hard to understand you are pretty amazing. i just saw the headers code you mentioned still looking at it. probably will have to give it some more time. As i don't own a iOS/Mac i'm pretty much doomed.

[afkcodes]

@nuc134r understood thanks mate.

[nuc134r]

Was thinking a lot about extending existing proxy to allow realtime caching (and even tried) and then stumbled upon #172. Considering that I want to provide data from disk on repeating requests, get notified of caching progress and have an index of cached data I figured that it would be better to set up my own proxy between API and just_audio.

Also I see it as a better approach in terms of plugin ecosystem architecture. I think I'm going to try to implement it and if I reach any success I'll update this issue.

[ryanheise]

@nuc134r that's an interesting thought to contemplate, although the plan is to eventually have caching support directly in the plugin as I think there are benefits in this case to having it tightly coupled. The proxy is required for other reasons, and it would not actually be difficult to tack caching onto the existing proxy, rather than have a situation where we have proxy upon proxy.

[zxl777]

Maybe using https://pub.dev/packages/flutter_cache_manager

[ryanheise]

I've just created a new branch called proxy_improvements which refactors the proxy code to make it more extensible. Using it, just_audio now streams assets directly from the bundle via the proxy, and there is now also a new StreamAudioSource for dynamically streaming data which also supports range requests (See #172 ).

Caching could potentially be plugged in by subclassing this class, however ultimately I would like to have such a class offered by just_audio, or alternatively add an option to UriAudioSource to request that the downloaded audio be cached.

The way that caching support should work is that it should support partial caching, such that if a byte range request was made, that partial segment of the audio file should be cached.

@nuc134r I've left the caching part unimplemented for now in case you would be interested in submitting a pull request for it.

Otherwise, I'd be interested to have a discussion here about what sort of features people would like in this caching function besides the caching itself.

[defsub]

I'll share my use case. As of now the user can download songs to precache them for later local playback. The downloaded songs are stored in a cache keyed off of S3 etags. The metadata for all songs includes the etag so it's easy to check if a song is already cached and can be streamed locally. Before playback I have code that that checks whether to use file or https URLs depending on the cache lookup. I prefer using etags since the URLs may change due to reindexing, etc., however the songs themselves rarely change.

I really like the idea of being able to have an additional cache of songs that have been recently played so this feature is great. I'm not sure in my case if I'd want to combine the 2 caches since one is user controlled and the other I'd probably use as a size constrained LRU expired cache.

[agersant]

I can also chime in on this as another example use-case. I am currently writing a new version of the Android client for Polaris using audio_service and just_audio.

The set of features I am trying to support is:

• Songs played by the user are cached on disk, in a specific directory structure and alongside some Polaris-specific metadata (similar to a MediaItem).
• User can manually flag songs so they are never evicted from the cache.
• User can download songs for future offline use (storing them in the same cache).
• Upcoming songs in the current playlist are preemptively downloaded and added to the cache.
• User can run the app in offline mode and listen to cached-music only.

As a side note, storing partially downloaded files is not important for me (and even somewhat undesirable). Seeking is not a very frequent operation either.

The previous implementation in Java achieved all of this with a custom DataSource for ExoPlayer, but the code felt pretty hacky.

Changes to just_audio that can help reach these goals would be nice, but I must also point out I'm already pretty happy with what is there. I currently have my audio task running alongside a local HTTP server which it requests audio from. This server serves content from either the local disk cache, or from an actual remote server - taking care of caching what it downloads and the corresponding metadata (WIP).

[ryanheise]

Thanks for sharing your use cases @agersant . I understand seeking would not be very frequent in a music streaming app, compared, say, with a podcast app. A podcast episode might take in some cases minutes to download, and the user may want to seek around to check whether this is the right episode. And ultimately the cache should be able to present one contiguous file from the combined regions.

A very simple implementation of StreamAudioSource could just download the entire file into the cache and then if a seek occurs, it could block until the download reaches that byte range.

[nuc134r]

@ryanheise I appreciate the offer to make a PR and will try when I will have some coding time. I actually spent some time making asynchronous byte reading and writing so would like to share a solution. For now I didn't even check the code in the new branch.

In my current implementation data is served if it is written to disk and if user seeks beyond the cached range another request is made to the server. It's not that smart but I decided that downloading parts and then merging them together is a bit of an overhead. Also resulting gaps should be filled later with data as well.

I faced performance (even FPS) issues with furiously skipping tracks that are served from cache. That can be avoided with separate isolate and/or providing file urls once data is fully(?) downloaded. Providing file urls going be even easier if routing can be handled on AudioSource layer rather than always having proxy url.

Will look into it, looks intriguing. And try to PR what I'll come with. Thank you, Ryan.

[defsub]

I agree with @agersant that partial files aren't that useful with music. If a user seeks toward the end of a file that hasn't been pre-cached yet, I'd be okay with having the player wait for the download to catch up at the benefit of having the entire song cached. This is not great for podcasts but is reasonable for music, I think.

[afkcodes]

What if someone is playing a live radio? it will be a continuous stream of data. I wanted to implement this on my app where users can start or stop caching and the portions will be saved to the device for playing later. the second approach looks better for this scenario not sure though. Ryan can shed some light on this.

[ryanheise]

I see no reason why both Nginx caching approaches can't be implemented, with an additional option in the cache slicing approach to automatically download all segments.

As for caching radio streams, I'm not sure how to make that work, particularly if there is no information on where a cached segment fits into the whole stream, position wise. Unless you can suggest something, I think this use case is best handled by creating a custom subclass of StreamAudioSource.

[afkcodes]

@ryanheise both can be implemented with an optional flag which one to choose from, and about the live radio stream data the streams will be a sequence for a particular radio stream as it can't bee seeked so only option is to get the latest chunk from the stream. I have seen some native android app giving an option to record a particular playing radio and save it on device as mp3 or aac file but the underlying technique is to just save the chunk and adding upcoming chunk until the user stops it. I may not be 100% right here but this is what i have found. the above mentions approach might work here.

[ryanheise]

I think the cache logic could be abstracted out into a separate class with a subclass for each caching strategy. These could be plugged into the same caching subclass of StreamAudioSource.

As for your use case @afkcodes I would suggest that you create your own subclass of StreamAudioSource and in it you could provide a switch to turn on recording. If you're not sure how to do that, you can wait until the subclass for the more common use case is implemented and you could use that as a guide.

[afkcodes]

yeah cool @ryanheise i will wait for the implementation. you are awesome.

[ryanheise]

I just tried making a simple implementation that subclasses StreamAudioSource and realised that the lengthInBytescannot be a synchronous getter, so I'm going to rework the API. I'll post a sample subclass demonstrating how you might go about caching as soon as that is done.

[ryanheise]

The latest commit fixes the StreamAudioSource API and also provides a LockCachingAudioSource subclass which implements a very simple approach to caching where the first request results in the file being downloaded and cached, and subsequent byte-range requests will block until enough audio has been downloaded. This should be improved in the future to follow the Nginx approach of making additional byte-range requests to the origin server (without caching those). The reason is that if you're caching a podcast and it's in a format that isn't optimised for streaming, the player may need to make a byte range request near the end of the file to find the metadata (including the duration information). For a podcast, it could take quite a while for the initial request from offset zero to reach the end and we don't want to block for that long. Pull requests are welcome for anyone who wants to implement that, or even perhaps another subclass implementing the other Nginx caching scheme (the one that cuts the audio into slices).

I would appreciate if anyone also has feedback on the API, particularly the naming. In StreamingAudioSource I had used start and end as range parameters, but in StreamingAudioResponse I used offset which feels inconsistent:

/// The response for a [StreamAudioSource]. This API is experimental.
@experimental
class StreamAudioResponse {
  /// The total number of bytes available.
  final int sourceLength;

  /// The number of bytes returned in this response.
  final int contentLength;

  /// The starting byte position of the response data.
  final int offset;

  /// The audio content returned by this response.
  final Stream<List<int>> stream;

  StreamAudioResponse({
    @required this.sourceLength,
    @required this.contentLength,
    @required this.offset,
    @required this.stream,
  });
}

I'm also not completely decided on the other field names, like sourceLength and contentLength.

[zxl777]

@ryanheise

I tested the LockCachingAudioSource,
When there is a Wi-Fi network, it worked, and there is a slight delay in seeking.
When I turn off Wi-Fi, the sound cannot be played.

Please consider the availability of cached file when the network is closed, thank you.

    LockCachingAudioSource(
      Uri.parse("https://static.v86.co/audiobook/MY5SatbZMAo/MY5SatbZMAo.opus" //test for android
      // "https://static.v86.co/audiobook/MY5SatbZMAo/MY5SatbZMAo.caf" //test for ios
      ),
      tag: AudioMetadata(
        album: "Science Friday",
        title: "A Salute To Head-Scratching Science",
        artwork:
            "https://media.wnyc.org/i/1400/1400/l/80/1/ScienceFriday_WNYCStudios_1400.jpg",
      ),
    ),

[ryanheise]

@zxl777 what behaviour would you like when Wi-Fi is turned off while the resource is being downloaded? I think if it's turned off after the download is complete, there should be no problem, but if it's turned off before the download is complete, obviously the parts that aren't downloaded yet can't be played. As for playing the parts that are downloaded, depending on how your audio file is encoded the player may need to make byte-range requests to the end of the file to grab metadata. If your audio file is encoded for streaming, that should not be a problem. If you have no control over the encoding, then you would need to wait for an implementation of the cache slicing strategy which would also cache the end slice containing the metadata. You'll sill of course get silence if you seek to a byte range that hasn't been downloaded yet.

[zxl777]

@ryanheise @gtu-myowin
The problem I hope to solve is that I can start playing after a few seconds instead of waiting for the download to complete.
LockCachingAudioSource seems to have solved this problem.

However, I also hope to play the downloaded local files directly during the second repeat play. Especially when the network is offline, the local files can also be played.

And LockCachingAudioSource may not download completely when there is a seek jump. It is also expected that it cannot be played in offline mode. I tested the YouTube App and after it was cached, I disconnected Wi-Fi and YouTube could not play. This also explains the problem.

I think, according to @gtu-myowin's suggestion, if I need a complete local file, I can start a download process while being cached.

After all the downloads are complete, I can switch to the local file playback mode.

LockCachingAudioSource only needs to be able to pre-load a few seconds of content into the cache to satisfy fast playback.

[ryanheise]

One easy way to experiment with this is to copy the code for LockCachingAudioSource into your own project and modify it as desired. It is built using entirely public APIs which anyone can use, so it is perfectly feasible to create your own implementation based on this original code. Of course, if you create something generally useful, it would be great to contribute that back to the project.

In the meantime, I will see what I can do about seeking during the download.

[ryanheise]

@afkcodes thanks! There's actually a page here where you can nominate just_audio for flutter favorites, and I'm not going to object to people clicking that and saying something nice about just_audio ;-)

At the same time, there are many great audio plugins built by the community, and I wonder whether Flutter Favorites is actually the right approach. I fear that favoring one plugin over others would ultimately have an effect over time of encouraging everyone to use just one audio plugin. But I think app developers are much better served by having choices and competition between alternatives.

So far, the Flutter team have done an amazing job of encouraging a thriving ecosystem of open source community plugins, and that has come about just through the natural, open marketplace. One thought that came to mind is that the same criteria that the Flutter favorites committee use when considering a package for adoption into Flutter Favorites could be simply used to enhance the automatic scoring and ranking of packages on pub.dev. There would be no need to distinguish between favored and non-favored plugins, but rather, the various computed metrics on pub.dev could allow users to evaluate each individual plugin in its own right.

P.S. I was already contacted by the Flutter Favorites team actually, and I shared the above views with them. Still, with all that said, and until something better comes along, Flutter Favorites might not be all that bad if it is allowed for multiple alternative audio plugins to be awarded "favorite" status. So on that basis, you are welcome to add your nomination.

[afkcodes]

yeah, that will be cool adding to README will also make aware how to use this feature and hence more people will use it and also can report any issues.

[esiqveland]

Hey, I tried the LockCachingAudioSource by giving it a https url.

On android this crashes in the background task with:

E/ExoPlayerImplInternal( 4179): Playback error
E/ExoPlayerImplInternal( 4179):   com.google.android.exoplayer2.ExoPlaybackException: Source error
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:579)
E/ExoPlayerImplInternal( 4179):       at android.os.Handler.dispatchMessage(Handler.java:102)
E/ExoPlayerImplInternal( 4179):       at android.os.Looper.loop(Looper.java:193)
E/ExoPlayerImplInternal( 4179):       at android.os.HandlerThread.run(HandlerThread.java:65)
E/ExoPlayerImplInternal( 4179):   Caused by: com.google.android.exoplayer2.upstream.HttpDataSource$CleartextNotPermittedException: Cleartext HTTP traffic not permitted. See https://exoplayer.dev/issues/cleartext-not-permitted
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.upstream.DefaultHttpDataSource.open(DefaultHttpDataSource.java:354)
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.upstream.DefaultDataSource.open(DefaultDataSource.java:201)
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.upstream.StatsDataSource.open(StatsDataSource.java:84)
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.source.ProgressiveMediaPeriod$ExtractingLoadable.load(ProgressiveMediaPeriod.java:1015)
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:415)
E/ExoPlayerImplInternal( 4179):       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
E/ExoPlayerImplInternal( 4179):       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
E/ExoPlayerImplInternal( 4179):       at java.lang.Thread.run(Thread.java:764)
E/ExoPlayerImplInternal( 4179):   Caused by: java.io.IOException: Cleartext HTTP traffic to 127.0.0.1 not permitted
E/ExoPlayerImplInternal( 4179):       at com.android.okhttp.HttpHandler$CleartextURLFilter.checkURLPermitted(HttpHandler.java:115)
E/ExoPlayerImplInternal( 4179):       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:458)
E/ExoPlayerImplInternal( 4179):       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:127)
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.upstream.DefaultHttpDataSource.makeConnection(DefaultHttpDataSource.java:641)
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.upstream.DefaultHttpDataSource.makeConnection(DefaultHttpDataSource.java:543)
E/ExoPlayerImplInternal( 4179):       at com.google.android.exoplayer2.upstream.DefaultHttpDataSource.open(DefaultHttpDataSource.java:349)
E/ExoPlayerImplInternal( 4179):       ... 7 more

This was a bit unexpected, and I'm not sure what is happening in the background, but this part of the documentation gives a hint:

  ///
  /// If headers are set, just_audio will create a cleartext local HTTP proxy on
  /// your device to forward HTTP requests with headers included.

I don't actually set any headers, but though that is not very important. I am just wondering on the background why a separate webserver proxy is needed.

As for input to the API:

• Setting the cacheKey manually (a String?) would be good to help with consistent caching. We already have known checksums for all files.
• Maybe add a callback to verify an expected checksum after download is completed?
• I would love to be able to provide the HttpClient used to download, so I can instrument it with my logging [low priority]

[ryanheise]

The proxy is used to implement headers, but it is also used to implement the cache.

Thanks also for your feedback on the API. One thing you can do is copy the LockCachingAudioSource class and then modify it to your liking. It sounds like what's needed in general is a way of listening to the download progress. If you manage to make improvements that you think could be worth sharing, it could be a good idea to submit a pull request and I will be able to include those changes for everyone in the public release.

Regarding your first suggestion, the cacheKey, you can already pass in the cache file which I think should address that requirement. In terms of the HttpClient, I think you can look at the way the HttpClient can be substituted during testing, and you may be able to use the same facility to do a substitution for your own purposes, too.

[esiqveland]

Hi, thank you for all your hard work! This project is immensely helpful.

I have been testing LockCachingAudioSource for a couple of days now with small-ish files. Every audio file is less than ~10 minutes though so I don't expect it to do too much real streaming wrt to the streaming feature, but the goal was to avoid downloading the file twice and offer immediate playback.

Initially I experienced a lot of issues, but it's been working very well now for the last 2 days so I think it was because of other errors. When used with ConcatenatingAudioSource it has worked very well.

Some of the things I found:

• LockCachingAudioSource seems to be dropping the Host header. This causes the load balancer of the api to be less happy. I had to set the host manually as a explicit header to make it work. I think this is bug, as I see attempts to propagate the Host header elsewhere.
• On the above error: In case of errors it would be nice to include the status code, URL called and any string body from the response in the exception that is thrown. That would have made this much easier to debug. Right now it either gives a obscure error from ExoPlayer or just the status code. I guess an instrumented client might have helped with this.

I am new to mobile development, but am a little worried about performance. I have playlists with potentially thousands of tracks, which leaves me a bit worried for the cacheFile. I now supply the cacheFile eagerly, but I am not sure of the cost of opening a File object. Maybe it's fine, but for performance reasons it seems reasonable to lazily create the file when the AudioSource actually needs it.

Again, this is a great project, and everything worked flawlessly on iOS as well. Great stuff!

[ryanheise]

Hi @esiqveland Thanks for trying it out.

This API is still marked as experimental and that was to give people a chance to try it out and suggest improvements. In fact what I'm hoping is that people copy this class and modify it to their own liking, as there is absolutely no problem with creating a custom subclass of StreamAudioSource, and if you happen to make a particularly good modification, you might consider contributing that back to the project to make just_audio better.

[chengyuhui]

I have tried to implement a custom StreamAudioSource, the progress has been good so far. However, I noticed that the proxy always returns 200 or 206 without any way to signal an error (possibly from the network), is that intentional or an oversight?

Sorry for bothering you guys with so many issues related to errors, but when it comes to mobile network, anything can happen and we need to account for that for an optimal user experience.

[ryanheise]

That's an oversight. Maybe what I could do is make it so that if the overridden request method throws an exception, that can be used to indicate an error HTTP status.

In that case, the question arises as to whether you would need the flexibility to indicate different types of error statuses.

There's also the question of whether you might want to return non-error statuses, in which case throwing an exception would be the wrong design. Instead, maybe we could add an error code to the StreamAudioResponse object.

One more consideration is that StreamAudioSource was intended to be independent of the transport layer, since in the case of the web implementation, it doesn't go through the HTTP proxy at all. So that may mean we don't want the flexibility to indicate all sorts of HTTP statuses.

[chengyuhui]

I personally think that it is enough to have a way to indicate that "something went wrong when generating the response" by grabbing the error from the returned future in request and return something like 500, since it will be quickly turned into a platform-specific error that can not be recognized ever again in the dart side.

In this way we can also catch unexpected exceptions and handle them gracefully, without hanging other parts on the chain (which is what it seems to be doing now).

BTW, certain types of stream behaves differently when the player gets different HTTP status code (most notably HLS on iOS), it might also be interesting to pass through the status code

[ryanheise]

I personally think that it is enough to have a way to indicate that "something went wrong when generating the response" by grabbing the error from the returned future in request and return something like 500, since it will be quickly turned into a platform-specific error that can not be recognized ever again in the dart side.

The result of request actually gets consumed first by the audio decoder, and so in theory we only need to be concerned with the types of response codes that the audio decoder might care about. I was originally thinking about things like redirect status codes which the decoder may use, but those are better handled internally by the particular StreamAudioSource without the decoder ever knowing that a redirect happened. So I am left thinking that simply throwing an exception would be enough, and the plugin can automatically convert this into a 500 status code if transporting over HTTP.

BTW, certain types of stream behaves differently when the player gets different HTTP status code (most notably HLS on iOS), it might also be interesting to pass through the status code

Maybe there is something here... But do you have a use case for streaming HLS through a StreamAudioSource rather than just letting the player directly access it via HlsAudioSource?

[chengyuhui]

Uh...now that you say it, probably not (we probably should not cache HLS in this way?).

It should be fine to return a 500, just like how these extemely simple HTTP servers do in case of an uncaught error.

[zhangruiyu]

Hello, I'm trying to use LockCachingAudioSource, it works fine on Android, but it doesn't work on iOS, and I'm using Just_Audio's example playback (instead of my own audio link) in the audio format m4a


@ryanheise

[zhangruiyu]

There are no problems with using the Audiosource.

[zhangruiyu]

6e3c8a09-5fc5-4119-a7ba-27ef503959b5.m4a.zip
can try this audio

[ryanheise]

@zhangruiyu If you are up for doing a little investigation, I would recommend copying this class into your own project, then using your own copy of it in your app, and then you could insert some log messages into it to see if you can identify where it is going wrong (maybe compare the logs for one audio file that works and one audio file that doesn't work.)

[zhangruiyu]

I'm sorry that I didn't make it clear. Actually, my project has been nearly completed. In my own project, Android is OK, but IOS is not

…

---Original--- From: ***@***.***> Date: 2021/4/10 23:35:34 To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [ryanheise/just_audio] Playing a remote file and cache the file at the same time (#47) @zhangruiyu If you are up for doing a little investigation, I would recommend copying this class into your own project, then using your own copy of it in your app, and then you could insert some log messages into it to see if you can identify where it is going wrong (maybe compare the logs for one audio file that works and one audio file that doesn't work.) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ryanheise]

I understood you, but to help move things along faster I was checking whether you were up for doing a little investigation. I am currently focused on other more urgent issues but as this is an open source project, people are welcome to help and contribute to the open source effort.

[zhangruiyu]

I understood you, but to help move things along faster I was checking whether you were up for doing a little investigation. I am currently focused on other more urgent issues but as this is an open source project, people are welcome to help and contribute to the open source effort.

0-0 Sorry, I tried to check it out, but I don't know anything about iOS development, all I can see is Flutter's error

[zhangruiyu]

I'm really sorry, but I can't even find exceptions in Object-C, probably because my skills are so poor...

[ryanheise]

That's OK, what I'm suggesting is that it would be possible to do some investigation of the LockCachingAudioSource class itself since it is all written in pure Dart code. It would be interesting to know whether the mime type is being set correctly and whether the file is being written to the cache correctly. You could even examine the cache file afterwards to see if it is a valid file, and whether it will successfully play on second try after it has downloaded. Things like that could be investigated just on the Dart side.

[chengyuhui]

If memory serves me right, there’s indeed problems in MIME handling when I was adapting the code to my business logic (it wasn’t being written properly), I’m looking into opening a PR. ryanheise ***@***.***>于2021年4月11日 周日10:24写道：

…

That's OK, what I'm suggesting is that it would be possible to do some investigation of the LockCachingAudioSource class itself since it is all written in pure Dart code. It would be interesting to know whether the mime type is being set correctly and whether the file is being written to the cache correctly. You could even examine the cache file afterwards to see if it is a valid file, and whether it will successfully play on second try after it has downloaded. Things like that could be investigated just on the Dart side. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#47 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKRK2EGUG527BCYAF3EPTLTIEB4RANCNFSM4K4WYTDQ> .

[zhangruiyu]

I found out where the exception was, but I probably won't be able to fix it (really not iOS development).

[ryanheise]

I would guess this is not an issue caused by the iOS side, but rather that the iOS side will correctly report errors when it is fed invalid data. So my theory is that the Dart side is feeding it incorrect data.

Regarding how to debug on iOS, there is something equivalent to a stack trace, so you want to keep clicking around to find the line higher in the stack that called this sendErrorForItem method. But it is most likely again that iOS is being fed invalid data from the Dart side.

[zhangruiyu]

I would guess this is not an issue caused by the iOS side, but rather that the iOS side will correctly report errors when it is fed invalid data. So my theory is that the Dart side is feeding it incorrect data.

Regarding how to debug on iOS, there is something equivalent to a stack trace, so you want to keep clicking around to find the line higher in the stack that called this sendErrorForItem method. But it is most likely again that iOS is being fed invalid data from the Dart side.

I can see it will automatically download, but I still don't know why it won't play. There's nothing I can do. It's too hard for me

[ryanheise]

Is the downloaded file playable outside of the app, e.g. through your mac? Or is the file corrupted?

[zhangruiyu]

After downloading can play, the file does not have any problems

…

---Original--- From: ***@***.***&gt; Date: 2021/4/13 18:56:00 To: ***@***.***&gt;; Cc: ***@***.******@***.***&gt;; Subject: Re: [ryanheise/just_audio] Playing a remote file and cache the file at the same time (#47) Is the downloaded file playable outside of the app, e.g. through your mac? Or is the file corrupted? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ryanheise]

To clarify, is it playable within the app after it has downloaded?

[zhangruiyu]

It can play on a Mac, but not by playing LockCachingAudioSource

…

---Original--- From: ***@***.***&gt; Date: 2021/4/13 19:02:22 To: ***@***.***&gt;; Cc: ***@***.******@***.***&gt;; Subject: Re: [ryanheise/just_audio] Playing a remote file and cache the file at the same time (#47) To clarify, is it playable within the app after it has downloaded? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[zhangruiyu]

1. Open my app 2. Open the app download directory on my Mac 3. Wait for the download to complete 4. Restart the app 5. Unable to play

…

---Original--- From: ***@***.***&gt; Date: 2021/4/13 19:02:22 To: ***@***.***&gt;; Cc: ***@***.******@***.***&gt;; Subject: Re: [ryanheise/just_audio] Playing a remote file and cache the file at the same time (#47) To clarify, is it playable within the app after it has downloaded? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[chengyuhui]

Now that audio_service has been moving to an approach that runs all code in the same isolate, I'm worried that all the existsSync operations in LockCachingAudioSource might cause unwanted problems if disk I/O is slow. Should we change them?

[zhangruiyu]

Any progress on the issue that the M4A format does not work with LockCachingAudioSource on Mac and iOS platforms 0-0

[chengyuhui]

@zhangruiyu M4A works differently from MP3 as the player needs to access data at the end of the file (because of how the container works by default), which hits a different code path. However, I suppose the code for notReadyRequests works fine (at least for me, as all our media library is now M4A).
However, you should check if the server returned the correct Content-Type, as the iOS player relies heavily on this (ExoPlayer does not always care and plays anything in any way it can).

I would recommend you enable "Pause on all exceptions" in your IDE and check if any error is thrown in the caching code.

[zhangruiyu]

@zhangruiyu M4A works differently from MP3 as the player needs to access data at the end of the file (because of how the container works by default), which hits a different code path. However, I suppose the code for notReadyRequests works fine (at least for me, as all our media library is now M4A).
However, you should check if the server returned the correct Content-Type, as the iOS player relies heavily on this (ExoPlayer does not always care and plays anything in any way it can).

I would recommend you enable "Pause on all exceptions" in your IDE and check if any error is thrown in the caching code.

emmm,can you add QQ

[chengyuhui]

@zhangruiyu I can personally tell you how to troubleshoot this, but please leave other discussions there so the author and others can fix the issue (I think we DO have some problems when it comes to M4A).

[BytesZero]

I use this：
https://pub.dev/documentation/just_audio/latest/just_audio/LockCachingAudioSource-class.html

[balasubramanian1612s]

Are there any possibilities to Concatenate streaming Internet Audios and Local audios(that I've downloaded)? @ryanheise

Thanks in Advance.

[ryanheise]

@esiqveland the latest just_audio release merges @chengyuhui 's pull request to better handle errors. It may still take some logging to understand where exactly it's failing, but this release will hopefully be better behaved in the event of errors.

@balasubramanian1612s your question doesn't mention caching so I'm not sure if it is about the same feature being developed in this issue.

[afkcodes]

@ryanheise i see the caching is done for the streams which is kind of finite like a mp3 or podcast show. is it possible to cache and store a radio stream so that i can be able to play it at a later point of time ?

[ryanheise]

@afkcodes I think this should be possible by creating your own custom subclass of StreamAudioSource as suggested in my earlier reply:

As for caching radio streams, I'm not sure how to make that work, particularly if there is no information on where a cached segment fits into the whole stream, position wise. Unless you can suggest something, I think this use case is best handled by creating a custom subclass of StreamAudioSource.

[chengyuhui]

I've noticed random network failures (connection errors) when playing in background on iOS, but have no idea if this is something on my device (we are only beta testing on Android, so no large-scale logs for iOS).

Maybe there is some network restriction in place?

[ryanheise]

Assuming the iOS background mode has been set in Info.plist, I guess this is related to pausing playback. During a pause, iOS may be killing or suspending the Dart process and it's having trouble resuming the connection after getting woken up again from a play button click. In this case, some more sophisticated logic in LockCacheAudioSource would be needed to detect the error and retry.

[chengyuhui]

In that case, I think we would need more customizability for LockCacheAudioSource, as it would be increasingly difficult to keep our custom versions in sync with the official one.

My primary use case is that our application uses a different HTTP client (dio) for logging and HTTP/2 support, which can not be easily hacked into the current implementation.

[afkcodes]

@ryanheise checked on my side in my use case its doesn't throw any errors, just wanted to know how can i store the cached data to device as an playable audio file is there any way this can be achieved.

[ryanheise]

@afkcodes I'm a little confused by your question because that is precisely what LockCachingAudioSource does. It stores the cached data to a file. If I misunderstood your question, would you mind clarifying?

[afkcodes]

yeah probably i didn't framed the question rightly. Basically is there a way where i can access the stored file and play it with other music players ?

[ryanheise]

@afkcodes I think that's covered in the documentation so if it's not, I would suggest opening a new issue for the documentation request.