rutgersc / m3u8-rs Goto Github PK
View Code? Open in Web Editor NEWm3u8 parser for rust
License: MIT License
m3u8 parser for rust
License: MIT License
For simplicity I'm going to refer to one of your example manifests
https://docs.rs/crate/m3u8-rs/1.0.5/source/sample-playlists/master-with-i-frame-stream-inf.m3u8
if simply ingested and playlist.write_to() it converts
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=86000,CODECS="c1",RESOLUTION=1x1,VIDEO="1",URI="low/iframe.m3u8"
into
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=86000,CODECS="c1",RESOLUTION=1x1,VIDEO="1"URI="low/iframe.m3u8"
I believe the code responsible to this is the follow:
if self.is_i_frame {
write!(w, "#EXT-X-I-FRAME-STREAM-INF:")?;
self.write_stream_inf_common_attributes(w)?;
writeln!(w, "URI=\"{}\"", self.uri)
}
else {
write!(w, "#EXT-X-STREAM-INF:")?;
self.write_stream_inf_common_attributes(w)?;
write_some_attribute_quoted!(w, ",AUDIO", &self.audio)?;
write_some_attribute_quoted!(w, ",SUBTITLES", &self.subtitles)?;
write_some_attribute_quoted!(w, ",CLOSED-CAPTIONS", &self.closed_captions)?;
write!(w, "\n")?;
writeln!(w, "{}", self.uri)
}
}
The RFC8216 states
Every EXT-X-I-FRAME-STREAM-INF tag MUST include a BANDWIDTH attribute
and a URI attribute.
Since BANDWIDTH is already included in the write_stream_inf_common_attributes, so it is safe to simply add comma "," in front of the URI attribute.
Hi, I recently upgraded from 3.0.0
to 5.0.3
and now my audio playlist ends up in unknown_tags
instead of in alternatives
.
What am I doing wrong?
Here is the master playlist, slightly anonymized
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:LANGUAGE="en",AUTOSELECT=YES,FORCED=NO,TYPE=AUDIO,URI="https://xyz.com/a/playlist.m3u8",GROUP-ID="audio",DEFAULT=YES,NAME="Audio"
#EXT-X-STREAM-INF:RESOLUTION=3840x2160,FRAME-RATE=25.0,BANDWIDTH=6878544,AUDIO="audio"
https://xyz.com/b/playlist.m3u8
When running cargo deny
to check for advisories on this crate, a failure is reported due the chrono
crate dependency, which has a dependency on a problematic version of the time
crate which has an outstanding CVE (https://rustsec.org/advisories/RUSTSEC-2020-0071)
cargo deny check advisories
This can be resolved by disabling default-features
on the chrono crate and enabling only the std
feature:
chrono = { version = "0.4", default-features = false, features = [ "std" ] }
Please consider making this change as I'd like to try using this crate in a project which requires checking advisories on every build.
I'm happy to fork the repo and open a PR myself if preferred.
While it's unlikely that more than 2 billion segments or byte ranges spanning 2GB (this one is more likely though...) are used, it would make sense to switch all these two u64
as it costs basically nothing, reduces the need to worry about negative numbers and would allow handling e.g. a playlist that works via byte ranges into a multi-GB file.
Here is the original manifest for posterity.
#EXTM3U
#EXT-X-VERSION:6
## Created with Unified Streaming Platform (version=1.11.12-25516)
# AUDIO groups
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-64",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2",URI="1645124292737-output_hls_dref-audio_eng=64000.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-96",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2",URI="1645124292737-output_hls_dref-audio_eng=96000.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-128",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2",URI="1645124292737-output_hls_dref-audio_eng=128000.m3u8"
# SUBTITLES groups
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="textstream",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,URI="1645124292737-output_hls_dref-textstream_eng=1000.m3u8"
# variants
#EXT-X-STREAM-INF:BANDWIDTH=228000,CODECS="mp4a.40.2,avc1.64000D",RESOLUTION=320x240,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-64",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=150000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=312000,CODECS="mp4a.40.2,avc1.64000D",RESOLUTION=416x312,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-96",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=197000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=467000,CODECS="mp4a.40.2,avc1.640015",RESOLUTION=480x360,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-128",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=311000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=746000,CODECS="mp4a.40.2,avc1.64001E",RESOLUTION=640x480,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-128",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=574000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=946000,CODECS="mp4a.40.2,avc1.64001E",RESOLUTION=768x576,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-128",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=763000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1127000,CODECS="mp4a.40.2,avc1.64001E",RESOLUTION=768x576,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-128",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=934000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2055000,CODECS="mp4a.40.2,avc1.64001F",RESOLUTION=1280x960,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-128",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=1809000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2489000,CODECS="mp4a.40.2,avc1.64001F",RESOLUTION=1280x960,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-128",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=2219000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=4315000,CODECS="mp4a.40.2,avc1.640028",RESOLUTION=1920x1440,FRAME-RATE=29.97,VIDEO-RANGE=SDR,AUDIO="audio-aacl-128",SUBTITLES="textstream",CLOSED-CAPTIONS=NONE
1645124292737-output_hls_dref-video=3941000.m3u8
# keyframes
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=20000,CODECS="avc1.64000D",RESOLUTION=320x240,VIDEO-RANGE=SDR,URI="keyframes/1645124292737-output_hls_dref-video=150000.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=27000,CODECS="avc1.64000D",RESOLUTION=416x312,VIDEO-RANGE=SDR,URI="keyframes/1645124292737-output_hls_dref-video=197000.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=42000,CODECS="avc1.640015",RESOLUTION=480x360,VIDEO-RANGE=SDR,URI="keyframes/1645124292737-output_hls_dref-video=311000.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=77000,CODECS="avc1.64001E",RESOLUTION=640x480,VIDEO-RANGE=SDR,URI="keyframes/1645124292737-output_hls_dref-video=574000.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=102000,CODECS="avc1.64001E",RESOLUTION=768x576,VIDEO-RANGE=SDR,URI="keyframes/1645124292737-output_hls_dref-video=763000.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=240000,CODECS="avc1.64001F",RESOLUTION=1280x960,VIDEO-RANGE=SDR,URI="keyframes/1645124292737-output_hls_dref-video=1809000.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=523000,CODECS="avc1.640028",RESOLUTION=1920x1440,VIDEO-RANGE=SDR,URI="keyframes/1645124292737-output_hls_dref-video=3941000.m3u8"
The problem here is that a in variant attribute CLOSED-CAPTIONS=NONE
, the NONE
is ingested as is and when rendered out by write_to()
as double quoted, so it becomes a name for a GROUP-ID="NONE"
, which obviously is not present. Here is how the standard reads about this:
CLOSED-CAPTIONS
The value can be either a quoted-string or an enumerated-string
with the value NONE. If the value is a quoted-string, it MUST
match the value of the GROUP-ID attribute of an EXT-X-MEDIA tag
elsewhere in the Playlist whose TYPE attribute is CLOSED-CAPTIONS,
and it indicates the set of closed-caption Renditions that can be
used when playing the presentation. See [Section 4.4.6.2.1]
If the value is the enumerated-string value NONE, all EXT-X-
STREAM-INF tags MUST have this attribute with a value of NONE,
indicating that there are no closed captions in any Variant Stream
in the Multivariant Playlist. Having closed captions in one
Variant Stream but not another can trigger playback
inconsistencies.
https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc8216bis-10#section-4.4.6.2
I will attach a PR with a proposed fix
const PLAYLIST: &str = r"#EXTM3U
#EXT-PLAYLIST-UNKNOWN-TAG
#EXT-X-TARGETDURATION:10
#EXT-SEGMENT-UNKNOWN-TAG
#EXTINF:10,
seg-1.ts
#EXT-SEGMENT-UNKNOWN-TAG
#EXTINF:10,
seg-2.ts
#EXT-X-ENDLIST
";
fn main() {
let mut playlist = m3u8_rs::parse_media_playlist_res(PLAYLIST.as_bytes()).unwrap();
println!(
"{:#?}\n{:-^40}\n{:#?}\n{:-^40}",
playlist.unknown_tags, "-", playlist.segments[0].unknown_tags, "-"
);
// FORCE PLAYLIST UNKNOWN TAG
playlist.segments[0].unknown_tags.remove(0);
playlist.unknown_tags.push(m3u8_rs::ExtTag {
tag: "PLAYLIST-UNKNOWN-TAG".to_owned(),
rest: None,
});
playlist.write_to(&mut std::io::stdout()).unwrap();
}
[]
----------------------------------------
[
ExtTag {
tag: "PLAYLIST-UNKNOWN-TAG",
rest: None,
},
ExtTag {
tag: "SEGMENT-UNKNOWN-TAG",
rest: None,
},
]
----------------------------------------
#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-SEGMENT-UNKNOWN-TAG
#EXTINF:10,
seg-1.ts
#EXT-SEGMENT-UNKNOWN-TAG
#EXTINF:10,
seg-2.ts
#EXT-X-ENDLIST
The use of CUEs are breaking the parser, maybe you would be interested in adding top-level support for those tags?
, unknown_tags: [ExtTag { tag: "X-CUE-OUT", rest: "DURATION=30" }, ExtTag { tag: "X-CUE-IN\n#EXTINF", rest: "4,"
Other libraries also support that tag:
I will be happy to send a PR if you are interested in considering it. :)
Key info should be added to the following Media Initialization Sections and every segment until the next key.
4.3.2.4. EXT-X-KEY
Media Segments MAY be encrypted. The EXT-X-KEY tag specifies how to
decrypt them. It applies to every Media Segment and to every Media
Initialization Section declared by an EXT-X-MAP tag that appears
between it and the next EXT-X-KEY tag in the Playlist file with the
same KEYFORMAT attribute (or the end of the Playlist file). Two or
more EXT-X-KEY tags with different KEYFORMAT attributes MAY apply to
the same Media Segment if they ultimately produce the same decryption
key.
At the moment, AlternativeMedia get placed into the 'alternatives' member of the next VariantStream/EXT-X-MEDIA-INF
tag encountered after the EXT-X-MEDIA
tag that defines each alternative stream, even if other VariantStream's reference the same AlternateMedia group.
Either the AlternativeMedia should be duplicated into every VariantStream that references them, or they should be all be output once as a member of the top-level MasterPlaylist struct instead. Even better, the alternatives could be grouped by GROUP-ID
in an AlternativeMediaGroup struct.
For example, this master playlist from Apple's Bipbop test media (https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8):
#EXTM3U
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac_sinewave/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/eng/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English (Forced)",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,LANGUAGE="en",URI="subtitles/eng_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="fr",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/fra/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français (Forced)",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,LANGUAGE="fr",URI="subtitles/fra_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="es",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/spa/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español (Forced)",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,LANGUAGE="es",URI="subtitles/spa_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="ja",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/jpn/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語 (Forced)",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,LANGUAGE="ja",URI="subtitles/jpn_forced/prog_index.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=263851,CODECS="mp4a.40.2, avc1.4d400d",RESOLUTION=416x234,AUDIO="bipbop_audio",SUBTITLES="subs"
gear1/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=28451,CODECS="avc1.4d400d",URI="gear1/iframe_index.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=577610,CODECS="mp4a.40.2, avc1.4d401e",RESOLUTION=640x360,AUDIO="bipbop_audio",SUBTITLES="subs"
gear2/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=181534,CODECS="avc1.4d401e",URI="gear2/iframe_index.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=915905,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,AUDIO="bipbop_audio",SUBTITLES="subs"
gear3/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=297056,CODECS="avc1.4d401f",URI="gear3/iframe_index.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=1030138,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1280x720,AUDIO="bipbop_audio",SUBTITLES="subs"
gear4/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=339492,CODECS="avc1.4d401f",URI="gear4/iframe_index.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=1924009,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1920x1080,AUDIO="bipbop_audio",SUBTITLES="subs"
gear5/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=669554,CODECS="avc1.4d401f",URI="gear5/iframe_index.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=41457,CODECS="mp4a.40.2",AUDIO="bipbop_audio",SUBTITLES="subs"
gear0/prog_index.m3u8
All the variant streams reference the same set of bipbop_audio
and subs
alternative media group streams, but the AlternativeMedia structs only get attached to the first variant stream.
See 4.4.4.2:
If o is not present, the sub-range begins at the next byte following
the sub-range of the previous Media Segment.
and also
If o is not present, a previous Media Segment MUST appear in the
Playlist file and MUST be a sub-range of the same media resource, or
the Media Segment is undefined and the client MUST fail to parse the
Playlist.
Questions for this:
media_playlist_from_tags
(which would have to be fallible then)?ByteRange
non-optional as it would always be set now?The current code allows the duplication of #X-EXT-KEY and #X-EXT-MAP tags when a manifest is injected in and then written out to another file. For example, the following
EXTM3U
#EXT-X-TARGETDURATION:5
#EXT-X-VERSION:7
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-KEY:METHOD=AES-128,URI="./slow_loading.php?delay=5&resource=crypt0.key",IV=0xbf9840dc7d7fa163301a6c38844d6239
#EXT-X-MAP:URI="main.mp4",BYTERANGE="560@0"
#EXTINF:4.96907,
#EXT-X-BYTERANGE:25312@560
main.mp4
#EXTINF:4.96907,
#EXT-X-BYTERANGE:25440@25872
main.mp4
#EXTINF:4.96907,
#EXT-X-BYTERANGE:25440@51312
main.mp4
#EXTINF:4.96907,
#EXT-X-BYTERANGE:25440@76752
main.mp4
becomes
#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:0,EXTM3U
#EXT-X-BYTERANGE:25312@560
#EXT-X-KEY:METHOD=AES-128,URI="./slow_loading.php?delay=5&resource=crypt0.key",IV=0xbf9840dc7d7fa163301a6c38844d6239
#EXT-X-MAP:URI="main.mp4",BYTERANGE=560@0
#EXTINF:4.96907,
main.mp4
#EXT-X-BYTERANGE:25440@25872
#EXT-X-KEY:METHOD=AES-128,URI="./slow_loading.php?delay=5&resource=crypt0.key",IV=0xbf9840dc7d7fa163301a6c38844d6239
#EXT-X-MAP:URI="main.mp4",BYTERANGE=560@0
#EXTINF:4.96907,
main.mp4
#EXT-X-BYTERANGE:25440@51312
#EXT-X-KEY:METHOD=AES-128,URI="./slow_loading.php?delay=5&resource=crypt0.key",IV=0xbf9840dc7d7fa163301a6c38844d6239
#EXT-X-MAP:URI="main.mp4",BYTERANGE=560@0
#EXTINF:4.96907,
main.mp4
#EXT-X-BYTERANGE:25440@76752
#EXT-X-KEY:METHOD=AES-128,URI="./slow_loading.php?delay=5&resource=crypt0.key",IV=0xbf9840dc7d7fa163301a6c38844d6239
#EXT-X-MAP:URI="main.mp4",BYTERANGE=560@0
#EXTINF:4.96907,
main.mp4
(the example is from https://stackoverflow.com/questions/12906820/any-m3u8-playlist-example-to-use-ext-x-map-tag)
Currently this crate still depends on nom 5. Updating will take a bit of effort though because nom 7 finally got rid of the macro based parser combinators, which are still in use here.
As per https://tools.ietf.org/html/rfc8216#section-4.3.2.4, there may be multiple EXT-X-KEY tags, as long as they have different KEYFORMAT.
Further, a key tag with a given KEYFORMAT applies to every media segment following until it is replaced. At the moment, the tag is only attached to the first segment following it.
Media Segments MAY be encrypted. The EXT-X-KEY tag specifies how to
decrypt them. It applies to every Media Segment and to every Media
Initialization Section declared by an EXT-X-MAP tag that appears
between it and the next EXT-X-KEY tag in the Playlist file with the
same KEYFORMAT attribute (or the end of the Playlist file). Two or
more EXT-X-KEY tags with different KEYFORMAT attributes MAY apply to
the same Media Segment if they ultimately produce the same decryption
key.
In a master playlist, EXT-X-SESSION-DATA or EXT-X-SESSION-KEY tags can appear more than once.
The only constraints are:
A Playlist MAY contain multiple EXT-X-SESSION-DATA tags with the same
DATA-ID attribute. A Playlist MUST NOT contain more than one EXT-X-
SESSION-DATA tag with the same DATA-ID attribute and the same
LANGUAGE attribute.
and
A Master Playlist MUST NOT contain more than one EXT-X-SESSION-KEY
tag with the same METHOD, URI, IV, KEYFORMAT, and KEYFORMATVERSIONS
attribute values.
Hi,
Thank you for fixing the CLOSED-CAPTIONS issue, I just ran on it yesterday. Somehow it works only if I clone and reference the library locally and not when built from crates.io. Probably deserves 3.0.1 version and publishing?
.... as you there :) Can you please update README.md ? Thanks a bunch!! ❤️
I found a playlist that is mistakenly parsed as a media playlist instead of a master playlist. It turns out a blank line after the initial #EXTM3U
tag is the trigger.
e.g.
#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=1280000,AVERAGE-BANDWIDTH=1000000
http://example.com/low.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2560000,AVERAGE-BANDWIDTH=2000000
http://example.com/mid.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=7680000,AVERAGE-BANDWIDTH=6000000
http://example.com/hi.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=65000,CODECS="mp4a.40.5"
http://example.com/audio-only.m3u8
will be parsed incorrectly. Removing the blank line makes it parse correctly.
Hello!
I was reading your source code and I came across master playlist detection here: https://github.com/rutgersc/m3u8-rs/blob/master/src/lib.rs#L229
Are #EXT-X-SESSION-KEY
and #EXT-X-SESSION-DATA
master playlist tags?
https://github.com/rutgersc/m3u8-rs/blob/master/src/playlist.rs#L423
println!("Tag: {:?}\n", &tag);
Need to change println!
to debug!
or delete it.
Unlike the segment duration in EXTINF, EXT-X-TARGETDURATION can only be a decimal-integer according to the RFC.
The motivation behind this issue is that players can refuse playback if the user of the library sets this to a non-integer value and it is better to eliminate the possibility of this happening.
const PLAYLIST: &str = r#"#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:6
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MAP:URI="video-H264-1080-3000k-video-avc1.mp4",BYTERANGE="867@0"
#EXTINF:6.000,
#EXT-X-BYTERANGE:59987@2375
video-H264-1080-3000k-video-avc1.mp4
"#;
fn main() {
let media = m3u8_rs::parse_media_playlist(PLAYLIST.as_bytes()).unwrap().1;
println!("{:#?}", media);
}
Output with v5.0.3
MediaPlaylist {
version: Some(
6,
),
target_duration: 6.0,
media_sequence: 0,
segments: [
MediaSegment {
uri: "video-H264-1080-3000k-video-avc1.mp4",
duration: 6.0,
title: None,
byte_range: Some(
ByteRange {
length: 59987,
offset: Some(
2375,
),
},
),
discontinuity: false,
key: None,
map: None,
program_date_time: None,
daterange: None,
unknown_tags: [
ExtTag {
tag: "X-MAP",
rest: Some(
"URI=\"video-H264-1080-3000k-video-avc1.mp4\",BYTERANGE=\"867@0\"",
),
},
],
},
],
discontinuity_sequence: 0,
end_list: false,
playlist_type: Some(
Vod,
),
i_frames_only: false,
start: None,
independent_segments: false,
unknown_tags: [],
}
Output with v4.0.0
MediaPlaylist {
version: 6,
target_duration: 6.0,
media_sequence: 0,
segments: [
MediaSegment {
uri: "video-H264-1080-3000k-video-avc1.mp4",
duration: 6.0,
title: None,
byte_range: Some(
ByteRange {
length: 59987,
offset: Some(
2375,
),
},
),
discontinuity: false,
key: None,
map: Some(
Map {
uri: "video-H264-1080-3000k-video-avc1.mp4",
byte_range: Some(
ByteRange {
length: 867,
offset: Some(
0,
),
},
),
},
),
program_date_time: None,
daterange: None,
unknown_tags: [],
},
],
discontinuity_sequence: 0,
end_list: false,
playlist_type: Some(
Vod,
),
i_frames_only: false,
start: None,
independent_segments: false,
}
Would be nice to standardize the source code of this library using the cargo fmt
tool. Additionally, adding some check in the CI server will help to keep consistency moving forward.
Random HTML content (like below) can still get parsed as a playlist succesfully. I think this is because the parser setup treats the required #EXTM3U start tag the same as any other tag, and doesn't require its presence as the first tag.
<!doctype html>
<html>
<head>
<title>Example Domain</title>
<meta charset="utf-8" />
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<style type="text/css">
body {
background-color: #f0f0f2;
margin: 0;
padding: 0;
font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
}
div {
width: 600px;
margin: 5em auto;
padding: 2em;
background-color: #fdfdff;
border-radius: 0.5em;
box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
}
a:link, a:visited {
color: #38488f;
text-decoration: none;
}
@media (max-width: 700px) {
div {
margin: 0 auto;
width: auto;
}
}
</style>
</head>
<body>
<div>
<h1>Example Domain</h1>
<p>This domain is for use in illustrative examples in documents. You may use this
domain in literature without prior coordination or asking for permission.</p>
<p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
There are two users of this crate (the streaming servers rectangle-device and javelin) which only use the types (MediaPlaylist
, MediaPlaylistType
, MediaSegment
). It would be nice to either split them out or put the parsing functionality behind a feature.
Issue can be reproduced with this code:
const DATA: &'static str = include_str!("../test.m3u8");
fn main() {
let parsed = m3u8_rs::parse_playlist_res(DATA.as_bytes());
match parsed {
Ok(m3u8_rs::playlist::Playlist::MasterPlaylist(mp)) => {
for variant in mp.variants {
println!("{}: {}", variant.alternatives[0].name, variant.uri);
}
},
_ => { }
}
}
If the input file ends with a new line, all URLs are returned, the above code snippet would output:
...
audio_only: https://video-weaver.cph01.hls.ttvnw.net/v1/playlist/CvADyY7WIxbtDSJ0njAsWODtSlU3th3xEoqA4xy0hB37DMPbQ-UOcK9ipvr4PSV5JZdacS0zHw1B7M7qMxNkGTz-4GresX5omdJK25JFJ3R57TdES2QqOYr6s9Njiov9VfYboWl93cTLeDC4bOk6jWa7CROjMQOdzZQFs7ItRvKWiIOWHJWmp20pkeGJGzKn7djFb1QYrmhEWpel36muLe62rV1iXDeAw2d6wzNH0cXF-Ub09fIawnTwECXvHaQAYMK6Ij4hqdDchOVewvLJQj7vxDLMQhus6bKPbgulojTO0CLvxHdrGydswSbiYYTib_5nIIuMXTNlZbTQ1vyPJgFZlOUHPDy8znoQanX4YQnTdx-9jQ35YA2AhdKJ6edHF3oKy3c-CDxBJIJavZqsMj2Zc_o6oEUQXJfrKAlmWUaYz0FFm6j8PDRGKxpyaiUYokYZ-W0wx_sNJDKOr_Fd_MN8sRCxwiTFRWJ1ht7tumnfpP177P3n3prplM__3cgleAdVwzULqx_5bjVJYbHkcVxCE0iGz21n2UaOWkW1CaFR9QxQunEfDuiTh_w_Ver3M9kWnFpU8S0LxYtjRNlvqU8o1ds4_vgaPNFXrMaN7fAnoN1ETawu5I0yDAWIX6PclLW6K5fyjSaCiCpk3M8O5KYCTRIQqSaiIAjvvYr_qGRuzVDHrRoMJSflyQR53h9-9ZMn.m3u8
but if the files does not end with a new line, we get this:
...
audio_only:
Here's the test file I used, it's from a Twitch Livestream:
test.m3u8
const PLAYLIST: &str = r"#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-KEY:METHOD=AES-128
#EXTINF:10,
seg-1.ts
#EXT-X-KEY:METHOD=SOME-UNKNOWN-ENC
#EXTINF:10,
seg-2.ts
";
fn main() {
let playlist = m3u8_rs::parse_media_playlist_res(PLAYLIST.as_bytes()).unwrap();
println!("{:#?}", playlist.segments);
// PASSES
assert_eq!(
playlist.segments[0].key,
Some(m3u8_rs::Key {
method: m3u8_rs::KeyMethod::AES128,
..Default::default()
})
);
// FAILS
assert_eq!(
playlist.segments[1].key,
Some(m3u8_rs::Key {
method: m3u8_rs::KeyMethod::Other("SOME-UNKNOWN-ENC".to_owned()),
..Default::default()
})
);
}
[
MediaSegment {
uri: "seg-1.ts",
duration: 10.0,
title: None,
byte_range: None,
discontinuity: false,
key: Some(
Key {
method: AES128,
uri: None,
iv: None,
keyformat: None,
keyformatversions: None,
},
),
map: None,
program_date_time: None,
daterange: None,
unknown_tags: [],
},
MediaSegment {
uri: "seg-2.ts",
duration: 10.0,
title: None,
byte_range: None,
discontinuity: false,
key: None,
map: None,
program_date_time: None,
daterange: None,
unknown_tags: [
ExtTag {
tag: "X-KEY",
rest: Some(
"METHOD=SOME-UNKNOWN-ENC",
),
},
],
},
]
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `None`,
right: `Some(Key { method: Other("SOME-UNKNOWN-ENC"), uri: None, iv: None, keyformat: None, keyformatversions: None })`', src\main.rs:26:5
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.