pion / webrtc

Pure Go implementation of the WebRTC API

Pion WebRTC

A pure Go implementation of the WebRTC API

Usage

Go Modules are mandatory for using Pion WebRTC. So make sure you set export GO111MODULE=on, and explicitly specify /v4 (or an earlier version) when importing.

example applications contains code samples of common things people build with Pion WebRTC.

example-webrtc-applications contains more full featured examples that use 3rd party libraries.

awesome-pion contains projects that have used Pion, and serve as real world examples of usage.

GoDoc is an auto generated API reference. All our Public APIs are commented.

FAQ has answers to common questions. If you have a question not covered please ask in Slack we are always looking to expand it.

Now go build something awesome! Here are some ideas to get your creative juices flowing:

• Send a video file to multiple browser in real time for perfectly synchronized movie watching.
• Send a webcam on an embedded device to your browser with no additional server required!
• Securely send data between two servers, without using pub/sub.
• Record your webcam and do special effects server side.
• Build a conferencing application that processes audio/video and make decisions off of it.
• Remotely control a robots and stream its cameras in realtime.

Want to learn more about WebRTC?

Join our Office Hours. Come hang out, ask questions, get help debugging and hear about the cool things being built with WebRTC. We also start every meeting with basic project planning.

Check out WebRTC for the Curious. A book about WebRTC in depth, not just about the APIs. Learn the full details of ICE, SCTP, DTLS, SRTP, and how they work together to make up the WebRTC stack.

This is also a great resource if you are trying to debug. Learn the tools of the trade and how to approach WebRTC issues.

This book is vendor agnostic and will not have any Pion specific information.

Features

PeerConnection API

• Go implementation of webrtc-pc and webrtc-stats
• DataChannels
• Send/Receive audio and video
• Renegotiation
• Plan-B and Unified Plan
• SettingEngine for Pion specific extensions

Connectivity

• Full ICE Agent
• ICE Restart
• Trickle ICE
• STUN
• TURN (UDP, TCP, DTLS and TLS)
• mDNS candidates

DataChannels

• Ordered/Unordered
• Lossy/Lossless

Media

• API with direct RTP/RTCP access
• Opus, PCM, H264, VP8 and VP9 packetizer
• API also allows developer to pass their own packetizer
• IVF, Ogg, H264 and Matroska provided for easy sending and saving
• getUserMedia implementation (Requires Cgo)
• Easy integration with x264, libvpx, GStreamer and ffmpeg.
• Simulcast
• SVC
• NACK
• Sender/Receiver Reports
• Transport Wide Congestion Control Feedback
• Bandwidth Estimation

Security

• TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 and TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA for DTLS v1.2
• SRTP_AEAD_AES_256_GCM and SRTP_AES128_CM_HMAC_SHA1_80 for SRTP
• Hardware acceleration available for GCM suites

Pure Go

• No Cgo usage
• Wide platform support
  • Windows, macOS, Linux, FreeBSD
  • iOS, Android
  • WASM see examples
  • 386, amd64, arm, mips, ppc64
• Easy to build Numbers generated on Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
  • Time to build examples/play-from-disk - 0.66s user 0.20s system 306% cpu 0.279 total
  • Time to run entire test suite - 25.60s user 9.40s system 45% cpu 1:16.69 total
• Tools to measure performance provided

Roadmap

The library is in active development, please refer to the roadmap to track our major milestones. We also maintain a list of Big Ideas these are things we want to build but don't have a clear plan or the resources yet. If you are looking to get involved this is a great place to get started! We would also love to hear your ideas! Even if you can't implement it yourself, it could inspire others.

Sponsoring

Work on Pion's congestion control and bandwidth estimation was funded through the User-Operated Internet fund, a fund established by NLnet made possible by financial support from the PKT Community/The Network Steward and stichting Technology Commons Trust.

Community

Pion has an active community on the Slack.

Follow the Pion Twitter for project updates and important WebRTC news.

We are always looking to support your projects. Please reach out if you have something to build! If you need commercial support or don't want to use public methods you can contact us at [email protected]

Contributing

Check out the contributing wiki to join the group of amazing people making this project possible

License

MIT License - see LICENSE for full text

webrtc's Issues

ICE: gather candidates per media section

According to the JSEP spec ICE candidates should be gathered per media section (track):

Each m= section, provided it is not marked as bundle-only, MUST
generate a unique set of ICE credentials and gather its own unique
set of ICE candidates. Bundle-only m= sections MUST NOT contain any
ICE credentials and MUST NOT gather any candidates.

API: Implement the Peer-to-peer Data API

Implement the Peer-to-peer Data API as documented in the spec

mobile: support compiling with go-mobile

TBD if we want to try this before porting DTLS (#20).

cc @chrisprobst @gedw99

[example]: save-to-disk problem

Wonderful Work!

I want to try the example, save to disk. But I can't connect the problem run at mac pro and the jsfiddle app.

the sdp generated by jsfiddle code is below:

v=0
o=- 1019280199810234193 3 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS 5wDoAUg4d1Kyl4H9L2jUOzepsUDnuPMgqJc9
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Wb7e
a=ice-pwd:LbjppN1g6w91Sj5dp8K5t25G
a=ice-options:trickle
a=fingerprint:sha-256 C1:75:2D:5A:80:34:CE:02:47:6C:AA:E3:D8:B6:9C:05:42:92:8B:8D:A6:72:18:27:39:C1:86:BF:4E:60:B9:B3
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:3150470829 cname:CO6FaHPJY9JOWrKp
a=ssrc:3150470829 msid:5wDoAUg4d1Kyl4H9L2jUOzepsUDnuPMgqJc9 00d1f007-6b52-4069-98d0-f0830f769aab
a=ssrc:3150470829 mslabel:5wDoAUg4d1Kyl4H9L2jUOzepsUDnuPMgqJc9
a=ssrc:3150470829 label:00d1f007-6b52-4069-98d0-f0830f769aab
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 123 127 122 125 107 108 109 124
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Wb7e
a=ice-pwd:LbjppN1g6w91Sj5dp8K5t25G
a=ice-options:trickle
a=fingerprint:sha-256 C1:75:2D:5A:80:34:CE:02:47:6C:AA:E3:D8:B6:9C:05:42:92:8B:8D:A6:72:18:27:39:C1:86:BF:4E:60:B9:B3
a=setup:actpass
a=mid:video
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:4 urn:3gpp:video-orientation
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 H264/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:123 rtx/90000
a=fmtp:123 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032
a=rtpmap:122 rtx/90000
a=fmtp:122 apt=127
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 red/90000
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:124 ulpfec/90000
a=ssrc-group:FID 1456106671 1460088707
a=ssrc:1456106671 cname:CO6FaHPJY9JOWrKp
a=ssrc:1456106671 msid:5wDoAUg4d1Kyl4H9L2jUOzepsUDnuPMgqJc9 befc3004-8e1b-45bc-843e-fa3aea93329c
a=ssrc:1456106671 mslabel:5wDoAUg4d1Kyl4H9L2jUOzepsUDnuPMgqJc9
a=ssrc:1456106671 label:befc3004-8e1b-45bc-843e-fa3aea93329c
a=ssrc:1460088707 cname:CO6FaHPJY9JOWrKp
a=ssrc:1460088707 msid:5wDoAUg4d1Kyl4H9L2jUOzepsUDnuPMgqJc9 befc3004-8e1b-45bc-843e-fa3aea93329c
a=ssrc:1460088707 mslabel:5wDoAUg4d1Kyl4H9L2jUOzepsUDnuPMgqJc9
a=ssrc:1460088707 label:befc3004-8e1b-45bc-843e-fa3aea93329c

it seems like the o=- 1019280199810234193 3 IN IP4 127.0.0.1 is not right, the ip address is empty.

Improve SDP code so that we properly generate response

Currently we statically set our supported codecs. This is incorrect.

As described in section 5.3.1 of the JSEP spec: The SDP answer should contain a MediaDescription corresponding the each MediaDescription found in the SDP offer. If the PeerConnection can't comply with the MediaDescription in the offer, the media section should be marked as rejected by setting the port in the m= line to zero.

Examples

Offer Audio/Video -> Respond Audio/Video
Offer Audio -> Respond Audio
Offer Video -> Respond Video

TODO

• Improve SDP generation to support rejected a media section
• Adapt the SDP answer generation to match the offer.

How to send and receive at the same time?

Hi, following your example I was able to receive media from the client to the server, but when I try to add another track to send media from server to client, the handler in OnTrack is not called anymore.

If I remove the call to AddTrack (that I used to add the track to send data to the client), then it works again

Support unified-plan

firefox supports unified-plan, latest chrome supports unified-plan too.

https://tools.ietf.org/html/draft-roach-mmusic-unified-plan-00

https://bugs.chromium.org/p/chromium/issues/detail?id=799030

API: Implement Session Negotiation Model

Implement Session Negotiation Model

TODO (Among others)

• Determine how to 'debounce' the NegotiationNeeded event without operations queue
• NegotiationNeeded event

Port DTLS code to Native Go (instead of OpenSSL)

This is a huge task, but if someone is up for it I won't stop them :)

When I get around to this I will assign to myself and start the long task. Also more then happy to split the task/get guidance if you are familiar with DTLS/have opinions on how the API looks but don't have time to do the coding.

Find better way to generate contributors in README.md

Currently I am manually updating them, I see other projects have a nice way to credit everyone that has helped. It would be great if someone could implement this so everyone that helps out gets credit :)

QUIC and flutter

Adding something for the roadmap.

QUIC is a good modern alternative to webrtc that is 100% written in golang.
It what Google used for their Allo video calling app.

https://github.com/lucas-clemente/quic-go

Flutter is a way to build mobile apps GUI.
It has the ability to interoperable between flutter ( dart ) and golang using its platform channels. It's quite easy.

https://www.google.de/url?sa=t&source=web&rct=j&url=https://sites.google.com/a/athaydes.com/renato-athaydes/posts/buildingamobilefrontendforagoapplicationusingflutter&ved=2ahUKEwiv6Jyjq9vbAhVOL1AKHfFoClQQFjAAegQIABAB&usg=AOvVaw3SDogyzDpC13gcMYbgLe0f

So you could use it for P2P data and video !

Just putting here because I reckon it's very doable with all the other bits already in this GitHub org like stun and turn.

Would be happy to team up on this

Implement full ICE

Currently we only implement ice-lite we need to finish this so we can operate with other peers that are ice-lite only.

• We should be able to respond to DTLS handshakes (currently we always generate them)
• We should generate the STUN requests to check for ICE connectivity (instead of only responding to them)
• Accept the first connected peer

---

The full RFC is here you don't have to implement it fully, just enough so we have an MVP. Then we can keep improving it. It is a big RFC

Support sending H264

I do not understand the idea of ​​the project a little

I have rtp or raw stream from rtsp ip camera
I can send my rtp or raw h264 or rtsp stream to browser over webrtc?

I can write examples that will get the rtp video stream but do not know how to give it to the browser. Does your project plan such a decision?

[suggestion] some WebRTC knowledge for every issue or a overview introduction

Hi, @Sean-Der, Thanks For U Great Work! We have look for a Go native WebRTC implement long time.

I have see the issues you created for improving this repo. I hope to contribute some code and want to learn some protocol, but i don't know how to start up for there are series of protocols.

So i suggest to have some basic knowledge for current issues you have created, or a overview intruduction of protocols. Even recommend some blogs or posts relative is fine.

Implement Datagram Transport Layer Security (DTLS)

Abstract:
The DTLS protocol provides communications privacy for datagram protocols. The protocol allows client/server applications to communicate in a way that is designed to prevent eavesdropping, tampering, or message forgery. The DTLS protocol is based on the Transport Layer Security (TLS) protocol and provides equivalent security guarantees. Datagram semantics of the underlying transport are preserved by the DTLS protocol.

https://tools.ietf.org/html/rfc4347

https://github.com/xhs/librtcdc/blob/master/src/dtls.h
https://github.com/xhs/librtcdc/blob/master/src/dtls.c

API: Implement the RTCRtpReceiver

As described in the WebRTC API

Investigate why Chrome doesn't send ICE requests when there is only one candidate

Currently if Chrome gets a list of ICE candidates with only one it will not send any requests.

For now if we have only one interface we just duplicate the list, but this needs to be properly fixed/investigated.

media engine: generate payload types

When creating a codec, dynamically generate a payload type in the range 96-127 if one wasn't provided. The generation is described in RFC3551

Mobiles

So can I run this on a mobile ? I guess not since it wants gstreaner for the real time transcode ?

Web client

https://github.com/zserge/webview

It's a fantastic golang based webview for all desktops.
Very very light and we'll tested.

You can bundle any client side golang code running outside the webview too.

Quick start example:
https://github.com/zserge/webview/tree/master/examples/todo-go

On windows it uses the built in browser ( shtml or something) not the edge browser.

If you need edge browser let me know..

Move Github/CI activity to public Slack channel

Use gometalinter instead of using individual tools

Right now we manually use tooling, switch to Gometalinter (and use the defaults as much as possible)

I am honestly ok with the build breaking as upstream changes things, as long as our code is clean/idiomatic as possible

send/receive natively. CreateOffer TODO

Thanks for making this project.

I want to send / receive through datachannel natively. Seems like CreateOffer is TODO at master branch. So wonder if you have any plan to support?

Separate the stack into reusable modules

I've added the following overview of the WebRTC stack to the wiki:

Most of these components deserve their own package (or even repository). This issue can be used to track the progress to that long term goal. Please use the Golang Slack #pion channel or mailing list for long form discussions.

Implement Data Channel Establishment Protocol (DCEP)

Abstract:
Simple protocol for establishing symmetric Data Channels between the peers. It uses a two way handshake and allows sending of user data without waiting for the handshake to complete.

https://tools.ietf.org/html/draft-ietf-rtcweb-data-protocol-09

https://github.com/xhs/librtcdc/blob/master/src/dcep.c
https://github.com/xhs/librtcdc/blob/master/src/dcep.h

Run all examples with -race, add locks to RTCDataChannel

Implement SRTP

Implement Interactive Connectivity Establishment (ICE)

Abstract:

A protocol for Network Address Translator (NAT) traversal for UDP-based multimedia sessions established with the offer/answer model. This protocol is called Interactive Connectivity Establishment (ICE). ICE makes use of the Session Traversal Utilities for NAT (STUN) protocol and its extension, Traversal Using Relay NAT (TURN). ICE can be used by any protocol utilizing the offer/answer model, such as the Session Initiation Protocol (SIP).

https://tools.ietf.org/html/rfc5245

https://github.com/xhs/librtcdc/blob/master/src/ice.h
https://github.com/xhs/librtcdc/blob/master/src/ice.c

Implement WebRTC Data Channels (RTCDC)

Abstract:
Direct interactive rich communication using audio, video, and data between two peers' web-browsers. This document specifies the non-media data transport aspects of the WebRTC framework. It provides an architectural overview of how the Stream Control Transmission Protocol (SCTP) is used in the WebRTC context as a generic transport service allowing WEB-browsers to exchange generic data from peer to peer.

https://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-13

https://github.com/xhs/librtcdc/blob/master/src/rtcdc.h
https://github.com/xhs/librtcdc/blob/master/src/rtcdc.c

Implement Session Description Protocol (SDP)

Abstract:
SDP is intended for describing multimedia sessions for the purposes of session announcement, session invitation, and other forms of multimedia session initiation.

https://tools.ietf.org/html/rfc4566

https://github.com/xhs/librtcdc/blob/master/src/sdp.h
https://github.com/xhs/librtcdc/blob/master/src/sdp.c

Implement Stream Control Transmission Protocol (SCTP)

Abstract:
SCTP is a reliable transport protocol operating on top of a connectionless packet network such as IP

https://tools.ietf.org/html/rfc4960

https://github.com/xhs/librtcdc/blob/master/src/sctp.h
https://github.com/xhs/librtcdc/blob/master/src/sctp.c

The initial implementation should use the kernel-mode driver for this (limiting portability for the short-term).

examples: exchange entire RTCSessionDescription

Currently we only exchange the SDP between the browser (JSFiddle) and the example, hard-coding the session description type on both sides. This should be changed so the entire RTCSessionDescription is exchanged. In addition, all examples should be adapted so the offer/answer direction can be changed.

Update: The pion-to-pion example exchanges the entire RTCSessionDescription in json format. This can be used as an example.

TODO

• Adapt gstreamer-receive
• Adapt gstreamer-send
• Adapt save-to-disk
• Adapt data-channels

examples: Create a pions to pions example

Right now we only have examples of pions connecting to a browser. It would be nice to have an example of two pions clients connecting to each other.

Firefox Support

Firefox is using sdparta_N in their SDP offers as a bundle & mid value instead of the hardcoded 'data' value.
Additionally Firefox only supports a max 62bit value as a sessionID, currently the 64bit value in the answer causes Firefox to error during SDP answer parsing.

community: related projects

This ticket tracks projects related to pions. These can be projects using pions or projects we want to/are contributing WebRTC support to.

Media API

• vvspacetime/webrtc
• maxhawkins/asciirtc (incubated)

DataChannel

• go-libp2p (issue) [WIP]
• WebTorrent support for anacrolix/torrent (issue)
• CoreOS Sean-Der reached out but didn't hear. Someone should email the dev list to see if there is still interest maybe? project
• LivePeer (issue, issue)
• decentraland/communications-server-go
• nobonobo/ssh-p2p
• cretz/webrtc-ipfs-signaling

Unknown

• 0xProject/webrtc

Other language implementations

• JS/Node: nodertc/nodertc
• Python: aiortc/aiortc
• Erlang (no link yet)

Organize TODO

• https://github.com/Antonito/gfile
• https://github.com/pion/asciirtc
• https://github.com/maxmcd/webtty
• https://github.com/rtctunnel/rtctunnel
• https://github.com/nobonobo/ssh-p2p
• https://github.com/mxseba/rtc-ssh
• https://github.com/poi5305/go-yuv2webRTC
• https://github.com/rian-hotate/tetoris
• https://github.com/schollz/webrtc-transfer
• https://github.com/machinesworking/myhome
• https://github.com/wwl33695/myrtc
• https://github.com/takutakahashi/wg-connect
• https://github.com/open-domophone/soft-omega2
• https://github.com/mtojek/filegate
• https://github.com/porjo/babelcast
• https://github.com/porjo/mumblet
• https://github.com/cretz/myscreen.live
• https://github.com/deepch/RTSPtoWebRTC
• https://github.com/hoernschen/goat
• https://github.com/discordianfish/infisk8-server
• https://github.com/edwintcloud/classmate
• https://github.com/nacgarg/stereo
• https://github.com/mtharp/gunk
• https://github.com/bashawhm/TheHarmonicInternetNoiseGiver
• https://github.com/s8sg/satellite

This issue can be used for tracking. Please use the Golang Slack #pion channel or mailing list for discussions.

net: build net interfaces around datachannels

Build net interfaces around datachannels. These will significantly simplify building higher level go apps.

• Determine where to put them (package/repository)
• net.Addr
• net.Conn
• net.Listener
• Build corresponding Javascript library.

SDP code should be able to generate offers, not only answers

Currently we only generate answers.

The SDP code in general needs a good review. Someone should look at the WebRTC API and make sure we match the Javascript API.

TODO

• Implement basic offer generation
• Create an example of offer generation

Project is too good

You're going to make competitors feel bad about themselves. Try making this project a little less good.

Roadmap

This is a covering ticket for the Pion WebRTC roadmap. If you have any requests feel free to leave them here/vote on others.

Planet Strike -- 1.0.0 (2018-06-20)

• SRTP (in Go)
• ICE-lite (We only respond to binding requests, requires host candidates. No STUN or TURN)
• DTLS (In C, using OpenSSL)
• API that allows users to receive media that closely matches the Javascript WebRTC API
• API that allows users to send media that closely matches the Javascript WebRTC API

Aliens of Gold -- 1.1.0 (2018-09-00)

• DataChannels
• STUN
• Full ICE (currently ice-lite only)
• Bundling

Quest for the Sigil -- 1.2.0 (2018-12-00)

• Port DTLS to Go from cgo (currently using OpenSSL)
• Mobile Support
• Transport Refactor

Spear of Destiny -- 2.0.0 (2019-04-00)

• ORTC
• Unified Plan
• DataChannel improvements (configurable reliability)
• Logging and ICE debugging
• Experimental QUIC support
• Experimental WebAssembly (WASM) support
• API Cleanup to better match the WebRTC Spec.
• Reliability improvements to SCTP and therefore Data Channels.
• ICE Regular Nomination

Beyond Heretic -- 2.1.0 (2019-08-00)

• TURN
• Trickle ICE
• mDNS Candidates

The HUNT Begins - 2.2.0 (2019-11-00)

• SCTP Performance Improvements (16x in some cases!)
• Extend SettingEngine to support SFU use cases
• TCP TURN Support
• PCM Support
• Add/Remove tracks at runtime
• IVF Reader to enable saving to disk

Future Shock - 3.0.0 (2020-12-14)

• ICE Restarts
• ICE TCP Candidates
• Remove non-Trickle ICE. Provide utility that emulates old behavior.
• SRTP AEAD_AES_128_GCM. Significant performance improvement in media crypto!
• Simulcast.
• Improved Media API so user doesn't need to managed PayloadType and SSRC
• Interceptor API that allows users to implement NACK, FEC and Congestion Control via Public API

Plutonia - 3.1.0 (2021-08-01)

• Transport Wide Congestion Control Interceptor, better Congestion Control then Sender+Receiver Reports
• H265 Packetizer
• FireFox Simulcast Support
• Sender/Recevier Report interceptors, can be used in pion/webrtc or outside
• UDPMux, Multiple PeerConnections can be served via the same UDP Port

System Shock 3.2.0 (2023-04-25)

• Congestion Control Interceptor
• AV1 Support
• Simulcast Sender
• Stats Interceptor

4.0.0 (No Date Picket)

• Active ICE TCP
• JitterBuffer Interceptor

No date picked

• Serialize/Deserialize PeerConnection API
• DTLS Restart
• Performance/Allocation testing to GitHub Actions
• Auto Change log generation/Remove manual process of releasing
• Automate fuzz testing, submit Pion to oss-fuzz
• H264 Interceptor (Analyze and attempt to fix common issues like no SPS/PPS with every IDR)
• Harden security (fuzzing and other automated tooling)
• TinyGo support. Pion for IoT devices

---

These things are ordered in the importance we have gotten from feedback. If you are building something and need a feature sooner please comment and we can move things around!

Allow user to pass supported codecs

I don't know exactly what this will look like API wise but we should allow users to specify what codecs they support. This is needed to support cases where user doesn't support all codecs (has hardware decoder for only some codecs)

• Update RTCPeerConnection to allow user to pass supported codecs
• Update SDP generation to take this into account. Also error if user creates a track for a codec we don't support.

media: support sending MediaStreams with multiple tracks

Currently the media stream labels are hardcoded in the SDP offer and answer generation. The RTCPeerConnection.AddTrack should be extended to support adding media stream labels.

TODO

• Add a variadic streams argument to the AddTrack method as specified in the RTP Media API
• Store the media stream ids on the RTCTrack object as specified here
• Add the media streams to the SDP offer and answer. Some of the required logic already exists. However, the stream label is currently hardcoded to the track label. The procedure is specified in the JSEP spec

Can we have an example for attach current stream to other webRTC

Hi Pions

could you create an example show how to attach a stream track (track *webrtc.RTCTrack) from webRTC call A to another WebRTC call B and send it to B browser ?

• Plus could we have sample to push stream track to another RTMP server :)

Best regards,
Toan

Implement better SDP builder

SDP construction is not great currently. It is a loose collection of SDP methods. https://github.com/pions/webrtc/blob/master/internal/sdp/util.go#L26

Implement a Go idiomatic way so I can build an SDP given things like

IceUsername, IcePassword, Fingerprint string
Candidates []string
Tracks []*SessionBuilderTrack

Implement DataChannels

This is the big covering ticket, will include links to RFC/design docs as needed.

As far as pion-WebRTC is concerned we need to start with a SCTP library to parse/create packets

Sending to the browser a RTP Stream source

Hi there, and thanks for the excellent work!
So far, this project looks very promissing!

I am wondering if there is anything on works to get a generic RTP source to the browser. It would be similar to the gstreamer example, but reading the media from an RTP source which could be the output of gstreamer itself, ffmpeg or vlc.

If there is none, in about a month I should be able to contribute a bit, if some tips out given on how to get that working!

Implement STUN

Currently pion-WebRTC only generates host candidates. Adding STUN shouldn't be too difficult and mostly just involves modifying SDP generation and a basic STUN client.

• Add a way for a user to pass a STUN server to pion-WebRTC add a new argument to RTCPeerConnection to match Javascript.

• Communicate with the STUN servers to get our public IP see https://github.com/pions/turn/tree/master/internal/client and how it communicates with a STUN server and parses the response to the binding request https://github.com/pions/turn/blob/master/internal/server/stun.go#L7-L17

• Start listening on a local port, but return the public IP in the candidate https://github.com/pions/webrtc/blob/master/rtcpeerconnection.go#L107 instead of a host address

Issue with profile-level-id for h264 on chome android

Current profile-level-id doesn't work for chrome android.
With current implementation video playing does not start on such devices.
If works fine if it's replaced from "42001f" to "42e01f"

Read example jsFiddles from GitHub

According to this documentation jsFiddles can be backed by GitHub. This would allow for easier contributions.

testing: integration testing

Automate integration tests between pions and web browsers. Example setup: Use a headless browser in combination with a simple web server to serve a test web page and exchange the SDP offer/answer. Validate the success using a browser API like the Google Chrome Remote API. Finally make the test part of the CI.

Fix panic in chunk_select_ack unmarshal

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x650d0d]
goroutine 22 [running]:
github.com/pions/webrtc/internal/sctp.(*chunkSelectiveAck).unmarshal(0xc420119aa0, 0xc42021992c, 0x14, 0x14, 0x0, 0x20)
    /root/go/src/github.com/pions/webrtc/internal/sctp/chunk_selective_ack.go:85 +0x28d
github.com/pions/webrtc/internal/sctp.(*packet).unmarshal(0xc420053dc0, 0xc420219920, 0x20, 0x20, 0xc420053de0, 0x0)
    /root/go/src/github.com/pions/webrtc/internal/sctp/packet.go:93 +0x214
github.com/pions/webrtc/internal/sctp.(*Association).HandleInbound(0xc420118180, 0xc420219920, 0x20, 0x20, 0x7fd4e0003c40, 0x7fd4e000a1c0)
    /root/go/src/github.com/pions/webrtc/internal/sctp/association.go:114 +0x75
github.com/pions/webrtc/internal/network.(*port).handleSCTP(0xc42009a740, 0xc420219920, 0x20, 0x20, 0xc420118180)
    /root/go/src/github.com/pions/webrtc/internal/network/port-receive.go:119 +0x4d
github.com/pions/webrtc/internal/network.(*port).handleDTLS(0xc42009a740, 0xc420016c40, 0x3d, 0x3d, 0xc420181a10)
    /root/go/src/github.com/pions/webrtc/internal/network/port-receive.go:126 +0x188
github.com/pions/webrtc/internal/network.(*port).networkLoop(0xc42009a740)
    /root/go/src/github.com/pions/webrtc/internal/network/port-receive.go:190 +0x3d5
created by github.com/pions/webrtc/internal/network.newPort
    /root/go/src/github.com/pions/webrtc/internal/network/port.go:47 +0x199
exit status 2