Get Raw Audio from reat-native-webrtc tracks. about react-native-webrtc

[saghul]

It's not currently possible to provide that.

Given how the RN JS bridge works, it would have abysmal performance even if we could get the raw audio frames to send them to userspace.

Something like Reanimated's worklets might work, but the complexity of that system would make it hard to integrate.

[saghul]

You can't opus decode a track. A track is just a JS object. Audio data is a few layers down and not accessible.

Using react I would've possible, not because of react itself, but because when running in an actual browser you have the APIs you need to what you want, such as stream transforms.

[8BallBomBom]

Totally looking at the other end of things, into the future.
Conversion to the new RN architecture might mean implementing things such as this would be more viable without much of the bumpy performance issues 🤔 would need some looking into though.

[saghul]

Unfortunately that is not possible.

You might try sending audio frames via a datachannel, but due to the way the RN bridge works binary data needs to be base64 encoded so I think that overhead will add up.

[sharjeelBokhari1]

Thank you for your help @saghul

[sharjeelBokhari1]

Oh okay.

Question 1:

Could you please tell me what would happen if i tried to apply opus decoding on the tracks? Since they give me back something like this when I console.log them:

Track 


 {"_constraints": {}, "_enabled": true, "_muted": false, "_peerConnectionId": 0, "_readyState": "live", "_settings": {}, "id": "aafa8615-c9f9-4aca-8d3e-5f73f6330773", "kind": "audio", "label": "", "remote": true}

What would happen if I applied opus decoding to this particular Track.

Question 2:

Would it be possible to achieve what I am doing if I used React JS instead of React Native.

I really appreciate your help man @saghul.

[sharjeelBokhari1]

@saghul Alright but is there a way I can make an audio, being streamed/recorded from a different library such as "react-native-audio", a MediaStream track? I mean I don't want to use the getUserMedia tracks I want to send the audio using webrtc but it isn't done using the getUserMedia, since getUserMedia is the one that accesses the microphone and other media devices if im not wrong. I want to use audio from another api and i want that to be heard to the remote/local peer using react-native-webrtc

Thank you so much for cooperating! Its very helpful!