Not functioning on iPad or iPhone about jacktrip-webrtc

[rezonant]

I think we have two issues: 1. Lack of support for AudioWorklet in iOS 2. Something not correct in the WebRTC implementation. If I try https://test.webrtc.org on iOS devices, it works fine but the jacktrip-webrtc does not. I am STILL (!!!!) trying to debug the iOS version but there is a lack of console tools on iOS ☹ David

If you happen to have a Mac, you can connect a remote DevTools running in Safari on the Mac to your iOS device. The iOS device must be connected via USB and there are some other steps to enable the functionality. Here's a guide that may help: https://medium.com/better-programming/debugging-your-iphone-mobile-web-app-using-safari-development-tools-71240657c487

Apparently it's also possible to use Chrome's DevTools if needed, here's an article about that: https://medium.com/@nikoloza/how-to-debug-remote-ios-device-using-chrome-devtools-f44d697003a7

Best of luck!

[matteosacchetto]

Hi David,
thanks for checking out the application.

So, I am not very familiar with the Apple ecosystem, but, if I am correct, on both iPads and iPhones the default browser is Safari.

Unluckily, as reported by caniuse: https://caniuse.com/?search=AudioWorklet, AudioWorklets, which is the technology which allows us to extract uncompressed audio from MediaStreams, is not yet supported on Safari (and I do not know if they will ever be supported)

I don't know if it is possible on iPads and iPhone, but if you can, try again using a different browser. You can try with either Mozilla Firefox or any Chromim based web browser (both do support AudioWorklets) and see if it is working or not

Cheers,
Matteo

[dlake02]

Hi Matteo Thank you but I think it is probably something to do with iOS rather than the browser. I've tried with Chrome, Opera, Firefox and Safari and none of them work. Safari on MacOS doesn't work but Chrome on MacOS does. WebRTC works on Safari on iOS - I use MeetEcho and Jitsi quite regularly - so I wonder if there is another problem? Safari support WebRTC (as of October 2020) so it isn't the basic framework. The interesting thing is that when I go to a room I can join the meeting but I never see the request to use camera and microphone on any iOS based browser. I can across this posting: https://webrtchacks.com/guide-to-safari-webrtc/ So it may not JUST be AudioWorklet - it may be the WebRTC as well! David

[matteosacchetto]

Hi David,
well... It may be something related to both AudioWorklets and WebRTC.

Unfortunately I do not have any iOS device I can test this on, so I need your help to debug this.

If you can access the developer console, can you post a screenshot of it? It may report some useful information about what it is not working

[matteosacchetto]

Additionally, JackTrip-WebRTC has also a WebRTC-only mode, which in order to be activated requires you to modify a line of code...

If you want you can test if the WebRTC-only mode works.

To activate it you need to modify line 6 of client/public/js/room/client.js changing

const USE_MEDIA_AUDIO = false;

to

const USE_MEDIA_AUDIO = true;

Cheers,
Matteo

[dlake02]

Thanks! I've not done any debug with iOS before but I'll have a look when I can in the next few days. I've tried with the USE_MEDIA_AUDIO set to true and the first page fails to load - I see a repeated "warning" error message in the URL box but it is too quick to read. I'll try and setup a debug session to Safari - this is not something I've done before so it will be new for me to learn! David

[matteosacchetto]

I've tried with the USE_MEDIA_AUDIO set to true and the first page fails to load - I see a repeated "warning" error message in the URL box but it is too quick to read.

Ok, that's kinda strange...

I've not done any debug with iOS before but I'll have a look when I can in the next few days.

Ok, thank you :)

I'll try and setup a debug session to Safari - this is not something I've done before so it will be new for me to learn!

If you can, try (also) with Chrome or Firefox, since those should support AudioWorklets.
During the development I tested this web app with Chrome on an Android smartphone and everything was working

Do you connect to the http or https web server?

Matteo

[dlake02]

It's the same problem with Chrome, Firefox and Opera (and the same failure mode which makes me think it is iOS rather than the browser at the moment). I've also tested WebRTC with https://test.webrtc.org and using Safari on iOS and that does work. There is a big difference though - as the site loads I see a request to use camera and microphone. I never see these in jacktrip-webrtc. I've just retested with HTTP but with just HTTP my PC also doesn't ask for camera/microphone access (Chrome and Firefox). With both the false and true setting I see the same problem - I can create a room but permission to use the camera and microphone is never asked for on iOS devices but it is on Windows/MacOS. I'm going to try and run some debug session on Safari on iOS and see what the differences are between test.webrtc.org and JackTrip. David

[matteosacchetto]

I've just retested with HTTP but with just HTTP my PC also doesn't ask for camera/microphone access (Chrome and Firefox).

If you connect from a different device from where the web server is launched you need to use HTTPS. If the connection is not HTTPS then the getUserMedia function will not work. If i recall correctly, in Android, in order to access mic and camera you need to be on HTTPS. I guess the same is for iOS

If you can, try (also) with Chrome or Firefox, since those should support AudioWorklets.

What I mean is that while you do a debug session on Safari, please do also one debug session on Chrome or Firefox, to see if there are the same errors

[dlake02]

Complete failure here I'm afraid. iOS 14 has locked-down the amount of debug you can do so I'm REALLY struggling to get anything for you. I'll keep trying but it's not straightforward.... David

[matteosacchetto]

Hi David,
well that's unfortunate...

In the meantime, have you tried to connect to the HTTPS version instead of the HTTP one? If not, please give it a try and let me know if over HTTPS you experience the same behaviour

Thanks,
Matteo

[dlake02]

Hi Yes - the HTTPS connection behaves in the same way on Opera, Safari, Firefox and Edge on iOS. David

[matteosacchetto]

Hi David,
Maybe this can help you with the debugging: https://stackoverflow.com/questions/38179396/is-there-any-way-to-debug-chrome-in-any-ios-device

It seems that you can access Chrome console directly from the iOS device by accessing chrome://inspect

Hope it helps 😃

Cheers,
Matteo

[dlake02]

Hi Thanks – yes, this worked until iOS 14 but now no longer works ☹ Trying to find a solution ….. David

[matteosacchetto]

this worked until iOS 14 but now no longer works frowning_face

That's unlucky... ☹️

[capnmidnight]

Chrome and Firefox on iOS are just UI and user data sync service wrappers around a hobbled version of Safari. Apple doesn't allow any browser engine other that Safari on iOS, so Google and Mozilla have opted to at least have a branding foothold there. Apple also imposes some limitations to the Web APIs that third party browser apps on iOS have access to, so non-Safari browsers on iOS will always have strictly inferior Web API support.

[omarcostahamido]

oh wow @capnmidnight I did not know that. That is terrible! ha!

[matteosacchetto]

Hi @capnmidnight,
Thank you very much for the clarification! I actually didn't know that.

So, since, from what I understood, they are kind of based on Safari, and since, as reported previously, https://caniuse.com/?search=AudioWorklet, Safari does not support AudioWorklets, I guess that in order to have this application work on iOS, we have to wait that Safari support AudioWorklets (if it will ever support them...)

It is kind of strange though that the implementation which uses only WebRTC is not working as well... May be due to some kind of missing elements which I am currently not aware of. Could be interesting to see a screenshot of the browser console on iOS (if it is something possible)

Thank you again 😃

[dlake02]

I think we have two issues: 1. Lack of support for AudioWorklet in iOS 2. Something not correct in the WebRTC implementation. If I try https://test.webrtc.org on iOS devices, it works fine but the jacktrip-webrtc does not. I am STILL (!!!!) trying to debug the iOS version but there is a lack of console tools on iOS ☹ David

[matteosacchetto]

I am STILL (!!!!) trying to debug the iOS version but there is a lack of console tools on iOS frowning_face

Yeah, no worries! Try and if you find something useful I will be pleased to have a look at it 😃
Thank you @dlake02

[matteosacchetto]

Hi @rezonant,
thank you for the help 😃

[gristow]

A much delayed response, but I just happened to come across this issue. It sounds like, for the way worklets are being used, it may be possible to use a workaround:

Safari on iOS and macOS does implement the WebAudio API script processor node which also allows you to extract uncompressed audio data. While it's deprecated and runs on-thread, it's a good fallback for browsers that don't yet have web audio worklets available. And, if all you're doing is extracting the raw data and sending it, the hit for running on-thread should be minimal.

[matteosacchetto]

I recently discovered that Safari is supporting AudioWorklets now

As reported by can i use, Safari on iOS and macOS it seems that it does now, or at least they are adding support for them (🎆 🎉 ), so I guess we should check this again