random video interrupts about aiortc

[JRafiei]

Yes, sure!
After testing with the latest version, the video quality has increased significantly and it is more stable. Sometimes the video get pixelated and stopped for a short time but it resumed to the normal state after that. Thanks a lot for your efforts.
during testing the video call we get two new errors that may be informative.

Task exception was never retrieved
future: <Task finished coro=<RTCDtlsTransport.__run() done, defined at /usr/local/lib/python3.6/site-packages/aiortc-0.9.1-py3.6-linux-x86_64.egg/aiortc/rtcdtlstransport.py:365> exception=Error('replay check failed (bad index)',)>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/aiortc-0.9.1-py3.6-linux-x86_64.egg/aiortc/rtcdtlstransport.py", line 368, in __run
await self._recv_next()
File "/usr/local/lib/python3.6/site-packages/aiortc-0.9.1-py3.6-linux-x86_64.egg/aiortc/rtcdtlstransport.py", line 427, in _recv_next
data = self._rx_srtp.unprotect(data)
File "/usr/local/lib/python3.6/site-packages/pylibsrtp/__init__.py", line 178, in unprotect
return self.__process(packet, lib.srtp_unprotect)
File "/usr/local/lib/python3.6/site-packages/pylibsrtp/__init__.py", line 198, in __process
_srtp_assert(func(self._srtp[0], self._cdata, len_p))
File "/usr/local/lib/python3.6/site-packages/pylibsrtp/__init__.py", line 52, in _srtp_assert
raise Error(ERRORS[rc])
pylibsrtp.Error: replay check failed (bad index)

and

Exception in callback Transaction.__retry()
handle: <TimerHandle when=1566464.094515949 Transaction.__retry()>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/asyncio/events.py", line 127, in _run
self._callback(*self._args)
File "/usr/local/lib/python3.6/site-packages/aioice-0.6.2-py3.6.egg/aioice/stun.py", line 257, in __retry
self.__future.set_exception(exceptions.TransactionTimeout())
asyncio.base_futures.InvalidStateError: invalid state

After these errors, video playback completely stops and the ice connection become disconnected and then failed.

[jlaine]

Looks like it might be a duplicate of #26

[JRafiei]

Yeah you are right. Actually I was reffering to this issue in the first place. But I can't understand the rationale behind it. Is it safe to apply those changes? By the way could you direct me to the neccesary informations about creating video frames from the incoming packets?

[jlaine]

If you re-read the code, I don't believe we are "discarding" any frames. We:

• run over the chunks present in the jitter buffer to determine if we have a complete frame (one video frame may span several RTP packet), and if so we set got_frame to True

• if we did receive a complete frame, we remove it from the jitter buffer and pass it to the codec for decoding

• otherwise we need to wait for the missing packets to arrive

What kind of resolution are you using? If you are having issues with the jitter buffer capacity I'm guessing you have very large images. One interesting metric would be to print count in the if got_frame branch.

[bl1nder]

It seems i have something similar . Based on your server example I'm streaming video from browser to server. on low resolution it works fine. After setup constraints to 1280 x 720 , framerate:5. After received about 50 frames server stuck. Server host machine profiling with tcpdump shows packets which still coming to server.
See code below

 @pc.on('track')
    def on_track(track):
        if track.kind == 'video':            
            pc._consumers.append(asyncio.ensure_future(consume_video(track)))

async def consume_video(track):
    
    Drain incoming video, and echo it back.
    logging.error('consume_video')
    count = 0
    while True:

        logging.error('start loop')
        frame_remote = await track.recv()
        count += 1
        logging.error(count)
        logging.error(datetime.datetime.now().time())

Could you help me with this?(add some thoughts any help) I'm beginners with python. Thanks in advance

[jlaine]

Could you please try running the example in verbose mode (python examples/server/server.py -v), and paste some of the output (especially towards the end)?

[jlaine]

@bl1nder I believe that's a different issue: I noticed that when you request a 1280x720 resolution from getUserMedia, Chrome will initially send the VP8 video out at a lower resolution (for instance 640x360), then progressively increase the resolution. Decoding works fine, but echoing these frames back failed when the resolution changed as the encoder needs to be torn down / recreated.

That should be fixed by ad0dadf

[JRafiei]

Thanks jlane for explaing the code and sorry for late response. I checked the parameters that you mentioned and here are the results:
I printed the Video frames size and they are 480x640. (portrait)
The count is 1 or 2 most of the times but occasionally it becomes higher values like 15 or 19.
By the way we changed the buffer capacity to 128 and MAX_DROPOUT to 500 to reduce the freeze time. And it seems we are getting better results. Do you think these are resonable changes?

[jlaine]

Could you please just try the latest code from git? It uses a buffer capacity of 128, and MAX_DROPOUT is gone.

[bl1nder]

Hi i had the same error as @JRafiei mentioned. @JRafiei Did you handle this type replay check failed (bad index) of error? Did add some reconnect code for this case? it's hard to reproduce. Now it work fine for me, but i'm not sure on 100% this sutiation will not happen again. Please share any info about this. thanks in advance.

[jlaine]

Ok can we please open separate tickets to track the new issues?