Worker hits ASSERT condition in production about mediasoup

[ibc]

Honestly there is no change since 3.12.12 that could have fix this issue. Let me reopen to not forget. I think we need a fuzzer test for this class.

[jmillan]

@gitamirp,

• Which mediasoup version are you using?
• Can you repro? can you provide an STR?
• Please provide as much context info as you can.

[gitamirp]

@jmillan thanks for looking into it.
We are using version 3.12.12.
Unfortunately, it happens in a production environment therefore its harder to get detailed logs or STR for it.
It rarely happens but I can try to get some more info around it.

[nazar-pc]

You should upgrade to latest release first, 3.12.12 is very old and the issue you are hitting might have been long fixed

[jmillan]

Yes please, you're using quite an old version. Upgrade it as soon as you can.

Don't hesitate to reopen the issue if it comes out in the future.

[gitamirp]

thanks, fwiw there were no significant changes to RateCalculator.cpp over the last 2 years.

https://github.com/versatica/mediasoup/commits/v3/worker/src/RTC/RateCalculator.cpp

I suspect it might occur when

https://github.com/versatica/mediasoup/blob/v3/worker/src/RTC/RateCalculator.cpp#L27

nowMs - this->newestItemStartTime >= this->itemSizeMs

and

https://github.com/versatica/mediasoup/blob/v3/worker/src/RTC/RateCalculator.cpp#L32

this->newestItemIndex >= this->windowItems

and

this->oldestItemIndex == 0

probably just logging an error and resting the RateCalculator will be more appropriate error handling

[ibc]

probably just logging an error and resting the RateCalculator will be more appropriate error handling

Defensive programming hides real bugs. We don't do that. If something should never happen then it must never happen, otherwise it's a bug that needs to be fixed.

[nazar-pc]

If there was a memory corruption somewhere, side-effects might be unexpected. This needs to be confirmed on latest version first.

[gitamirp]

probably just logging an error and resting the RateCalculator will be more appropriate error handling

Defensive programming hides real bugs. We don't do that. If something should never happen then it must never happen, otherwise it's a bug that needs to be fixed.

also true

[gitamirp]

If there was a memory corruption somewhere, side-effects might be unexpected. This needs to be confirmed on latest version first.

I am not sure how fast we can get these hosts on latest release. for now I will just patch it to suit our needs.

I'll let you know if I find more about this condition.

thanks

[ibc]

@gitamirp any news? I assumed you patched version 3.12.12 so it's not a problem for you anymore but perhaps you updated to latest version (without patching it)?

[gitamirp]

We patched 3.12.12 by using

MS_ASSERT( this->newestItemIndex != this->oldestItemIndex || this->oldestItemIndex == -1 || this->newestItemIndex, "newest index overlaps with the oldest one");

I will let you know if we run into this ASSERT again with this patch.
If you are interested we can add some logs when

this->newestItemIndex == this->oldestItemIndex

I believe that in 3.12.12 both newest && oldest can be zero at the same time but I don't have logs to support it.

[ibc]

Yes please. Add those logs and comment here when you get something. We will investigate this next week. Too busy these days.

[gitamirp]

Sure, we'll do. thanks