Lag can cause Audio to get behind Video about openviii-monogame

[Sebanisu]

Well I thought of one optimization while eating dinner. I bypassed a Marshal.Copy. By using a byte array to store the resampled sound data instead of a byte pointer. I didn't realize I could use fixed to point to the array.

fixed (byte* tmp = &_convertedData[0])
{
outSamples = ffmpeg.swr_convert(ResampleContext, &tmp, ResampleFrame->nb_samples, null, 0);
}

So it skips the whole convert byte* to byte[] bit. Might be able to do more of that in some other spots.

[Sebanisu]

On saturday night I did some small optimization. I minimized some coping to buffer. I put all the needed header data into a byte[]. So it's ready to go and just need to send it to ffcc. with the info of where the data is in the data file and the data file's name. On the step where it reads from the file it only reads enough to fill the buffer for ffmpeg and closes the file.

--edit--
This Introduced a crash that only happened when running through the audio channels. But it's fixed. Just had to Dispose before leaving fixed.

[Sebanisu]

So we got threading last week. And increased the buffer to 100 chunks of samples. That seemed to help. I thought of another change to make when I'm home.

I can track how many samples are sent to the audio buffer, subtract the precached buffer, divide that by sample rate. And that gives us the seconds of audio sent to buffer. I think I can use that to detect if video got ahead of the audio. And basically pause the video till audio catches up.

There might be a need to check number of chunks in the buffer. But I don't know if all chunks will always be 32 samples.

If you resize the window during playback it's an easy way to force this to happen.