owncast crashes after incoming stream about owncast

[gabek]

Not at all, I've defaulted to panics simply so it would draw attention to things that fail so they can be fixed.

[gabek]

Awesome, thank you for sending that along, it helped me figure out what's going on with creating a thumbnail from that specific segment. It tries to pull a thumbnail 1 second into a segment, and this particular segment is less than a second long. Changing it to pull from 00:00:00 fixes it:

ffmpeg -i stream-20200618-1592522663.ts -ss 00:00:00.000 -vframes 1 test.png

It would actually make the most sense to find a way to pull the last frame out of the segment instead of a specific timestamp. The last frame would be most relevant for a thumbnail, anyway. And then regardless if a segment is a minute long or less than a second long it won't care. I'm going to look into that.

Thanks again!

[gabek]

Interesting! That's a good find.

If you want to uncomment https://github.com/gabek/owncast/blob/master/thumbnailGenerator.go#L68 you can see the actual ffmpeg invocation. It should be using the most recent segment generated and build a static image out of it. If you want to send along that invocation and that .ts file that it's trying to use I can reproduce it on my side and find what it's unhappy about.

Also, as an experiment, could you try turning off passthrough in the config file? I'm curious if re-encoding the incoming stream would fix it.

[graywolf336]

stream-20200618-1592522663.ts.zip

ffmpeg -y -i webroot/hls/0/stream-20200618-1592522663.ts -ss 00:00:01.000 -vframes 1 webroot/thumbnail.png

Curiously, I noticed that I was getting a ton of dropped frames when running locally. And generally when the dropped frames went up this happened.

I turned off the passthrough and it happened a lot less but still occurred every so often.

[graywolf336]

Does the server need to panic when the thumbnail generation fails? Or can that fail silently or without panic?

[gabek]

As for thumbnail generation causing additional issues with dropped frames, I guess it could make sense, since it's doing more work at that particular point in time. Since thumbnail generation is low-priority maybe there's a way we can tell it to not work so hard. I'll look into that too.

[graywolf336]

Makes sense to me.

I keep getting it now that I've moved to a DO $5 machine. And what's really interesting is that whenever I start streaming, it will be good for a while but then it suddenly starts dropping a ton of frames. Here's a screen recording of it:

Screen Recording 2020-06-18 at 7.17.25 PM.zip

Edit: Which is exactly what you said, it starts working hard for a thumbnail. Maybe have the thumbnail generation an option to turn on/off and a static image instead?

[gabek]

I just pushed a couple thumbnail related changes:

• Use JPG instead of PNG since it seems it's a little faster.
• Limit ffmpeg to using 1 thread.
• Pull the first frame out of the segment instead of a hardcoded location to fix the original crash. I'd still like to pull the last frame, but that requires seeking, and would take longer.

ffmpeg -hide_banner -threads 1 -y -t 1 -i stream-20200618-1592522663.ts -f     0.09s user 0.02s system 97% cpu 0.110 total

ffmpeg -hide_banner -y -t 1 -i stream-20200618-1592522663.ts -f image2  1   0.18s user 0.05s system 161% cpu 0.140 total

It takes longer, but takes less CPU to do so.

Hopefully these changes make some difference. I agree that turning it off completely for people is a valid option, as well as a customizable time (it's hardcoded for every 20 seconds now), but I'd like to see if these changes help.

Let me know if these help!

[graywolf336]

So far locally, it is working amazing! 👏

However, my DO droplet isn't doing so hot. Both OBS and ProPresenter show they are working fine for about six seconds but then they drop down to under 100 bits per second. So, I don't know if it is the droplet, my internet, or owncast :/ I have poor upload speed, so I'm going to try again tomorrow when I'm somewhere that has ample upload speed.

Will close this issue as it is fixed, so far, and the above paragraph is about something else (seemingly).

[gabek]

Awesome, every little bit helps. Thanks so much for helping with this. I look forward to continue to squeeze more performance out of things so it can work well on a $5 machine. I'm running it under a Linode $5 machine and while it runs, it certainly pegs things. Let me know if there's any other performance increases you can think of when trying it out in your particular environments.