Comments (15)
I had to move away from espressif's streaming implementation completely because their code uses "chunked" stream which VLC and Blynk video players choke on. I needed something that can play on multiple clients and be compatible with VLC/Blynk.
As for use of cores - yes, and I have done some experimentation in this project:
https://www.hackster.io/anatoli-arkhipenko/minecraft-interactive-do-not-enter-sword-sign-esp32-cam-cd1b07
Also, there are multiple approaches for using RTOS tasks here:
https://github.com/arkhipenko/esp32-mjpeg-multiclient-espcam-drivers
from esp32-cam-webserver.
push, any news on this topic?
from esp32-cam-webserver.
Thanks, I didnt do a good job on merging myconfig up to the sample. Sigh... fixed.
I've very much focussed on this playing in a browser window, In keeping with it being an expanded example. The stream code is essentially untouched from the original.
I'm kind of assuming that those who need rtsp streams etc would be looking at the more advanced projects focussing on specific security and home-automation. That said; a more solid streaming part of the code is in the plan; which is why this issue is open and active 😉
from esp32-cam-webserver.
Hi All. May be an old subject but just waned to understand the idea of the multi-client behind the proposed solution a bit better.
I have refactored this whole project in PR #280 and where the frame capture is timer-based. This works pretty well and the frames could be broadcasted to all the connected clients theoretically, this is pretty easy. However, there are several questions to be understood. For example, assume we have 2 clients connected simultaneously, for the sake of simplicity. Let's consider the following scenarios:
- One has started the video stream while the other decides to take a still image. We could take one frame and send to the 2nd guy, not a problem. However, flash lamp cannot be triggered in this case, because this will affect the streaming for the 1st one.
- Should we allow 2nd guy to change any settings (camera/frame rate etc) or not? I'd say no, since the 1st client is already running the stream and any changes will affect his experience. This means that all the settings for the 2nd client should not be possible while the stream is running.
- if the 1st client has started the stream, the second should only be able to connect and see the same image/stream, which is already served for the 1st one. No settings changes should be allowed and no flash photography either.
- If the 2nd client connects to the stream started by the 1st one, then the 1st one should also be blocked from doing any changes, which may impact the experience for the 2nd guy. This means the 1st client also should not be able to change any settings until and unless he is the only one served.
- If one of the clients disconnects, and the disconnection message delivery fails for some reason then there have to be some sort of client ping/auto-cleanup on timeout, otherwise the settings will remain locked for the 1st guy even after the 2nd one is dropped.
Overall, I believe implementing this feature in the ESP32CAM sketch doesn't worth the effort, for the following reasons:
- The solution will be complex and the overall user experience will not be as nice, since parallel clients will be competing for resources of the board, which cannot be shared effectively (i.e. flash lamp, camera sensor etc).
- Feeding the camera video stream for multiple clients can be implemented on a proxy server much more efficiently and serve as many parallel clients as required, subject to proxy server capacity. In this case, the ESP CAM will have to push the frames to one client only (the proxy server). Moreover, one could also implement a video compression on the proxy so that clients would receive an MPEG stream instead of frame series - this should help on the server bandwidth requirements quite significantly.
Would be great to hear your thoughts on this.
from esp32-cam-webserver.
By co-incidence I was looking at that sketch earlier today and wondering If this sketch could do something similar.
Well.. not co-incidence, I was checking out people who follow this, and saw it listed in your recent activity ;-)
It's an excellent suggestion; but the architecture they use to serve the stream is really different to how this sketch currently works; it's not a copy/paste/modify job to replace the stream handler. And they do not use face detection, further complicating things.
I have a vague idea that the existing stream handler could be made thread-safe and multiple handlers run via RTOS, and we have 2 cores in the ESP to leverage too, the bottleneck should be WiFi not CPU.. Especially if (as I believe) face recognition can be done before the image is copied to be returned to each stream client.
I'm going to tentitively peg this to V5, which is where I intend to strip face recognition anyway, and should make this easier to implement using the code in the repo from @arkhipenko
At present V5 is pure vaporware, I'm not sure if I will ever have time to get there, but I'm very open to PR's from anybody who can get multi-client streaming running on top of the existing code architecture.
from esp32-cam-webserver.
for the moment I solved the issue for me, that I just call for a still picture every 1 sec and show it on my tablet UI. This I can do from several clients at the same time.
from esp32-cam-webserver.
I had to move away from espressif's streaming implementation completely because their code uses "chunked" stream which VLC and Blynk video players choke on. I needed something that can play on multiple clients and be compatible with VLC/Blynk.
...
As info, just for another data point, I was able to get VLC to receive and display the stream using this build. It would not work on the example webcam sketch. https://i.imgur.com/O9JHfPL.jpeg
As such, I suspect that VLC can transcode this in a multicast construct. Granted, it is not ideal, but could be a viable workaround for those with a system topology that supports doing same.
Also, @easytarget, be advised that there are a couple of typos in the myconfig.sample.h file - seems you dropped the #define on the serial debug #def (Line ~139). And, on line 26, seems to be an extra 'station' word in the structure definition construct (yes, it being a comment is acknowledged. :).'
from esp32-cam-webserver.
@easytarget are you still working on this project? Please do not let it go down....
from esp32-cam-webserver.
fyi, Tasmota added rtsp to the tasmota32-webcam.bin, so this added multiclient streaming to ESP32cam.
May be this helps you as well?
Also a nice page in Tasmota for the cam, but missing some options:
https://cgomesu.com/blog/Esp32cam-tasmota-webcam-server/
from esp32-cam-webserver.
push ;-)
from esp32-cam-webserver.
@abratchik thx for deep looking into this.
Speaking for myself I would love to see multiclient streaming from clients like tinyCam using MJPEG URL (http://:81). Currently I can only watch the stream from one client eg my tablet. A second client eg my mobile can not access the stream, unless the first client stopped the streaming. All my other webcams do support this.
I was not referring to the GUI and a second client/user is browsing the frontend and being able to change parameters in parallel.
Does this help and make sense?
from esp32-cam-webserver.
from esp32-cam-webserver.
Hi @abratchik, not sure what you mean by proxying the stream.
In my setup max 2-3 clients would call the stream in parallel. Currently one client can call the stream, a second client is not showing the stream, unless the first client stops the streaming.
To be honest I do not fully understand the way it is working now in the new version done by you.
How can a client like TinyCam app on Android call the stream to display the video? Which protocoll is used instead of MJPEG in new version?
from esp32-cam-webserver.
Hi @TungstenE2
In theory, one could develop a websocket frame relay and deploy it on the proxy server. So the clients would connect to the proxy rather than to the ESP32 board directly. In this case, number of clients could be much higher than 2-3, and there is no need to modify the ESP32 sketch for that. This require the relay development, may not be very trivial. Doable though.
The way the new version is working is very simple. First, the client needs to open a websocket. This is done in JS code but can be any other language supporting WebSocket API.
Once the websocket connection is established, the client sends a command to the server to start the stream. The server then starts generating frames based on timer events (for example 25 frames per second) and pushes it to the client.
Since the websocket is a stateful protocol, it can consume the messages from the server and update it on the page. When the client doesn't want the frames anymore, it just sends a relevant command to the server or simply disconnects from the websocket - in this case the frame timer is stopped.
In a way, this is less cumbersome than MJPEG because there is no need for the client to request every frame. From the server side, there is also no need to handle chunked responses. All frames are generated by timer events, no delay() calls are used anywhere in the code, which is also a big plus, since the video streaming is not blocking any threads.
One can also allow multiple websocket clients and all of them will receive the same frame simultaneously, I just don't need it for my project.
from esp32-cam-webserver.
@abratchik thx for explanation. Do you also plan to main or support the project after @easytarget integrated your refactoring?
@easytarget are you able to proceed with the project if @abratchik decides not to support in future anymore? Seems he has a certain project and multi clients support is not part of his project plan.
from esp32-cam-webserver.
Related Issues (20)
- Flashing went well but no wifi network
- cam hangs after 2-3 days HOT 2
- Save and restore SSID and Passwd in config file
- Support mic and speaker
- Compiler warnings and errors in Arduino2.2.1
- started getting this HOT 5
- Error while compiling with Arduino IDE 1.8.19 HOT 1
- ssid issue
- init error
- Error when compiling for Xiao ESP32S3 HOT 1
- Internet access with Wifi-sta mode
- flash bin to device via ota
- Custom hostname not working
- how to get working with ov2640 HOT 1
- Streaming on Andorid Phone not possible HOT 1
- exit status 1 esp_int_wdt.h: No such file or directory HOT 6
- Start Recording and led on off issues HOT 1
- Stream Window is not rotated on port 81
- Sending data to Serial2 from the video stream.
- Watchdog API changes in arduino-esp32 version 3.x.x
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from esp32-cam-webserver.