Using Webcam instead of raspberry Pi camera. about mmm-face-reco-dnn

[bojuelkjeldsen]

Hi Nischi
Thank you, I will check on the error logs! I do run the MM with PM2 as automatic startup.

[nischi]

Hi @bojuelkjeldsen

Do you have a little bit more information whats wrong and maybe error message you got?

Kind regards,

[bojuelkjeldsen]

Hi @nischi
Thank you very much for your reply.
I have had great use of your previous version of MMM-Face-Reco-DNN! My family likes the different versions made for each of them.
In the previous version, you could set the "source" in confic.js to "1" if it was not a pi-cam. This option I cannot find in the new version?
Regarding the error-messages, I am sorry, but I am not sure where to look for the error messages?
The mirror runs fin, but no faces are recognized. The dataset and coding also runs without problems.
While the mirror runs with the MMM-Face-Reco-DNN module, I can still access the webcam with Cheese, so I am not sure that webcam is active for the MMM-Face-Reco-DNN module? I am not sure that the module gets any pictures from the camera?

Best

Bo

[nischi]

Sorry i really dont know what could be wrong without some errors. If you run MM with pm2 you can check the error logs there.

https://pm2.keymetrics.io/docs/usage/log-management/

[bojuelkjeldsen]

Hi Nischi
Now, I checked the PM2 mm-error.log. It looks like the module "libcamera" is not installed for Python? (PythonShellError).
I am not quite sure how to correct this? Reinstall OpenCV?

Best Bo

The Error log file:

[4587:0329/154802.262639:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.270248:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.271143:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.272030:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.272885:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.274088:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.274878:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.281378:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.282353:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.283281:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.284224:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.285183:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.286195:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[4587:0329/154802.287095:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[29.03.2024 15:48.20.641] [ERROR] Whoops! There was an uncaught exception...
[29.03.2024 15:48.20.647] [ERROR] PythonShellError: ModuleNotFoundError: No module named 'libcamera'
at PythonShell.parseError (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:303:21)
at terminateIfNeeded (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:193:32)
at ChildProcess. (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:185:13)
at ChildProcess.emit (node:events:514:28)
at Process.onexit (node:internal/child_process:291:12)
----- Python Traceback -----
File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in
from picamera2 import Picamera2
File "/home/bo/.local/lib/python3.10/site-packages/picamera2/init.py", line 3, in
import libcamera {
traceback: 'Traceback (most recent call last):\n' +
' File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in \n' +
' from picamera2 import Picamera2\n' +
' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/init.py", line 3, in \n' +
' import libcamera\n' +
"ModuleNotFoundError: No module named 'libcamera'\n",
executable: 'python3',
options: null,
script: 'modules/MMM-Face-Reco-DNN/tools/recognition.py',
args: [
'--cascade=modules/MMM-Face-Reco-DNN/model/haarcascade_frontalface_default.xml',
'--encodings=modules/MMM-Face-Reco-DNN/model/encodings.pickle',
'--rotateCamera=-1',
'--method=dnn',
'--detectionMethod=hog',
'--interval=1500',
'--output=0',
'--outputmm=0',
'--extendDataset=True',
'--dataset=modules/MMM-Face-Reco-DNN/dataset/',
'--tolerance=0.6',
'--brightness=0',
'--contrast=0',
'--resolution=1920,1080',
'--processWidth=500'
],
exitCode: 1
}
[29.03.2024 15:48.20.647] [ERROR] MagicMirror² will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
[29.03.2024 15:48.20.649] [ERROR] If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues
/home/bo/MagicMirror/node_modules/electron/dist/electron exited with signal SIGINT

[nischi]

i think you have not installed the picamera properly. try to reinstall all the dependencies

[bojuelkjeldsen]

Hi Nischi

Sorry for asking about this picamera2 and libcamera module. It seems to give some trouble. I have tried to reinstall these dependencies several times and in several ways.
My system is ubuntu 22.04, 64 bit, Pi 4.
Should I somehow add a location for "libcamera" ? Perhaps try to add it to the path?
Regarding the dependencies, I have tried to use npm ci, but that seems not to work on my Pi?

Best

Bo

bo@mm:~$ npm ci
npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /home/bo/package.json
npm ERR! errno -2
npm ERR! enoent Could not read package.json: Error: ENOENT: no such file or directory, open '/home/bo/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

The error.log from PM2 looks like this:

[31.03.2024 00:01.27.684] [ERROR] Whoops! There was an uncaught exception...
[31.03.2024 00:01.27.708] [ERROR] PythonShellError: ImportError: cannot import name 'ControlType' from 'libcamera' (unknown location)
at PythonShell.parseError (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:303:21)
at terminateIfNeeded (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:193:32)
at Socket. (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:173:17)
at Socket.emit (node:events:526:35)
at endReadableNT (node:internal/streams/readable:1359:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
----- Python Traceback -----
File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in
from picamera2 import picamera2
File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/init.py", line 5, in
from .configuration import CameraConfiguration, StreamConfiguration
File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/configuration.py", line 1, in
from .controls import Controls
File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/controls.py", line 4, in
from libcamera import ControlType, Rectangle, Size {
traceback: 'Traceback (most recent call last):\n' +
' File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in \n' +
' from picamera2 import picamera2\n' +
' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/init.py", line 5, in \n' +
' from .configuration import CameraConfiguration, StreamConfiguration\n' +
' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/configuration.py", line 1, in \n' +
' from .controls import Controls\n' +
' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/controls.py", line 4, in \n' +
' from libcamera import ControlType, Rectangle, Size\n' +
"ImportError: cannot import name 'ControlType' from 'libcamera' (unknown location)\n",
executable: 'python3',
options: null,
script: 'modules/MMM-Face-Reco-DNN/tools/recognition.py',
args: [
'--cascade=modules/MMM-Face-Reco-DNN/model/haarcascade_frontalface_default.xml',
'--encodings=modules/MMM-Face-Reco-DNN/model/encodings.pickle',
'--rotateCamera=-1',
'--method=dnn',
'--detectionMethod=hog',
'--interval=1500',
'--output=0',
'--outputmm=0',
'--extendDataset=True',
'--dataset=modules/MMM-Face-Reco-DNN/dataset/',
'--tolerance=0.6',
'--brightness=0',
'--contrast=0',
'--resolution=1920,1080',
'--processWidth=500'
],
exitCode: 1

[bojuelkjeldsen]

[ERROR] PythonShellError: ModuleNotFoundError: No module named 'libcamera'
at PythonShell.parseError (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:303:21)
at terminateIfNeeded (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:193:32)
at ChildProcess. (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:185:13)
at ChildProcess.emit (node:events:514:28)
at Process.onexit (node:internal/child_process:291:12)
----- Python Traceback -----
File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in
from picamera2 import Picamera2
File "/home/bo/.local/lib/python3.10/site-packages/picamera2/init.py", line 3, in
import libcamera {
traceback: 'Traceback (most recent call last):\n' +
' File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in \n' +
' from picamera2 import Picamera2\n' +
' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/init.py", line 3, in \n' +
' import libcamera\n' +
"ModuleNotFoundError: No module named 'libcamera'\n",
executable: 'python3',
options: null,
script: 'modules/MMM-Face-Reco-DNN/tools/recognition.py',
args: [
'--cascade=modules/MMM-Face-Reco-DNN/model/haarcascade_frontalface_default.xml',
'--encodings=modules/MMM-Face-Reco-DNN/model/encodings.pickle',
'--rotateCamera=-1',
'--method=dnn',
'--detectionMethod=hog',
'--interval=1500',
'--output=0',
'--outputmm=0',
'--extendDataset=True',
'--dataset=modules/MMM-Face-Reco-DNN/dataset/',
'--tolerance=0.6',
'--brightness=0',
'--contrast=0',
'--resolution=1920,1080',
'--processWidth=500'
],
exitCode: 1

[bojuelkjeldsen]

Hi Nischi
I do hope that your weekend has treated you well!
It does seem that the libcamera module cannot be found. I have tried several ways to install it. When I manage to install it, it ends up installed in the root and not under the ".local/lib/python3.10/site-packages" which is where the picamera2 module is installed.
The libcamera path has been added to the pythonpath. Unfortunately, this does not help?
I am not sure how to install all the dependencies for the module in the right way?
It runs on Raspberry pi 4, ubuntu 22.04, python 3.10,
Suggestions are very much appriciated!
Kind Regards
Bo

[nischi]

Hi @bojuelkjeldsen

It seems you try to run npm ci in your home directory and not in the directory of the module itself.

And about the python dependencies. I'm not sure how it works with ubuntu, but could be that this is working same as on bookworm. Maybe you need to create a new virtual environment and install the dependencies there. Like described here: https://github.com/nischi/MMM-Face-Reco-DNN?tab=readme-ov-file#some-additional-steps-for-bookworm-and-above-to-run-it-with-an-virtual-environment

[bojuelkjeldsen]

Hi Nischi

Yes, thank you, it did help running the "npm ci" in the right directory!

Also, I did have "some" issues installing libcamera module. I could not get permission with pip to write to the normal directory:
(for me) " .local/lib/python3.10/site-packages. So I use this solution so kindly shared by nbaldy: raspberrypi/picamera2#563

So, now the picamera2 and libcamera modules are working. However, it seems that the usb webcam cannot be found by the modules?

Perhaps, there ss there a way to set the correct camera?

Kind regards and any help is very much appreciated :-)

[2024-04-04 10:47:47.322] [ERROR] Whoops! There was an uncaught exception...
[2024-04-04 10:47:47.330] [ERROR] PythonShellError: [0:01:35.870026963] [2882] �[1;32m INFO �[1;37mCamera �[1;34mcamera_manager.cpp:284 �[0mlibcamera v0.2.0+47-46b868fe
Traceback (most recent call last):
File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 40, in
picam2 = Picamera2()
File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/picamera2.py", line 242, in init
camera_num = self.global_camera_info()[camera_num]['Num'] # 'Num'
IndexError: list index out of range
Exception ignored in: <function Picamera2.del at 0xffff8bb51f30>
Traceback (most recent call last):
File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/picamera2.py", line 404, in del
self.close()
File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/picamera2.py", line 604, in close
if self._preview:
AttributeError: 'Picamera2' object has no attribute '_preview'

at PythonShell.parseError (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:311:21)
at terminateIfNeeded (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:193:32)
at ChildProcess.<anonymous> (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:185:13)
at ChildProcess.emit (node:events:514:28)
at ChildProcess._handle.onexit (node:internal/child_process:294:12) {

executable: 'python3',
options: null,
script: 'modules/MMM-Face-Reco-DNN/tools/recognition.py',
args: [
'--cascade=modules/MMM-Face-Reco-DNN/model/haarcascade_frontalface_default.xml',
'--encodings=modules/MMM-Face-Reco-DNN/model/encodings.pickle',
'--rotateCamera=-1',
'--method=dnn',
'--detectionMethod=hog',
'--interval=1500',
'--output=0',
'--outputmm=0',
'--extendDataset=True',
'--dataset=modules/MMM-Face-Reco-DNN/dataset/',
'--tolerance=0.6',
'--brightness=0',
'--contrast=0',
'--resolution=1920,1080',
'--processWidth=500'
],
exitCode: 1
}

The camera is detected by ubuntu:

bo@mm:~$ ls -ltrh /dev/video*
crw-rw----+ 1 root video 81, 2 Apr 4 10:44 /dev/video15
crw-rw----+ 1 root video 81, 1 Apr 4 10:44 /dev/video14
crw-rw----+ 1 root video 81, 6 Apr 4 10:44 /dev/video12
crw-rw----+ 1 root video 81, 0 Apr 4 10:44 /dev/video13
crw-rw----+ 1 root video 81, 3 Apr 4 10:44 /dev/video10
crw-rw----+ 1 root video 81, 10 Apr 4 10:44 /dev/video21
crw-rw----+ 1 root video 81, 8 Apr 4 10:44 /dev/video20
crw-rw----+ 1 root video 81, 12 Apr 4 10:44 /dev/video23
crw-rw----+ 1 root video 81, 7 Apr 4 10:44 /dev/video18
crw-rw----+ 1 root video 81, 13 Apr 4 10:44 /dev/video0
crw-rw----+ 1 root video 81, 9 Apr 4 10:44 /dev/video31
crw-rw----+ 1 root video 81, 5 Apr 4 10:44 /dev/video11
crw-rw----+ 1 root video 81, 4 Apr 4 10:44 /dev/video16
crw-rw----+ 1 root video 81, 11 Apr 4 10:44 /dev/video22
crw-rw----+ 1 root video 81, 14 Apr 4 10:44

bo@mm:~$ lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 005: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 009: ID aaec:c021 Multi touch Multi touch overlay device
Bus 001 Device 003: ID 1bcf:2284 Sunplus Innovation Technology Inc. Full HD webcam
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The webcam does work in the Ubuntu 22.04

[nischi]

Hi @bojuelkjeldsen

Boah this looks more like an issue in picamera module itself. raspberrypi/picamera2#866 I found this and seems there was something wrong wired, but no more information.

Are you able to test with another camera to check if this is the issue?

[bojuelkjeldsen]

Hi @nischi

Thank you for your reply!

Unfortunately, other web cameras did not work either. I found 3 other cameras to test, one of which was an old Logitech. All worked fine, but not in Picamera/libcamera modules. They gave the same error:

File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/picamera2.py", line 242, in init
camera_num = self.global_camera_info()[camera_num]['Num']
IndexError: list index out of range

I did try with the cameras connected to both USB2 and USB3, with same result. They did work on Ubuntu using the Cheese camera app.

Can USB perhaps be forced only to look for the USB web cam?

Kind regards

Bo

[nischi]

Hi @bojuelkjeldsen

Hmm okay, strange. I need to have a look into the documentation if we can configure some stuff. Need a little bit of time for this.

Kind regards

[nischi]

Hi @bojuelkjeldsen

Proberly i did found a solution for the issue. But not sure if it's working. Can you try something?

In this file
/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py

change on line 40 from
picam2 = Picamera2()
to
picam2 = Picamera2(1)

Maybe the usb-camera has another index and 0 (default) is not available. But the error say out of index, so properly it does not work. But worth a try.

[bojuelkjeldsen]

Hi @nischi

Thank you for your response! I did try your suggestions with 1, 2, 3, 4 ... but it did not work.

So, one of my friends (professional IT-guy), also had a look at it, and finally suggested me to start over (again). This time, however, I did use Debian 12 (Bookworm) on my raspberry Pi 4 (Previously, I used Ubuntu 22.04). Also, I did use the virtual environment. AND - after some hours - I also changed the "PythonPath" in the MagicMirror config.js file, as already instructed by you in that file (remember the " " around the path).

SUCCESS!!!

The MMM-Face-Reco-DNN is up and running again!

[nischi]

hi @bojuelkjeldsen

Awesome if had worked 👍