iperov / deepfacelive Goto Github PK

I tried to download the model from your codes, but it seems the url is not work:

dfm_models = [
            #DFMModelInfo(name='Tom Cruise', model_path=models_path / f'Tom Cruise.dfm', url=rf'https://github.com/iperov/DeepFaceLive/releases/download/test/TOM_CRISE.onnx'),#TODO https://github.com/iperov/DeepFaceLive/releases/download/dfm/TOM_CRUISE.dfm'),
            #DFMModelInfo(name='Vladimir Putin', model_path=models_path / f'Vladimir Putin.dfm', url=rf'https://github.com/iperov/DeepFaceLive/releases/download/dfm/VLADIMIR_PUTIN.dfm'),
        ]

Will the project provided a default dfm model (such as Tom_Cruise.dfm)?
Or a public model download link is OK.
Thanks a lot.

Thanks a lot for your amazing work!
In China, we can't download software from Mega even if we use VPN QAQ
So can you share Google drive links about this release?
Thanks again!

Hello,
Thank you for sharing this wonderful work with us.

I need to configure the project in PyCharm IDE using Anaconda.
My Basic question is
"Can I run the whole project by typing one complete python command with command-line arguments by providing the link to the image and video. This means can I run the project by calling the main.py file only and then generate the output in a separate folder."
OR
Do I need to run the DeepFaceLive.py file and then run the project from the user interface?

Thank You

Добрый день!
Спасибо за крутое приложение!
Возможно ли использовать в качестве источника камеры - виртуальную камеру OBS? Пытался транслировать, но при всех конфигурациях получал только черный экран. OBS virtualcam plugin не помог.

Heres the error , newest python instaled, gtx drivers 472.12
`Running DeepFaceLive.
FaceDetector error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceDetector.py", line 234, in on_tick
rects = self.YoloV5Face.extract (frame_image, threshold=detector_state.threshold, fixed_window=detector_state.fixed_window_size)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 74, in extract
preds = self._get_preds(ip.get_image('NCHW'))
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 106, in _get_preds
preds = self._sess.run(None, {self._input_name: img})
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device

FaceDetector error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceDetector.py", line 234, in on_tick
rects = self.YoloV5Face.extract (frame_image, threshold=detector_state.threshold, fixed_window=detector_state.fixed_window_size)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 74, in extract
preds = self._get_preds(ip.get_image('NCHW'))
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 106, in _get_preds
preds = self._sess.run(None, {self._input_name: img})
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device

FaceMarker error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'conv2d_1__50' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceMarker.py", line 173, in on_tick
lmrks = self.google_facemesh.extract(face_image)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\FaceMesh\FaceMesh.py", line 57, in extract
lmrks = self._sess.run(None, {self._input_name: feed_img})[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'conv2d_1__50' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device

FaceSwapper error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'transpose_21' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceSwapper.py", line 280, in on_tick
celeb_face, celeb_face_mask_img, face_align_mask_img = dfm_model.convert(face_align_image, morph_factor=model_state.morph_factor)
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\DFLive\DFMModel.py", line 114, in convert
out_face_mask, out_celeb, out_celeb_mask = self._sess.run(None, {'in_face:0': img})
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'transpose_21' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device

FaceMarker error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'conv2d_1__50' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceMarker.py", line 173, in on_tick
lmrks = self.google_facemesh.extract(face_image)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\FaceMesh\FaceMesh.py", line 57, in extract
lmrks = self._sess.run(None, {self._input_name: feed_img})[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'conv2d_1__50' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device

FaceDetector error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceDetector.py", line 234, in on_tick
rects = self.YoloV5Face.extract (frame_image, threshold=detector_state.threshold, fixed_window=detector_state.fixed_window_size)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 74, in extract
preds = self._get_preds(ip.get_image('NCHW'))
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 106, in _get_preds
preds = self._sess.run(None, {self._input_name: img})
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device

Press any key to continue . . .`

Hi

I'd like to know if there is a python script or example code that can invoke DeepFaceLive from the command line.
I Know that this functionality is available in deepfacelab but as far as I know deepfacelive has 2 main advantages:
1- It can work with any target video
2- It is real time fast (30 fps is possible)

So I'd like to know how to load dfm model and point deepfacelive to a video file to instantly convert it.
Is there any script (python of course) or example code that can do this

Thanks

Excuse me, do you need the linux networking version? Windows I don't think it can be better commercialized. I have changed the linux interfaceless version.

Destination path can't have spaces, otherwise you get the error 'not recognized as an internal or external command'

I can't open the faceset.pak file . i think it is corrupted

Hello God, now the framework can only be used on the PC side to realize synchronous face change with streaming.
Can it be optimized on the VR glasses side and mobile phone side.

Hi I am facing this issue and don't know how to solve it. How to fix this?

Hi! When I try to change the face opacity from the frame merger and the selected device for it is the CPU, frame merger module stop to work and it raises an exception.

FaceMerger error: Iterator requested dtype could not be cast from dtype('float64') to dtype('float32'), the operand 0 dtype, according to the rule 'safe' Traceback (most recent call last):
  File "U:\DeepFake_softwares\DeepFaceLive_NVIDIA\_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
    self.on_tick()
  File "U:\DeepFake_softwares\DeepFaceLive_NVIDIA\_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceMerger.py", line 350, in on_tick
    merged_frame = self._merge_on_cpu(out_merged_frame, frame_image, face_resolution, face_align_img, face_align_mask_img, face_align_lmrks_mask_img, face_swap_img, face_swap_mask_img, aligned_to_source_uni_mat, frame_width, frame_height )
  File "U:\DeepFake_softwares\DeepFaceLive_NVIDIA\_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceMerger.py", line 252, in _merge_on_cpu
    ne.evaluate('frame_image*(one_f-frame_face_mask) + frame_image*frame_face_mask*(one_f-opacity) + frame_face_swap_img*frame_face_mask*opacity', out=out_merged_frame)
  File "U:\DeepFake_softwares\DeepFaceLive_NVIDIA\_internal\python\lib\site-packages\numexpr\necompiler.py", line 836, in evaluate
    return compiled_ex(*arguments, **kwargs)
TypeError: Iterator requested dtype could not be cast from dtype('float64') to dtype('float32'), the operand 0 dtype, according to the rule 'safe'

With GPU is alright.

During windows installation it appears this error

I've download some .npy models either from Internet and local DeepFaceLab, but they are not detected from DFL workscreen
All models from DFL are .npy!

How to use deepfacelive on linux

Just a tip, that someone is monetizing your tech. They are charging $250 for a two minute video. https://theindiandeepfaker.com/pricing/

How can I generate a dfm file?:)

can the project work on ubuntu/linux environment? if so, could you point me to the doc or how could i help to port.

Автор, как изменить разрешение источника видео, хочу добиться характеристик мобильного телефона прямой трансляции, 720х1280

I use my phone's camera as a virtual camera for my PC via OBS using https://vdo.ninja/

Is it possible to use this virtual camera as a Camera Source?

Hi,
first off, this is so cool, thanks for your marvellous work!

This is more of a suggestion than an issue.
When I saw your excellent pipeline, the extent on how this is all optimized to run in real time and the ease of configurability, I was wondering if you know of Spout.
It could save some resources to just output the final stream as a SpoutSender. This could then easily be imported into OBS by the Spout Plugin.
The advantage would be that all the buffers already in the GPU would get directly transferred to OBS and not via a window capture.

Hi !
I heard about your wonderful creation and I went to download it !
But I have a big problem, I can't find the file "DeepFaceLive.bat" or a similar file as you show in your setup tutorial.
I wonder if the software is still totally accessible or just the source code.
Thanks in advance !

can you plan to add xseg face mask ?

This technology is only going to be used for evil purposes. To deceive people, steal their money, possessions, undercover operations. Nothing good. It is already being used for scams like the pig-butchering plate, in which hundreds of thousands of lives around the world are being left without money, causing a lot of suffering.

Awesome work! i throw modles into DFMfolder, the software can't detect it.

Affirmation from a Chinese 💪

I've download some .npy models either from Internet and local DeepFaceLab, but they are not detected from DFL workscreen

What settings and resolution are the ready-to-use public face models created with?
Is the resolution better than the user-faq supplied settings "res:224, WF, archi:liae-udt, ae_dims:512, e_dims:64, d_dims:64, d_mask_dims:32, eyes_mouth_prio:Y, blur_out_mask:Y, uniform_yaw:Y, lr_dropout:Y, batch:8. Others by default"?

And whats the difference with the Google FaceMesh models, is there any code for that?

Thanks for your great work.
Is it possible to run it in colab?
Or, do you have any plans to release colab demo?

Running DeepFaceLive.
Traceback (most recent call last):
File "internal\DeepFaceLive\main.py", line 95, in
main()
File "internal\DeepFaceLive\main.py", line 88, in main
args.func(args)
File "internal\DeepFaceLive\main.py", line 30, in run_DeepFaceLive
from apps.DeepFaceLive.DeepFaceLiveApp import DeepFaceLiveApp
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\DeepFaceLiveApp.py", line 14, in
from . import backend
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend_init.py", line 1, in
from .BackendBase import (BackendConnection, BackendConnectionData, BackendDB,
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\BackendBase.py", line 7, in
from xlib import time as lib_time
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\time_init.py", line 1, in
from .time import timeit, measure, FPSCounter, AverageMeasurer
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\time\time_.py", line 11, in
if not kernel32.QueryPerformanceFrequency(_perf_freq):
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\api\win32\wintypes\wintypes.py", line 32, in wrapper
raise RuntimeError(f'Unable to load {dll_name} library.')
RuntimeError: Unable to load kernel32 library.
Press any key to continue . . .

I've tried both versions. I have an rtx 2070 with 8gig of vram. All other programs work just fine. Deepfacelab works great to. This is the only program that won't run. I've tried everything. Even tried replacing the kernel32.dll with another windows 10 kernel.dll.

Does this have MacOs support?

Hi Team,

The mega.nz download link isn't working (it times out). Has the installer been moved?

Автор вроде бы знает русский, поэтому буду писать на русском)
Как я могу добавить свою модель? Как отсканировать свое лицо? Или как вообще можно расширить те 8 лиц, которые идут по умолчанию?
How can I add my face model? How to scan my face? Or how can you expand those 8 faces that come by default?

i clone the code and try to run on pycharm and find there is no module xlib.image.misc

请问deepfacelive可以在换脸的同时将头发一起更换吗，请看这个例子 https://www.youtube.com/watch?v=xr5FHd0AdlQ

Downloaded latest version from Mega: DeepFaceLive_build_07_24_2021.exe

Is it possible to swap multiple faces?

I have made a model and using xseg I have excluded the mouth when open and trained it, and it masks as desired. But when I export this model to dfm and use it in deepfacelive the mouth is not excluded. I can however use deepfacelabs merge SAEHD and set mask_mode to Xseg-dst. This gives the desired result.

Is there any way to achieve the same in deepfacelive?

Thanks for your sharing of your wonderful work!
But I get some trouble when using. Hope you can give some help. Thank you very much!
I cannot import trained dfl model by click the 'select' button in Face swapper.
Actually, I can find my model by the small eye on the right.
So I think mybe the structure of my model files is not correct. But I dont know how to fix it.
My folder is as following:

in apps/DeepFaceLive/DeepFaceLiveApp.py

line about 164.

menu_help_action_github.triggered.connect(lambda: qtx.QDesktopServices.openUrl(qtx.QUrl('https://github.com/iperov/DeepFaceLive' )))

you miss the code qtx.
so you change your code QUrl to qtx.QUrl

Hello!

I am trying to run the demo program, following this tutorial, however at the point when I choose the device for the face detector, it goes red and the terminal has this output:

FaceDetector error: CUDAExecutionProvider is not avaiable in onnxruntime Traceback (most recent call last):
  File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\CSWBase.py", line 472, in _start_proc
    self.on_start(*worker_start_args, **worker_start_kwargs)
  File "D:\Desktop\DeepFaceLive-master\apps\DeepFaceLive\backend\FaceDetector.py", line 81, in on_start
    cs.detector_type.select(state.detector_type)
  File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\DynamicSingleSwitch.py", line 99, in select
    result = self._set_selected_idx(idx_or_choice)
  File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\DynamicSingleSwitch.py", line 33, in _set_selected_idx
    self._on_selected_evl.call(selected_idx, self.get_selected_choice() )
  File "D:\Desktop\DeepFaceLive-master\xlib\python\EventListener.py", line 24, in call
    func(*args, **kwargs)
  File "D:\Desktop\DeepFaceLive-master\apps\DeepFaceLive\backend\FaceDetector.py", line 99, in on_cs_detector_type
    cs.device.select(state.YoloV5_state.device)
  File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\DynamicSingleSwitch.py", line 99, in select
    result = self._set_selected_idx(idx_or_choice)
  File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\DynamicSingleSwitch.py", line 33, in _set_selected_idx
    self._on_selected_evl.call(selected_idx, self.get_selected_choice() )
  File "D:\Desktop\DeepFaceLive-master\xlib\python\EventListener.py", line 24, in call
    func(*args, **kwargs)
  File "D:\Desktop\DeepFaceLive-master\apps\DeepFaceLive\backend\FaceDetector.py", line 144, in on_cs_devices
    self.YoloV5Face = onnx_models.YoloV5Face(device)
  File "D:\Desktop\DeepFaceLive-master\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 34, in __init__
    self._sess = sess = InferenceSession_with_device(str(path), device_info)
  File "D:\Desktop\DeepFaceLive-master\xlib\onnxruntime\InferenceSession.py", line 23, in InferenceSession_with_device
    raise Exception('CUDAExecutionProvider is not avaiable in onnxruntime')
Exception: CUDAExecutionProvider is not avaiable in onnxruntime

I have CUDA 11.4 installed with the cudnn support. My graphics card is GTX 1080 Ti.

Looking forward to hearing from you! Thanks.

Hello, on Linux (ubuntu 20.04) I got this error:

[ WARN:[email protected]] global /io/opencv/modules/videoio/src/cap_v4l.cpp (889) open VIDEOIO(V4L2:/dev/video0): can't open camera by index
[ WARN:[email protected]] global /io/opencv/modules/videoio/src/cap_v4l.cpp (889) open VIDEOIO(V4L2:/dev/video1): can't open camera by index

I have test with one camera & one webcam:
ls /dev/video*

/dev/video0 /dev/video1

I training with RTM model for the first time
I'm wondering should the dst(rtm wf faceset in the torrent) be xseg and apply?
In the FAQ it only said that i should xseg and apply in src but don't known should it be apply in dst

Previus versions with old my models works 3x faster, with new only Tom kruse is working ok and my own dfl old models works very slow, its very strange, how correct train and convert model? And where get other pretrained models like Tom Kruse and why its works on all faces how to traint it like that?