hukkelas / deepprivacy Goto Github PK
View Code? Open in Web Editor NEWDeepPrivacy: A Generative Adversarial Network for Face Anonymization
License: MIT License
DeepPrivacy: A Generative Adversarial Network for Face Anonymization
License: MIT License
Could you help me?
python3 anonymize.py -s/home/daqua/Pictures/h.png -t/home/daqua/Pictures/h2.png
None
Traceback (most recent call last):
File "anonymize.py", line 5, in
cli.main()
File "/home/daqua/Videos/DeepPrivacy/deep_privacy/cli.py", line 111, in main
anonymizer, cfg = build_anonymizer(
File "/home/daqua/Videos/DeepPrivacy/deep_privacy/build.py", line 62, in build_anonymizer
cfg = get_config(config_urls[model_name])
File "/home/daqua/Videos/DeepPrivacy/deep_privacy/build.py", line 36, in get_config
torch.hub.download_url_to_file(config_url, cfg_path)
File "/home/daqua/.local/lib/python3.8/site-packages/torch/hub.py", line 394, in download_url_to_file
u = urlopen(req)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 308: Permanent Redirect
Is it possible to anonymize only a part of a face using bounding boxes? Using a custom im_bboxes with a high truncation level gives this traceback because z_shape is 0 in the first dimension
Traceback (most recent call last):
File "DeepPrivacy/anonymize.py", line 5, in <module>
cli.main()
File "DeepPrivacy/deep_privacy/cli.py", line 142, in main
anonymizer.anonymize_image_paths(image_paths, image_target_paths, [np.array([[250,400,500,550]])])
File "DeepPrivacy/deep_privacy/inference/anonymizer.py", line 61, in anonymize_image_paths
anonymized_images, image_annotations = self.detect_and_anonymize_images(
File "DeepPrivacy/deep_privacy/inference/anonymizer.py", line 43, in detect_and_anonymize_images
anonymized_images = self.anonymize_images(
File "/usr/lib/python3/dist-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "DeepPrivacy/deep_privacy/inference/deep_privacy_anonymizer.py", line 74, in anonymize_images
z = infer.truncated_z(
File "DeepPrivacy/deep_privacy/inference/infer.py", line 96, in truncated_z
while z.abs().max() >= truncation_level:
RuntimeError: operation does not have an identity.
Without a truncation level the model outputs an identical image without changes
Not sure why this issue is popping up now...I've run this code in the past to anonymize thousands of images...
CUDA version: 10.1
Linux learner001 4.15.0-118-generic #119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
>>> torch.__version__
'1.7.0.dev20200923+cu101'
>>> torchvision.__version__
'0.8.0.dev20200923+cu101'
(deep-privacy) (base) lrm@learner001:~/repos/DeepPrivacy$ python -m deep_privacy.inference.anonymize_folder models/large/config.yml --source_path /home/lrm/data/inline_2020-09-17_2020-09-18/ --target_path /home/lrm/data/anon_inline_2020-09-17_2020-09-18/
CONFIG USED:
================================================================================
logging.num_ims_per_log 500
logging.num_ims_per_save_image 100000
logging.num_ims_per_checkpoint 200000
max_imsize 128
use_full_validation False
load_fraction_of_dataset False
dataset yfcc100m128
train_config.transition_iters 1200000
train_config.learning_rate 0.00175
train_config.amp_opt_level O1
train_config.batch_size_schedule {4: 256, 8: 256, 16: 256, 32: 128, 64: 96, 128: 64}
models.start_channel_size 512
models.pose_size 14
models.image_channels 3
models.generator.running_average_decay 0.999
models.discriminator.structure normal
config_path models/large/config.yml
checkpoint_dir models/large/checkpoints
generated_data_dir models/large/generated_data
summaries_dir models/large/summaries
source_path /home/lrm/data/inline_2020-09-17_2020-09-18/
target_path /home/lrm/data/anon_inline_2020-09-17_2020-09-18/
================================================================================
[*] Loading checkpoint from models/large/checkpoints/step_40000000.ckpt succeed!
extending G 512
extending G 512
extending G 512
extending G 256
extending G 128
Anonymizer initialized. Keypoint threshold: 0.3Face threshold: 0.1
Batch detecting faces: 0%| | 0/25359 [00:00<?, ?it/s]Finished loading DSFD model!
Batch detecting faces: 60%|███████████████████████████████▏ | 15202/25359 [14:15<09:31, 17.78it/s]
Traceback (most recent call last):
File "/home/lrm/.pyenv/versions/3.7.3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/lrm/.pyenv/versions/3.7.3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/lrm/repos/DeepPrivacy/deep_privacy/inference/anonymize_folder.py", line 10, in <module>
anonymizer.anonymize_folder(source_path, save_path)
File "/home/lrm/repos/DeepPrivacy/deep_privacy/inference/anonymizer.py", line 35, in anonymize_folder
im_bboxes=im_bboxes)
File "/home/lrm/repos/DeepPrivacy/deep_privacy/inference/anonymizer.py", line 42, in anonymize_image_paths
face_threshold=self.face_threshold
File "/home/lrm/repos/DeepPrivacy/deep_privacy/detection/detection_api.py", line 43, in batch_detect_faces_with_keypoints
im_bboxes = batch_detect_faces(images, face_threshold)
File "/home/lrm/repos/DeepPrivacy/deep_privacy/detection/detection_api.py", line 25, in batch_detect_faces
det = face_detector.detect_face(im[:, :, ::-1], face_threshold)
File "/home/lrm/repos/DeepPrivacy/deep_privacy/detection/dsfd/detect.py", line 46, in detect_face
y = self.net(x, confidence_threshold, self.nms_iou_threshold)
File "/home/lrm/.pyenv/versions/deep-privacy/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/lrm/repos/DeepPrivacy/deep_privacy/detection/dsfd/face_ssd.py", line 193, in forward
nms_threshold
File "/home/lrm/repos/DeepPrivacy/deep_privacy/detection/dsfd/utils.py", line 61, in forward
indices = (conf_scores >= confidence_threshold).nonzero().squeeze()
RuntimeError: CUDA error: misaligned address
In the README it states that the model fdf128_rcnn512 uses DSFD as face detector. However, the respective config file at: https://folk.ntnu.no/haakohu/configs/fdf_512.json has RetinaNetResNet50 as face detector. Why exactly is that?
When you start anonymize.py you get an ImportError
Hello, thank you for this great work.
I am trying to train retinanet512 model on my own dataset, but there aren't examples of usage of train.py in version 2.
How do I train on my own dataset in version 2 ?
I am trying to train on a new dataset using train.py but it is showing me this error continuously. I am not sure or aware what this .debug/before.torch file is and is trying to do
I am very interested in your work and have studied it carefully.But there are some questions when I run the program in docker.I wonder if there is a problem with the docker I created.
Traceback (most recent call last):
File "/workspace/DeepPrivacy/deep_privacy/inference/infer.py", line 44, in load_model_from_checkpoint
ckpt = get_checkpoint(cfg.output_dir, validation_checkpoint_step)
File "/workspace/DeepPrivacy/deep_privacy/engine/checkpointer.py", line 48, in get_checkpoint
return load_checkpoint(path)
File "/workspace/DeepPrivacy/deep_privacy/engine/checkpointer.py", line 29, in load_checkpoint
raise FileNotFoundError(f"Did not find path: {ckpt_path}")
FileNotFoundError: Did not find path: outputs/workspace/deep_privacy_cache/fdf_512/checkpoints
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "anonymize.py", line 5, in
cli.main()
File "/workspace/DeepPrivacy/deep_privacy/cli.py", line 113, in main
return_cfg=True)
File "/workspace/DeepPrivacy/deep_privacy/build.py", line 66, in build_anonymizer
generator = load_model_from_checkpoint(cfg)
File "/workspace/DeepPrivacy/deep_privacy/inference/infer.py", line 47, in load_model_from_checkpoint
ckpt = load_checkpoint_from_url(cfg.model_url)
File "/workspace/DeepPrivacy/deep_privacy/engine/checkpointer.py", line 18, in load_checkpoint_from_url
model_url, map_location=_get_map_location())
File "/opt/conda/lib/python3.6/site-packages/torch/hub.py", line 494, in load_state_dict_from_url
raise RuntimeError('Only one file(not dir) is allowed in the zipfile')
RuntimeError: Only one file(not dir) is allowed in the zipfile
Hey, thank you for your research and publishing your source code.
Are you aware of brighter.ai, a Berlin based startup - they are doing more or less what your paper does, but as a product. Maybe you should get in contact and/or mention them in the paper?
Best,
Christoph
Thank you very much for the impressive work
It is not clear to me:
1) How to fine-tune the existing models on another dataset?
2) How to train a deep privacy model from scratch?
In both cases, what are the exact steps, scripts, formats, etc
Regards
Thanks for the great repository!
I just wonder, could the anonymization network generate different results on the same input image?
There is no manual seed selection in code, that I could find in your repository.
The only thing that works for me:
anonymizer = build_anonymizer(model_name="fdf128_rcnn512", truncation_level=100)
torch.manual_seed(int(time.time()))
np.random.seed(int(time.time()))
res = anonymizer.detect_and_anonymize_images([in_image])
python3.6 -m deep_privacy.inference.anonymize_folder model/default/config.yml --source_path lina2.jpg --target_path lina2_anonymized.jpg
Traceback (most recent call last):
File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/DeepPrivacy/deep_privacy/inference/anonymize_folder.py", line 1, in <module>
from deep_privacy.inference import deep_privacy_anonymizer, infer
File "/opt/DeepPrivacy/deep_privacy/inference/deep_privacy_anonymizer.py", line 7, in <module>
from .anonymizer import Anonymizer
File "/opt/DeepPrivacy/deep_privacy/inference/anonymizer.py", line 7, in <module>
from deep_privacy.inference import infer
File "/opt/DeepPrivacy/deep_privacy/inference/infer.py", line 5, in <module>
from deep_privacy import config_parser, utils
File "/opt/DeepPrivacy/deep_privacy/utils.py", line 4, in <module>
from apex.amp._amp_state import _amp_state
File "/usr/local/lib/python3.6/site-packages/apex/__init__.py", line 18, in <module>
from apex.interfaces import (ApexImplementation,
File "/usr/local/lib/python3.6/site-packages/apex/interfaces.py", line 10, in <module>
class ApexImplementation(object):
File "/usr/local/lib/python3.6/site-packages/apex/interfaces.py", line 14, in ApexImplementation
implements(IApex)
File "/usr/local/lib/python3.6/site-packages/zope/interface/declarations.py", line 483, in implements
raise TypeError(_ADVICE_ERROR % 'implementer')
**TypeError: Class advice impossible in Python3. Use the @implementer class decorator instead.**
why?
Hi,
I am quite new to GANs and experimenting a bit with loss functions. While trying to train my own model I struggled a bit unraveling the code of your losses. Could you please explain what the loss function with id = 1 stands for?
And is it correct to say that loss function with id 0 represents the wasserstein distance, loss function 2 the true positive rate of the discriminator of real images and loss function 3 the adversarial loss for the generator (fakes scores of discriminator)?
Best regards and thanks in advance!
As I understand, this algorithm works well on small faces in the image. But for large faces (larger than 128 x 128), the output is not suitable.
Is there a solution?
Is this a problem because the output of the generator network is 128 x 128?
I tried use it with docker, but I am stuck on the command with nvidia-docker: nvidia-docker run --rm -it -v $PWD:/workspace -e CUDA_VISIBLE_DEVICES=0 deep_privacy python -m deep_privacy.train models/large/config.yml
After installation of nvidia-docker, I could command like this docker run --gpus all nvidia/cuda:10.0-base nvidia-smi
but not like nvidia-docker run
.
I am not familiar with nvidia-docker, I'd really appreciate that if you could tell me how to run your program on docker.
nvidia-docker: command not found · Issue #1028 · NVIDIA/nvidia-docker
Thank you for this work
It is mentioned in the paper, temporal consistency can be one of the proposed improvements.
Regards
I tried to install that on Windows, all packages were successful downloaded.
Now i'm trying to run command
python -m deep_privacy.inference.anonymize_folder model/default/config.yml --source_path testim.jpg --target_path testim_anonymized.jpg
I changed after failure model to models
python -m deep_privacy.inference.anonymize_folder models/default/config.yml --source_path testim.jpg --target_path testim_anonymized.jpg
after running this command i get
`C:\Users\Alexander\Work\NN\DeepPrivacy>python -m deep_privacy.inference.anonymize_folder models/default/config.yml --source_path testim.jpg --target_path testim_anonymized.jpg
CONFIG USED:
logging.num_ims_per_log 500
logging.num_ims_per_save_image 100000
logging.num_ims_per_checkpoint 200000
max_imsize 128
use_full_validation False
load_fraction_of_dataset False
dataset yfcc100m128
train_config.transition_iters 1200000
train_config.learning_rate 0.00175
train_config.amp_opt_level O1
train_config.batch_size_schedule {4: 256, 8: 256, 16: 256, 32: 128, 64: 96, 128: 64}
train_config.criterion.type wgan
train_config.criterion.gradient_penalty_weight 10
train_config.criterion.epsilon_penalty_weight 0.001
train_config.criterion.different_images False
models.start_channel_size 256
models.pose_size 14
models.image_channels 3
models.generator.running_average_decay 0.999
models.discriminator.structure normal
config_path models/default/config.yml
checkpoint_dir models/default\checkpoints
generated_data_dir models/default\generated_data
summaries_dir models/default\summaries
source_path testim.jpg
target_path testim_anonymized.jpg
Traceback (most recent call last):
File "C:\Users\Alexander\Miniconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Users\Alexander\Miniconda3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\Alexander\Work\NN\DeepPrivacy\deep_privacy\inference\anonymize_folder.py", line 5, in
generator, imsize, source_path, image_paths, save_path = infer.read_args()
File "C:\Users\Alexander\Work\NN\DeepPrivacy\deep_privacy\inference\infer.py", line 176, in read_args
ckpt = utils.load_checkpoint(config.checkpoint_dir)
File "C:\Users\Alexander\Work\NN\DeepPrivacy\deep_privacy\utils.py", line 43, in load_checkpoint
with open(os.path.join(ckpt_dir_or_file, 'latest_checkpoint')) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'models/default\checkpoints\latest_checkpoint'`
How can i fix it?
Hello,
I was wondering how the resolution of the output images is chosen? Is this a parameter of the model, which could be trained to output higher resolution images?
Alternatively do you think it is reasonable to cut-out the face from a higher resolution image and input this into your program, then re-integrate into the original resolution image? If so, how much context surrounding the face would be good to include in these subsets?
My goal is to benchmark how this tool works on high resolution images.
Thanks,
James
Hello,
I just tried to build the docker image, but sadly it fails:
=> ERROR [ 8/14] RUN pip install -e /detectron2_repo 3.6s
------
> [ 8/14] RUN pip install -e /detectron2_repo:
#12 1.137 Obtaining file:///detectron2_repo
#12 3.374 ERROR: Command errored out with exit status 1:
#12 3.374 command: /opt/conda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/detectron2_repo/setup.py'"'"'; __file__='"'"'/detectron2_repo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
#12 3.374 cwd: /detectron2_repo/
#12 3.374 Complete output (6 lines):
#12 3.374 Traceback (most recent call last):
#12 3.374 File "<string>", line 1, in <module>
#12 3.374 File "/detectron2_repo/setup.py", line 15, in <module>
#12 3.374 assert torch_ver >= [1, 6], "Requires PyTorch >= 1.6"
#12 3.374 AssertionError: Requires PyTorch >= 1.6
#12 3.374 No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
#12 3.374 ----------------------------------------
#12 3.375 ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
------
executor failed running [/bin/sh -c pip install -e /detectron2_repo]: exit code: 1
It says No CUDA runtime is found
but that's not mentioned in the requirements.
Do I need to install it manually or what's the case here?
hello,
https://folk.ntnu.no/haakohu/RetinaFace_ResNet50.pth is not available
First of all, I should say thanks a lot for a detailed write-up of your amazing work. It is really useful for me
I have some questions. I want to train this model from scratch with the FDF dataset. I download it from the Github that you mentioned:
Whenever you get some time, I appreciate your eyes on this.
Hello @cclauss @hardmaru @hukkelas @StianHanssen @SajjadAemmi
Great work on the paper and even the results look amazing!
Can you please guide me with step by step process on how we need to prepare a dataset for a custom image dataset with a custom mask (if possible)?
How the masks will look like in the sense that whether it will be a colour image on which white patches (the part that will be inpainted from the model) is drawn or it will be a completely black image on which white patches (the part that will be inpainted from the model) is drawn? (in short how to prepare mask and then how to supply it for model training)
I referred to your steps but couldn't completely follow them.
Thank you in advance!
Thanks for your code! I am trying to train a model from scratch using my own facial keypoint detection network (rather than using the standard pre-trained mask RCNN provided in this repository). My keypoint network provides 63 facial points as compared to the 7 points used by your code. Is it possible to train the network with 63 keypoints? Where should I make the changes to your code? Thanks in advance!
Hello, thanks for the great work !
I'am currently new to this, but im trying to use your code to anonymize face on cctv footage, do you think i should use it as it is or i should re-train your model on a cctv dataset ( using your pre-trained weights obviously ) ?
I have two other questions :
use_static_z
, max_face_size
?stitch_face
method ? (if you can explain what stitch_face
and replace_face
methods exactly do i will be very grateful).Thank you so much !
Hi, I apply your creative code on my own data, but find some results of the face swap are scary. I think going back to targeted training might help. Could you teach me how to train with my own data? thank you so much!
Dears
Regarding keypoint_threshold
and face_threshold
, could you please explain in some detial how they work and affect the anonymization.
anonymizer = deep_privacy_anonymizer.DeepPrivacyAnonymizer(generator,
batch_size=32,
use_static_z=True,
keypoint_threshold=.1,
face_threshold=.6)
For example, I noticed that making face_threshold
too small, encourage the network to predict anything as face. and making it too large, leads to missing faces. How this will work with the keypoint_threshold
?
Additionally, what is replace_tight_bbox
used for?
Finally, kindly share any tips to enable detection and anonymization of small faces.
Thank you and best regards
Can't be used, I can't anonymize the pictures in the folder using "python3 anonymize.py -s input_path -t output_path" after using the command "python3 setup.py install"
Dear
Is there a way (API) to get the bounding boxes and key points detected? (For example, in terms of a list of x,y,w,h for bounding boxes and x,y for each key point)
Best Regards
This should be
anonymized_frames = self.anonymize_images(frames, im_bboxes=im_bboxes)
as im_boxes is a positional argument.
Otherwises this method throws an error when it is called with_keypoints = False which is the default
Hello, Thank you for this awesome work.
So, I'm trying to compute the SSIM metric between real and generated images inside the loss.py program to see it's evolution while training the model.
I tried to do this by transforming real_data tensors to RGB image but the result was not satisfying. So I was wondering if there's a way to get the images (in a pixelwise structure), since SSIM is using pixels not tensors.
Finally, kindly share any ideas or tips to complete this task.
Thank you for the great work!
Regarding the paper, it is mentioned that:
My question is how the model preserves the gender, age and other related features while generating the faces?
Regards
Hi,
it seems like
Reading of the images is successfully completed, dataset is loaded, validation of the model seems to be completed too, but then the following error happens:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec)
File "/opt/conda/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals)
File "/workspace/deep_privacy/train.py", line 484, in <module> trainer.train()
File "/workspace/deep_privacy/train.py", line 464, in train self.maybe_validate_model()
File "/workspace/deep_privacy/train.py", line 412, in maybe_validate_model self.validate_model()
File "/workspace/deep_privacy/train.py", line 326, in validate_model log_to_validation=True)
File "/workspace/deep_privacy/logger.py", line 58, in save_images torchvision.utils.save_image(images, filepath, nrow=10)
File "/opt/conda/lib/python3.6/site-packages/torchvision/utils.py", line 101, in save_image normalize=normalize, range=range, scale_each=scale_each)
File "/opt/conda/lib/python3.6/site-packages/torchvision/utils.py", line 31, in make_grid raise TypeError('tensor or list of tensors expected, got {}'.format(type(tensor)))
TypeError: tensor or list of tensors expected, got <class 'numpy.ndarray'>
I wrote a print statement to logger.py file to see which "images" are being forwarded to torchvision.utils.save_image function and I saw that the docker crashes every time fakes*.jpg images are forwarded into the save_image function. Their are of <class 'numpy.ndarray'> class, whereas all the others (at which the docker doesn't crash) are of torch.Tensor type. However, I wrote an if statement if the images are numpy array class, and I change their type when that happens into torch.Tensor, but then this error happens:
RuntimeError: The expanded size of the tensor (3) must match the existing size (8) at non-singleton dimension 0. Target sizes: [3, 8, 3]. Tensor sizes: [8, 8, 3]
What can I do that it will go through as it should?
Hi, it seems that the config.yml file for the large "best" model is missing. The other model folder seems to have this. Would it be possible to update the google drive with the large config file?
How to prepare your own dataset for training
When running the webcam module, it crashes after some time with a division by zero error.
python webcam.py None 2020-11-23 16:53:20,565 [INFO ] Loaded model:fdf_512 2020-11-23 16:53:23,362 [INFO ] Generator initialized with 47.39M parameters {'type': 'RCNNDetector', 'keypoint_threshold': 0.2, 'densepose_threshold': 0.3, 'simple_expand': False, 'align_faces': False, 'resize_background': True, 'face_detector_cfg': {'name': 'RetinaNetResNet50', 'confidence_threshold': 0.1, 'nms_iou_threshold': 0.3, 'max_resolution': 1080, 'fp16_inference': True, 'clip_boxes': True}, 'rcnn_batch_size': 8} Traceback (most recent call last): File "webcam.py", line 48, in <module> print("FPS:", frames / (time.time() - t), end="\r") ZeroDivisionError: float division by zero
The same happens with a different model:
python webcam.py -m fdf128_retinanet128 None 2020-11-23 16:52:23,881 [INFO ] Loaded model:retinanet128 2020-11-23 16:52:25,888 [INFO ] Generator initialized with 3.17M parameters {'type': 'BaseDetector', 'keypoint_threshold': 0.2, 'densepose_threshold': 0.3, 'simple_expand': True, 'align_faces': False, 'resize_background': True, 'face_detector_cfg': {'name': 'RetinaNetResNet50', 'confidence_threshold': 0.1, 'nms_iou_threshold': 0.3, 'max_resolution': 1080, 'fp16_inference': True, 'clip_boxes': True}} Traceback (most recent call last): File "webcam.py", line 48, in <module> print("FPS:", frames / (time.time() - t), end="\r") ZeroDivisionError: float division by zero
seems like the demo with face configuration does not allow the upload of image, but tries to access the webcam in both modalities
When running the program and doing a conversion (after downloading the model and setting up all dependencies), I keep getting a Killed error about what seems to be half way through the process:
(base) stephenlizcano@deepfacelab-vm:~/DeepPrivacy$ python -m deep_privacy.inference.anonymize_video models/isvc_large/config.yml --source_path /home/stephenlizcano/deepfakeResize.mp4 --target_path /home/stephenlizcano/video_anonymized.mp4
CONFIG USED:
================================================================================
logging.num_ims_per_log 500
logging.num_ims_per_save_image 100000
logging.num_ims_per_checkpoint 200000
max_imsize 128
use_full_validation False
load_fraction_of_dataset False
dataset yfcc100m128
train_config.transition_iters 1200000
train_config.learning_rate 0.00175
train_config.amp_opt_level O1
train_config.batch_size_schedule {4: 256, 8: 256, 16: 256, 32: 128, 64: 96, 128: 64}
models.start_channel_size 512
models.pose_size 14
models.image_channels 3
models.generator.running_average_decay 0.999
models.discriminator.structure normal
config_path models/isvc_large/config.yml
checkpoint_dir models/isvc_large/checkpoints
generated_data_dir models/isvc_large/generated_data
summaries_dir models/isvc_large/summaries
source_path /home/stephenlizcano/deepfakeResize.mp4
target_path /home/stephenlizcano/video_anonymized.mp4
anonymize_source False
max_face_size 1.0
without_source False
================================================================================
[*] Loading checkpoint from models/isvc_large/checkpoints/step_40000000.ckpt succeed!
extending G 512
extending G 512
extending G 512
extending G 256
extending G 128
Anonymizer initialized. Keypoint threshold: 0.1Face threshold: 0.6
================================================================================
Anonymizing video.
Duration: 90.38. Total frames: 2259, FPS: 25.0
Anonymizing from: 0(0.0), to: 2259(90.36)
Reading frames: 100%|███████████████████████| 2259/2259 [00:30<00:00, 72.91it/s]
Batch detecting faces: 0%| | 0/2259 [00:00<?, ?it/s]Finished loading DSFD model!
Batch detecting faces: 100%|████████████████| 2259/2259 [10:02<00:00, 3.79it/s]
Killed
Hello,
I am trying to build the docker container using proposed docker file, but I am getting this error:
-- cut here --
Found link https://files.pythonhosted.org/packages/cb/28/91f26bd088ce8e22169032100d4260614fc3da435025ff389ef1d396a433/pip-20.2.4-py2.py3-none-any.whl#sha256=51f1c7514530bd5c145d8f13ed936ad6b8bfcb8cf74e10403d0890bc986f0033 (from https://pypi.org/simple/pip/) (requires-python:>=2.7,!=3.0.,!=3.1.,!=3.2.,!=3.3.,!=3.4.), version: 20.2.4
Found link https://files.pythonhosted.org/packages/0b/f5/be8e741434a4bf4ce5dbc235aa28ed0666178ea8986ddc10d035023744e6/pip-20.2.4.tar.gz#sha256=85c99a857ea0fb0aedf23833d9be5c40cf253fe24443f0829c7b472e23c364a1 (from https://pypi.org/simple/pip/) (requires-python:>=2.7,!=3.0.,!=3.1.,!=3.2.,!=3.3.,!=3.4.), version: 20.2.4
The command '/bin/sh -c pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./' returned a non-zero code: 1
-- cut here --
This is related to this line in the Dockerfile:
RUN pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
Any advise?
FM
Does this work on ears since they are increasingly being used for identification?
Hello, thank you very much for your work!
I have a problem with loading the checkpoint: when I use checkpoint places_E.ckpt
with config configs/gcpr/places/places_E_conv.py
, I end up with RuntimeError which cannot match the weights of my model with pretrained one:
RuntimeError: Error(s) in loading state_dict for MSGGenerator:
Missing key(s) in state_dict: "decoder.skip_connection8.conv.layers.0.weight", "decoder.skip_connection8.conv.layers.0.bias", "decoder.skip_connection16.conv.layers.0.weight", "decoder.skip_connection16.conv.layers.0.bias", "decoder.skip_connection32.conv.layers.0.weight", "decoder.skip_connection32.conv.layers.0.bias", "decoder.skip_connection64.conv.layers.0.weight", "decoder.skip_connection64.conv.layers.0.bias", "decoder.skip_connection128.conv.layers.0.weight", "decoder.skip_connection128.conv.layers.0.bias", "decoder.skip_connection256.conv.layers.0.weight", "decoder.skip_connection256.conv.layers.0.bias".
Unexpected key(s) in state_dict: "decoder.basic_block4.layers.0.expected_value_updater.input_updater.weight", "decoder.basic_block4.layers.0.expected_value_updater.input_updater.bias", "decoder.basic_block4.layers.0.mask_updater.weight", "decoder.basic_block4.layers.3.expected_value_updater.input_updater.weight", "decoder.basic_block4.layers.3.expected_value_updater.input_updater.bias", "decoder.basic_block4.layers.3.mask_updater.weight", "decoder.skip_connection8.beta0", "decoder.skip_connection8.beta1", "decoder.basic_block8.layers.0.expected_value_updater.input_updater.weight", "decoder.basic_block8.layers.0.expected_value_updater.input_updater.bias", "decoder.basic_block8.layers.0.mask_updater.weight", "decoder.basic_block8.layers.3.expected_value_updater.input_updater.weight", "decoder.basic_block8.layers.3.expected_value_updater.input_updater.bias", "decoder.basic_block8.layers.3.mask_updater.weight", "decoder.skip_connection16.beta0", "decoder.skip_connection16.beta1", "decoder.basic_block16.layers.0.expected_value_updater.input_updater.weight", "decoder.basic_block16.layers.0.expected_value_updater.input_updater.bias", "decoder.basic_block16.layers.0.mask_updater.weight", "decoder.basic_block16.layers.3.expected_value_updater.input_updater.weight", "decoder.basic_block16.layers.3.expected_value_updater.input_updater.bias", "decoder.basic_block16.layers.3.mask_updater.weight", "decoder.skip_connection32.beta0", "decoder.skip_connection32.beta1", "decoder.basic_block32.layers.0.expected_value_updater.input_updater.weight", "decoder.basic_block32.layers.0.expected_value_updater.input_updater.bias", "decoder.basic_block32.layers.0.mask_updater.weight", "decoder.basic_block32.layers.3.expected_value_updater.input_updater.weight", "decoder.basic_block32.layers.3.expected_value_updater.input_updater.bias", "decoder.basic_block32.layers.3.mask_updater.weight", "decoder.skip_connection64.beta0", "decoder.skip_connection64.beta1", "decoder.basic_block64.layers.0.expected_value_updater.input_updater.weight", "decoder.basic_block64.layers.0.expected_value_updater.input_updater.bias", "decoder.basic_block64.layers.0.mask_updater.weight", "decoder.basic_block64.layers.3.expected_value_updater.input_updater.weight", "decoder.basic_block64.layers.3.expected_value_updater.input_updater.bias", "decoder.basic_block64.layers.3.mask_updater.weight", "decoder.skip_connection128.beta0", "decoder.skip_connection128.beta1", "decoder.basic_block128.layers.0.expected_value_updater.input_updater.weight", "decoder.basic_block128.layers.0.expected_value_updater.input_updater.bias", "decoder.basic_block128.layers.0.mask_updater.weight", "decoder.basic_block128.layers.3.expected_value_updater.input_updater.weight", "decoder.basic_block128.layers.3.expected_value_updater.input_updater.bias", "decoder.basic_block128.layers.3.mask_updater.weight", "decoder.skip_connection256.beta0", "decoder.skip_connection256.beta1", "decoder.basic_block256.layers.0.expected_value_updater.input_updater.weight", "decoder.basic_block256.layers.0.expected_value_updater.input_updater.bias", "decoder.basic_block256.layers.0.mask_updater.weight", "decoder.basic_block256.layers.3.expected_value_updater.input_updater.weight", "decoder.basic_block256.layers.3.expected_value_updater.input_updater.bias", "decoder.basic_block256.layers.3.mask_updater.weight", "encoder.basic_block256.layers.0.expected_value_updater.input_updater.weight", "encoder.basic_block256.layers.0.expected_value_updater.input_updater.bias", "encoder.basic_block256.layers.0.mask_updater.weight", "encoder.basic_block256.layers.3.expected_value_updater.input_updater.weight", "encoder.basic_block256.layers.3.expected_value_updater.input_updater.bias", "encoder.basic_block256.layers.3.mask_updater.weight", "encoder.basic_block128.layers.0.expected_value_updater.input_updater.weight", "encoder.basic_block128.layers.0.expected_value_updater.input_updater.bias", "encoder.basic_block128.layers.0.mask_updater.weight", "encoder.basic_block128.layers.3.expected_value_updater.input_updater.weight", "encoder.basic_block128.layers.3.expected_value_updater.input_updater.bias", "encoder.basic_block128.layers.3.mask_updater.weight", "encoder.basic_block64.layers.0.expected_value_updater.input_updater.weight", "encoder.basic_block64.layers.0.expected_value_updater.input_updater.bias", "encoder.basic_block64.layers.0.mask_updater.weight", "encoder.basic_block64.layers.3.expected_value_updater.input_updater.weight", "encoder.basic_block64.layers.3.expected_value_updater.input_updater.bias", "encoder.basic_block64.layers.3.mask_updater.weight", "encoder.basic_block32.layers.0.expected_value_updater.input_updater.weight", "encoder.basic_block32.layers.0.expected_value_updater.input_updater.bias", "encoder.basic_block32.layers.0.mask_updater.weight", "encoder.basic_block32.layers.3.expected_value_updater.input_updater.weight", "encoder.basic_block32.layers.3.expected_value_updater.input_updater.bias", "encoder.basic_block32.layers.3.mask_updater.weight", "encoder.basic_block16.layers.0.expected_value_updater.input_updater.weight", "encoder.basic_block16.layers.0.expected_value_updater.input_updater.bias", "encoder.basic_block16.layers.0.mask_updater.weight", "encoder.basic_block16.layers.3.expected_value_updater.input_updater.weight", "encoder.basic_block16.layers.3.expected_value_updater.input_updater.bias", "encoder.basic_block16.layers.3.mask_updater.weight", "encoder.basic_block8.layers.0.expected_value_updater.input_updater.weight", "encoder.basic_block8.layers.0.expected_value_updater.input_updater.bias", "encoder.basic_block8.layers.0.mask_updater.weight", "encoder.basic_block8.layers.3.expected_value_updater.input_updater.weight", "encoder.basic_block8.layers.3.expected_value_updater.input_updater.bias", "encoder.basic_block8.layers.3.mask_updater.weight", "encoder.basic_block4.layers.0.expected_value_updater.input_updater.weight", "encoder.basic_block4.layers.0.expected_value_updater.input_updater.bias", "encoder.basic_block4.layers.0.mask_updater.weight", "encoder.basic_block4.layers.3.expected_value_updater.input_updater.weight", "encoder.basic_block4.layers.3.expected_value_updater.input_updater.bias", "encoder.basic_block4.layers.3.mask_updater.weight", "from_rgb.0.expected_value_updater.input_updater.weight", "from_rgb.0.expected_value_updater.input_updater.bias".
size mismatch for from_rgb.0.weight: copying a param with shape torch.Size([16, 3, 1, 1]) from checkpoint, the shape in current model is torch.Size([16, 5, 1, 1]).
Process finished with exit code 1
Will be grateful for any help!
Hey quite new to this , can you help in a doubt . when you say install pythorch and python , where exactly should I type those codelines .
Hello,
I was wondering how the resolution of the output images is chosen? Is this a parameter of the model, which could be trained to output higher resolution images?
Alternatively do you think it is reasonable to cut-out the face from a higher resolution image and input this into your program, then re-integrate into the original resolution image? If so, how much context surrounding the face would be good to include in these subsets?
My goal is to benchmark how this tool works on high resolution images.
Thanks,
James
i have Nvidia GPU 8 GB total capacity. i run anonymize video on a mp4 video with resolution 240x160 and lenght 00:00:01.
and get this error:
RuntimeError: CUDA out of memory. Tried to allocate 850.00 MiB (GPU 0; 8.00 GiB total capacity; 4.64 GiB already allocated; 298.65 MiB free; 5.96 GiB reserved in total by PyTorch)
what is wrong?
thank you
Thanks for the detailed write up of your amazing work. I have a few questions, in particular relating to "Image Inpainting with Learnable Feature Imputation".
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.