minivision-ai / photo2cartoon Goto Github PK

您好，感谢您分享这个有趣的项目，我关于卡通画像关键点检测那边有一些疑问
现在的关键点检测大多都是在现实图像里面的，貌似不同直接套用在卡通画像里面，想问一下这边你们是怎么处理的，有用到迁移学习什么的吗？还是自己标注数据呢？

如题...
这可 excited

Traceback (most recent call last):
File "test.py", line 59, in
cartoon = c2p.inference(img)
File "test.py", line 38, in inference
mask = face_rgba[:, :, 3][:, :, np.newaxis].copy() / 255.
IndexError: index 3 is out of bounds for axis 2 with size 3

这个要怎么解决呢？谢谢你们

1.请问在README.md,您说"首先训练模型20万步，然后使用FID指标挑选出最优模型，最终挑选出的模型为迭代9万步时的模型。"
为什么要这样做而不是直接训练到代码里默认的100万步？以及如何用FID指标挑选呢？

2.请问小程序中演示的效果，您在训练时，真实照片和卡通画像各大约多少张呢？

3.之前看抖音的宝宝特效(比如把明星变成宝宝) 很火，就就很想尝试一下，请问用您的代码可以实现类似的效果吗？您对尝试这块，数据或代码上有什么建议吗？

恳请回复，万分感谢！

作者您好，感谢你分享此项目。
1.请教您一个问题，训练模型时，男女生图片是要单独训练一个模型，还是图片混合到一起，只训练一个模型？

每10个iteration大概耗时16s，200k iteration耗时有点长，

一般gan的batch_size很多设置为1， 有其他方法提升训练速度吗？

Traceback (most recent call last):
File "test.py", line 56, in
cartoon = c2p.inference(img)
File "test.py", line 43, in inference
cartoon = self.net(face)[0][0]
File "C:\ProgramData\Anaconda3\envs\gpu\lib\site-packages\torch\nn\modules\module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "C:\Users\24238\Desktop\zhangting\photo2cartoon\models\networks.py", line 120, in forward
x = self.DecodeBlock1(x, content_features4, style_features)
File "C:\ProgramData\Anaconda3\envs\gpu\lib\site-packages\torch\nn\modules\module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "C:\Users\24238\Desktop\zhangting\photo2cartoon\models\networks.py", line 296, in forward
out = self.norm1(out, content_features, style_features)
File "C:\ProgramData\Anaconda3\envs\gpu\lib\site-packages\torch\nn\modules\module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "C:\Users\24238\Desktop\zhangting\photo2cartoon\models\networks.py", line 354, in forward
out = self.norm(x, soft_gamma, soft_beta)
File "C:\ProgramData\Anaconda3\envs\gpu\lib\site-packages\torch\nn\modules\module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "C:\Users\24238\Desktop\zhangting\photo2cartoon\models\networks.py", line 366, in forward
in_mean, in_var = torch.mean(input, dim=[2, 3], keepdim=True), torch.var(input, dim=[2, 3], keepdim=True)
TypeError: var(): argument 'dim' must be int, not list

dear maintainer,
你好，当我测试的时候在face.seg的tf.session报错如下
*** Error in `/mnt/lustre/shanyuhao/.conda/envs/yh0524/bin/python': munmap_chunk(): invalid pointer: 0x00007f17d33bed40 ***
请问如何解决？
谢谢！

报错：Tried to allocate 20.00 MiB (GPU 0; 5.94 GiB total capacity; 5.22 GiB already allocated; 20.81 MiB free; 5.42 GiB reserved in total by PyTorch
是因为我的显存太小吗 T.T

hi，你好，我在windows下进行安装测试，但是一直在：
Downloading the Face Alignment Network(FAN). Please wait...

然后下载中断，有什么其他的解决方式吗，比如我先下载这个FAN之类的

另外，你们小程序是输入512，然后训练的trainb部分也是只有200多张吗？

I mean when it's working good, but sometimes it fail. The best.
Good job.

您好，请问如果要训练自己想要风格的模型，训练数据的真人图片和卡通图片需要对应吗

在window10环境中调用test.py：
python test.py --photo_path ./images/photo_test.jpg --save_path ./images/cartoon_result.png
程序没有报错，也无法得出结果。

Warning: this detector is deprecated. Please use a different one, i.e.: S3FD.
WARNING:tensorflow:From E:\DL_with_coding\photo2cartoon\utils\face_seg.py:13: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.
ConfigProto instead.

WARNING:tensorflow:From E:\DL_with_coding\photo2cartoon\utils\face_seg.py:16: The name tf.Session is deprecated. Please use tf.compat.v1.Sess
ion instead.

2020-04-22 11:46:56.162121: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary w
as not compiled to use: AVX AVX2
2020-04-22 11:46:56.167602: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library nvcuda.dll
2020-04-22 11:46:57.083446: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX 1060 with Max-Q Design major: 6 minor: 1 memoryClockRate(GHz): 1.3415
pciBusID: 0000:01:00.0
2020-04-22 11:46:57.090947: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, ski
p dlopen check.
2020-04-22 11:46:57.096467: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2020-04-22 11:46:58.398108: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge
matrix:
2020-04-22 11:46:58.403445: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0
2020-04-22 11:46:58.406841: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N
2020-04-22 11:46:58.410200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task
:0/device:GPU:0 with 4714 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1060 with Max-Q Design, pci bus id: 0000:01:00.0, compute
capability: 6.1)
WARNING:tensorflow:From E:\DL_with_coding\photo2cartoon\utils\face_seg.py:26: FastGFile.init (from tensorflow.python.platform.gfile) is d
eprecated and will be removed in a future version.
Instructions for updating:
Use tf.gfile.GFile.
WARNING:tensorflow:From E:\DL_with_coding\photo2cartoon\utils\face_seg.py:27: The name tf.GraphDef is deprecated. Please use tf.compat.v1.Gra
phDef instead.

仅仅运行test.py的话只安装这一个就可以了

网上找了很多地方，终于找到了python3.8的dlib，放在这里供大家参阅（博主真的好人一个）
https://blog.csdn.net/landing_guy_/article/details/115228797

请问能否提供头像分割模型的训练数据集？

请问小程序上的多种效果是用本项目生成后再做后处理的吗？

我是Python新手，我在使用 test.py 时碰到了问题。一直卡在 self.fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D, flip_input=False)这一句上，我猜测的原因为pytorch的模型没加载好。

万分感谢！

centos下跑出来的效果和win7上面不一样，centos下跑出来的卡通图有些失真的感觉，但win7下跑出来示例的结果和你给出的效果就比较一致。centos下使用的是 nvida p4，win7使用的是Quadro K620，ps: 显卡驱动检查没问题

how can I finetune the existing photo2cartoon weights, I loaded the weights but then the training starts from 0 iterations

还有待改进的地方，不少。

和你们demo里的图片不一样？
https://s1.ax1x.com/2020/06/23/NUcwr9.png

你好，我想问下，trainA人脸照片和trainB动漫人脸， 两份数据是需要对应人吗(或者说 trainB里面都是trainA里人的动漫脸)？
比如 trainA 目录下的 a.jpg 和 trainB目录下的 a.jpg 是同一个人，因为我看你用了feature计算特征距离

I train the model with the given dataset, but found that some faces with light get a bad result at the edge of face.

想问下你们小程序后端部署也是用的torch模型么 能简单说下部署小细节么 感谢

请问提供的模型训练集中只有女性的数据吗？输入男性的图片生成的图片都被女性化了

你好，在readme的Q&A中，你们描述小程序中的效果是使用了200张定制卡通图训练，这个数据量是不是太小了，还是说这200张定制的卡通图只是用来最后微调的？
此外，你们定制卡通图的时候应该是参照真实人脸图像找专业人士绘制的吧，这样的话是不是等于有了成对的训练数据，你们用成对的数据进行有监督学习么，这样对最后的效果会不会有提升呢？

您好，非常不错的工作。
由于开源模型基于女性数据训练，因此结果带有明显的女性特征，而小程序的结果泛化能力更强。在 Readme 中，提到内部训练数据是200多张定制化的卡通画，想请问一下，这200多张是包含了男、女总共的数据大小吗？

ONNX export failed on ATen operator var because torch.onnx.symbolic_opset9.var does not exist，有大佬遇见过吗

I try to run it in google colab.
https://colab.research.google.com/drive/1kt3JpWFvmlmjv4lGQrKNNRKVoOqsw6uV?usp=sharing
But getting an error.

Traceback (most recent call last):
  File "test.py", line 5, in <module>
    from models import ResnetGenerator
  File "/content/photo2cartoon/models/__init__.py", line 2, in <module>
    from .UGATIT_sadalin_hourglass import UgatitSadalinHourglass
  File "/content/photo2cartoon/models/UGATIT_sadalin_hourglass.py", line 7, in <module>
    from utils import *
  File "/content/photo2cartoon/utils/__init__.py", line 1, in <module>
    from .preprocess import Preprocess
  File "/content/photo2cartoon/utils/preprocess.py", line 1, in <module>
    from .face_detect import FaceDetect
  File "/content/photo2cartoon/utils/face_detect.py", line 4, in <module>
    import face_alignment
ModuleNotFoundError: No module named 'face_alignment'

hello!Where can i find the paper?

/usr/bin/python test.py --photo_path ./images/photo_test.jpg --save_path ./images/cartoon_result.png
2020-05-05 19:57:44.377745: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.2
Traceback (most recent call last):
File "test.py", line 5, in
from models import ResnetGenerator
File "/home/pankaj/Learning/photo2cartoon/models/init.py", line 2, in
from .UGATIT_sadalin_hourglass import UgatitSadalinHourglass
File "/home/pankaj/Learning/photo2cartoon/models/UGATIT_sadalin_hourglass.py", line 7, in
from utils import *
File "/home/pankaj/Learning/photo2cartoon/utils/init.py", line 1, in
from .preprocess import Preprocess
File "/home/pankaj/Learning/photo2cartoon/utils/preprocess.py", line 2, in
from .face_seg import FaceSeg
File "/home/pankaj/Learning/photo2cartoon/utils/face_seg.py", line 4, in
import tensorflow as tf
File "/usr/lib/python3.8/site-packages/tensorflow/init.py", line 41, in
from tensorflow.python.tools import module_util as _module_util
File "/usr/lib/python3.8/site-packages/tensorflow/python/init.py", line 64, in
from tensorflow.python.framework.framework_lib import * # pylint: disable=redefined-builtin
File "/usr/lib/python3.8/site-packages/tensorflow/python/framework/framework_lib.py", line 52, in
from tensorflow.python.framework.importer import import_graph_def
File "/usr/lib/python3.8/site-packages/tensorflow/python/framework/importer.py", line 28, in
from tensorflow.python.framework import function
File "/usr/lib/python3.8/site-packages/tensorflow/python/framework/function.py", line 37, in
from tensorflow.python.ops import resource_variable_ops
File "/usr/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 41, in
from tensorflow.python.ops import gen_logging_ops
File "/usr/lib/python3.8/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 312, in
def image_summary(tag, tensor, max_images=3, bad_color=_execute.make_tensor("""dtype: DT_UINT8 tensor_shape { dim { size: 4 } } int_val: 255 int_val: 0 int_val: 0 int_val: 255""", "bad_color"), name=None):
File "/usr/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 229, in make_tensor
text_format.Merge(v, pb)
File "/usr/lib/python3.8/site-packages/google/protobuf/text_format.py", line 696, in Merge
return MergeLines(
File "/usr/lib/python3.8/site-packages/google/protobuf/text_format.py", line 770, in MergeLines
return parser.MergeLines(lines, message)
File "/usr/lib/python3.8/site-packages/google/protobuf/text_format.py", line 795, in MergeLines
self._ParseOrMerge(lines, message)
File "/usr/lib/python3.8/site-packages/google/protobuf/text_format.py", line 817, in _ParseOrMerge
self._MergeField(tokenizer, message)
File "/usr/lib/python3.8/site-packages/google/protobuf/text_format.py", line 942, in _MergeField
merger(tokenizer, message, field)
File "/usr/lib/python3.8/site-packages/google/protobuf/text_format.py", line 1105, in _MergeScalarField
setattr(message, field.name, value)
TypeError: descriptor to field 'tensorflow.TensorProto.dtype' doesn't apply to 'TensorProto' object

文件：https://github.com/minivision-ai/photo2cartoon/blob/master/models/networks.py
行：224-237
是不是应该是：

 skip1 = self.ConvBlock1_1(x)
down1 = F.avg_pool2d(x, 2)
down1 = self.ConvBlock1_2(down1)

skip2 = **self.ConvBlock2_1(down1)**
down2 = F.avg_pool2d(down1, 2)
down2 = **self.ConvBlock2_2(down2)**

skip3 = **self.ConvBlock3_1(down2)**
down3 = F.avg_pool2d(down2, 2)
down3 = **self.ConvBlock3_2(down3)**

skip4 = **self.ConvBlock4_1(down3)**
down4 = F.avg_pool2d(down3, 2)
down4 = **self.ConvBlock4_2(down4)**

按照你们的要求，请问你是怎么成功把pytorch1.4和tensorflow1.14装到同一个环境里面的，因为pytorch1.14需要cuda10.2，而tensorflow1.14需要cuda10.0，这个问题你们是怎么解决的？

在另一台机器能跑出正确结果，但今天在另一台配置的机器上跑发现mask出不来。
发现｀utils/face_seg.py｀里output_op run出来的值并不是0-1之前，最大值有1.5300853e+26，非常奇怪，经过sigmoid的值怎么会大于1?

版本应该是没错的：
Python 3.6.9
tensorflow-gpu 1.14.0
感谢提供下排查思路！

您好我有两个问题：（1）如果重新训练一个模型，trainA和trainB数据集中大概有多少张图片才可以？
（2）一般要训练到多少代（iteations）才能基本达到给出的转换效果？

我的电脑没有GPU，我想直接用您的模型，不做训练，这样仅有CPU是否可以呢？

what would be the steps to export this to a model.onnx file?

Great work!
请问一下为什么不直接使用dlib的shape_predictor_68_face_landmarks.dat获得人脸landmarks，而使用face_alignment？

2020-04-30 15:42:14.563154: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-04-30 15:42:14.571325: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
等待响应时间长，报错
urllib.error.ContentTooShortError
求解决方法！！！

运行:
python test.py --photo_path ./images/photo_test.jpg --save_path ./images/cartoon_result.png

一直提示:
Downloading the Face Alignment Network(FAN). Please wait...

国内下载不动

有什么办法快速下载这个包吗？

您好，我现在训练的d_loss在1.0-2.0徘徊，g_loss在40-80徘徊，训练了8k左右个iteration。为了训练稳定，我把face_id loss的weight调成0了，其他权重保持默认值。想问下您的实验中最终的d_loss和g_loss长啥样吗？或者如果您有画loss曲线，能否分享下呢？感谢。

作者，您好，您的工作做的非常棒，给了我很多启发。我想请问设置两个分别在layer=5和layer=7的判别器的用意是什么呢？另外这个参数的设置是否有什么经验可循？