ryuukeisyou / comfyui_face_parsing

A set of custom nodes for ComfyUI. The nodes utilize the face parsing model to provide detailed segmantation of face.

Introduction

This is a set of custom nodes for ComfyUI. The nodes utilize the face parsing model to provide detailed segmantation of face. To improve face segmantation accuracy, yolov8 face model is used to first extract face from an image.

There are also auxiliary nodes for image and mask processing. A guided filter is also provided for skin smoothing.

If you encounter any problems, feel free to open an issue. I'll try to fix it when I'm not busy. Besides, if you like the nodes, please give a star, thanks!

Installation

Just clone the repo https://github.com/Ryuukeisyou/comfyui_face_parsing.git into ComfyUI's "custom_nodes" directory.

On first run, some required files will be downloaded to ComfyUI's "models/face_parsing/" folder and "models/ultralytics/bbox" folder. If you have difficulty connecting to Huggingface, you can try mannually:

1. Download the .json and .safetensors files from the face parsing model repo and save them to the "models/face_parsing/" folder.
2. Download the face_yolov8m.pt from the Bingsu/adetailer repo and save it to "models/ultralytics/bbox" folder.

Requirements

See requirements.txt. Specially, opencv-contrib-python is needed for guided filter. If you have other versions of opencv e.g. opencv-python-headless installed after open-contrib-python, you may need to uninstall opencv-contrib-python and reinstall it again.

Workflow

a demo work flow is shown below.

Original	Processed

This image contains the workflow which can be loaded into ComfyUI.

comfyui_face_parsing's Issues

FaceBBoxDetectorLoader(FaceParsing)

Starting server

To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: D:\NTest\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json
[ERROR] An error occurred while retrieving information for the 'FaceBBoxDetectorLoader(FaceParsing)' node.
Traceback (most recent call last):
File "D:\NTest\ComfyUI\ComfyUI\server.py", line 414, in get_object_info
out[x] = node_info(x)
^^^^^^^^^^^^
File "D:\NTest\ComfyUI\ComfyUI\server.py", line 392, in node_info
info['input'] = obj_class.INPUT_TYPES()
^^^^^^^^^^^^^^^^^^^^^^^
File "D:\NTest\ComfyUI\ComfyUI\custom_nodes\comfyui_face_parsing\face_parsing_nodes.py", line 24, in INPUT_TYPES
files = folder_paths.get_filename_list("ultralytics_bbox")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\NTest\ComfyUI\ComfyUI\folder_paths.py", line 220, in get_filename_list
out = get_filename_list_(folder_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\NTest\ComfyUI\ComfyUI\folder_paths.py", line 187, in get_filename_list_
folders = folder_names_and_paths[folder_name]
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'ultralytics_bbox'

Error while deserializing header: MetadataIncompleteBuffer

I get following error when using faceparsing: (my comfui is updated to latest commit)

Error occurred when executing FaceParsingModelLoader(FaceParsing):

Error while deserializing header: MetadataIncompleteBuffer

File "c:\comfyui\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "c:\comfyui\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "c:\comfyui\ComfyUI\execution.py", line 70, in map_node_over_list
results.append(getattr(obj, func)())
File "C:\comfyui\ComfyUI\custom_nodes\ComfyUI_face_parsing\face_parsing_nodes.py", line 534, in main
model = AutoModelForSemanticSegmentation.from_pretrained(face_parsing_path)
File "C:\stable-diffusion-webui\system\python\lib\site-packages\transformers\models\auto\auto_factory.py", line 484, in from_pretrained
return model_class.from_pretrained(
File "C:\stable-diffusion-webui\system\python\lib\site-packages\transformers\modeling_utils.py", line 2604, in from_pretrained
state_dict = load_state_dict(resolved_archive_file)
File "C:\stable-diffusion-webui\system\python\lib\site-packages\transformers\modeling_utils.py", line 450, in load_state_dict
with safe_open(checkpoint_file, framework="pt") as f:

workflow.

Can you share the workflow as the workflow.png doesnot load it into comfyui.

Error occurred when executing GuidedFilter(FaceParsing)

Error occurred when executing GuidedFilter(FaceParsing):

module 'cv2.ximgproc' has no attribute 'guidedFilter'

File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 154, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 84, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_ezXY\autoCastPatch.py", line 299, in map_node_over_list
return _map_node_over_list(obj, input_data_all, func, allow_interrupt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\execution.py", line 77, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SD\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_face_parsing\face_parsing_nodes.py", line 733, in guided_filter
result_cv2 = cv2.ximgproc.guidedFilter(guide_cv2, image_cv2, radius, eps)
^^^^^^^^^^^^^^^^^^^^^^^^^

not work well for find the face

when use it, bbox does not find the right face,

Error in GuidedFilter(FaceParsing) Node: No Attribute 'guidedFilter'

Description

Encountered an error while executing the GuidedFilter(FaceParsing) node within a workflow shared in another post, which includes a comparison operation. It's unclear what should be provided in the guide input of that node. Also, what is the version of opencv-contrib-python to be used? In the requirements.txt there is no version therefore the latest is installing (4.9.0.80).

Error Message:

The following error message is received when executing the node:
Error occurred when executing GuidedFilter(FaceParsing):

module 'cv2.ximgproc' has no attribute 'guidedFilter'

Traceback (most recent call last):
File "M:\AIgenerated\StableDiffusion\ComfyUI2\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "M:\AIgenerated\StableDiffusion\ComfyUI2\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "M:\AIgenerated\StableDiffusion\ComfyUI2\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "M:\AIgenerated\StableDiffusion\ComfyUI2\custom_nodes\comfyui_face_parsing\face_parsing_nodes.py", line 923, in guided_filter
result_cv2 = cv2.ximgproc.guidedFilter(guide_cv2, image_cv2, radius, eps)
AttributeError: module 'cv2.ximgproc' has no attribute 'guidedFilter'

Console output:

ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "M:\AIgenerated\StableDiffusion\ComfyUI2\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "M:\AIgenerated\StableDiffusion\ComfyUI2\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "M:\AIgenerated\StableDiffusion\ComfyUI2\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "M:\AIgenerated\StableDiffusion\ComfyUI2\custom_nodes\comfyui_face_parsing\face_parsing_nodes.py", line 923, in guided_filter
result_cv2 = cv2.ximgproc.guidedFilter(guide_cv2, image_cv2, radius, eps)
^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'cv2.ximgproc' has no attribute 'guidedFilter'

Prompt executed in 0.05 seconds

Environment

ComfyUI version: (specify the version if known)
Python version: 3.11.6
CUDA/CuDNN version: cudnn 12.1
Hardware specs:
VRAM: 24563 MB
RAM: 32693 MB
GPU: NVIDIA GeForce RTX 4090
Additional Libraries:
torch: 2.1.2+cu121
torchaudio: 2.1.2+cu121
torchmetrics: 1.3.0.post0
torchsde: 0.2.6
torchvision: 0.16.2+cu121
xformers: 0.0.23.post1
opencv-contrib-python: 4.9.0.80
opencv-python: 4.9.0.80
opencv-python-headless: 4.7.0.72

Steps to Reproduce

-Normal installation of ComfyUI (non-portable version).
-Execute the GuidedFilter(FaceParsing) node with the mentioned configuration.
-Encounter the attribute error regarding 'cv2.ximgproc'.

Expected Behavior

GuidedFilter should execute without errors, processing the inputs as intended.

Actual Behavior

The execution fails due to the missing 'guidedFilter' attribute in 'cv2.ximgproc'.

the readme is not suitable

When I perform manual operations with reference to the current readme, the following error will occur. After checking the code, I found that it was a problem with the wrong path. Then I found that you updated the code two days ago. The operation of the readme should not be consistent with the current code. If the author If you have time, I hope you can update the readme file. Thank you very much.