irisrainbowneko / ml-danbooru

Anime image tags detector

ML-Danbooru: Anime image tags detector

Introduction

An anime image tag detector based on modified ML-Decoder. Model trained with cleaned danbooru2021.

• Designed a new TResNet-D structure as backbone to enhance the learning of low-level features.
• Replace the ReLU in backbone with FReLU.
• Using learnable queries for transformer decoder.

Model Structure

Model-Zoo

https://huggingface.co/7eu7d7/ML-Danbooru

Usage

Download the model and run below command:

python demo.py --data <path to image or directory> --model_name tresnet_d --num_of_groups 32 --ckpt <path to ckpt> --thr 0.7 --image_size 640 

Keep the image ratio invariant:

python demo.py --data <path to image or directory> --model_name tresnet_d --num_of_groups 32 --ckpt <path to ckpt> --thr 0.7 --image_size 640 --keep_ratio True

ML_CAFormer

python demo_ca.py --data <path to image or directory> --model_name caformer_m36 --ckpt <path to ckpt> --thr 0.7 --image_size 448

ml-danbooru's Issues

模型微调求助

我完全不懂深度学习模型方面的知识，我已经准备好了训练用数据（总共255个样本），训练集和验证集分好了，文件夹内是训练图片和对应的.txt标签文件（和你的数据读取有点区别，不过这个我可以自己更改）。目前在更改代码的过程中遇到了一些问题：
1.最主要的问题是新增的标签不在class.json中，这意味着分类数比12547更大。这种情况下，在我读取预训练模型ml_caformer-97527.ckpt后应该更改哪些层，冻结哪些层进行训练？
2.除了上述提到的数据预处理部分和训练层的问题，我还需要更改什么地方的？

这个模型cpu能跑吗？

渣电脑没gpu

[功能请求] SD-WebUI的脚本插件。

大佬不考虑整合到 SD-WebUI吗？写个脚本配合wd-tagger的插件使用模型。

Couldn't run Usage through requirement.txt

(venv) F:\ML-Danbooru>python demo.py --data F:\collapse_ben_train_ml --model_name tresnet_d --num_of_groups 32 --ckpt F:\ML-Danbooru\model --thr 0.7 --image_size 512
xformers not find
Traceback (most recent call last):
File "F:\ML-Danbooru\demo.py", line 12, in
from src_files.models import create_model
File "F:\ML-Danbooru\src_files\models_init_.py", line 1, in
from .utils import create_model
File "F:\ML-Danbooru\src_files\models\utils_init_.py", line 1, in
from .factory import create_model
File "F:\ML-Danbooru\src_files\models\utils\factory.py", line 9, in
from ..caformer import build_caformer
File "F:\ML-Danbooru\src_files\models\caformer_init_.py", line 1, in
from .ml_caformer import build_caformer
File "F:\ML-Danbooru\src_files\models\caformer\ml_caformer.py", line 8, in
from .metaformer_baselines import MetaFormer
File "F:\ML-Danbooru\src_files\models\caformer\metaformer_baselines.py", line 27, in
from timm.models.layers.helpers import to_2tuple
ModuleNotFoundError: No module named 'timm.models.layers.helpers'

(venv) F:\ML-Danbooru>pip list
Package Version

---

certifi 2023.5.7
charset-normalizer 3.1.0
colorama 0.4.6
einops 0.6.1
filelock 3.12.0
fsspec 2023.5.0
huggingface-hub 0.14.1
idna 3.4
inplace-abn 1.1.0
Jinja2 3.1.2
loguru 0.7.0
MarkupSafe 2.1.2
mpmath 1.3.0
networkx 3.1
numpy 1.24.3
packaging 23.1
Pillow 9.5.0
pip 22.2.1
PyYAML 6.0
requests 2.30.0
safetensors 0.3.1
setuptools 63.2.0
sklearn 0.0.post5
sympy 1.12
timm 0.9.2
torch 2.0.1
torchvision 0.15.2
tqdm 4.65.0
typing_extensions 4.5.0
urllib3 2.0.2
win32-setctime 1.1.0

大佬请问，对于特殊的打标tag，怎么训练成一个可以反推的tag模型

如题？