zhangzjn / ader

ADer is an open source visual anomaly detection toolbox based on PyTorch, which supports multiple popular AD datasets and approaches.

Jiangning Zhang (张江宁) works as a Principal Researcher of two teams (Industry Perception and AIGC) at YouTu Lab, Tencent, Shanghai. I receive Ph.D. degree in College of Control Science and Engineering, Zhejiang University, Hangzhou, China, under the supervision of Prof. Yong Liu. My major is Computer Vision and my research interests include:
🌱 GAN-/Diffusion-based AIGC researches with LLM, e.g., multi-modal image/video generation, 2D/3D virtual digital human related researches (3D face/body/hand reconstruction, multi-modal digital human drive, motion generation, etc.), text-to-image generation, multi-modal human-centric editing and generation, etc.
🌱 Neural Architecture Design (NAD), e.g., transformer-based architecture, light-wight vision model, etc.
🌱 Anomaly Classification and Segmentation.

• 💬 Feel free to drop me emails ([email protected]) if you have interests on above topics, and remote cooperations are welcomed.
• 💬 You can contact me if you are applying for a Research Intern or a B.S./Ph.D. student in computer vision / robotic perception, and I co-supervise students with Prof. Yong Liu at Zhejiang University.
• 💖 I love 📷photography, 🍲cooking, and 🌏traveling, enjoy together!!!!!!

ader's Issues

Training is just dangling

I made my data as exactlysame as format of mvtec. And then did modification on some codes to follow training my new data.

But after a bit of training, it is stuck and never processing forward like below showing 2/3. No more running GPU but GPU memory is still taken. Can I get some help out of this?

The whole thing after running the training code is shown as below

`
CUDA_VISIBLE_DEVICES=0 python run.py -c configs/vitad/vitad_cj.py -m train
08/30 10:46:56 AM - ==> Logging on master GPU: 0
08/30 10:46:56 AM - ==> Running Trainer: ViTADTrainer
08/30 10:46:56 AM - ==> Using GPU: [0] for Training
08/30 10:46:56 AM - ==> Building model
08/30 10:46:56 AM - Loading pretrained weights from Hugging Face hub (timm/vit_small_patch16_224.dino)
08/30 10:46:57 AM - [timm/vit_small_patch16_224.dino] Safe alternative available for 'pytorch_model.bin' (as 'model.safetensors'). Loading weights using safetensors.
08/30 10:46:57 AM - Resized position embedding: (14, 14) to (16, 16).
08/30 10:46:57 AM -
------------------------------------ ViTAD ------------------------------------

module	#parameters or shape	#flops
model	38.586M	9.668G
net_t	21.689M	5.544G
net_t.cls_token	(1, 1, 384)
net_t.pos_embed	(1, 257, 384)
net_t.patch_embed.proj	0.295M	75.497M
net_t.patch_embed.proj.weight	(384, 3, 16, 16)
net_t.patch_embed.proj.bias	(384,)
net_t.blocks	21.294M	5.469G
net_t.blocks.0	1.774M	0.456G
net_t.blocks.0.norm1	0.768K	0.493M
net_t.blocks.0.norm1.weight	(384,)
net_t.blocks.0.norm1.bias	(384,)
net_t.blocks.0.attn	0.591M	0.152G
net_t.blocks.0.attn.qkv	0.444M	0.114G
net_t.blocks.0.attn.proj	0.148M	37.896M
net_t.blocks.0.norm2	0.768K	0.493M
net_t.blocks.0.norm2.weight	(384,)
net_t.blocks.0.norm2.bias	(384,)
net_t.blocks.0.mlp	1.182M	0.303G
net_t.blocks.0.mlp.fc1	0.591M	0.152G
net_t.blocks.0.mlp.fc2	0.59M	0.152G
net_t.blocks.1	1.774M	0.456G
net_t.blocks.1.norm1	0.768K	0.493M
net_t.blocks.1.norm1.weight	(384,)
net_t.blocks.1.norm1.bias	(384,)
net_t.blocks.1.attn	0.591M	0.152G
net_t.blocks.1.attn.qkv	0.444M	0.114G
net_t.blocks.1.attn.proj	0.148M	37.896M
net_t.blocks.1.norm2	0.768K	0.493M
net_t.blocks.1.norm2.weight	(384,)
net_t.blocks.1.norm2.bias	(384,)
net_t.blocks.1.mlp	1.182M	0.303G
net_t.blocks.1.mlp.fc1	0.591M	0.152G
net_t.blocks.1.mlp.fc2	0.59M	0.152G
net_t.blocks.2	1.774M	0.456G
net_t.blocks.2.norm1	0.768K	0.493M
net_t.blocks.2.norm1.weight	(384,)
net_t.blocks.2.norm1.bias	(384,)
net_t.blocks.2.attn	0.591M	0.152G
net_t.blocks.2.attn.qkv	0.444M	0.114G
net_t.blocks.2.attn.proj	0.148M	37.896M
net_t.blocks.2.norm2	0.768K	0.493M
net_t.blocks.2.norm2.weight	(384,)
net_t.blocks.2.norm2.bias	(384,)
net_t.blocks.2.mlp	1.182M	0.303G
net_t.blocks.2.mlp.fc1	0.591M	0.152G
net_t.blocks.2.mlp.fc2	0.59M	0.152G
net_t.blocks.3	1.774M	0.456G
net_t.blocks.3.norm1	0.768K	0.493M
net_t.blocks.3.norm1.weight	(384,)
net_t.blocks.3.norm1.bias	(384,)
net_t.blocks.3.attn	0.591M	0.152G
net_t.blocks.3.attn.qkv	0.444M	0.114G
net_t.blocks.3.attn.proj	0.148M	37.896M
net_t.blocks.3.norm2	0.768K	0.493M
net_t.blocks.3.norm2.weight	(384,)
net_t.blocks.3.norm2.bias	(384,)
net_t.blocks.3.mlp	1.182M	0.303G
net_t.blocks.3.mlp.fc1	0.591M	0.152G
net_t.blocks.3.mlp.fc2	0.59M	0.152G
net_t.blocks.4	1.774M	0.456G
net_t.blocks.4.norm1	0.768K	0.493M
net_t.blocks.4.norm1.weight	(384,)
net_t.blocks.4.norm1.bias	(384,)
net_t.blocks.4.attn	0.591M	0.152G
net_t.blocks.4.attn.qkv	0.444M	0.114G
net_t.blocks.4.attn.proj	0.148M	37.896M
net_t.blocks.4.norm2	0.768K	0.493M
net_t.blocks.4.norm2.weight	(384,)
net_t.blocks.4.norm2.bias	(384,)
net_t.blocks.4.mlp	1.182M	0.303G
net_t.blocks.4.mlp.fc1	0.591M	0.152G
net_t.blocks.4.mlp.fc2	0.59M	0.152G
net_t.blocks.5	1.774M	0.456G
net_t.blocks.5.norm1	0.768K	0.493M
net_t.blocks.5.norm1.weight	(384,)
net_t.blocks.5.norm1.bias	(384,)
net_t.blocks.5.attn	0.591M	0.152G
net_t.blocks.5.attn.qkv	0.444M	0.114G
net_t.blocks.5.attn.proj	0.148M	37.896M
net_t.blocks.5.norm2	0.768K	0.493M
net_t.blocks.5.norm2.weight	(384,)
net_t.blocks.5.norm2.bias	(384,)
net_t.blocks.5.mlp	1.182M	0.303G
net_t.blocks.5.mlp.fc1	0.591M	0.152G
net_t.blocks.5.mlp.fc2	0.59M	0.152G
net_t.blocks.6	1.774M	0.456G
net_t.blocks.6.norm1	0.768K	0.493M
net_t.blocks.6.norm1.weight	(384,)
net_t.blocks.6.norm1.bias	(384,)
net_t.blocks.6.attn	0.591M	0.152G
net_t.blocks.6.attn.qkv	0.444M	0.114G
net_t.blocks.6.attn.proj	0.148M	37.896M
net_t.blocks.6.norm2	0.768K	0.493M
net_t.blocks.6.norm2.weight	(384,)
net_t.blocks.6.norm2.bias	(384,)
net_t.blocks.6.mlp	1.182M	0.303G
net_t.blocks.6.mlp.fc1	0.591M	0.152G
net_t.blocks.6.mlp.fc2	0.59M	0.152G
net_t.blocks.7	1.774M	0.456G
net_t.blocks.7.norm1	0.768K	0.493M
net_t.blocks.7.norm1.weight	(384,)
net_t.blocks.7.norm1.bias	(384,)
net_t.blocks.7.attn	0.591M	0.152G
net_t.blocks.7.attn.qkv	0.444M	0.114G
net_t.blocks.7.attn.proj	0.148M	37.896M
net_t.blocks.7.norm2	0.768K	0.493M
net_t.blocks.7.norm2.weight	(384,)
net_t.blocks.7.norm2.bias	(384,)
net_t.blocks.7.mlp	1.182M	0.303G
net_t.blocks.7.mlp.fc1	0.591M	0.152G
net_t.blocks.7.mlp.fc2	0.59M	0.152G
net_t.blocks.8	1.774M	0.456G
net_t.blocks.8.norm1	0.768K	0.493M
net_t.blocks.8.norm1.weight	(384,)
net_t.blocks.8.norm1.bias	(384,)
net_t.blocks.8.attn	0.591M	0.152G
net_t.blocks.8.attn.qkv	0.444M	0.114G
net_t.blocks.8.attn.proj	0.148M	37.896M
net_t.blocks.8.norm2	0.768K	0.493M
net_t.blocks.8.norm2.weight	(384,)
net_t.blocks.8.norm2.bias	(384,)
net_t.blocks.8.mlp	1.182M	0.303G
net_t.blocks.8.mlp.fc1	0.591M	0.152G
net_t.blocks.8.mlp.fc2	0.59M	0.152G
net_t.blocks.9	1.774M	0.456G
net_t.blocks.9.norm1	0.768K	0.493M
net_t.blocks.9.norm1.weight	(384,)
net_t.blocks.9.norm1.bias	(384,)
net_t.blocks.9.attn	0.591M	0.152G
net_t.blocks.9.attn.qkv	0.444M	0.114G
net_t.blocks.9.attn.proj	0.148M	37.896M
net_t.blocks.9.norm2	0.768K	0.493M
net_t.blocks.9.norm2.weight	(384,)
net_t.blocks.9.norm2.bias	(384,)
net_t.blocks.9.mlp	1.182M	0.303G
net_t.blocks.9.mlp.fc1	0.591M	0.152G
net_t.blocks.9.mlp.fc2	0.59M	0.152G
net_t.blocks.10	1.774M	0.456G
net_t.blocks.10.norm1	0.768K	0.493M
net_t.blocks.10.norm1.weight	(384,)
net_t.blocks.10.norm1.bias	(384,)
net_t.blocks.10.attn	0.591M	0.152G
net_t.blocks.10.attn.qkv	0.444M	0.114G
net_t.blocks.10.attn.proj	0.148M	37.896M
net_t.blocks.10.norm2	0.768K	0.493M
net_t.blocks.10.norm2.weight	(384,)
net_t.blocks.10.norm2.bias	(384,)
net_t.blocks.10.mlp	1.182M	0.303G
net_t.blocks.10.mlp.fc1	0.591M	0.152G
net_t.blocks.10.mlp.fc2	0.59M	0.152G
net_t.blocks.11	1.774M	0.456G
net_t.blocks.11.norm1	0.768K	0.493M
net_t.blocks.11.norm1.weight	(384,)
net_t.blocks.11.norm1.bias	(384,)
net_t.blocks.11.attn	0.591M	0.152G
net_t.blocks.11.attn.qkv	0.444M	0.114G
net_t.blocks.11.attn.proj	0.148M	37.896M
net_t.blocks.11.norm2	0.768K	0.493M
net_t.blocks.11.norm2.weight	(384,)
net_t.blocks.11.norm2.bias	(384,)
net_t.blocks.11.mlp	1.182M	0.303G
net_t.blocks.11.mlp.fc1	0.591M	0.152G
net_t.blocks.11.mlp.fc2	0.59M	0.152G
net_t.norm	0.768K
net_t.norm.weight	(384,)
net_t.norm.bias	(384,)
net_fusion.fc	0.148M	37.749M
net_fusion.fc.weight	(384, 384)
net_fusion.fc.bias	(384,)
net_s	16.75M	4.086G
net_s.cls_token	(1, 1, 384)
net_s.pos_embed	(1, 256, 384)
net_s.patch_embed.proj	0.295M
net_s.patch_embed.proj.weight	(384, 3, 16, 16)
net_s.patch_embed.proj.bias	(384,)
net_s.blocks	15.97M	4.086G
net_s.blocks.0	1.774M	0.454G
net_s.blocks.0.norm1	0.768K	0.492M
net_s.blocks.0.norm1.weight	(384,)
net_s.blocks.0.norm1.bias	(384,)
net_s.blocks.0.attn	0.591M	0.151G
net_s.blocks.0.attn.qkv	0.444M	0.113G
net_s.blocks.0.attn.proj	0.148M	37.749M
net_s.blocks.0.norm2	0.768K	0.492M
net_s.blocks.0.norm2.weight	(384,)
net_s.blocks.0.norm2.bias	(384,)
net_s.blocks.0.mlp	1.182M	0.302G
net_s.blocks.0.mlp.fc1	0.591M	0.151G
net_s.blocks.0.mlp.fc2	0.59M	0.151G
net_s.blocks.1	1.774M	0.454G
net_s.blocks.1.norm1	0.768K	0.492M
net_s.blocks.1.norm1.weight	(384,)
net_s.blocks.1.norm1.bias	(384,)
net_s.blocks.1.attn	0.591M	0.151G
net_s.blocks.1.attn.qkv	0.444M	0.113G
net_s.blocks.1.attn.proj	0.148M	37.749M
net_s.blocks.1.norm2	0.768K	0.492M
net_s.blocks.1.norm2.weight	(384,)
net_s.blocks.1.norm2.bias	(384,)
net_s.blocks.1.mlp	1.182M	0.302G
net_s.blocks.1.mlp.fc1	0.591M	0.151G
net_s.blocks.1.mlp.fc2	0.59M	0.151G
net_s.blocks.2	1.774M	0.454G
net_s.blocks.2.norm1	0.768K	0.492M
net_s.blocks.2.norm1.weight	(384,)
net_s.blocks.2.norm1.bias	(384,)
net_s.blocks.2.attn	0.591M	0.151G
net_s.blocks.2.attn.qkv	0.444M	0.113G
net_s.blocks.2.attn.proj	0.148M	37.749M
net_s.blocks.2.norm2	0.768K	0.492M
net_s.blocks.2.norm2.weight	(384,)
net_s.blocks.2.norm2.bias	(384,)
net_s.blocks.2.mlp	1.182M	0.302G
net_s.blocks.2.mlp.fc1	0.591M	0.151G
net_s.blocks.2.mlp.fc2	0.59M	0.151G
net_s.blocks.3	1.774M	0.454G
net_s.blocks.3.norm1	0.768K	0.492M
net_s.blocks.3.norm1.weight	(384,)
net_s.blocks.3.norm1.bias	(384,)
net_s.blocks.3.attn	0.591M	0.151G
net_s.blocks.3.attn.qkv	0.444M	0.113G
net_s.blocks.3.attn.proj	0.148M	37.749M
net_s.blocks.3.norm2	0.768K	0.492M
net_s.blocks.3.norm2.weight	(384,)
net_s.blocks.3.norm2.bias	(384,)
net_s.blocks.3.mlp	1.182M	0.302G
net_s.blocks.3.mlp.fc1	0.591M	0.151G
net_s.blocks.3.mlp.fc2	0.59M	0.151G
net_s.blocks.4	1.774M	0.454G
net_s.blocks.4.norm1	0.768K	0.492M
net_s.blocks.4.norm1.weight	(384,)
net_s.blocks.4.norm1.bias	(384,)
net_s.blocks.4.attn	0.591M	0.151G
net_s.blocks.4.attn.qkv	0.444M	0.113G
net_s.blocks.4.attn.proj	0.148M	37.749M
net_s.blocks.4.norm2	0.768K	0.492M
net_s.blocks.4.norm2.weight	(384,)
net_s.blocks.4.norm2.bias	(384,)
net_s.blocks.4.mlp	1.182M	0.302G
net_s.blocks.4.mlp.fc1	0.591M	0.151G
net_s.blocks.4.mlp.fc2	0.59M	0.151G
net_s.blocks.5	1.774M	0.454G
net_s.blocks.5.norm1	0.768K	0.492M
net_s.blocks.5.norm1.weight	(384,)
net_s.blocks.5.norm1.bias	(384,)
net_s.blocks.5.attn	0.591M	0.151G
net_s.blocks.5.attn.qkv	0.444M	0.113G
net_s.blocks.5.attn.proj	0.148M	37.749M
net_s.blocks.5.norm2	0.768K	0.492M
net_s.blocks.5.norm2.weight	(384,)
net_s.blocks.5.norm2.bias	(384,)
net_s.blocks.5.mlp	1.182M	0.302G
net_s.blocks.5.mlp.fc1	0.591M	0.151G
net_s.blocks.5.mlp.fc2	0.59M	0.151G
net_s.blocks.6	1.774M	0.454G
net_s.blocks.6.norm1	0.768K	0.492M
net_s.blocks.6.norm1.weight	(384,)
net_s.blocks.6.norm1.bias	(384,)
net_s.blocks.6.attn	0.591M	0.151G
net_s.blocks.6.attn.qkv	0.444M	0.113G
net_s.blocks.6.attn.proj	0.148M	37.749M
net_s.blocks.6.norm2	0.768K	0.492M
net_s.blocks.6.norm2.weight	(384,)
net_s.blocks.6.norm2.bias	(384,)
net_s.blocks.6.mlp	1.182M	0.302G
net_s.blocks.6.mlp.fc1	0.591M	0.151G
net_s.blocks.6.mlp.fc2	0.59M	0.151G
net_s.blocks.7	1.774M	0.454G
net_s.blocks.7.norm1	0.768K	0.492M
net_s.blocks.7.norm1.weight	(384,)
net_s.blocks.7.norm1.bias	(384,)
net_s.blocks.7.attn	0.591M	0.151G
net_s.blocks.7.attn.qkv	0.444M	0.113G
net_s.blocks.7.attn.proj	0.148M	37.749M
net_s.blocks.7.norm2	0.768K	0.492M
net_s.blocks.7.norm2.weight	(384,)
net_s.blocks.7.norm2.bias	(384,)
net_s.blocks.7.mlp	1.182M	0.302G
net_s.blocks.7.mlp.fc1	0.591M	0.151G
net_s.blocks.7.mlp.fc2	0.59M	0.151G
net_s.blocks.8	1.774M	0.454G
net_s.blocks.8.norm1	0.768K	0.492M
net_s.blocks.8.norm1.weight	(384,)
net_s.blocks.8.norm1.bias	(384,)
net_s.blocks.8.attn	0.591M	0.151G
net_s.blocks.8.attn.qkv	0.444M	0.113G
net_s.blocks.8.attn.proj	0.148M	37.749M
net_s.blocks.8.norm2	0.768K	0.492M
net_s.blocks.8.norm2.weight	(384,)
net_s.blocks.8.norm2.bias	(384,)
net_s.blocks.8.mlp	1.182M	0.302G
net_s.blocks.8.mlp.fc1	0.591M	0.151G
net_s.blocks.8.mlp.fc2	0.59M	0.151G
net_s.norm	0.768K
net_s.norm.weight	(384,)
net_s.norm.bias	(384,)
net_s.head	0.385M
net_s.head.weight	(1000, 384)
net_s.head.bias	(1000,)

---

08/30 10:46:57 AM - ==> Creating optimizer
08/30 10:46:57 AM - ==> Loading dataset: DefaultAD
08/30 10:46:57 AM - ==> ********** cfg **********
fvcore_is : True
fvcore_b : 1
fvcore_c : 3
epoch_full : 100
metrics : ['mAUROC_sp_max', 'mAP_sp_max', 'mF1_max_sp_max', 'mAUPRO_px', 'mAUROC_px', 'mAP_px', 'mF1_max_px', 'mF1_px_0.2_0.8_0.1', 'mAcc_px_0.2_0.8_0.1', 'mIoU_px_0.2_0.8_0.1', 'mIoU_max_px']
use_adeval : True
evaluator.kwargs : {'metrics': ['mAUROC_sp_max', 'mAP_sp_max', 'mF1_max_sp_max', 'mAUPRO_px', 'mAUROC_px', 'mAP_px', 'mF1_max_px', 'mF1_px_0.2_0.8_0.1', 'mAcc_px_0.2_0.8_0.1', 'mIoU_px_0.2_0.8_0.1', 'mIoU_max_px'], 'pooling_ks': [16, 16], 'max_step_aupro': 100}
vis : False
vis_dir : None
optim.lr : 0.0004
optim.kwargs : {'name': 'adamw', 'betas': (0.9, 0.999), 'eps': 1e-08, 'weight_decay': 0.0001, 'amsgrad': False}
trainer.name : ViTADTrainer
trainer.checkpoint : runs
trainer.logdir_sub :
trainer.resume_dir :
trainer.cuda_deterministic : False
trainer.epoch_full : 100
trainer.scheduler_kwargs : {'name': 'step', 'lr_noise': None, 'noise_pct': 0.67, 'noise_std': 1.0, 'noise_seed': 42, 'lr_min': 4e-06, 'warmup_lr': 4.0000000000000003e-07, 'warmup_iters': -1, 'cooldown_iters': 0, 'warmup_epochs': 0, 'cooldown_epochs': 0, 'use_iters': True, 'patience_iters': 0, 'patience_epochs': 0, 'decay_iters': 0, 'decay_epochs': 80, 'cycle_decay': 0.1, 'decay_rate': 0.1}
trainer.mixup_kwargs : {'mixup_alpha': 0.8, 'cutmix_alpha': 1.0, 'cutmix_minmax': None, 'prob': 0.0, 'switch_prob': 0.5, 'mode': 'batch', 'correct_lam': True, 'label_smoothing': 0.1}
trainer.test_start_epoch : 100
trainer.test_per_epoch : 10
trainer.find_unused_parameters : False
trainer.sync_BN : apex
trainer.dist_BN :
trainer.scaler : none
trainer.data.batch_size : 32
trainer.data.batch_size_per_gpu : 32
trainer.data.batch_size_test : 32
trainer.data.batch_size_per_gpu_test : 32
trainer.data.num_workers_per_gpu : 4
trainer.data.drop_last : True
trainer.data.pin_memory : True
trainer.data.persistent_workers : False
trainer.data.num_workers : 4
trainer.iter : 0
trainer.epoch : 0
trainer.iter_full : 1400
trainer.metric_recorder : {'mAUROC_sp_max_pizza': [], 'mAP_sp_max_pizza': [], 'mF1_max_sp_max_pizza': [], 'mAUPRO_px_pizza': [], 'mAUROC_px_pizza': [], 'mAP_px_pizza': [], 'mF1_max_px_pizza': [], 'mF1_px_0.2_0.8_0.1_pizza': [], 'mAcc_px_0.2_0.8_0.1_pizza': [], 'mIoU_px_0.2_0.8_0.1_pizza': [], 'mIoU_max_px_pizza': []}
loss.loss_terms : [{'type': 'CosLoss', 'name': 'cos', 'avg': False, 'lam': 1.0}]
loss.clip_grad : 5.0
loss.create_graph : False
loss.retain_graph : False
adv : False
logging.log_terms_train : [{'name': 'batch_t', 'fmt': ':>5.3f', 'add_name': 'avg'}, {'name': 'data_t', 'fmt': ':>5.3f'}, {'name': 'optim_t', 'fmt': ':>5.3f'}, {'name': 'lr', 'fmt': ':>7.6f'}, {'name': 'cos', 'suffixes': [''], 'fmt': ':>5.3f', 'add_name': 'avg'}]
logging.log_terms_test : [{'name': 'batch_t', 'fmt': ':>5.3f', 'add_name': 'avg'}, {'name': 'cos', 'suffixes': [''], 'fmt': ':>5.3f', 'add_name': 'avg'}]
logging.train_reset_log_per : 50
logging.train_log_per : 50
logging.test_log_per : 50
data.sampler : naive
data.loader_type : pil
data.loader_type_target : pil_L
data.type : DefaultAD
data.root : data/cj
data.meta : meta.json
data.cls_names : []
data.train_transforms : [{'type': 'Resize', 'size': (256, 256), 'interpolation': <InterpolationMode.BILINEAR: 'bilinear'>}, {'type': 'CenterCrop', 'size': (256, 256)}, {'type': 'ToTensor'}, {'type': 'Normalize', 'mean': (0.485, 0.456, 0.406), 'std': (0.229, 0.224, 0.225), 'inplace': True}]
data.test_transforms : [{'type': 'Resize', 'size': (256, 256), 'interpolation': <InterpolationMode.BILINEAR: 'bilinear'>}, {'type': 'CenterCrop', 'size': (256, 256)}, {'type': 'ToTensor'}, {'type': 'Normalize', 'mean': (0.485, 0.456, 0.406), 'std': (0.229, 0.224, 0.225), 'inplace': True}]
data.target_transforms : [{'type': 'Resize', 'size': (256, 256), 'interpolation': <InterpolationMode.BILINEAR: 'bilinear'>}, {'type': 'CenterCrop', 'size': (256, 256)}, {'type': 'ToTensor'}]
data.train_size : 14
data.test_size : 3
data.train_length : 477
data.test_length : 93
model_t.name : vit_small_patch16_224_dino
model_t.kwargs : {'pretrained': True, 'checkpoint_path': '', 'pretrained_strict': False, 'strict': True, 'img_size': 256, 'teachers': [3, 6, 9], 'neck': [12]}
model_f.name : fusion
model_f.kwargs : {'pretrained': False, 'checkpoint_path': '', 'strict': False, 'dim': 384, 'mul': 1}
model_s.name : de_vit_small_patch16_224_dino
model_s.kwargs : {'pretrained': False, 'checkpoint_path': '', 'strict': False, 'img_size': 256, 'students': [3, 6, 9], 'depth': 9}
model.name : vitad
model.kwargs : {'pretrained': False, 'checkpoint_path': '', 'strict': True, 'model_t': Namespace(name='vit_small_patch16_224_dino', kwargs={'pretrained': True, 'checkpoint_path': '', 'pretrained_strict': False, 'strict': True, 'img_size': 256, 'teachers': [3, 6, 9], 'neck': [12]}), 'model_f': Namespace(name='fusion', kwargs={'pretrained': False, 'checkpoint_path': '', 'strict': False, 'dim': 384, 'mul': 1}), 'model_s': Namespace(name='de_vit_small_patch16_224_dino', kwargs={'pretrained': False, 'checkpoint_path': '', 'strict': False, 'img_size': 256, 'students': [3, 6, 9], 'depth': 9})}
seed : 42
size : 256
warmup_epochs : 0
test_start_epoch : 100
test_per_epoch : 10
batch_train : 32
batch_test_per : 32
lr : 0.0004
weight_decay : 0.0001
cfg_path : configs.vitad.vitad_cj
mode : train
sleep : 0
memory : -1
dist_url : env://
logger_rank : 0
opts : []
command : python3 -m torch.distributed.launch --nproc_per_node=$nproc_per_node --nnodes=$nnodes --node_rank=$node_rank --master_addr=$master_addr --master_port=$master_port --use_env run.py -c configs.vitad.vitad_cj -m train --sleep 0 --memory -1 --dist_url env:// --logger_rank 0
task_start_time : 5769699.826605973
dist : False
world_size : 1
rank : 0
local_rank : 0
ngpus_per_node : 1
nnodes : 1
master : True
logdir : runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656
logger.filters : []
logger.name : root
logger.level : 20
logger.parent : None
logger.propagate : True
logger.disabled : False
logdir_train : runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656/show_train
logdir_test : runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656/show_test
08/30 10:46:57 AM - ==> Starting training with 1 nodes x 1 GPUs
08/30 10:47:01 AM - ==> Total time: 0:00:04 Eta: 0:07:55 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
08/30 10:47:05 AM - ==> Total time: 0:00:09 Eta: 0:07:26 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
08/30 10:47:10 AM - ==> Total time: 0:00:13 Eta: 0:07:10 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
08/30 10:47:12 AM - Train: 3.57% [50/1400] [3.6/100.0] [batch_t 0.053 (0.265)] [data_t 0.002] [optim_t 0.051] [lr 0.000400] [cos 0.592 (0.631)]
08/30 10:47:14 AM - ==> Total time: 0:00:17 Eta: 0:07:01 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
08/30 10:47:18 AM - ==> Total time: 0:00:21 Eta: 0:06:57 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
08/30 10:47:22 AM - ==> Total time: 0:00:26 Eta: 0:06:48 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
08/30 10:47:27 AM - ==> Total time: 0:00:30 Eta: 0:06:44 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
08/30 10:47:28 AM - Train: 7.14% [100/1400] [7.1/100.0] [batch_t 0.053 (0.530)] [data_t 0.002] [optim_t 0.050] [lr 0.000400] [cos 0.348 (0.351)]
08/30 10:47:31 AM - ==> Total time: 0:00:34 Eta: 0:06:41 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
08/30 10:47:35 AM - ==> Total time: 0:00:39 Eta: 0:06:36 Logged in 'runs/ViTADTrainer_configs_vitad_vitad_cj_20240830-104656'
2/3
`

Thank you very much.

model文件下没有backbones文件或文件夹

without vitad/single_cls/vitad_mvtec_bs16.py

Confuse MACs and FLOPs

I just found that you may mistake FLOPs by MACs. The FLOPs reported in ADer are actually MACs. As MACs~0.5*FLOPs, the reported FLOPs is around half of the real FLOPs. (Some FLOPs calculation packages also confuse this, including the one you use).

You may want to try the flops-profiler python package.

An error encountered while replicating the code

Dear author, thank you for your great work and open source its code. When I reproduced your work, I encountered an error:
Traceback (most recent call last): File "run.py", line 5, in <module> from trainer import get_trainer File "/home/401229/ADer-main/trainer/__init__.py", line 9, in <module> model_lib = importlib.import_module(file.split('.')[0].replace('/', '.')) File "/root/anaconda3/envs/ldm/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/401229/ADer-main/trainer/invad_trainer.py", line 14, in <module> from model import get_model File "/home/401229/ADer-main/model/__init__.py", line 86, in <module> model_lib = importlib.import_module(file.split('.')[0].replace('/', '.')) File "/root/anaconda3/envs/ldm/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/401229/ADer-main/model/backbone_factory.py", line 1, in <module> import model.backbones.pvt_v2 ModuleNotFoundError: No module named 'model.backbones'
I have checked the code in the model file and found no function related to backbone. How can I solve this problem? I am looking forward to your reply

Code release?

Thanks for your great work! When would you release the code? Thank you!

METHOD doesnt exist in configs/METHOD path.

There is no configs/METHOD/METHOD_CFG.py when I clone your repository. Is it me or you are missing something?

Thank you.

self.data.anomaly_source_path = 'data/dtd/images/'，这个参数不知道如何设置，或者如何获得这个文件

我使用destseg_256_100e.py，模型为destseg数据集为mvtec，已经生成了meta.json。

其中一个参数self.data.anomaly_source_path = 'data/dtd/images/'中，这个参数不知道如何设置，或者如何获得这个文件。

导致了一下错误：

06/12 03:27:47 PM - ==> Starting training with 1 nodes x 1 GPUs
Traceback (most recent call last):
  File "/home/ubuntu/code_space/ADer/run.py", line 31, in <module>
    main()
  File "/home/ubuntu/code_space/ADer/run.py", line 27, in main
    trainer.run()
  File "/home/ubuntu/code_space/ADer/trainer/_base_trainer.py", line 252, in run
    self.train()
  File "/home/ubuntu/code_space/ADer/trainer/_base_trainer.py", line 177, in train
    train_data = next(train_loader)
  File "/home/ubuntu/anaconda3/envs/ADer/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 630, in __next__
    data = self._next_data()
  File "/home/ubuntu/anaconda3/envs/ADer/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data
    return self._process_data(data)
  File "/home/ubuntu/anaconda3/envs/ADer/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
    data.reraise()
  File "/home/ubuntu/anaconda3/envs/ADer/lib/python3.9/site-packages/torch/_utils.py", line 694, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/ADer/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/ubuntu/anaconda3/envs/ADer/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ubuntu/anaconda3/envs/ADer/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ubuntu/code_space/ADer/data/ad_dataset.py", line 919, in __getitem__
    anomaly_source_idx = torch.randint(0, len(self.anomaly_source_paths), (1,)).item()
RuntimeError: random_ expects 'from' to be less than 'to', but got from=0 >= to=0

错误出现在ad_dataset.py中919行。

def __getitem__(self, index):
	if self.train:
		idx = torch.randint(0, len(self.data_all), (1,)).item()
		anomaly_source_idx = torch.randint(0, len(self.anomaly_source_paths), (1,)).item()
		data = self.data_all[idx]
		img_path, mask_path, cls_name, specie_name, anomaly = data['img_path'], data['mask_path'], data['cls_name'], \
																  data['specie_name'], data['anomaly']
		image, augmented_image, anomaly_mask, has_anomaly = self.transform_image(os.path.join(self.root, img_path),
																					 self.anomaly_source_paths[
																						 anomaly_source_idx])
		image = self.transform(image) if self.transform is not None else image
		augmented_image = self.transform(augmented_image) if self.transform is not None else augmented_image
		sample = {'img': image, "img_mask": anomaly_mask, 'cls_name': cls_name,
				  'augmented_image': augmented_image, 'anomaly': has_anomaly}
		return sample

anomaly_source_idx = torch.randint(0, len(self.anomaly_source_paths), (1,)).item()中self.anomaly_source_paths为0。
请问应该如何做呢。

vitad_trainer.py is empty?

How to prepare custom dataset

I want to train and test on my dataset,how to make it.Thanks

Can it be converted to tensorrt?

导出模型为onnx，vitAD成功，invAD及InvAD-lite均报错？

首先非常惊赞叹作者们的杰出创作！感谢无私开源！

在拜读论文及源码之后，我尝试将模型导出为onnx，但是发现只有vitAD成功，invAD及InvAD-lite均报错，其它还没来得及测试。
torch版本 2.3.0 和 1.13.1 都测试了，报错相同，大致如下：

torch.onnx.export(self.net, x, "inv.onnx")
Traceback (most recent call last):
File "", line 1, in
File "f:\pyenv\Torchgpu\lib\site-packages\torch\onnx\utils.py", line 516, in export
_export(
File "f:\pyenv\Torchgpu\lib\site-packages\torch\onnx\utils.py", line 1612, in _export
graph, params_dict, torch_out = _model_to_graph(
File "f:\pyenv\Torchgpu\lib\site-packages\torch\onnx\utils.py", line 1138, in _model_to_graph
graph = _optimize_graph(
File "f:\pyenv\Torchgpu\lib\site-packages\torch\onnx\utils.py", line 619, in _optimize_graph
_C._jit_pass_peephole(graph, True)
IndexError: Argument passed to at() was not in the map.

百度谷歌了一圈也没甚收效，麻烦你们给看看，有没有什么发现或线索，谢谢！

文件缺失

请问为啥没有这个文件

checkpoint

KeyError: 'img_path'

It seems that after updating the version, this problem has arisen

CUDA_VISIBLE_DEVICES=0 python3 run.py -c configs/benchmark/destseg/destseg_256_300e.py -m train

Traceback (most recent call last):
File "run.py", line 31, in
main()
File "run.py", line 27, in main
trainer.run()
File "/workspace/mycode/ADer-main/trainer/_base_trainer.py", line 252, in run
self.train()
File "/workspace/mycode/ADer-main/trainer/_base_trainer.py", line 178, in train
self.set_input(train_data)
File "/workspace/mycode/ADer-main/trainer/destseg_trainer.py", line 64, in set_input
self.img_path = inputs['img_path']
KeyError: 'img_path'

About single-class unsupervised defect detection

Dear author, I have successfully verified the relevant data set on your code and achieved good results. Thank you again for your contribution. But I have a question, in reality, in most cases, more single-class defect detection, the code should also be able to achieve, but your README mentioned that it is not recommended at present, can you tell me the reason？Thanks!

AttributeError: 'DistributedDataParallel' object has no attribute 'destseg'

there is an error when i run
python -m torch.distributed.launch --nproc_per_node=$nproc_per_node --nnodes=$nnodes --node_rank=$node_rank --master_port=$master_port --use_env run.py -c configs/benchmark/destseg/destseg_256_300e.py -m train with multiple GPUs.

Traceback (most recent call last):
File "run.py", line 31, in
main()
File "run.py", line 26, in main
trainer = get_trainer(cfg)
File "/workspace/mycode/04-anomaly-detection/ADer/trainer/init.py", line 13, in get_trainer
return TRAINER.get_module(cfg.trainer.name)(cfg)
File "/workspace/mycode/04-anomaly-detection/ADer/trainer/destseg_trainer.py", line 41, in init
self.optim.de_st = get_optim(cfg.optim.de_st.kwargs, self.net.destseg.student_net,
File "/workspace/mysoftware/miniconda3/envs/ader/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1695, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'DistributedDataParallel' object has no attribute 'destseg'

so, what should i do?

slightly lower results than reported

Thank you very much for sharing the code. I am trying to reproduce the results reported in the paper and also this github repo. I am an A100 GPU, and the provided weight to reproduce the results of ViTAD, however the results are slightly different from the reported ones. Is this normal? My reprodeuced results are as following:

Name	mAUROC_sp_max	mAUROC_sp_max (Max)	mAP_sp_max	mAP_sp_max (Max)	mF1_max_sp_max	mF1_max_sp_max (Max)	mAUPRO_px	mAUPRO_px (Max)	mAUROC_px	mAUROC_px (Max)	mAP_px	mAP_px (Max)	mF1_max_px	mF1_max_px (Max)	mF1_px_0.2_0.8_0.1	mF1_px_0.2_0.8_0.1 (Max)	mAcc_px_0.2_0.8_0.1	mAcc_px_0.2_0.8_0.1 (Max)	mIoU_px_0.2_0.8_0.1	mIoU_px_0.2_0.8_0.1 (Max)	mIoU_max_px	mIoU_max_px (Max)
Avg	98.091	98.091 (1 epoch)	99.061	99.061 (1 epoch)	96.949	96.949 (1 epoch)	91.855	91.855 (1 epoch)	97.582	97.582 (1 epoch)	55.358	55.358 (1 epoch)	58.611	58.611 (1 epoch)	31.047	31.047 (1 epoch)	40.695	40.695 (1 epoch)	20.489	20.489 (1 epoch)	42.549	42.549 (1 epoch)

Unused forward modules result in RuntimeError

I encounter this RuntimeError during parallel training:

RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument find_unused_parameters=True to torch.nn.parallel.DistributedDataParallel, and by
making sure all forward function outputs participate in calculating loss.

According to this reply and its follow-up, I think it is safe to multiply the unused CNN layers with zero to avoid the RuntimeError.

Would you agree with the fix method, and would you mind me making a fix for this and making a pull request?

自定义数据集训练

已经按数据说明进行数据处理但是还是出现这个问题，这是哪里配置没有修改正确吗

Following MVTEC format for custom data.

MVTEC has mask image data. But My custom data does not have it. Must we have mask image data to run ViTAD? I just made my data in exactly the same format as MVTEC by populating mask data with just black uniform image for all image data just to follow the MVTEC data but my mask image data is just meaning less. Is it going to be okay this way or is there some way you recommend?

Thank you in advance.

单图推理及缺陷位置可视化

请问能否提供一下单图推理及缺陷位置可视化热力图的代码呢

你好，我在配属环境时候出现了些意外

请问，是不是python版本不对？

No module named 'hilbert'

I get this error below when I run CUDA_VISIBLE_DEVICES=0 python run.py -c configs/vitad/vitad_mvtec.py -m train

File "/home/sdt/Workspace/hojun/anomaly_detection/ader/model/mambaad.py", line 24, in <module> from hilbert import decode, encode ModuleNotFoundError: No module named 'hilbert'

And as far as I see, there is nothing like hilbert in pip.

How do I install hilbert or work around this?

Thank you.

请问下在文章中，这个红色框的Test Time是针对于该数据集中的相同测试数据，得到的总和测试时间吗？

About visualization

Dear author, how can I visualize the results of the test？

no `meta.json`