fishandwasabi / yolo-ms

YOLO-MS: Rethinking Multi-Scale Representation Learning for Real-Time Object Detection

🎓 I’m currently a first-year graduate student of VCIP, College of Computer Science, Nankai University, supervised by Prof. Ming-Ming Cheng & Prof. Qi-Bin Hou.

😆 My research interests are Vision Understanding, Object Detection and Knowledge Distillation.

❤️ My homepage: FishAndWasabi.

📃 All my research works: Google scholar.

📫 Email contact: [email protected].

Visitor Count

yolo-ms's Issues

请问一下这两个代码在哪里啊？

from mmdet.utils import OptConfigType
from mmcv.cnn import ConvModule

请问旋转目标检测什么时候可以做出来呢

十分期待

How to deploy yolo ms

Hello, we would like to deploy YOLO MS. However, we have noticed that the public code is missing the docker/GPU/ directory and ${DEPLOY_CONFIG_FILE}. How can we resolve this issue?

请问实例分割什么时候可以出来呢？

请问关于 YOLO-MS 的实例分割什么时候可以出来呢？

How to merge the pred bboxs and labels into one output feature

1、deploy to onnx failed. cannot export pth to onnx.
05/14 15:10:10 - mmengine - WARNING - Failed to search registry with scope "mmyolo" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmyolo" is a correct scope, or whether the registry is initialized.
05/14 15:10:10 - mmengine - WARNING - Failed to search registry with scope "mmyolo" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmyolo" is a correct scope, or whether the registry is initialized.
The given version [15] is not supported, only version 1 to 14 is supported in this build.

2、How to merge the pred bboxs and labels into one output feature.
I couldn't find a way to modify the output nodes in this project. I don't know how the model is constructed. Does mmengine provide an API that allows modifying nodes during the model export to ONNX？

By the way, using mmengine seems has made this project less user-friendly.

训练到第十轮时报错：IndexError: list index out of range

return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
09/04 19:35:45 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:35:53 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:00 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:07 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:13 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:21 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:27 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:34 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:41 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:48 - mmengine - INFO - Exp name: yoloms_syncbn_fast_8xb8-300e_coco_20230904_193458
09/04 19:36:48 - mmengine - INFO - Saving checkpoint at 10 epochs
Traceback (most recent call last):
File "/home/sysu/program/python/terrorismFlag/YOLO-MS/tools/train.py", line 114, in
main()
File "/home/sysu/program/python/terrorismFlag/YOLO-MS/tools/train.py", line 110, in main
runner.train()
File "/home/sysu/software/anaconda3/envs/pytorch2.0/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1706, in train
model = self.train_loop.run() # type: ignore
File "/home/sysu/software/anaconda3/envs/pytorch2.0/lib/python3.10/site-packages/mmengine/runner/loops.py", line 102, in run
self.runner.val_loop.run()
File "/home/sysu/software/anaconda3/envs/pytorch2.0/lib/python3.10/site-packages/mmengine/runner/loops.py", line 366, in run
metrics = self.evaluator.evaluate(len(self.dataloader.dataset))
File "/home/sysu/software/anaconda3/envs/pytorch2.0/lib/python3.10/site-packages/mmengine/evaluator/evaluator.py", line 79, in evaluate
_results = metric.evaluate(size)
File "/home/sysu/software/anaconda3/envs/pytorch2.0/lib/python3.10/site-packages/mmengine/evaluator/metric.py", line 115, in evaluate
_metrics = self.compute_metrics(results) # type: ignore
File "/home/sysu/software/anaconda3/envs/pytorch2.0/lib/python3.10/site-packages/mmdet/evaluation/metrics/coco_metric.py", line 419, in compute_metrics
result_files = self.results2json(preds, outfile_prefix)
File "/home/sysu/software/anaconda3/envs/pytorch2.0/lib/python3.10/site-packages/mmdet/evaluation/metrics/coco_metric.py", line 239, in results2json
data['category_id'] = self.cat_ids[label]
IndexError: list index out of range

替换v6相关

您好，请问有尝试过替换v6更小尺寸，比如v6n结构吗？我看config里面是silu，不知道relu效果会掉多少？我复现v8的时候发现激活换成relu➕去掉dfl会掉2-3个点。

有人测试跑通过吗？

什么时候可以提供预训练权重啊

some questions about HKS protocol

Have you done any comparative experiments using HKS protocol in YOLOMSPAFPN and whether you can also improve performance?

训练测试coco128数据集，不收敛是什么原因

/home/wansan/PycharmProjects/pythonProject/YOLO-MS-main/tools/work_dirs/ms-srobotdata/20240307_154846/vis_data/config.py
2024/03/07 15:49:44 - mmengine - INFO - Epoch(val) [10][13/13] coco/bbox_mAP: 0.4280 coco/bbox_mAP_50: 0.6050 coco/bbox_mAP_75: 0.5030 coco/bbox_mAP_s: 0.2120 coco/bbox_mAP_m: 0.4820 coco/bbox_mAP_l: 0.5560 data_time: 0.0044 time: 0.1565
2024/03/07 15:52:14 - mmengine - INFO - Epoch(val) [40][13/13] coco/bbox_mAP: 0.1460 coco/bbox_mAP_50: 0.2380 coco/bbox_mAP_75: 0.1540 coco/bbox_mAP_s: 0.0440 coco/bbox_mAP_m: 0.2050 coco/bbox_mAP_l: 0.1850 data_time: 0.0010 time: 0.0556

训练log

您好，请问是否可以提供YOLOMS模型的训练log？

TypeError: yolov5_collate() got an unexpected keyword argument '_scope_'

Traceback (most recent call last):
File "tools/train.py", line 114, in
main()
File "tools/train.py", line 110, in main
runner.train()
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1706, in train
model = self.train_loop.run() # type: ignore
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/mmengine/runner/loops.py", line 96, in run
self.run_epoch()
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/mmengine/runner/loops.py", line 111, in run_epoch
for idx, data_batch in enumerate(self.dataloader):
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 681, in next
data = self._next_data()
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data
return self._process_data(data)
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data
data.reraise()
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/torch/_utils.py", line 461, in reraise
raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/data/.conda/envs/YOLO-MS/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
return self.collate_fn(data)
TypeError: yolov5_collate() got an unexpected keyword argument 'scope'

Questions about Training Time and Test Latency

Hi, I'm attempting to reproduce YOLO-MS-XS (with SE attn). It shows that training from scratch (300 epochs) will take almost 10 days. I'm using RTX 3090 * 8. The training command, CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 bash tools/dist_train.sh configs/yoloms/yoloms-xs-se_syncbn_fast_8xb8-300e_coco.py 8, aligns with the README file.

Is it common for training to take this long? Could you share your training time? btw, which GPU device are you using to test the model's latency?

如何将MSBlock加入到YOLO-X中呢

请问如何构造YOLO-X-MS网络结构

Train my own dataset: ValueError: class `YOLOv5CocoDataset`

HI Thank you for great work!
When I train my own dataset, there some error:
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "/opt/conda/lib/python3.8/site-packages/mmengine/runner/loops.py", line 44, in init
super().init(runner, dataloader)
File "/opt/conda/lib/python3.8/site-packages/mmengine/runner/base_loop.py", line 26, in init
self.dataloader = runner.build_dataloader(
File "/opt/conda/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1346, in build_dataloader
dataset = DATASETS.build(dataset_cfg)
File "/opt/conda/lib/python3.8/site-packages/mmengine/registry/registry.py", line 545, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "/opt/conda/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 135, in build_from_cfg
raise type(e)(
ValueError: class YOLOv5CocoDataset in mmyolo/datasets/yolov5_coco.py: need at least one array to concatenate

I modify the classes_num and name_clasess.
My config as flow:
yoloms-xs_syncbn_fast_8xb8-300e_coco.txt

A question about the HKS

Your work is excellent, but I have a question about the HKS . In the paper, why does HKS have no effect on shallow features, because eventually PAFPN fuses the features so that the coarse-grained features of the large receptive field affect the fine-grained features of the shallow layer ?

ms只下采样到16倍吗？

KeyError: 'YOLOMS is not in the mmyolo::model registry.

python tools/train.py configs/yoloms/yoloms-xs_syncbn_fast_8xb8-300e_coco.py
运行时报错：
Traceback (most recent call last):
File "tools/train.py", line 123, in
main()
File "tools/train.py", line 112, in main
runner = Runner.from_cfg(cfg)
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\runner\runner.py", line 445, in from_cfg
runner = cls(
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\runner\runner.py", line 412, in init
self.model = self.build_model(model)
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\runner\runner.py", line 819, in build_model
model = MODELS.build(model)
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\registry\registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\registry\build_functions.py", line 232, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\registry\build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "d:\mmyolo\mmyolo\models\detectors\yolo_detector.py", line 41, in init
super().init(
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmdet\models\detectors\single_stage.py", line 30, in init
self.backbone = MODELS.build(backbone)
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\registry\registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\registry\build_functions.py", line 232, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "D:\anaconda\envs\mmyolo_ms\lib\site-packages\mmengine\registry\build_functions.py", line 100, in build_from_cfg
raise KeyError(
KeyError: 'YOLOMS is not in the mmyolo::model registry. Please check whether the value of YOLOMS is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/e
n/latest/advanced_tutorials/config.html#import-the-custom-module'

请问怎么预测啊，将预测可视化

训练自己的数据集，报错提示为TypeError: FormatCode() got an unexpected keyword argument 'verify'

请问如何训练自己的数据集

请问如何训练自己的数据集，包括数据路径写在哪里？超参数应该如何调整

TypeError: yolov5_collate() got an unexpected keyword argument '_scope_'

你好，我在训练过程中遇到该报错，请问有解决办法嘛，谢谢。TypeError: yolov5_collate() got an unexpected keyword argument 'scope'