yuto3o / yolox Goto Github PK

View Code? Open in Web Editor NEW

149.0 7.0 46.0 42.19 MB

More Than YOLO（v3, v4, v3-tiny, v4-tiny）

Python 100.00%

tensorflow keras yolov4 yolov4-tiny yolov3 yolov3-tiny

yolox's Introduction

More Than YOLO

English | 中文

TensorFlow & Keras & Python

YOLOv3, YOLOv3-tiny, YOLOv4, YOLOv4-tiny

[Unofficial] YOLOv4-tiny, YOLOX

requirements: TensorFlow 2.1 (not test on 1.x), OpenCV, Numpy, PyYAML

News !

Small batch size is used, because available GPU (8G) has small memory. Please use big batch size as possible.
Online High Level augmentation will slow down training speed.
When I tried to train yolov3 or yolov4, 'NaN' problem made me crazy.
Support AccumOptimizer, Similar to 'subdivisions' in darknet.
When I tried to train yolov3 or yolov4, I found that if I set weight decay to 5e-4，the result is unsatisfactory; if I set it to 0, everything is OK.
Note that both yolov3-tiny and yolov4-tiny don't use anchor 0, so they use only anchors 1-7.[link]

This repository have done:

[toc]

0. Please Read Source Code for More Details

You can get official weight files from https://github.com/AlexeyAB/darknet/releases or https://pjreddie.com/darknet/yolo/.

1. Samples

1.1 Data File

We use a special data format like that,

path/to/image1 x1,y1,x2,y2,label x1,y1,x2,y2,label 
path/to/image2 x1,y1,x2,y2,label 
...

Convert your data format firstly. We present a script for Pascal VOC and a script for COCO.

More details and a simple dataset could be got from https://github.com/YunYang1994/yymnist.

1.2 Configure

# coco_yolov4.yaml
yolo:
  type: "yolov4"  # must be 'yolov3', 'yolov3_tiny', 'yolov4', 'yolov4_tiny' ‘unofficial_yolov4_tiny’ and 'yolox'.
  iou_threshold: 0.5
  score_threshold: 0.005
  max_boxes: 100
  strides: "32,16,8"
  anchors: "12,16 19,36 40,28 36,75 76,55 72,146 142,110 192,243 459,401"
  mask: "6,7,8 3,4,5 0,1,2"
  name_path: "./data/coco/coco.name"

train:
  label: "coco_yolov4" # any thing you like
  anno_path: "./data/coco/train2017.txt"
  image_size: "320,352,384,416,448,480,512,544,576,608"  # "416" for single mini batch size, "352,384,416,448,480" for Dynamic mini batch size.

  batch_size: 4
  init_weight_path: "./ckpts/yolov4.weights"
  save_weight_path: "./ckpts"

  loss_type: "CIoU+FL" # Must be "L2", "DIoU", "GIoU", "CIoU" or something like "L2+FL" for focal loss
  
  # turn on hight level data augmentation
  mosaic: false
  label_smoothing: false
  normal_method: true

  ignore_threshold: 0.7

test:
  anno_path: "./data/coco/val2017.txt"
  image_size: "608"
  batch_size: 1
  init_weight_path: "./ckpts/yolov4.weights"

1.3 K-Means

Edit the kmeans.py as you like

# kmeans.py
# Key Parameters
K = 6 # num of clusters
image_size = 416
dataset_path = './data/pascal_voc/train.txt'

1.4 Inference

A Simple Script for Video, Device or Image

Only support mp4, avi, device id, rtsp, png, jpg (Based on OpenCV)

python detector.py --config=./cfgs/coco_yolov4.yaml --media=./misc/street.mp4 --gpu=false

A simple demo for YOLOv4

from core.utils import decode_cfg, load_weights
from core.model.one_stage.yolov4 import YOLOv4
from core.image import draw_bboxes, preprocess_image, postprocess_image, read_image, Shader

import numpy as np
import cv2
import time

# read config
cfg = decode_cfg('cfgs/coco_yolov4.yaml')
names = cfg['yolo']['names']

model, eval_model = YOLOv4(cfg)
eval_model.summary()

# assign colors for difference labels
shader = Shader(cfg['yolo']['num_classes'])

# load weights
load_weights(model, cfg['test']['init_weight_path'])

img_raw = read_image('./misc/dog.jpg')
img = preprocess_image(img_raw, (512, 512))
imgs = img[np.newaxis, ...]

tic = time.time()
boxes, scores, classes, valid_detections = eval_model.predict(imgs)
toc = time.time()
print((toc - tic)*1000, 'ms')

# for single image, batch size is 1
valid_boxes = boxes[0][:valid_detections[0]]
valid_score = scores[0][:valid_detections[0]]
valid_cls = classes[0][:valid_detections[0]]

img, valid_boxes = postprocess_image(img, img_raw.shape[1::-1], valid_boxes)
img = draw_bboxes(img, valid_boxes, valid_score, valid_cls, names, shader)

cv2.imshow('img', img[..., ::-1])
cv2.imwrite('./misc/dog_v4.jpg', img)
cv2.waitKey()

2. Train

!!! Please Read the above guide (e.g. 1.1, 1.2).

python train.py --config=./cfgs/coco_yolov4.yaml

3. Experiment

3.1 Speed

i7-9700F+16GB

Model	416x416	512x512	608x608
YOLOv3	219 ms	320 ms	429 ms
YOLOv3-tiny	49 ms	63 ms	78 ms
YOLOv4	344 ms	490 ms	682 ms
YOLOv4-tiny	51 ms	66 ms	83 ms
Unofficial-YOLOv4-tiny	64 ms	86 ms	110 ms
YOLOX	67 ms	83 ms	104 ms

i7-9700F+16GB / RTX 2070S+8G

Model	416x416	512x512	608x608
YOLOv3	59 ms	66 ms	83 ms
YOLOv3-tiny	28 ms	30 ms	33 ms
YOLOv4	73 ms	74 ms	91 ms
YOLOv4-tiny	30 ms	32 ms	35 ms
Unofficial-YOLOv4-tiny	30 ms	31 ms	34 ms
YOLOX	42 ms	45 ms	50 ms

3.2 Logs

Augmentations

Name	Abbr
Standard Method	SM
Dynamic mini batch size	DM
Label Smoothing	LS
Focal Loss	FL
Mosaic	M
Warm-up LR	W
Cosine Annealing LR	CA

Standard Method Package includes Flip left and right, Crop and Zoom(jitter=0.3), Grayscale, Distort, Rotate(angle=7).

YOLOv3-tiny(Pretrained on COCO; Trained on VOC)

SM	DM	LS	FL	M	Loss	AP	AP@50	AP@75
✔					L2	26.6	61.8	17.2
✔	✔				L2	27.3	62.4	17.9
✔	✔	✔			L2	26.7	61.7	17.1
✔	✔				CIoU	30.9	64.2	25.0
✔	✔		✔		CIoU	32.3	65.7	27.6
✔	✔		✔	✔	CIoU

YOLOv3(TODO; Pretrained on COCO; Trained on VOC; only 15 epochs)

SM	DM	LS	FL	M	Loss	AP	AP@50	AP@75
✔	✔		✔		CIoU	46.5	80.0	49.0
✔	✔		✔	✔	CIoU

YOLOv4-tiny(TODO; Pretrained on COCO; Trained on VOC)

SM	DM	LS	FL	M	Loss	AP	AP@50	AP@75
✔	✔		✔		CIoU	35.1	70.2	30.0
✔	✔		✔	✔	CIoU

YOLOv4(TODO; Pretrained on COCO; Trained on VOC)

SM	DM	LS	FL	M	Loss	AP	AP@50	AP@75
✔	✔		✔		CIoU
✔	✔		✔	✔	CIoU

Unofficial-YOLOv4-tiny(TODO; Pretrained on COCO, part of YOLOv3-tiny weights; Trained on VOC)

SM	DM	LS	FL	M	Loss	AP	AP@50	AP@75
✔	✔		✔		CIoU	35.0	65.7	33.8
✔	✔		✔	✔	CIoU

YOLOX(TODO; Pretrained on COCO, part of YOLOv4-tiny weights; Trained on VOC)

SM	DM	LS	FL	M	Loss	AP	AP@50	AP@75
✔	✔		✔		CIoU	40.6	72.2	40.3
✔	✔		✔	✔	CIoU

3.3 Details

Tiny Version

Stage	Freeze Backbone	LR	Steps
0	Yes	1e-3 (w/ W)	4000
1	Yes	-	32*4000
2	No	1e-3 to 1e-6 (w/ CA)	48*4000

Common Version

Stage	Freeze Backbone	LR	Steps
0	Yes	1e-3 (w/ W)	4000
1	Yes	-	80*4000
2	No	1e-3 to 1e-6 (w/ CA)	120*4000

Training a complete network is time-consuming ...

4. Reference

5. History

Slim version: https://github.com/yuto3o/yolox/tree/slim
Tensorflow2.0-YOLOv3: https://github.com/yuto3o/yolox/tree/yolov3-tf2

yolox's People

Contributors

Stargazers

Watchers

yolox's Issues

利用单gpu训练，速度很慢，GPU利用率很低，CPU经常跑满

大佬，我是初学者，有个问题想问下，我这边训练的时候，GPU占有使用只有零点几G 利用率最多百分之十，而CPU通常利用率百分之百，请问下是什么情况，该如何解决啊

Does your code support all the functions of yolov4 and can it train normally ?

GPU usage

I could notice that when i trigger training, gpu is not being used enen 10% also. would you have any suggestions to improve and fasten the training?

Reason for calling fit thrice in train.py

Hi, thanks for nice code!

I was trying to understand the reason for calling fit method thrice in train.py, would you mind explaining a bit about it?

yolov4-tiny pretrained model?

Hi @yuto3o,

Is it okay to publish YOLOv4-tiny pre-trained model? I would like to test it.

Thanks a lot!

Handling non-square images

Greetings,

I am using YOLOv4_tiny to detect objects on images in the shape 2704x1520. How to configure yaml cfg files?

When I am using the images in the shape 2704x1520, the Keras throws a dimension mismatch error.

Ratko

where do I find ./ckpts/yolov4.weights

convert weights to .h5

thanks for your great code. i have downloaded yolov4.weights from alexey ab directory, but i can not convert it to .h5 format as you have suggested. please help.
thanks.

train.py not working

Hey,

I am currently having a problem with train.py

After some debugging I found that the error occurs at the following line:

model.fit(train_dataset, steps_per_epoch=len(train_dataset), epochs=warmup_epochs, callbacks=warmup_callback)

and the Error message obtained is the following:
self._value = int(value)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'tuple'

any idea what is causing this? or how to investigate further where the error is comming from?
Thank you in advance

Merwan

hello
thanks for your coding
i get this error when i run the train.py files for yolov3 tiny models
###########################################################################
Traceback (most recent call last):
File "train.py", line 133, in
app.run(main)
File "C:\ProgramData\Anaconda3\envs\tf1g2\lib\site-packages\absl\app.py", line 299, in run
_run_main(main, args)
File "C:\ProgramData\Anaconda3\envs\tf1g2\lib\site-packages\absl\app.py", line 250, in _run_main
sys.exit(main(argv))
File "train.py", line 69, in main
load_weights(model, init_weight)
File "F:\1-project\object detection\yolox\yolox-master\core\utils\weight.py", line 25, in load_weights
model.load_weights(weights_file_path, by_name=True, skip_mismatch=True)
TypeError: load_weights() got an unexpected keyword argument 'skip_mismatch'
#####################################################################################
thanks for helping

AP is not improved!!!

hello
thanks for codes
i trained yolov3 tiny version by yymnist dataset but in all epochs AP is zero!!!
How can I solve this problem?

backbone yolov4

I have couple of things to understand a bit more.

looks like yolov4 also has darknet53 as backbone with Mish activation, is my understanding correct?
having mish activation and cspdarknet53 backbone improves the accuracy? suppose to improve accuracy based on the paper.

looks like there are different variations of loss functions, would you mind describing scenarios in which a particular loss function recommended or best loss function in ideal cases?

yolox initial weight

hello
thanks for your coding
which initial wieght we use in the running code with yolox network at cfg files?

Transform Model to Real_Time: Serialize Model

It is possible to Save the trained model in tensorflow format for real_time use with OpenCV-OpenVino?

As far as I know Tf.Keras runs on top of TensorFlow and it doesn´t create the graph to run this way.

While the detail of graph and session is outside of the scope of this question, how can we deploy it woth OpenCV module:
can still use the following code when deploying your model to be accessible during scheduled sessions.
This is an example. In darknet models it was quite easy and it was compatible with OpenCV. I am not shure if this way it is possible.

Load a model imported from Tensorflow

tensorflowNet = cv2.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'graph.pbtxt')

To finish, the rest of the code and they it is structures is impecable.
P.D:
Thank you a lot for all the help to make it easier for us to keep learning from this incredible technology!

Getting Error while executing yolo_utils.non_max_suppression(predictions)

Hi,
A really great piece of work done by you here.
I was able to successfully convert my weights and cfg files from darknet to tensorflow ckpt. I was using 3 classes so, I had to change filters in a few layers from 255 to 24 and I was able to get a prediction using the model. But after getting the prediction when I am using yolo_utils.non_max_suppression, I am getting the following error :

File "/Users/asdhiman/DataAnalysis/darkflow_yolo/yolov3-tensorflow/yolo_utils.py", line 92, in non_max_suppression
image_pred = image_pred.reshape(-1, shape[-1])
ValueError: cannot reshape array of size 722 into shape (8)

I am using an input image of shape 608 X 608 as my model was trained on the same dimensions in the darknet yolov3. Do I need to change anything in order for non_max_suppression to run?

conv2d_17_loss 和conv2d_20_loss

@yuto3o 你好，请问这两个loss是啥？

Weights file used in the implementation

Could you provide pretrained weights file you have used? When I tried to use the files in the provided link, the assertion fails and gives "AssertionError: failed to read all data". I believe the weights file must be in the form of h5 yet the files in the provided link have the extension of .weights. How can I convert them to h5 and feed to your models?

IndexError: index 3 is out of bounds for axis 0 with size 3

@yuto3o
Thank you for your great code. I did everything as explained, but while training i keep receiving the error: "IndexError: index 3 is out of bounds for axis 0 with size 3" after some train steps. what do you think could be the problem? Thanks.

通过关闭cfg中的数据增强手段和减小epch大小确实加快了训练速度但map值却一直没有上涨

大佬，感谢你上期的回答，但是现在这个问题导致训练的效率没有提升，如果可以的话能不能加一下你的微信或qq向你学习一下代码和调参，我现在很想把yolov4-tiny吃透