jwchoi384 / gaussian_yolov3 Goto Github PK

Hi, I talked with you at your poster in ICCV2019!
I just released PyTorch implementation of Gaussian YOLOv3 with training code on COCO dataset. Would it be possible to include link to our repo in third-party-implementations section of your README? If it is okay for you, I can send a pull request to update your README.

Though Gaussian YOLOv3 in our repo shows significant improvement of COCO mAP (2.7 point) on COCO2017 val, this improvement is still smaller than the one reported in your paper (3.1 point).
I'm wondering if this difference comes from the hyper parameter sigma_const set to 0.3 in your implementation (our implementation does not have this parameter).

Do you think sigma_const affects the result a lot?
How did you find the value 0.3 for this parameter?

Hi, @jwchoi384. Thanks for this wonderful project.
Since I did not find any confidence score threshold in the bdd evaluation script, I wonder if you set confidence score threshold when evaluate.

If the answer is yes, how do you set the thresholds when you evaluate YOLOv3 and Gaussian_YOLOv3? After multiply the box certainty score, the final confidence score in Gaussian_YOLOv3 would be a little smaller than the one in YOLOv3.

Can we adapt this approach to the other networks -like tiny yolo- by changing yolo layer to the gaussian yolo layer and adjusting filter size?

I got 0.47map when the IOU is 0.5, I am wondering if your result is correct or not.

Hi Choi,
which IDE for the paper?
Best regards,
Peter Pham

Hi, could you please upload model weight to repository? Sorry, I cannot download it from the google.drive.
Thanks!

Can I use the Gaussian_yolov3_BDD.weights to be my training pre-trained weights?

Thanks.

In the equation (5), Loss function is defined as -log(N(x|μ,σ). I think N(x|μ，σ）is the value of probability density function of gaussian distribution. What does that mean? Does it reflect the probability?

I don't think the low FP will lead a accident. because when the car with the ADAS system detects a obstacle that does not really exists, it will stop. I think the high FN will lead to the accident.

In the paper, "w_scale in equation (8) is calculated based on the width and height
ratios of the GT bbox in an image, as shown in (9). It provides different weights according to the object size during training."
I don't understand why loss should be weighted according to the size of the objects? Thanks.

Hi, a strange idea happens to me: the test result on server doesn't give the class name corresponding to mAP value, so how can I know which value belongs to its class? And further, when I make txt training file, could the ordering of class name be consistent with that of official test?
Thanks in advance!

Hi, jwchoi284. I tried to upload test result of single class to the server fro evaluation as you said before, but still no any result could be returned. I am really helpless, and do you have some ideas for me?
Thanks in advance!

Thanks for the work! I trained the Gaussian Yolov3 on my own dataset. When I tried to validate the model by using: ./darknet detector valid cfg/xx.data cfg/xx.cfg backup/xxx.weights I found all the results are saved in the /results directory with separate txt files corresponding to different categories, which may be related to "eval = xx" in cfg file. May I know what are the choices for eval? And which one is suggested for the evaluation of darknet format labels?

@jwchoi384 ,
I want to print out TP and FP of my results. How can i do it ?
print fp[-1] after this line https://github.com/jwchoi384/Gaussian_YOLOv3/blob/master/bdd_evaluation/evaluate.py#L169, right ?

hi ，i try to train yolov2 by your code(Gaussian), but report error“Floating point exception (core dumped)”

Hi, I run the code trained on BDD100K dataset from scratch, but the result returns loss nan. What's the problems? Could you please give me some advice for mode training in such case?
Thanks!

Hi @jwchoi384 ,
I want to calculate TP/FP of KITTI dataset in your paper. How can I do it ?

Hi, nice work!

I use it to train yolov3-tiny.After training 150 batch, the loss became nan,Have you encountered this problem? Thank you!

batch=32,subdivisions=16,learning_rate=0.0001

Region 15 Avg IOU: nan, Class: 0.441899, Obj: 0.010084, No Obj: 0.009767, .5R: 0.478261, .75R: 0.043478, count: 23
Region 22 Avg IOU: 0.392249, Class: 0.575245, Obj: 0.002243, No Obj: 0.001650, .5R: 0.236842, .75R: 0.000000, count: 38
Region 15 Avg IOU: nan, Class: 0.458486, Obj: 0.010441, No Obj: 0.009767, .5R: 0.428571, .75R: 0.000000, count: 7
Region 22 Avg IOU: 0.464182, Class: 0.519555, Obj: 0.001945, No Obj: 0.001650, .5R: 0.460317, .75R: 0.015873, count: 63
Region 15 Avg IOU: nan, Class: 0.461843, Obj: 0.010374, No Obj: 0.009767, .5R: 0.320000, .75R: 0.040000, count: 25
Region 22 Avg IOU: 0.386944, Class: 0.502782, Obj: 0.001714, No Obj: 0.001650, .5R: 0.274510, .75R: 0.019608, count: 51
Region 15 Avg IOU: nan, Class: 0.463781, Obj: 0.010159, No Obj: 0.009767, .5R: 0.285714, .75R: 0.071429, count: 28
Region 22 Avg IOU: 0.439150, Class: 0.492993, Obj: 0.001829, No Obj: 0.001650, .5R: 0.428571, .75R: 0.000000, count: 49
Region 15 Avg IOU: nan, Class: 0.466690, Obj: 0.010356, No Obj: 0.009767, .5R: 0.150000, .75R: 0.050000, count: 20
Region 22 Avg IOU: 0.346722, Class: 0.426868, Obj: 0.001468, No Obj: 0.001650, .5R: 0.240000, .75R: 0.040000, count: 50
Region 15 Avg IOU: nan, Class: 0.466256, Obj: 0.010191, No Obj: 0.009767, .5R: 0.136364, .75R: 0.000000, count: 22
Region 22 Avg IOU: 0.382021, Class: 0.544101, Obj: 0.002096, No Obj: 0.001650, .5R: 0.247059, .75R: 0.000000, count: 85
Region 15 Avg IOU: nan, Class: 0.465827, Obj: 0.010348, No Obj: 0.009767, .5R: 0.222222, .75R: 0.000000, count: 18
Region 22 Avg IOU: 0.444984, Class: 0.587952, Obj: 0.001802, No Obj: 0.001650, .5R: 0.467532, .75R: 0.025974, count: 77
Region 15 Avg IOU: nan, Class: 0.466738, Obj: 0.009839, No Obj: 0.009767, .5R: 0.300000, .75R: 0.000000, count: 10
Region 22 Avg IOU: 0.371075, Class: 0.510235, Obj: 0.002036, No Obj: 0.001650, .5R: 0.300000, .75R: 0.000000, count: 30
Region 15 Avg IOU: nan, Class: 0.460950, Obj: 0.010231, No Obj: 0.009767, .5R: 0.473684, .75R: 0.105263, count: 19
Region 22 Avg IOU: 0.397543, Class: 0.496430, Obj: 0.001451, No Obj: 0.001650, .5R: 0.337662, .75R: 0.025974, count: 77
Region 15 Avg IOU: nan, Class: 0.451978, Obj: 0.010417, No Obj: 0.009767, .5R: 0.666667, .75R: 0.166667, count: 6
Region 22 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.001650, .5R: -nan, .75R: -nan, count: 0
Region 15 Avg IOU: nan, Class: 0.463936, Obj: 0.010312, No Obj: 0.009767, .5R: 0.250000, .75R: 0.000000, count: 28
Region 22 Avg IOU: 0.527799, Class: 0.639013, Obj: 0.001620, No Obj: 0.001650, .5R: 0.596774, .75R: 0.064516, count: 62
Region 15 Avg IOU: nan, Class: 0.466003, Obj: 0.010128, No Obj: 0.009767, .5R: 0.240000, .75R: 0.040000, count: 25
Region 22 Avg IOU: 0.493470, Class: 0.549164, Obj: 0.001179, No Obj: 0.001650, .5R: 0.461538, .75R: 0.128205, count: 39
Region 15 Avg IOU: nan, Class: 0.464300, Obj: 0.010413, No Obj: 0.009767, .5R: 0.102564, .75R: 0.000000, count: 39
Region 22 Avg IOU: 0.478209, Class: 0.595393, Obj: 0.001984, No Obj: 0.001650, .5R: 0.486111, .75R: 0.027778, count: 72
Region 15 Avg IOU: nan, Class: 0.457656, Obj: 0.010366, No Obj: 0.009767, .5R: 0.240000, .75R: 0.000000, count: 25
Region 22 Avg IOU: 0.451578, Class: 0.537413, Obj: 0.002011, No Obj: 0.001650, .5R: 0.385714, .75R: 0.028571, count: 70
Region 15 Avg IOU: nan, Class: 0.457986, Obj: 0.010434, No Obj: 0.009767, .5R: 0.357143, .75R: 0.000000, count: 14
Region 22 Avg IOU: 0.343575, Class: 0.520320, Obj: 0.002263, No Obj: 0.001650, .5R: 0.233333, .75R: 0.000000, count: 30
Region 15 Avg IOU: nan, Class: 0.461090, Obj: 0.010450, No Obj: 0.009767, .5R: 0.222222, .75R: 0.000000, count: 9
Region 22 Avg IOU: 0.510054, Class: 0.517130, Obj: 0.001078, No Obj: 0.001650, .5R: 0.550000, .75R: 0.000000, count: 20
151: nan, -nan avg, 0.000100 rate, 2.439035 seconds, 4832 images

@jwchoi384
Thank you for your work done.
I don't quite understand why the uncertainty of regression can reduce FP.
float prob = objectness*predictions[class_index]*(1.0-uc_aver);
That only reduces the prob. Can you give me some explanation?Thank you!

I' m not familiar with the dataset and hope that someone can explain more about the leaderboard.
Thank you!

Hi, could you please provide script for us that converts txt to json format, so that we can evaluate on bdd-data? My ground-truth file is txt format due to being used for yolov3 model training.
Thanks a lot.

Can I use KITTI dataset that contains .txt and img files for car plate detection

use the same annotations as YOLO or need new annotations

Hi, nice work!

I want to use tf implementation the work, but i dont know loss function (5) in paper; Especially the vertical line symbol in Especially the vertical line symbol； I read you code, but i dont know the following code:

` float temp_x = (1./2.) * 1./(normal_dist_x+epsi) * normal_dist_x * scale;
float temp_y = (1./2.) * 1./(normal_dist_y+epsi) * normal_dist_y * scale;
float temp_w = (1./2.) * 1./(normal_dist_w+epsi) * normal_dist_w * scale;
float temp_h = (1./2.) * 1./(normal_dist_h+epsi) * normal_dist_h * scale;

delta[index + 0*stride] = temp_x * in_exp_x  * (1./x[index+1*stride]);
delta[index + 2*stride] = temp_y * in_exp_y  * (1./x[index+3*stride]);
delta[index + 4*stride] = temp_w * in_exp_w  * (1./x[index+5*stride]);
delta[index + 6*stride] = temp_h * in_exp_h  * (1./x[index+7*stride]);

delta[index + 1*stride] = temp_x * (in_exp_x_2/x[index+1*stride] - 1./(x[index+1*stride]+sigma_const));
delta[index + 3*stride] = temp_y * (in_exp_y_2/x[index+3*stride] - 1./(x[index+3*stride]+sigma_const));
delta[index + 5*stride] = temp_w * (in_exp_w_2/x[index+5*stride] - 1./(x[index+5*stride]+sigma_const));
delta[index + 7*stride] = temp_h * (in_exp_h_2/x[index+7*stride] - 1./(x[index+7*stride]+sigma_const));`

Can you give a detailed explanation? Thanks for your great work.

Hi, jwchoi

when I training Gaussian yolov3 model (lr = 0.0001 0.00001 0.000001), the avg loss value can't converge, after the loss value touched 1, then the loss start increase slowly.

have you met this problem, did I missed some details?

Hello, how does this algorithm apply to Yolov3 tiny

Maybe, the motivation of authors at beginning is to optimize YOLOv3's behavior on autonomous driving dataset. Well, if the explanation of Gaussian and location score is true, it seems workable for general object detection. So. I'm very curious about the mAP result of COCO.

If training from ImageNet pretrained model are expensive, I hope anyone can offer a result of fineture reult on a well-trained YOLOv3 model witch implement the modification proposed in this paper.

the Lx is negative log likelihood loss. But why the it is negative and log. May you use the log may get rid of the e of normal distribution.

Hi @jwchoi384
Thanks for you share.
I refer to your code, compiled a version, I am using your weight file and configuration file, when you reason, I can get the same result as you.
However, when I went to train myself , the network could not converge and fluctuated around 10 rounds.
Excuse me, what is the reason?
Single card, 1080ti, lr=0.001

batch=64
subdivisions=16
width=416
height=416
channels=3
momentum=0.9
decay=0.0005
angle=0
saturation = 1.5
exposure = 1.5
hue=.1
flip=0
learning_rate=0.001
burn_in=1000
max_batches = 50020
policy=steps
steps=40000,45000
scales=.1,.1

Thank you very much.

I have trained KITTI dataset around 20k iter with yolov3 to reproduce your result in paper and archive mAP like this with your setting in paper. Through training process, i saw loss value was converging but i don't know why mAP for Car at easy is quite low. Did you see it normal? Can you give me some ideas about this result?

I use eval = bdd and eval = coco in *.data file.
Try test and valid command.
test return nothing
valid return .json file, but json file is empty.
so how to test on my own dataset train weights?

I have trained on my dataset,found the training loss is much higher,it was 40 but ap 98.3.Compared to the original yolov3, the loss is less than 0.3 and ap 99.3.
So what range of training loss is normal by using new loss func and which tricks you have used?

Hi @jwchoi384 ,
After training process with 60k iteration, I got this result. I see a big gap between result from your paper and my result in Car Moderate mAP.And I want to hear your idea about the gap? Is it normal ?

Can I use tiny yolov3 cfg file with Gaussian Yolov3?
And if yes what changes I have to make to train images with tiny yolov3 cfg file?

Hi, could you please teach me to use KITTI evaluation code? Now, I have trained my model and obtained predicted results on test data based on trained model. And then the evaluation code have been downloaded from the official website. But I cannot use it, and how can I do next? Please help me to continue!
Thanks in advance!

my data is voc2007 format?how to train it

I want to ask about Python / darknet.py res.append([meta.names[i], dets[j].prob[i], [b.x, b.y, b.w, b.h]])
When output, when J > 1, error is always reported? It seems that there is no such value

Hello, i'm want to test this model on my data. After i cloned a files i tried to make it. And i have this error msg:

Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing `opencv.pc'
to the PKG_CONFIG_PATH environment variable
No package 'opencv' found
In file included from ./src/utils.h:5:0,
from ./src/gemm.c:2:
include/darknet.h:14:14: fatal error: curand.h: No such file or directory
#include "curand.h"
^~~~~~~~~~
compilation terminated.
Makefile:89: recipe for target 'obj/gemm.o' failed
make: *** [obj/gemm.o] Error 1

But darknet YOLOv3 compiles without this errors. Should paths of this project and darknet have some special structure?

Hi, I delivered the predicted result on test data to the BDD100K server, but I don't receive any response yet. And I sent the email to the official, there was also no any information returned to me. Do you know what's the problem? Is Does the server exist some bugs? How do you receive your test result?
Please help me! Thanks!

Hi jwchoi384,
Thanks for the awsome project. I wonder how to prepare the dataset for training. Except I need to generate test_bdd_list.txt, train_bdd_list.txt and val_bdd_list.txt, do I need to generate a label txt file for each image? And should the label txt file be in the same folder with responding image?

Finally, is there a script to reference?

Hi, nice work!

Did you compare Gaussian_YOLOv3 with GIoU?

• https://github.com/generalized-iou/g-darknet
• https://arxiv.org/abs/1902.09630v2
• https://giou.stanford.edu/
• AlexeyAB/darknet#3249

Or you can try to combine Gaussian_YOLOv3 + GIoU.

@jwchoi384

Hi, I'm confused about the loss function you proposed at equation (5) in your paper.

In my understanding, the goal of the NLL loss is trying to make the predicted mean value and GT value as close as possible, e.g and

and make the variance, e.g. close to 0 (which means high certainty).

However, at your equation, if the predicted mean value equals GT value and variance close to 0, the
NLL loss could be negative.

How do you think about this?

Thanks!

halo,thank you to open your code!
when I was get this code, I used the USB camera, and run below command:
./darknet detector demo cfg/BDD.data cfg/Gaussian_yolov3_BDD.cfg Gaussian_yolov3_BDD.weights
even the teminal display the FPS is 51
but in fact,the windows of image out show is too slow, process time is too long compared with the original YOLOV3,
so I want to know how to get better that use USB camera

and my computer's GPU is TITAN xp, i use cuda10.1
thank you!!

I run the python ./python/darknet.py， but an error see: Segmentation fault (core dumped)?!!!

First, thanks for your great work.

Can you provide model/cfg file/evaluate code for KITIT dataset used in your paper ?