jumabek / darknet_scripts Goto Github PK

Hi,

Thanks for sharing the scripts. To generate the anchors on own dataset, a file list needs to be prepared as suggested by the following command:
gen_anchors.py -filelist \path\to\voc-filelist\filelist.txt -num_clusters 5 -output_dir voc-anchors

My question is what are the contents and formats of filelist.txt? Could you elaborate on that? Somehow I am not able to find an example on your github. Thanks.

Another question is what is the "wider" folder? Which dataset is this referring to?

Hello @Jumabek

First of all, thanks for providing the necessary scripts to do some customizations of darknet.

I tried the gen_anchors script, with the following command python gen_anchors.py -filelist ~/Github/darknet/train.txt -num_clusters 5 -output_dir coconut_anchors/ but whenever running it, it seems I have an error through python

Traceback (most recent call last):
  File "gen_anchors.py", line 154, in <module>
    main(sys.argv)
  File "gen_anchors.py", line 128, in main
    w,h = line.split(' ')[3:]
ValueError: too many values to unpack

Any idea what could be the reason of this?

Hi @Jumabek

As stated in your manual

3. Download and install Python for Windows: https://www.python.org/ftp/python/3.5.2/python-3.5.2-amd64.exe

• If I use Python 2.7.14 then using gen_anchors.py I can get good result:

iter 1: dists = 3720.01823941
iter 2: dists = 1491.80522823
iter 3: dists = 653.572890569
iter 4: dists = 381.526341088
iter 5: dists = 284.587940623
iter 6: dists = 242.033800206
iter 7: dists = 155.795443854
iter 8: dists = 107.012408492
iter 9: dists = 57.3295218927
iter 10: dists = 31.8779793257
iter 11: dists = 10.0989472103
iter 12: dists = 3.2274733626
('Centroids = ', array([[0.0200242 , 0.02964734],
       [0.12974252, 0.12257372]]))
(2L, 2L)
('Anchors = ', array([[0.26031462, 0.38541545],
       [1.68665282, 1.59345842]]))
()
('centroids.shape', (2L, 2L))

• But if I use Python 3.5.2 or Python 3.6.4 on Windows 7x64 then using gen_anchors.py I get an error:

Traceback (most recent call last):
  File "D:\Darknet2\darknet\build\darknet\x64\gen_anchors.py", line 165, in <mod
ule>
    main(sys.argv)
  File "D:\Darknet2\darknet\build\darknet\x64\gen_anchors.py", line 161, in main

    kmeans(annotation_dims,centroids,eps,anchor_file)
  File "D:\Darknet2\darknet\build\darknet\x64\gen_anchors.py", line 74, in kmean
s
    k,dim = centroids.shape
ValueError: not enough values to unpack (expected 2, got 1)

I tried initially with anchors not sorted and the MaP looked very bad and then when i sorted based on width i got significantly better results. Any idea ?

Could you probably tell me how to use it? Is the input of this scripts the log file? But I failed to use log file as the input.

In visualise_anchors.py, the width and height of the anchors boxes are converted back to the image size (416x416) by multiplying by 32

w=int (w*stride)
h=int(h*stride)

However when plotting the rectangle you are using (w,h) as your bottom-right corner. Shouldn't it be

x = 10+i*stride_w
y = 10+i*stride_h
cv2.rectangle(blank_image, (x, y) ,(x+w, x+h), colors[i])

?

Hello,

I have a dataset with 1 class which is apple. Apple is an object with almost square annotations, in whatever scales of the object.

But when I visualize the anchors, it shows some long rectangles, I mean their width and their heights are very different, except than the yellow one. is this correct?

Hello,
I am training YOLO with my own data. I started with some available examples and they worked well. Nevertheless, when using my data set the network doesn't find any object (although loss value is around 0.06). Since my images have big size with small objects, I read that anchor values must be adjusted, although it is not clear how to do it.

Then, I actually did the following:

1. Get image width and size
2. Get every object width and size in any image
3. Calculate ratio object_width / image_width and object_height / image_height (here all values are less than 1)
4. Calculate k-means clustering for those ratios
5. Multiply the values by 13 (grid cell number) and use the values as anchors [w1, h1, w2, h2, ... w5, h5]

Is this process OK?

Thank you.

How to choise value of num_clusters ? what it mean ?
Thanks!!

Hi,

You have wrote the code in Python 2 and it is obvious from the print without parenthesis in the code. But in the description you mentioned about installing the python 3.5.

The code works with Python 2.7, but it does not work with Python 3.6.4 as I tested now, even after I modified the ( and ) for print operations.

Error:

Traceback (most recent call last):
  File "gen_anchors.py", line 165, in <module>
    main(sys.argv)
  File "gen_anchors.py", line 161, in main
    kmeans(annotation_dims,centroids,eps,anchor_file)
  File "gen_anchors.py", line 74, in kmeans
    k,dim = centroids.shape
ValueError: not enough values to unpack (expected 2, got 1)

Please fix.

Hi! First of all, thank you for your work!

I'd like to point that having a /32 there hard coded might be a little ugly. Of course this is for darknet19 and for nets that do 5 max-poolings, but maybe it would be better if an input could be parsed from outside, like the depth of your network or number of max-poolings.

Again, thanks for the work!

Upon running the script on my own dataset, I run into this error:

k,dim = centroids.shape
ValueError: not enough values to unpack (expected 2, got 1)

The filelist contains paths to only the txt files. It is reading them correctly, and no empty lines in the txt files either.

EDIT: Python3 issue. Used python2

I use the provided scripts to generate the anchors on the wider dataset. However, the performance is not improving much compared to the original anchors. Have you observed the similar trend? Thanks.

Hi,

I get the following error when I try to execute gen_anchors.py

Traceback (most recent call last):
  File "gen_anchors.py", line 165, in <module>
    main(sys.argv)
  File "gen_anchors.py", line 142, in main
    w,h = line.split(' ')[3:]
ValueError: need more than 0 values to unpack

I run the code in Windows

I have generated anchors for my own dataset for size 416x416.
For sanity check, I decided to resize my training images to 416x416 and draw the bounding boxes around it from the annotation text files and check if the sizes of these bounding boxes match the anchor box sizes when visualized using the visualize_anchors.py script. Surprisingly, they don't. The anchor sizes are a bit smaller when compared to the bounding box sizes around objects of interest(when images have also been resized to 416x416 which was used for anchor calculation).

I would like to produce anchors to represent my object sizes more accurately than the current case. Am I missing something here? Any inputs will be very much appreciated.

@Jumabek Hi
i have something about the plot_yolo_log.py
zxh@k40-2:~/darknet/darknet_scripts$ python plot_yolo_log.py /home/zxh/darknet/log/tiny-yolo-voc-train.log
Traceback (most recent call last):
File "plot_yolo_log.py", line 46, in
main(sys.argv)
File "plot_yolo_log.py", line 33, in main
loss.append(float(args[2]))
ValueError: could not convert string to float: weights

Hi,
I have a problem with saving to log file. The only thing saved to it is "yolov3-markers" (that's the content of the log file). I train with this commend:
./darknet detector train markers_detection/build/darknet/x64/obj_improved.data markers_detection/build/darknet/x64/yolov3-markers.cfg backup/yolov3-markers_9000.weights >> log/yolov3-markers.log

Thanks for the great script. But I could not figure out why the image size is not used? The image is read and the size of it is written to [im_h,im_w] but they are not used. I expected that before

annotation_dims.append(map(float,(w,h)))

w and h are multiplied with them to get unnormalized w and h. Or am I wrong about that anchors are in unnormalized sizes?
Should they be in relative size?

The second question is about ignoring the classes other than that with id=0
if cls_id!=0:
continue

Why this is so?
Thanks

Hi,

I get NaN dists for some reason. The scripts can´t read all the annotations and he can get all the information in there, but we can´t give me the dists correctly.
Can someone help me with this?

Regards,
NR

Hi I wanted to check I am using the generated anchors in the correct way (because they don't seem to improve my performance that much). Firstly, I have a custom dataset that has lots of cylindrical small objects so the ideal anchors are likely to be very different.

I generate 10 anchors (changing the width and height in the script to: 1248) and get:

0.52,0.83, 0.54,1.57, 0.63,2.12, 0.76,3.13, 0.87,1.22, 1.08,1.83, 1.67,2.40, 2.47,3.98, 3.03,2.07, 5.98,7.84
0.790824

However they look a bit funny; they are rectangular whereas most of my bounding boxes are perfect squares (since I am trying to detect the circular edge of many cylinders).

I'm also not sure about the significance of the number 0.790824? I then insert it into my yolo-voc.2.0.cfg based configuration file like so:

[convolutional]
size=1
stride=1
pad=1
filters=60
activation=linear

[region]
anchors =  0.52,0.83, 0.54,1.57, 0.63,2.12, 0.76,3.13, 0.87,1.22, 1.08,1.83, 1.67,2.40, 2.47,3.98, 3.03,2.07, 5.98,7.84
bias_match=1
classes=1
coords=4
num=10
softmax=1
jitter=.2
rescore=1

Just wanted to check I did this correctly, thanks!

Good work... I used anchor generated by your anchor generator, but it fails to detect any object in my model. I want to know why we divided width_in_cfg_file by 32 and also why we are using stride = 32

Edited:
got the answer here: https://github.com/Jumabek/darknet_scripts/issues/11
but wants to know more
Thanks in advance!

I know it decides on my own images，so how to get the number?
AND how to get the visualizations like you've showed to us?

THANK YOU in advance.

Hello guys, I am trying to run gen_anchors.py but I have this error:

I am using python 3.5, opencv 3.2 installed and I already fixed print function with parentheses. Please help me to fix it !

Hey. I have my own dataset labelled as .xml and .txt files and was I wondering how to use this repository to calculate best anchor boxes for YoloV2. Thanks in advances.