GithubHelp home page GithubHelp logo

jumabek / darknet_scripts Goto Github PK

View Code? Open in Web Editor NEW
223.0 10.0 96.0 11.58 MB

Auxilary scripts to work with (YOLO) darknet deep learning famework. AKA -> How to generate YOLO anchors?

Python 100.00%
anchor-box reproduce-yolov2-anchors anchors visualize-genereted-anchors object-detection-anchor generated-anchors

darknet_scripts's People

Contributors

jumabek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

darknet_scripts's Issues

Using anchors in .cfg

Hi I wanted to check I am using the generated anchors in the correct way (because they don't seem to improve my performance that much). Firstly, I have a custom dataset that has lots of cylindrical small objects so the ideal anchors are likely to be very different.

I generate 10 anchors (changing the width and height in the script to: 1248) and get:

0.52,0.83, 0.54,1.57, 0.63,2.12, 0.76,3.13, 0.87,1.22, 1.08,1.83, 1.67,2.40, 2.47,3.98, 3.03,2.07, 5.98,7.84
0.790824

However they look a bit funny; they are rectangular whereas most of my bounding boxes are perfect squares (since I am trying to detect the circular edge of many cylinders).

I'm also not sure about the significance of the number 0.790824? I then insert it into my yolo-voc.2.0.cfg based configuration file like so:

[convolutional]
size=1
stride=1
pad=1
filters=60
activation=linear

[region]
anchors =  0.52,0.83, 0.54,1.57, 0.63,2.12, 0.76,3.13, 0.87,1.22, 1.08,1.83, 1.67,2.40, 2.47,3.98, 3.03,2.07, 5.98,7.84
bias_match=1
classes=1
coords=4
num=10
softmax=1
jitter=.2
rescore=1

Just wanted to check I did this correctly, thanks!

Calculating anchor on custom dataset

Hey. I have my own dataset labelled as .xml and .txt files and was I wondering how to use this repository to calculate best anchor boxes for YoloV2. Thanks in advances.

Error in the gen_anchors.py script using Python 3.5.2 - 3.6.4 on Windows 7x64

Hi @Jumabek

As stated in your manual

  1. Download and install Python for Windows: https://www.python.org/ftp/python/3.5.2/python-3.5.2-amd64.exe
  • If I use Python 2.7.14 then using gen_anchors.py I can get good result:
iter 1: dists = 3720.01823941
iter 2: dists = 1491.80522823
iter 3: dists = 653.572890569
iter 4: dists = 381.526341088
iter 5: dists = 284.587940623
iter 6: dists = 242.033800206
iter 7: dists = 155.795443854
iter 8: dists = 107.012408492
iter 9: dists = 57.3295218927
iter 10: dists = 31.8779793257
iter 11: dists = 10.0989472103
iter 12: dists = 3.2274733626
('Centroids = ', array([[0.0200242 , 0.02964734],
       [0.12974252, 0.12257372]]))
(2L, 2L)
('Anchors = ', array([[0.26031462, 0.38541545],
       [1.68665282, 1.59345842]]))
()
('centroids.shape', (2L, 2L))

  • But if I use Python 3.5.2 or Python 3.6.4 on Windows 7x64 then using gen_anchors.py I get an error:
Traceback (most recent call last):
  File "D:\Darknet2\darknet\build\darknet\x64\gen_anchors.py", line 165, in <mod
ule>
    main(sys.argv)
  File "D:\Darknet2\darknet\build\darknet\x64\gen_anchors.py", line 161, in main

    kmeans(annotation_dims,centroids,eps,anchor_file)
  File "D:\Darknet2\darknet\build\darknet\x64\gen_anchors.py", line 74, in kmean
s
    k,dim = centroids.shape
ValueError: not enough values to unpack (expected 2, got 1)

Error using gen_anchors - not enough values to unpack

Upon running the script on my own dataset, I run into this error:

k,dim = centroids.shape
ValueError: not enough values to unpack (expected 2, got 1)

The filelist contains paths to only the txt files. It is reading them correctly, and no empty lines in the txt files either.

EDIT: Python3 issue. Used python2

Anchors possibly are not in harmony with the annotations

Hello,

I have a dataset with 1 class which is apple. Apple is an object with almost square annotations, in whatever scales of the object.

But when I visualize the anchors, it shows some long rectangles, I mean their width and their heights are very different, except than the yellow one. is this correct?

anchors5

Draw precision and recall curve

Could you probably tell me how to use it? Is the input of this scripts the log file? But I failed to use log file as the input.

more than 0 value to unpack - gen_anchors.py

Hi,

I get the following error when I try to execute gen_anchors.py

Traceback (most recent call last):
  File "gen_anchors.py", line 165, in <module>
    main(sys.argv)
  File "gen_anchors.py", line 142, in main
    w,h = line.split(' ')[3:]
ValueError: need more than 0 values to unpack

I run the code in Windows

Issue when executing gen_anchors

Hello @Jumabek

First of all, thanks for providing the necessary scripts to do some customizations of darknet.

I tried the gen_anchors script, with the following command python gen_anchors.py -filelist ~/Github/darknet/train.txt -num_clusters 5 -output_dir coconut_anchors/ but whenever running it, it seems I have an error through python

Traceback (most recent call last):
  File "gen_anchors.py", line 154, in <module>
    main(sys.argv)
  File "gen_anchors.py", line 128, in main
    w,h = line.split(' ')[3:]
ValueError: too many values to unpack

Any idea what could be the reason of this?

custom anchors not improving the performance

I use the provided scripts to generate the anchors on the wider dataset. However, the performance is not improving much compared to the original anchors. Have you observed the similar trend? Thanks.

error about plot_yolo_log.py

@Jumabek Hi
i have something about the plot_yolo_log.py
zxh@k40-2:~/darknet/darknet_scripts$ python plot_yolo_log.py /home/zxh/darknet/log/tiny-yolo-voc-train.log
Traceback (most recent call last):
File "plot_yolo_log.py", line 46, in
main(sys.argv)
File "plot_yolo_log.py", line 33, in main
loss.append(float(args[2]))
ValueError: could not convert string to float: weights

Plotting the anchor boxes

In visualise_anchors.py, the width and height of the anchors boxes are converted back to the image size (416x416) by multiplying by 32

w=int (w*stride)
h=int(h*stride)

However when plotting the rectangle you are using (w,h) as your bottom-right corner. Shouldn't it be

x = 10+i*stride_w
y = 10+i*stride_h
cv2.rectangle(blank_image, (x, y) ,(x+w, x+h), colors[i])

?

how to prepare file list for anchor generation?

Hi,

Thanks for sharing the scripts. To generate the anchors on own dataset, a file list needs to be prepared as suggested by the following command:
gen_anchors.py -filelist \path\to\voc-filelist\filelist.txt -num_clusters 5 -output_dir voc-anchors

My question is what are the contents and formats of filelist.txt? Could you elaborate on that? Somehow I am not able to find an example on your github. Thanks.

Another question is what is the "wider" folder? Which dataset is this referring to?

The code is not compatible with Python 3.x

Hi,

You have wrote the code in Python 2 and it is obvious from the print without parenthesis in the code. But in the description you mentioned about installing the python 3.5.

The code works with Python 2.7, but it does not work with Python 3.6.4 as I tested now, even after I modified the ( and ) for print operations.

Error:

Traceback (most recent call last):
  File "gen_anchors.py", line 165, in <module>
    main(sys.argv)
  File "gen_anchors.py", line 161, in main
    kmeans(annotation_dims,centroids,eps,anchor_file)
  File "gen_anchors.py", line 74, in kmeans
    k,dim = centroids.shape
ValueError: not enough values to unpack (expected 2, got 1)

Please fix.

Anchor sizes conflict with custom dataset

I have generated anchors for my own dataset for size 416x416.
For sanity check, I decided to resize my training images to 416x416 and draw the bounding boxes around it from the annotation text files and check if the sizes of these bounding boxes match the anchor box sizes when visualized using the visualize_anchors.py script. Surprisingly, they don't. The anchor sizes are a bit smaller when compared to the bounding box sizes around objects of interest(when images have also been resized to 416x416 which was used for anchor calculation).

I would like to produce anchors to represent my object sizes more accurately than the current case. Am I missing something here? Any inputs will be very much appreciated.

Invalid dists

Hi,

I get NaN dists for some reason. The scripts can´t read all the annotations and he can get all the information in there, but we can´t give me the dists correctly.
Can someone help me with this?
nandists

Regards,
NR

Does anchors need to be sorted ?

I tried initially with anchors not sorted and the MaP looked very bad and then when i sorted based on width i got significantly better results. Any idea ?

Anchors for custom net

Hi! First of all, thank you for your work!

I'd like to point that having a /32 there hard coded might be a little ugly. Of course this is for darknet19 and for nets that do 5 max-poolings, but maybe it would be better if an input could be parsed from outside, like the depth of your network or number of max-poolings.

Again, thanks for the work!

Generate anchors error !

Hello guys, I am trying to run gen_anchors.py but I have this error:
capture
I am using python 3.5, opencv 3.2 installed and I already fixed print function with parentheses. Please help me to fix it !

Why image size is not used? Why classes other than 0 are ignored? (gen_anchors)

Thanks for the great script. But I could not figure out why the image size is not used? The image is read and the size of it is written to [im_h,im_w] but they are not used. I expected that before

annotation_dims.append(map(float,(w,h)))

w and h are multiplied with them to get unnormalized w and h. Or am I wrong about that anchors are in unnormalized sizes?
Should they be in relative size?

The second question is about ignoring the classes other than that with id=0
if cls_id!=0:
continue

Why this is so?
Thanks

Problem with log file

Hi,
I have a problem with saving to log file. The only thing saved to it is "yolov3-markers" (that's the content of the log file). I train with this commend:
./darknet detector train markers_detection/build/darknet/x64/obj_improved.data markers_detection/build/darknet/x64/yolov3-markers.cfg backup/yolov3-markers_9000.weights >> log/yolov3-markers.log

General question

Hello,
I am training YOLO with my own data. I started with some available examples and they worked well. Nevertheless, when using my data set the network doesn't find any object (although loss value is around 0.06). Since my images have big size with small objects, I read that anchor values must be adjusted, although it is not clear how to do it.

Then, I actually did the following:

  1. Get image width and size
  2. Get every object width and size in any image
  3. Calculate ratio object_width / image_width and object_height / image_height (here all values are less than 1)
  4. Calculate k-means clustering for those ratios
  5. Multiply the values by 13 (grid cell number) and use the values as anchors [w1, h1, w2, h2, ... w5, h5]

Is this process OK?

Thank you.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.