lars76 / kmeans-anchor-boxes Goto Github PK

Hello, thank you for the code, but I don't quite understand how it works. Can you help me?
I not really understand the following step:
x = np.minimum(clusters[:, 0], box[0]) y = np.minimum(clusters[:, 1], box[1]) if np.count_nonzero(x == 0) > 0 or np.count_nonzero(y == 0) > 0: raise ValueError("Box has no area")
I also don’t understand the definition of the kmeans function:
`def kmeans(boxes, k, dist=np.median):
rows = boxes.shape[0]
distances = np.empty((rows, k))
last_clusters = np.zeros((rows,))
np.random.seed()

clusters = boxes[np.random.choice(rows, k, replace=False)]

while True:
    for row in range(rows):
        distances[row] = 1 - iou(boxes[row], clusters)
    nearest_clusters = np.argmin(distances, axis=1)
    if (last_clusters == nearest_clusters).all():
        break
    for cluster in range(k):
        clusters[cluster] = dist(boxes[nearest_clusters == cluster], axis=0)
    last_clusters = nearest_clusters

`
I seem to know its general principle. First, set d=1-IoU(s1, s2) as the distance formula for kmeans clustering. Then all the bounding boxes that are closer to the bounding box of the cluster center are treated as the same class. Constantly looping, and finally get k classes, which are the width and height of the bounding box. I understand it right?

But I really don't understand the details, such as I don't know how it chooses x and y, and how it chooses the cluster centroid for clustering, and I don't quite understand how to get the width and height of the bounding box. Like these arrays of the following, the first column represents width and the second column represents height. So how does it generate these widths and heights?

[[0.0234375 0.06041667]
[0.02109375 0.046875 ]
[0.01875 0.04166667]
[0.028125 0.05104167]
[0.0140625 0.028125 ]
[0.0421875 0.08854167]
[0.009375 0.02083333]
[0.02109375 0.04166667]
[0.0328125 0.06979167]]

I'm so confused about these questions , Can you tell me about how it is implemented? Or can you make some comments on these functions, sorry to bother you , please help me , thank you very much.

I get Boxes:
[[0.0671875 0.09814815]
[0.07552083 0.11388889]
[0.05572917 0.08796296]
[0.24010417 0.17222222]
[0.32552083 0.26759259]
[0.0953125 0.12685185]
[0.05833333 0.08333333]
[0.12604167 0.14074074]
[0.18177083 0.15740741]]

Then copy and paste the numbers above. Then flatten the array by hand and get a list, a:
(The numbers of Boxes and a are from different results, so they are indepent. Just show that a is from Boxes)
a=[0.452,0.69375,0.116,0.104,0.22,0.512,0.1,0.30303703,0.038,0.064,0.056,0.16266667,0.41,0.33866667,0.192,0.224,0.83766667,0.78469484]

#Training yolov3, the imgs are resized to 606x608
#My imgs' size is 1920*1080. After k-means, I can ignore 1920*1080, right?
b=[round(608*x) for x in a]
boxes=[]
areas=[]
for i in range(0,len(a),2):
    boxes.append([b[i],b[i+1]])
    areas.append([b[i]*b[i+1]])
#print(boxes)
#print(areas)
new_areas=sorted(areas)
new_boxes=[]
#print(new_areas)
for i in range(0,len(boxes)):
    mylist=list(range(0,len(boxes)))
    for j in mylist:
        if new_areas[i]==areas[j]:
            new_boxes.append(boxes[j])
            mylist.remove(j)
print(new_boxes)

new_boxes=
[[23, 39], [34, 99], [71, 63], [61, 184], [117, 136], [134, 311], [249, 206], [275, 422], [509, 477]]

Should the following part of gaussian_yolov3_default.cfg be like this?

  ANCHORS: [[23, 39], [34, 99], [71, 63], 
            [61, 184], [117, 136], [134, 311],
            [249, 206], [275, 422], [509, 477]]
  ANCH_MASK: [[6, 7, 8], [3, 4, 5], [0, 1, 2]]

Many thanks!

1、after we use the kmeans and find the bettest number of cluster is 5, then how to decide the anchor box width and height?
2、here we calculate the you is that to use the percent of width/height to instead of height/height

Hi, Thank you for your contribution.
I am getting a numpy array of the distances of every pixel in the bounding box. Now I am trying to apply k-means to get the median of distance to the object in my case person. Can you check if my approach is correct?

self.cluster = kmeans.kmeans(np.array(self.kmeans_box_data),2,np.median(self.person_bounding_box))

where self.kmeans_box_data is ((xmax-xmin)/w),(ymax-ymin)/h) 2 is the number of clusters and self.person_bounding_box is the array of distance at every pixel in the bounding box.
I am getting the following exception:

src/kmeans.py", line 11, in iou
    x = np.minimum(clusters[:, 0], box[0])
IndexError: too many indices for array

And when k is greater than 2, I get the following error:

kmeans.py", line 64, in kmeans
    clusters = boxes[np.random.choice(rows, k, replace=False)]
  File "mtrand.pyx", line 1168, in mtrand.RandomState.choice
ValueError: Cannot take a larger sample than population when 'replace=False'

Any suggestions are appreciated.

你好，我在使用你的代码的时候，有点困惑。得到的结果如下：
Accuracy: 87.11%
Boxes:
[[0.0234375 0.06041667]
[0.02109375 0.046875 ]
[0.01875 0.04166667]
[0.028125 0.05104167]
[0.0140625 0.028125 ]
[0.0421875 0.08854167]
[0.009375 0.02083333]
[0.02109375 0.04166667]
[0.0328125 0.06979167]]
Ratios:
[0.39, 0.45, 0.45, 0.45, 0.47, 0.48, 0.5, 0.51, 0.55]

结果是相对比例，但是YOLOv3是使用的绝对值。我的想法是应该对每个值乘上输入进网络的尺寸，比如输入网络尺寸是608x608。那anchor大小尺寸就是上面的每一个值乘以608。请问我的理解没错吧？

Hi，非常感谢你的代码。你之前的回答对我很有帮助，不过当我想在voc2007annotations上测试时却没有结果输出，我不知道那地方出错了，你能帮帮我吗？如下是我的结果：
dxt@dxt-System-Product-Name:~/kmeans-anchor-boxes-master$ python3 -m unittest discover -s tests/
..............

Ran 14 tests in 77.215s

OK

最后只出现了这个结果，我直接将annotations放在了kmeans-anchor-boxes-master文件夹下，只需要annotations吗，是否还需要其他的文件，你能指出哪些地方可能会出问题吗？非常感谢你，祝你生活愉快。

def iou(box, clusters):
    """
    Calculates the Intersection over Union (IoU) between a box and k clusters.
    :param box: tuple or array, shifted to the origin (i. e. width and height)
    :param clusters: numpy array of shape (k, 2) where k is the number of clusters
    :return: numpy array of shape (k, 0) where k is the number of clusters
    """
    x = np.minimum(clusters[:, 0], box[0])
    y = np.minimum(clusters[:, 1], box[1])
    if np.count_nonzero(x == 0) > 0 or np.count_nonzero(y == 0) > 0:
        raise ValueError("Box has no area")

    intersection = x * y
    box_area = box[0] * box[1]
    cluster_area = clusters[:, 0] * clusters[:, 1]

    iou_ = intersection / (box_area + cluster_area - intersection)

    return iou_

Hi~ I think when u use '/' to calculate iou_ may have a bug.
For example, box = np.array([250, 200]),
clusters = np.array([[200, 200], [100, 200], [200, 100], [150, 200], [200, 150]]),
the output is [0, 0, 0, 0, 0]. Actually, it should be [0.8, 0.4, 0.4, 0.6, 0.6].

I think there are two methods to solve this bug.
One is setting boxes' & clusters' data type.

boxes = boxes.astype(np.float32)
clusters = clusters.astype(np.float32)

The other is using np.true_divide() to calculate iou_.

iou_ = np.true_divide(intersection, box_area + cluster_area - intersection)

Do u think so?

In my own dataset, the func "kmeans" in kmeans.py went into endless loop because the condition “boxes[nearest_clusters == cluster]” was None. Maybe this condition should be considered. Thanks a lot for your codes. @lars76

你好，我运行后没有输出，请问是为什么呢？

asus@asus:~/c001/VOC2007/kmeans-anchor-boxes-master$ python3 -m unittest discover -s tests/
............FF
FAIL: test_kmeans_5 (test_voc2007.TestVoc2007)
Traceback (most recent call last):
File "/home/luvision/c001/VOC2007/kmeans-anchor-boxes-master/tests/test_voc2007.py", line 37, in test_kmeans_5
np.testing.assert_almost_equal(percentage, 0.61, decimal=2)
File "/home/luvision/anaconda3/lib/python3.6/site-packages/numpy/testing/nose_tools/utils.py", line 581, in assert_almost_equal
raise AssertionError(_build_err_msg())
AssertionError:
Arrays are not almost equal to 2 decimals
ACTUAL: 0.8394690411933133
DESIRED: 0.61

======================================================================
FAIL: test_kmeans_9 (test_voc2007.TestVoc2007)
Traceback (most recent call last):
File "/home/luvision/c001/VOC2007/kmeans-anchor-boxes-master/tests/test_voc2007.py", line 45, in test_kmeans_9
np.testing.assert_almost_equal(percentage, 0.672, decimal=2)
File "/home/luvision/anaconda3/lib/python3.6/site-packages/numpy/testing/nose_tools/utils.py", line 581, in assert_almost_equal
raise AssertionError(_build_err_msg())
AssertionError:
Arrays are not almost equal to 2 decimals
ACTUAL: 0.8947800941384673
DESIRED: 0.672

Ran 14 tests in 5.298s

FAILED (failures=2)

如果去掉tests/test_voc2007.py这两行：
np.testing.assert_almost_equal(percentage, 0.61, decimal=2)
np.testing.assert_almost_equal(percentage, 0.672, decimal=2)

就是这样
asus@asus:~/c001/VOC2007/kmeans-anchor-boxes-master$ python3 -m unittest discover -s tests/
..............jiji
Ran 14 tests in 3.390s

OK

@lars76
Hi,how are you.the code works very well,But I still confused about how does it works.
for example the results:
AP = 0.67
K=5
[[0.792 0.776 ]
[0.098 0.16516517]
[0.042 0.07466667]
[0.38133333 0.52535323]
[0.184 0.33866667]]
what does these [[0.792 0.776 ] ,[0.098 0.16516517] ...] mean?Are they refer to the width and height of the anchors,?How to get these anchors's width and height And why in the yolo9000 these wight and height are some coordinates? for example(10x13),(16x30),(33x23),(30x61),(62x45)...
Can you teach me about that？
thanks so much and Happy Dragon Boat Festival。

I ran the code with 10 boxes for testing, it completed quickly. However when I run with few hundred boxes and K=9, it is going on for ever. Is this a right approach?

When I use my data,This error occurred.Is that because of the problem with my data?

I am sorry to bother you. When I run: "python3 -m unittest discover -s tests/", the VOC dataset works fine, but when I use my own dataset (also in VOC format) I get the following error: AssertionError: Arrays are not almost equal to 2 decimals. Can you tell me why there is such an error? Thank you so much!

I was able to find the clusters, using that now how to find the bbox?

你好，代码写的很棒。
只是我想知道如何通过它来进行anchor的优化，例如我现在已经知道了设置5个anchor-boxs的效果要比faster-rcnn通过ratio设置的9个anchor-box好。
如果我想使用k-means来优化我的anchor-box设置，是通过ratio来增加anchor-box的个数吗，那么我如何得到ratio呢？我想知道你的聚类结果中有没有显示各聚类中的大致尺寸比例？
再次非常感谢。祝你一切顺利。

Hi,
In yout iou() function, you cannot calculate the interaction AREA by the following equation:
(Line16) intersection = x * y

In my opinion, we cannot calculate the overlap if we only know the width and heights of the two boxes.
Please help to double check and iou() function.

Hi,I want to use the code to reproduce yolov3 anchor:[[10, 13], [16, 30], [33, 23], [30, 61],[62, 45], [59, 119], [116, 90], [156, 198],[373, 326]].
I use kmeans.py code to generate 9 anchor on those dataset:

# resize=416,voc2007
# [[22, 33], [58, 45], [34, 79], [69, 110], [137, 94], [103, 186], [268, 146], [180, 242], [329, 268]]

# resize=416,voc2007+2012
# [[20, 27], [29, 64], [57, 42], [55, 108], [113, 82], [98, 171], [245, 134], [167, 237], [319, 269]]

# resize=416,coco2017
# [[5, 7], [10, 20], [19, 12], [19, 40], [38, 25], [39, 80], [75, 50],
#  [103, 129], [246, 228]]

The results is different from yolov3 anchor.
Could you tell me how to reproduce this anchor:[[10, 13], [16, 30], [33, 23], [30, 61],[62, 45], [59, 119], [116, 90], [156, 198],[373, 326]]?

Hi,

Why do you use np.median: kmeans(boxes, k, dist=np.median); when calculating the new cluster centres instead of using np.mean?

Thank you.

can this method applied in other data set, such as KITTI(has converted into VOC format)? and how can I get those cluster figure and final cluster output(five anchor ratio)?
thanks!

hi，@lars76
你好，好久不见，我最近想在coco数据集上进行训练和测试。我把coco的annotations格式制作成xml的格式，xml的格式如下：

-<annotation>
   <folder>1</folder>
   <filename>COCO_train2014_000000000036.jpg</filename>
  -<source>
     <database>CKdemo</database>
     <annotation>VOC</annotation>
     <image>CK</image>
   </source>
  -<size>
     <width>481</width>
     <height>640</height>
     <depth>3</depth>
   </size>
   <segmented>0</segmented>
  -<object>
     <name>person</name>
     <pose/>
     <truncated>0</truncated>
     <difficult>0</difficult>
    -<bndbox>
       <xmin>167</xmin>
       <ymin>162</ymin>
       <xmax>478</xmax>
       <ymax>628</ymax>
     </bndbox>
   </object>
 </annotation>

然后我使用你提供的k-means代码进行聚类，但是我得到了如下的错误：

dxt@dxt-System-Product-Name:~/kmeans-anchor-boxes-master$ python example.py
Traceback (most recent call last):
File "example.py", line 32, in
out = kmeans(data, k=CLUSTERS)
File "/home/dxt/kmeans-anchor-boxes-master/kmeans.py", line 68, in kmeans
distances[row] = 1 - iou(boxes[row], clusters)
File "/home/dxt/kmeans-anchor-boxes-master/kmeans.py", line 14, in iou
raise ValueError("Box has no area")
ValueError: Box has no area

然而我不清楚我的xml格式是哪地方出现的错误，我对比了一下我和voc的xml文件，不过还是没能找到问题，你能帮帮我吗？万分感谢，祝你生活愉快。

对了还有下面这一句话是什么意思，你能告诉我吗，谢谢啦。

    x = np.minimum(clusters[:, 0], box[0])
    y = np.minimum(clusters[:, 1], box[1])
    if np.count_nonzero(x == 0) > 0 or np.count_nonzero(y == 0) > 0:
        raise ValueError("Box has no area")