jrieke / shape-detection

🟣 Object detection of abstract shapes with neural networks

shape-detection

Read my Medium post on this project

Note: Some people reached out and told me that they can't replicate some of the results in this repo. I couldn't find the reason, potentially one of the dependencies changed since I did this project a few years ago. Unfortunately I don't have any notes on which exact setup I used back then. If you find anything, please reach out to me :)

This is an example of object detection with neural networks (implemented with keras). The training images contain abstract geometric shapes and can be easily bootstraped.

The code is split up into several Jupyter notebooks. They increase in complexity:

• Detection of rectangles in numpy arrays: single-rectangle, two-rectangles, multiple-rectangles
• Additional classification between rectangles and triangles: two-rectangles-or-triangles, multiple-rectangles-or-triangles
• Application to more complex images (using convnets): color-multiple-shapes

Requirements: Python 2.7, keras (v1.0.5), theano or tensorflow, numpy, matplotlib, jupyter, pycairo (only for color-multiple-shapes.ipynb)

shape-detection's Issues

dist wrong

In the file ``two-rectangles-or-triangles` you have the following code:

        iou = IOU(pred_bboxes[:4], exp_bboxes[:4]) + IOU(pred_bboxes[4:], exp_bboxes[4:])
        iou_flipped = IOU(pred_bboxes[:4], flipped_exp_bboxes[:4]) + IOU(pred_bboxes[4:], flipped_exp_bboxes[4:])
        
        dist = IOU(pred_bboxes[:4], exp_bboxes[:4]) + IOU(pred_bboxes[4:], exp_bboxes[4:])
        dist_flipped = IOU(pred_bboxes[:4], flipped_exp_bboxes[:4]) + IOU(pred_bboxes[4:], flipped_exp_bboxes[4:])
        
        if mse_flipped < mse:  # using iou or dist here leads to similar results

Here, dist is calculated the same as iou, which is wrong!a

Question about multiple classes in an image

Hello?

Thanks for uploading this and I have a question about having multiple classes in a training image. There could be many classes more than just 2 and the bboxes must contain all class coordinates per a training image.

Ex. train_image_0.jpg: 4 classes
train_image_1.jpg: 1 class
train_image_2.jpg: 10 classes
...

But you defined bboxes to have max. 2 classes per an image.
bboxes = np.zeros((num_imgs, num_objects, 4))

Then how to re-define the bboxes?

Thanks,

Rotated Regtangle

Hi,
In the single-rectangle.ipynb file, is it able to detect rectangles with for example 40 rotation?

Accuracy is much lower in my case

Output_performance_simple_model.txt
Hi,

I tried your notebook using tensorflow.keras instead of Keras. (Tensorflow-gpu v 2.1.0).
I only modified the code for the model in Cell [81], where I removed the parameter dim_ordering='tf' because it is not needed (and available) in tensorflow.keras.
I tried to train the model for 20 epochs multiple times and I get very different results with respect to your notebook, in terms of "Accuracy shapes" and "Accuracy colors". (see Output_performance_simple_model.txt ). Do you know what can be the problem?
Can it be only the different version of Keras that I used?

Best regards,
Giulia

data set

Hi, nice work. came across while looking for such approach.
one question, on what kind of images you have trained your model, that is, you have only small images of that figures or the figures are on entire images? give some hints.