plotly / dash-detr

A User Interface for DETR built with Dash. 100% Python.

Dash DETR Detection App

*A User Interface for DETR built with Dash. 100% Python.

The release of DETR: End-to-End Object Detection with Transformers showed significant improvement in real-time object detection and panoptic segmentation (PS), while greatly simplifying the architecture. As a mean to test the model, we decided to build a simple Dash app that let you experiment and play with the model through a user interface.

Overview

• The code is open-source and ready to be forked.
• Everything is in pure Python - not a single line of HTML, CSS, or JavaScript required.
• The app was written 200 lines of code (in addition to modelling), and only took a few hours.
• From development to production in minutes with Dash Kubernetes - no need to spend hours figuring out deployment.

Usage

1. Clone this repo:

git clone https://github.com/plotly/dash-detr
cd dash-detr

2. Create a fresh venv (with conda or virtualenv) and activate it:

conda create -n dash-detr python=3.7
conda activate dash-detr

3. Install the requirements:

pip install -r requirements.txt

4. Start the app:

python app.py

5. Try the app at localhost:8050!

Modifying or extending the app

To make it easy for you to extend the app by adding a custom model, we placed everything related to PyTorch and modeling in model.py, so you can add your own models without modifying the app. If you want to customize the layout or create new interactions, you can edit app.py (if you are not familiar with Dash, read the tutorials first.)

Productionizing Object Detection?

If you are interested in deploy apps like this one for production, check out our article on productionizing object detection models with Dash Enterprise, or reach out to us.

dash-detr's Issues

Issues on Launching it locally

Hi, Awesome work you have here!
When I try to setup the environment to play around I face some issues over my place. It would be great help if you can take a look.

I have followed the steps given and somehow have the error of "TypeError: cannot convert 'NoneType' object to bytes"

There is also a small issues when I try to install the required module with the requirement.txt file

• ERROR: Could not find a version that satisfies the requirement torch==1.5.0 (from -r requirements.txt (line 28)) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2)
• ERROR: No matching distribution found for torch==1.5.0 (from -r requirements.txt (line 28))

To solve this, I install the pytorch manually using

• pip install torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

Issue with filter_boxes

Hi,

Thanks a lot for the demo app, it looks very nice!

While playing with it, we noticed some unexpected behaviors of the model.

We initially were a bit confused by the presence of NMS in the model output, as DETR doesn't need NMS in general, so we looked a bit more into it.

It turns out that there is a couple of issues the way NMS is applied in filter_boxes.
Indeed, in there you applying NMS irrespective of the class label, and thus are removing important predictions.

In

dash-detr/model.py

Line 39 in 99474ba

keep = nms(boxes, scores, iou)

we could use the class labels to perform NMS on a per-class basis, using batched_nms. This is standard practice in object detectors that use NMS, such as Faster R-CNN.

Another thing is that nms expects a 1d tensor for the scores, but you are feeding a 2d tensor, so the values it is considering for NMS are most probably wrong. We should definitely add a check in torchvision so that this shows an error.

Here is the proposed change:

class_score, labels = scores.max(-1)
keep = torchvision.ops.boxes.batched_nms(boxes, class_score, labels, iou)

Comparison before and after

Results before (in the current state):

Results after the proposed fix:

---

Option to enable-disable NMS

As a separate comment, would it be possible to add a toggle flag to enable-disable NMS? DETR by default doesn't need NMS, so it would be great if this option could be added in the webapp.

Once again thanks for the great work!