Comments (2)
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
from object-detection-metrics.
In the example section, it mentions:
In some images there are more than one detection overlapping a ground truth (Images 2, 3, 4, 5, 6 and 7). For those cases, the predicted box with the highest IOU is considered TP (e.g. in image 1 "E" is TP while "D" is FP because IOU between E and the groundtruth is greater than the IOU between D and the groundtruth). This rule is applied by the PASCAL VOC 2012 metric: "e.g. 5 detections (TP) of a single object is counted as 1 correct detection and 4 false detections”.
I don't think we should decide which detection is TP by IOU only. In the original PASCAL VOC 2012 you sited, it says:
Detections output by a method were assigned to ground truth objects satisfying the overlap criterion in order ranked by the (decreasing) confidence output. Multiple detections of the same object in an image were considered false detections e.g. 5 detections of a single object counted as 1 correct detection and 4 false detections.
It means that we first decide a IOU threshold, then all bboxes that meets the threshold criteria are candidates. And then we select the one with the highest detection score. This one makes more sense because consider that when we are calculating the Precision/Recall, we are actually thresholding the confidence score. The bbox with score lower than the threshold would actually "disappear" from the image. Imagine a case when two detection matches with 1 groundtruth. One with IOU 90%, confidence score 0.2. One with IOU 80%, confidence score 0.8. If we select the IOU threshold to be 0.5, both should meet the criteria. Then let's say we are computing the recall and precision at 0.5. We would consider both detection as false positive, which is not the case because the latter is definitely a true positive.
i agree with you, and this calculation method was used in yolov5 that makes me confused.
from object-detection-metrics.
Related Issues (20)
- need to cast classId to str when saving PlotPrecisionRecallCurve HOT 2
- Question about the argument -imgsize HOT 1
- Confidence threshold HOT 8
- NMS ?? HOT 3
- Possible bug in BoundingBox.py? HOT 1
- How to calculate mAP@[0.5:0.99]? HOT 2
- red boxes represent detections with prediction label of that class or any detections whenever it prediction is that class ? HOT 2
- incorrect equation HOT 4
- ggggg
- Check performance of the trained model HOT 2
- How can i get TP,TN,FP,FN from it? HOT 1
- [question] support for 3d volumes? HOT 1
- Difference implementations between this repo and the faster_rcnn ones HOT 1
- What are the dimensions of precision represent? HOT 1
- Got `AP=0.00%` when running `pascalvoc.py` with samples
- image3 G iou ? HOT 2
- getting threshold values of confidence score which recall/precision calculated HOT 3
- I think a bug resulting in incorrect calculation for Average Precision HOT 1
- Definition of xywh is <left> <top> <width> <height> or <center-x> <center-y> <width> <height>
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from object-detection-metrics.