Comments (8)
We will rerun the experiments on +1M datasets, stay tuned. Also, have you found whether the numbers on ResNet101 have gaps or not?
from superglobal.
And whether the setting you use here can reproduce the results on ROxf 5k and RPar 6k?
from superglobal.
I can reproduce the results on ROxf and Rpar without R1M, but there is a gap with distractor.
from superglobal.
I can reproduce the results on ROxf and Rpar without R1M, but there is a gap with distractor.
@maolaoshi301 Cool! My Google collaborators are re-verificating the results on 1M because 1M datasets are stored at their side. Should be quick. Stay tuned!
from superglobal.
I reproduce R101 on Rpar/Roxf+R1M, there is also a gap.
Roxford+R1M:
Retrieval results: mAP E: 89.48, M: 77.69, H: 60.44
Rparis+R1M
Retrieval results: mAP E: 92.7, M: 82.34, H: 66.6
from superglobal.
I reproduce R101 on Rpar/Roxf+R1M, there is also a gap. Roxford+R1M: Retrieval results: mAP E: 89.48, M: 77.69, H: 60.44 Rparis+R1M Retrieval results: mAP E: 92.7, M: 82.34, H: 66.6
Without reranking?
Google is re-verificating results and I am also working on my side to reproduce the results on 1M.
from superglobal.
Yeah, this score is without reranking.
from superglobal.
Hi! Thanks for your patience. I just finished reproduce the 1M results on RN101 on my side. Here are the details:
ROxford+1M:
RN101 w/o reranking:
Retrieval results: mAP E: 90.09, M: 78.69, H: 62.27
RN101 w/ reranking:
Retrieval results: mAP E: 94.72, M: 84.2, H: 71.27
RN50 w/o reranking:
Retrieval results: mAP E: 88.72, M: 74.24, H: 52.69
RN50 w/ reranking:
Retrieval results: mAP E: 92.66, M: 79.95, H: 64.12
RParis+1M:
RN101 w/o reranking:
Retrieval results: mAP E: 93.25, M: 83.46, H: 68.53
RN101 w/ reranking:
Retrieval results: mAP E: 94.36, M: 85.5, H: 72.47
RN50 w/o reranking:
Retrieval results: mAP E: 91.11, M: 80.53, H: 64.6
RN50 w/ reranking:
Retrieval results: mAP E: 92.94, M: 83.23, H: 69.23
At this stage, you can see my reproduction is significantly different from yours, and is very close to the paper results (well, even slightly better, probably due to some implementation details between me and google). I plan to release the evaluation code for 1M as well in the next few days for you to check whether you or I made some mistakes. For now, I stongly encourage you to double check your reproduction code. And a probably tip would be, to firstly reproduce the 1M results on CVNet vanilla weights to see if your pipeline is correct, since we have checked this pretrained weight and it can reproduce the exact results on their paper. I will remain this issue open for now, until our full reproduction results and 1M reproduction code is released.
UPDATE 9/14/2023: Since all the reproduction results and evaluation codes were released, I closed this issue.
from superglobal.
Related Issues (17)
- Do you have any idea to publish the training codes? HOT 1
- how to train model on my own data HOT 1
- Custom dataset HOT 1
- Hello, could you please provide SuperGlobal training scripts and tutorials? Thank you HOT 1
- Issue Replicating SuperGlobal Results: "Override list has odd length" Error HOT 2
- Dataset issue HOT 1
- Code HOT 1
- Training CVNET Backbone With GeM+, Regional-GeM, Scale-GeM
- ไปฃ็ ็้ฎ HOT 1
- code for CVnet training HOT 1
- Re-ranking network HOT 3
- Reranking network HOT 1
- about DELG-pytouch HOT 2
- Understand the reordering network HOT 2
- GLDv2 test performance reproduction HOT 1
- how to set GeM when training?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from superglobal.