Leaderboard submission

#65 · closed · 5 comments

View on GitHub ↗

Chris10M

Hi @zc-alexfan, Thanks for your great work. I uploaded the test split zip following the template zip you provided, but the MR-E evaluation is spinning indefinitely. `MR-E evaluation: a new submission is being processed. The processing can take up to several minutes. You do not have to stay on this page and can close it. ` I used the full images for my method, but then changed the paths to `./data/arctic_data/data/cropped_images/s03/box_use_01/0/00010.jpg` matching the provided zip. Furthermore, I also changed `float32` values to `float16` values for the hand parameters as given in the zip. Can I please know how to solve this issue? Thank you, Christen

Comments

zc-alexfan

Hi Christen, I am reaching out to our engineer to take a closer look.

zc-alexfan

Hi Christen, could you share with me a zip file that you tried to upload but the server does not provide results? Having that file will help us to debug.

Chris10M

Hi @zc-alexfan, Thank you for the fast response! Here is the zip I used [Link](https://drive.google.com/file/d/14AILfqKRiLY8NF3Xk4hLBFkZHDO-juCj/view?usp=sharing) Best, Christen

zc-alexfan

Hi, I have checked with our engineer. There are some files missing in your submission -- all the `pred.object*` files. This is the specific stacktrace: ``` Traceback (most recent call last): File "/work/src/evaluate_metrics.py", line 125, in <module> main() File "/work/src/evaluate_metrics.py", line 119, in main evalute_results( File "/work/src/evaluate_metrics.py", line 43, in evalute_results data = prepare_data( File "/work/src/src/extraction/interface.py", line 28, in prepare_data batch = read_keys(gt_folder, folder_p, keys=data_keys, verbose=False) File "/work/src/src/extraction/interface.py", line 159, in read_keys data = torch.load(data_p) File "/work/venv/lib/python3.10/site-packages/torch/serialization.py", line 771, in load with _open_file_like(f, 'rb') as opened_file: File "/work/venv/lib/python3.10/site-packages/torch/serialization.py", line 270, in _open_file_like return _open_file(name_or_buffer, mode) File "/work/venv/lib/python3.10/site-packages/torch/serialization.py", line 251, in __init__ super(_open_file, self).__init__(open(name, mode)) FileNotFoundError: [Errno 2] No such file or directory: '/tmp/arctic/pose_p2_test/eval/s03_box_grab_01_0/preds/pred.object.rot.pt' ```

Chris10M

@zc-alexfan Thank you very much. After including the pred.object* files, the evaluation is successful. Even if we are only evaluating hand poses, the pred.object* files are neccessary