richermans / psl Goto Github PK

Hello! I am trying to obtain teacher model from this project recently. After some modification, I finally made it trained and obtained 4 checkpoint models from psl_balanced_chunk_10sec and teacher_student_chunk_5sec respectively, but there are some questions I am not sure:
1.
After training, how to perform "evaluate" part for my trained model exactly?
I tried to modify evaluate function like:
def evaluate( self, experiment_path: Path = '../src/experiments/psl_balanced_chunk_10sec/MobileNetV2_DM_MobileNetV2_DM/2022-05-27_16-01-05_2cd778a6dd9311ec98a1d542cc97d3a1', test_data_file: Path = '../data/labels/eval.csv', label_indices: Path = '../data/csvs/class_labels_indices.csv', num_classes=527, **kwargs, ):
but raised an error:
`
ssh: .../envs/PSL4/bin/python3.7 -u .../myProjects/PSL/src/run.py evaluate ../configs/psl_balanced_chunk_10sec.yaml
[2022-05-28 17:41:06] Using seed 42
Traceback (most recent call last):
File ".../myProjects/PSL/src/run.py", line 468, in
Fire(Runner)
File ".../envs/PSL4/lib/python3.7/site-packages/fire/core.py", line 127, in Fire
component_trace = _Fire(component, args, context, name)
File ".../envs/PSL4/lib/python3.7/site-packages/fire/core.py", line 366, in _Fire
component, remaining_args)
File ".../envs/PSL4/lib/python3.7/site-packages/fire/core.py", line 542, in _CallCallable
result = fn(*varargs, **kwargs)
File ".../myProjects/PSL/src/run.py", line 388, in evaluate
training_dump = torch.load(experiment_path, map_location='cpu')
File ".../envs/PSL4/lib/python3.7/site-packages/torch/serialization.py", line 608, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File ".../envs/PSL4/lib/python3.7/site-packages/torch/serialization.py", line 777, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: could not find MARK

Process finished with exit code 1
`
What should I do to make it work?
2.
What are the models I have obtained from training? Are they just student models?

First, thanks for your work.

I am trying to replicate the results. Having downloaded the Audioset(resampled to 16k) before, I prepare the hdf5 files as instructed in the code. However, directly running ./train_psl.sh configs/psl_balanced_chunk_10sec.yaml, the mAP on the eval subset is only 0.01794.
I also tried to run evaluation using pretained teacher model, the mAP on the eval subset is 0.02107.

About the python environment, I create a new conda environment using the same dependencies except numpy==1.22 and torch_audiomentations==0.11.0 to avoid conflicts (I am using Ubuntu 16.04)

I also found a possible bug in src/run.py, the outputdir is already included in saved_params_file

How can i get the pretrained_teacher model by myself instead of using the ones u provided?