yinkalario / ein-seld Goto Github PK

I read your paper(EVENT-INDEPENDENT NETWORK FOR POLYPHONIC SOUND EVENT LOCALIZATION AND DETECTION),and read your code. I find your implementation is different from your paper。
first, you use Transformer Encoder replace GRU.
second, it seems that you never use EAD Prediction.

When I run following command:
sh scripts/preproc.sh

I faced #6 issue.
and after using the solution from #6 which is as follows:
I replaced the code in line 148 of preprocess.py with the following,
batch_x = batch_sample.batch_out_dict['waveform']

then I faced this issue

Extracting scalar......

 74%|███████████████████████████████▏          | 84/113 [01:16<00:26,  1.10it/s]
Traceback (most recent call last):
  File "seld/main.py", line 59, in <module>
    sys.exit(main(args, cfg))
  File "seld/main.py", line 36, in main
    preprocessor.extract_scalar()
  File "/home/csinuc10i5/esp32/EIN-SELD/seld/learning/preprocess.py", line 145, in extract_scalar
    for it, batch_sample in iterator:
  File "/home/csinuc10i5/anaconda3/envs/ein/lib/python3.7/site-packages/tqdm/std.py", line 1180, in __iter__
    for obj in iterable:
  File "/home/csinuc10i5/anaconda3/envs/ein/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/csinuc10i5/anaconda3/envs/ein/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1179, in _next_data
    return self._process_data(data)
  File "/home/csinuc10i5/anaconda3/envs/ein/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
    data.reraise()
  File "/home/csinuc10i5/anaconda3/envs/ein/lib/python3.7/site-packages/torch/_utils.py", line 429, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 4.
Original Traceback (most recent call last):
  File "/home/csinuc10i5/anaconda3/envs/ein/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/csinuc10i5/anaconda3/envs/ein/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/home/csinuc10i5/esp32/EIN-SELD/seld/methods/data.py", line 134, in collate_fn
    return PinMemCustomBatch(batch_dict)
  File "/home/csinuc10i5/esp32/EIN-SELD/seld/methods/data.py", line 121, in __init__
    'waveform': torch.tensor(batch_x, dtype=torch.float32),
ValueError: expected sequence of length 240000 at dim 2 (got 150010)

My environment is as follows:
I am not using cuda.
I am using NUC11PAHi5()
Python version 3.7.11

I also managed to download the code and datasets from GitHub without issue and modified the code only to bypass issue#6 which I detailed above.

Hi yinkalario:
I counted the parameters of the model ，does it has about 9 millions?

In the paper, you reported that you considered AdamW, while in your config (seld.yml) the optimizer is Adam.

• Could you please tell me which optimizer you have considered for producing the reported results?
• In models/seld.py self.pe=False, did you use or did not use a PositionalEncoder for producing the reported results?

Many thanks in advance!
Sara

My audio files have 8 channels, but apparently, your code is written for data with four-channel.
Could you please explain how to change the code to apply my data?

Thanks

hello，I've read the paper on EINV2, but I still don't understand the orbital alignment problem. Does SELDnet's output format also have this problem? Can you give me an example to help me understand, thank you very much for your reply.

I don't have any GPU in my system and I'm facing with this error.
Would you please help me?

sh scripts/predict.sh

Traceback (most recent call last):
  File "seld/main.py", line 59, in <module>
    sys.exit(main(args, cfg))
  File "seld/main.py", line 48, in main
    infer.infer(cfg, dataset, **infer_initializer)
  File "/workspace/sed/EIN-SELD/seld/learning/infer.py", line 20, in infer
    model.module.load_state_dict(state_dict['model'])
  File "/opt/conda/envs/ein/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1052, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for EINV2:
        Missing key(s) in state_dict: "pe.pe".

When i run follwing line:
sh scripts/preproc.sh
I face with this error:
Would you please help me?

Dataset dcase2020task3 is being developed......

Extracting scalar......

Traceback (most recent call last):
  File "seld/main.py", line 59, in <module>
    sys.exit(main(args, cfg))
  File "seld/main.py", line 36, in main
    preprocessor.extract_scalar()
  File "/workspace/sed/EIN-SELD/seld/learning/preprocess.py", line 148, in extract_scalar
    batch_x = batch_sample['waveform']
TypeError: 'PinMemCustomBatch' object is not subscriptable

Dear author,

I would like to reproduce the results. Thus, I have re-trained the model with the provided source code with the following steps:
(1) python main.py -c XXX train
The output is
"valid: ER20: 0.436, F20: 0.660, LE20: 12.390, LR20: 0.762, seld20: 0.271, ER19: 0.348, F19: 0.763, LE19: 10.058, LR19: 0.768, seld19: 0.218"
then
(2) python main.py -c XXX infer
(3) python main.py -c XXX evaluate
The output is
"test: ER20: 0.373, F20: 0.704, LE20: 13.204, LR20: 0.816, seld20: 0.232, ER19: 0.272, F19: 0.818, LE19: 10.680, LR19: 0.794, seld19: 0.180, "

Both the test and validation results are diferent from the original paper.
Could you please let me know is there any mistakes I made?

Thanks a lot for your help!

What is the difference between SELDMetrics2020 and SELDMetrics2019? Which version of the metric did the paper use?

Thanks for your work!

Could you please let me know which GPU enviornment you are using?
And the batchsize for train/inference?

Thanks!

After downloading the pre-trained models following the instruction provided in the readme.
I tried to run the following shell command:

sh scripts/predict.sh && sh scripts/evaluate.sh
I received the following error:

File "/home/alrawis/EIN-SELD/seld/learning/infer.py", line 23, in infer pred = inferer.infer(test_generator) File "/home/alrawis/EIN-SELD/seld/methods/ein_seld/inference.py", line 50, in infer pred = self.model(batch_x) File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 155, in forward outputs = self.parallel_apply(replicas, inputs, kwargs) File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 165, in parallel_apply return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)]) File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 85, in parallel_apply output.reraise() File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) IndexError: Caught IndexError in replica 1 on device 1. Original Traceback (most recent call last): File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker output = module(*input, **kwargs) File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/alrawis/EIN-SELD/seld/methods/ein_seld/models/seld.py", line 95, in forward x_sed = torch.einsum('c, nctf -> nctf', self.stitch[0][:, 0, 0], x_sed) + \ File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/nn/modules/container.py", line 426, in __getitem__ idx = self._get_abs_string_index(idx) File "/home/alrawis/miniconda3/envs/ein/lib/python3.7/site-packages/torch/nn/modules/container.py", line 409, in _get_abs_string_index raise IndexError('index {} is out of range'.format(idx)) IndexError: index 0 is out of range

My pc has 2 gpus:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 435.21 Driver Version: 435.21 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 TITAN X (Pascal) Off | 00000000:03:00.0 On | N/A |
| 23% 29C P8 10W / 250W | 145MiB / 12189MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 TITAN X (Pascal) Off | 00000000:04:00.0 Off | N/A |
| 23% 31C P8 9W / 250W | 2MiB / 12196MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 2753 G /usr/lib/xorg/Xorg 57MiB |
| 0 2795 G /usr/bin/gnome-shell 84MiB |
+-----------------------------------------------------------------------------+

I appreciate your quick reply.

In your paper, section 3.3, you mentioned and also reported results from cPIT training.
Unfortunately, I could not find any cPIT config also, in losses.py there is only tPIT function.

Could you please point out the cPIT training?

Best,
Sara