Reproduce problems that do not reproduce the accuracy of the paper about cwvae-jax

[juliuskunze]

The default configuration uses original parameters from the paper, listed in config.py. There are no other settings. num_examples=100 and num_samples=1, perhaps you are confusing the two?

[fengpany]

The default configuration uses original parameters from the paper, listed in config.py. There are no other settings. num_examples=100 and num_samples=1, perhaps you are confusing the two?

How are the numerical results in the paper calculated? Is it obtained from the PSNR value of all frames after averaging 36 frames (i.e. the average of the ssim_all array)? I got a result of 0.556 on the Minerl dataset (a big difference from the paper's 0.65), but I came all by setpoint. I would also like to ask, what is your version of Jax and Jaxlib of this code?

[juliuskunze]

eval.py and https://github.com/juliuskunze/cwvae-jax/blob/main/tools.py#L50 should have the details you are looking for.

On the day of release it was up to date, so it must have been (roughly) jax 0.2.20 and jaxlib 0.1.71.

[fengpany]

OK,I also want to ask, I encountered the following prompt when running tensorflow 2.4.1 with cuda 11.0 and cudnn 8.0, does it have an impact on accuracy?

And do you have a weights file for CWVAE's tensorflow code that is thesis-accurate? I still want to reproduce the code for TensorFlow to thesis precision.

[juliuskunze]

To me that looks like it has no impact on accuracy.

[fengpany]

To me that looks like it has no impact on accuracy.

Ok, thank you very much for your patience answer.

[juliuskunze]

You're welcome, I hope it helped!

[fengpany]

I reproduced the jax version of the CWVAE code, but the accuracy I got was close to what I got when I reproduced tensorflow. Specifically, when testing the minerl_nvigate dataset (where num_examples=225, num_samples=1), the PSNR and SSIM are 0.55 and 17.3, respectively (which I averaged all the frames predicted in the minerl_nvigate test set), but this is much different from the results of the paper of 0.65 and 23.23. Where is the problem?How can I get thesis accuracy？

[juliuskunze]

Since the JAX version seems to be consistent with the original code, it's probably best if you directly ask the paper authors, i. e. @vaibhavsaxena11.

(As you have done already in vaibhavsaxena11/cwvae#5.) I hope they can help!

[fengpany]

由于JAX版本似乎与原始代码一致，因此最好直接询问论文作者，即。

（正如您在vaibhavsaxena11 / cwvae#5中所做的那样。我希望他们能提供帮助！

OK，thank you