mpi4py-fft and numpy differ for a noisy field about mpi4py-fft

[mikaem]

Hi Francis,

Yes, numpy and mpi4py-fft should give the same result. But you are using rfftn/irfftn for mpi4py-fft and fftn/ifftn for numpy. Normally that's not a problem, but you have to do it correctly, and for that you have to know what is going on under the hood. When a real array X is transformed (with rfftn) you get a complex array Y with Hermitian symmetry. See, e.g., here. For this transformed array item Y[N/2+1] will be real. Problem is that when you multiply this same array with k1j, then Y[N/2+1] will become complex and not real. And when you do irfftn(1jk*rfftn(x)), then irfftn assumes that it is transforming a complex array with Hermitian symmetry and assumes item N/2+1 is real. Since it is actually now complex it is treated as zero and that is why you get a difference between irfftn and ifftn (not mpi4py-fft and numpy). If you do numpy with rfftn/irfftn as well, then you should get the same results.

[francispoulin]

Hello Mikael,

Ah, that makes sense. Thanks! I can confirm that when I use rfftn in numpy I do get the same result.
One last question, with this issue I hope, does that mean to maintain the symmetry we should multiply the last entry by k1 instead of k1j, to keep it real?

[mikaem]

No. You should set it to 0, but only for N even. For N odd you don't have to do anything. Takes a while to explain, but you could search for Fourier interpolator. Trefethen's 'spectral methods for matlab' has a decent explanation and so does most textbooks on spectral methods.

[francispoulin]

Thanks. I tried playing with that and didn't have success but will keep at it. I know Trefethen's book quite well from my doctoral days but clearly need to brush up on it. Thanks for the reference.

On a related note, when I comment out the PFFT line and uncomment this, which I think uses the fftn and ifftn, I still get the same results. I still have that the transformed arrays are not square. How should I change the syntax to get the c2c transforms?

`r2c = PFFT(MPI.COMM_WORLD, N, axes=(0, 1), dtype=np.float,

grid=(-1,), transforms={(0,1): (rfftn, irfftn)} )`

[mikaem]

Use 'dtype=np.complex'

[francispoulin]

That makes sense, and I did try that but then it seems I have to define the wavenmbers differently. I guess the get_local_wavenumbermesh assumes it's going to be using r2c. Does that make sense?

Traceback (most recent call last): File "example2.py", line 124, in <module> main() File "example2.py", line 61, in main K = get_local_wavenumbermesh(r2c, L) File "example2.py", line 33, in get_local_wavenumbermesh return [np.broadcast_to(k, FFT.shape(True)) for k in Ks] File "example2.py", line 33, in <listcomp> return [np.broadcast_to(k, FFT.shape(True)) for k in Ks] File "<__array_function__ internals>", line 5, in broadcast_to File "/home/fpoulin/software/anaconda3/envs/mpi4py-fft/lib/python3.8/site-packages/numpy/lib/stride_tricks.py", line 182, in broadcast_to return _broadcast_to(array, shape, subok=subok, readonly=True) File "/home/fpoulin/software/anaconda3/envs/mpi4py-fft/lib/python3.8/site-packages/numpy/lib/stride_tricks.py", line 125, in _broadcast_to it = np.nditer( ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (1,5) and requested shape (8,8)

[mikaem]

Yes, that is correct. Use fftfreq for both directions.

[francispoulin]

Thanks! I simply changed the rfft to fft and I can confirm that it does yield the correct answer.
I will try and get the rfft working as that is the better way to go but I am happy to say that I am getting agreement.

I will close this issue.

From,

k.append(np.fft.rfftfreq(N[-1], 1./N[-1]).astype(int))

to,

k.append(np.fft.fftfreq(N[-1], 1./N[-1]).astype(int))