maxcuda / cans Goto Github PK

Hi,

I am trying to run the code with the default setup.h90 and bc.h90 files on a Tesla K80, with CUDA & MPI; one MPI task.

The case is compiled with the following flags (pgi 18.10):
OPT := -O0 -Mcuda
OTH := -DEPHC -DEPA2A -DTIMING -DUSE_CUDA -DUSE_NVTX -DGPU_MPI
(If I instead use -Mcuda=cc70, the bug reported here is triggered)

I find that if I don't add a synchronization step in between setting up the pressure BCs, i.e.:

call set_bc(cbc(0,3),0,n(3),3,.true.,bc(0,3),dzc(0)   ,p)
!@cuf ierr = cudaDeviceSynchronize()
call set_bc(cbc(1,3),1,n(3),3,.true.,bc(1,3),dzc(n(3)),p)

the code will crash at the first call of subroutine boundp:

[t03n02:30004] *** Process received signal ***
[t03n02:30004] Signal: Bus error (7)
[t03n02:30004] Signal code: Non-existant physical address (2)
[t03n02:30004] Failing at address: 0x8105d9c200

When I add this synchronization step the code seems to run fine.

I don't understand why this synchronization is required, since something very similar is done in the subroutine bounduvw and there seems to be no problem there.

Thanks in advance!

Best,
Pedro