ecp-warpx / impactx Goto Github PK
View Code? Open in Web Editor NEWImpactX: an s-based beam dynamics code including space charge effects
Home Page: https://impactx.readthedocs.io
License: Other
ImpactX: an s-based beam dynamics code including space charge effects
Home Page: https://impactx.readthedocs.io
License: Other
We could include a model for Alpha Magnet elements.
This might need an magnetostatic model on top of the electrostatic model (to be confirmed).
For t-based, we have both in WarpX.
References:
Turn Python on by default and update all according workflows.
Activate the option to use embedded boundaries for our space charge solver.
We already have the interfaces exposed in ABLASTR for the Poisson solver.
Add openPMD output support.
It would be super useful if we could do a simple z-x visualization of an accelerator lattice, e.g., from the Python side.
We could write a function that takes our list of Python elements and saves them to SVG/PDF/PNG/TiX, etc. I am not yet sure if matplotlib or another tool would be ideal for this job.
At the moment (23.01), it looks like the ctest
runs of pytest
for ImpactX segfault on Perlmutter when doing multiple finalize-init test cycles.
This is likely due to a leaking MultiFab object (or another object that uses the AMReX arena and is not fully cleaned when finalize of a test is called).
Let's add helper functions to the reference particle to:
We need to improve the particle initialization: to allow having a first few boxes initially, initialize in parallel & redistribute.
Currently, we will be limited to initializing as many particles as a single GPU can handle (even if we run on multiple GPUs).
Add a Python version of each example (and show py version first in each example's README).
While running in debug (#43), I realized the ResizeMesh
function does assert the simulation:
https://github.com/ECP-WarpX/impactx/pull/19/files#r778820245
0::Assertion `hi > lo' failed, file "/home/axel/src/impactx/build/_deps/fetchedamrex-src/Src/Base/AMReX_Algorithm.H", line 107, Msg: "Error - calling bisect but lo and hi don't describe a reasonable interval." !!!
I temporarily disabled calling it and enabled debug builds in CI (#43). The function probably needs to be fixed.
Use https://github.com/AMReX-Codes/pyamrex to establish Python bindings.
Add an option to load beams from an openPMD series.
In ECP-WarpX/WarpX#3063, @dpgrote adds a simple plasma lens element.
plasmalens
for a field modeling a plasma lens
This applies a radially directed plasma lens field that is uniform within the z
extent of the element with
a sharp cut off at the ends.
This uses residence corrections, with the field scaled by the amount of time within the element for particles entering
or leaving it, to increase the accuracy.
Let us add the same element as a map to ImpactX.
Is this similar to our ConstF
element?
Implement an outer loop for turns in a ring.
Use symplectic integrators for the treatment of soft-edge quads, solenoids, (possibly) dipoles, and any other elements that may require this treatment for SciDAC applications.
We use pti
for the ParIter
of AMReX.
@cemitch99 mentioned that this can be confusing for new readers, because it might also be the initial value of pt
in a readers mind.
We could change this throughout the code base and call these variables pit
, p_it
, par_it
or par_iter
or so.
@dpgrote did a clever recursive way to define segments in WarpX. Do we want to add the same in ImpactX?
lattice.elements = lattice1 lattice2
lattice1.type = lattice
lattice1.elements = drift1 quad1
lattice2.type = lattice
lattice2.elements = drift2 quad2
drift1.type = drift
drift1.ds = 0.2
quad1.type = quad
quad1.ds = 0.2
quad1.dEdx = 1.e4
...
Implement Python callbacks similar to the functionality in WarpX.
Check if we can further modernize the callback logic. pybind11 has this beautiful thing:
https://pybind11.readthedocs.io/en/stable/advanced/cast/functional.html
🗒️ do not forget to add profiling to each hook. Rationale: people will add slow stuff there and need to know how long the hooks are running.
When running ctest
on the release build of ImpactX (30f93ec, AMReX 23.04) on a MacBook Pro 35 tests out of 94 are failing after the run and TinyProfiler output, after AMReX announces that it finalizes.
E.g. here the error from FODO.run
Pinned Memory Usage:
------------------------------------------------------------------
Name Nalloc Nfree AvgMem MaxMem
------------------------------------------------------------------
The_Pinned_Arena::Initialize() 1 1 95 KiB 8192 KiB
ImpactX::AddNParticles 27 27 2524 B 853 KiB
ParticleContainer::addParticles 846 846 400 KiB 853 KiB
------------------------------------------------------------------
AMReX (23.04) finalized
[MacBookPro:16898] *** Process received signal ***
[MacBookPro:16898] Signal: Segmentation fault: 11 (11)
[MacBookPro:16898] Signal code: Address not mapped (1)
[MacBookPro:16898] Failing at address: 0x10
[MacBookPro:16898] [ 0] 0 libsystem_platform.dylib 0x00007ff80e4055ed _sigtramp + 29
[MacBookPro:16898] [ 1] 0 ??? 0x0000000000000021 0x0 + 33
[MacBookPro:16898] [ 2] 0 libomp.dylib 0x000000010ff74342 __kmp_release_64 + 34
[MacBookPro:16898] [ 3] 0 libomp.dylib 0x000000010ff4d027 _ZL17__kmp_reap_threadP8kmp_infoi + 471
[MacBookPro:16898] [ 4] 0 libomp.dylib 0x000000010ff4a399 _ZL18__kmp_internal_endv + 185
[MacBookPro:16898] [ 5] 0 libomp.dylib 0x000000010ff4a226 __kmp_internal_end_library + 518
[MacBookPro:16898] [ 6] 0 libsystem_c.dylib 0x00007ff80e2a9ba1 __cxa_finalize_ranges + 409
[MacBookPro:16898] [ 7] 0 libsystem_c.dylib 0x00007ff80e2a99bb exit + 35
[MacBookPro:16898] [ 8] 0 libdyld.dylib 0x00007ff80e3ef8d3 _ZNK5dyld416LibSystemHelpers4exitEi + 11
[MacBookPro:16898] [ 9] 0 dyld 0x00007ff80e07e458 start + 1960
[MacBookPro:16898] *** End of error message ***
<end of output>
MacOS Version 13.3.1 (22E261) (just updated today but also failed with previous version)
I am using Spack for the dependencies. This is the content of spack.yaml
:
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
# add package specs to the `specs` list
specs: [cmake, fftw, mpi, llvm-openmp, adios2, hdf5, pkgconfig, python, py-pip,
py-pybind11, py-mpi4py]
view: true
concretizer:
unify: true
compilers:
- compiler:
spec: [email protected]
paths:
cc: /usr/bin/clang
cxx: /usr/bin/clang++
f77: null
fc: null
flags: {}
operating_system: monterey
target: x86_64
modules: []
environment: {}
extra_rpaths: []
Needed to join local coordinate systems for non-ideal elements.
Add checkpoint-restart functionality to ImpactX.
Needs to restore:
states, incl. written files from in situ diagnostics, etc.
Let's write a user guide section in our manual on how to transition IMPACT-Z inputs to ImpactX.
Help needed by @qianglbl and @cemitch99 pls :)
Implement the first 3D space-charge solver.
Adding in-situ calculated beam parameters to ImpactX would reduce I/O and make typically used quantities like beta-function, emittance and distribution momenta readily available without additional post-processing.
When compiling with the OpenMP backend, we do not yet use particle tiling it seems.
With our current distribution strategy, we start with one block per device.
Add a script running WarpX as a source and ImpactX for beam transport.
We received an email request for a combined function magnets (a dipole or “SBEND” with a quadrupole strenght associated with) and will also require such an element to model ALS-U.
MAD-X has this as a a quadrupole field that one can set to a dipole.
After a first discussion, we could implement this as:
Add a first data-driven element (map).
Design for collective effects: take into account to also pass global beam parameters, such as beam charge/current and beam moments.
@cemitch99 found in #114 that if we set the constant focusing element in kt
to zero that we get a division by zero somewhere.
Draft a 0.1 "beta" release.
Based on #101, ensure that MR capabilities work.
Cross-Link of AMReX-Codes/pyamrex#35
If you compile with the Conda-Forge compilers on Apple M1 processors, you need to add
cmake -S . -B build -DImpactX_PYTHON=ON -DCMAKE_STRIP=$(which strip)
to avoid errors of the kind:
zsh: killed python -c "import impactx"
This is because the llvm-strip
command is not compatible to the XCode strip
command yet, which we use to reduce the pyAMReX
binary size significantly.
LLVM 14 should fix this (see above linked issue for updates).
Hi, I tried to run the FODO example without changes on the Perlmutter GPU partition and encountered the following error:
amrex::Abort::0::CUDA error 700 in file /global/homes/m/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base/AMReX_GpuDevice.cpp line 660: an illegal memory access was encountered !!!
SIGABRT
See Backtrace.0 file for details
MPICH ERROR [Rank 0] [job id 2519166.0] [Wed Jun 29 18:07:42 2022] [nid001512] - Abort(6) (rank 0 in comm 480): application called MPI_Abort(comm=0x84000001, 6) - process 0
srun: error: nid001512: task 0: Exited with exit code 6
srun: launch/slurm: _step_signal: Terminating StepId=2519166.
I first tried the example with the submit script that is provided by the docs (however, had to change the naming a bit since it's still copied from WarpX).
This configuration used 4 nodes but I also tried a single node and then just a single GPU per node. All fail with the same error.
The backtrace reads the following:
=== If no file names and line numbers are shown below, one can run
addr2line -Cpfie my_exefile my_line_address
to convert `my_line_address` (e.g., 0x4a6b) into file name and line number.
Or one can use amrex/Tools/Backtrace/parse_bt.py.
=== Please note that the line number reported by addr2line may not be accurate.
One can use
readelf -wl my_exefile | grep my_line_address'
to find out the offset for that line.
0: /pscratch/sd/m/mgarten/impactx/001_FODO_single-GPU/./impactx() [0x5d63b6]
amrex::BLBackTrace::print_backtrace_info(_IO_FILE*) at ??:?
1: /pscratch/sd/m/mgarten/impactx/001_FODO_single-GPU/./impactx() [0x5d875c]
amrex::BLBackTrace::handler(int) at ??:?
2: /pscratch/sd/m/mgarten/impactx/001_FODO_single-GPU/./impactx() [0x5c13e9]
amrex::Gpu::Device::streamSynchronizeAll() at ??:?
3: /pscratch/sd/m/mgarten/impactx/001_FODO_single-GPU/./impactx() [0x5b6165]
amrex::MFIter::~MFIter() at ??:?
4: /pscratch/sd/m/mgarten/impactx/001_FODO_single-GPU/./impactx() [0x473e79]
impactx::Push(impactx::ImpactXParticleContainer&, std::__cxx11::list<std::variant<impactx::None, impactx::ConstF, impactx::DipEdge, impactx::Drift, impactx::Multipole, impactx::NonlinearLens, impactx::Quad, impactx::Sbend, impactx::ShortRF>, std::allocator<std::variant<impactx::None, impactx::ConstF, impactx::DipEdge, impactx::Drift, impactx::Multipole, impactx::NonlinearLens, impactx::Quad, impactx::Sbend, impactx::ShortRF> > > const&) at ??:?
5: /pscratch/sd/m/mgarten/impactx/001_FODO_single-GPU/./impactx() [0x432b06]
impactx::ImpactX::evolve(int) at ??:?
6: /pscratch/sd/m/mgarten/impactx/001_FODO_single-GPU/./impactx() [0x41ecfe]
main at ??:?
7: /lib64/libc.so.6(__libc_start_main+0xef) [0x7f2ab60502bd]
8: /pscratch/sd/m/mgarten/impactx/001_FODO_single-GPU/./impactx() [0x42da0a]
_start
../sysdeps/x86_64/start.S:122
===== TinyProfilers ======
main()
ImpactX::evolve
ImpactX::evolve::step
"Backtrace.0" 43L, 1979C 1,66 Top
I am adding the machine/system
label since an earlier Slack message from @ax3l said that it runs on Summit V100s without fail.
We want to support reading accelerator lattices directly from MAD-X input files.
For this, we will need to support at least the following syntax elements:
from MAD-X definitions.
For that, we could explore if @janfschmidt's palattice
could be used as a library :)
https://github.com/janfschmidt/palattice
References on MAD-X:
Example files for MAD-X lattices:
Make use of the Line
element introduced in #335 and show its nesting capabilities in an example.
Hey folks, I was trying to get a test example running (specifically the Constant Focusing Channel example). I am able to successfully install the package through conda. However, executing the script gives the following error.
Traceback (most recent call last):
File "/home/chris/physics-ml-priors-linux/20230402-impactx-test.py", line 41, in <module>
monitor = elements.BeamMonitor("monitor", "h5")
AttributeError: module 'impactx.impactx_pybind.elements' has no attribute 'BeamMonitor'
Placing diagnostics properly in evolve-loops is always a hazzle that complicates the core logic of PIC codes (before/after the loop, what is step zero, diagnostics at a fixed position s that might not coincide with the end of an element, etc.).
With our lattice elements as main outer loop of the code, we could do the following approach: we could treat at diagnostics as zero-width element and just put it in the lattice elements list. That way, order is 100% clear.
This also has a nice analogy to actual, physical beam monitors.
As syntactic sugar for users, we could still provide input options that let them set the diagnostics at a given reference s (or time/phase) and then we find the corresponding element that might be cut and slice it up automatically into two at the location. If the position fits the end of an element, then no slicing is needed.
We want to encourage Pythonic ImpactX by showing the Python version of examples first in each example readme file
The particle distributions implemented in ImpactX are s-based, but can be re-interpreted also in t-based, mostly by passing different values for s/z. Thus, we can move their implementations (not their initializations) to ABLASTR to share them with WarpX.
During the OLCF crusher hackathon I had tried to build/run ImpactX for profiling.
I tried a CPU-only version first to then go ahead and use AMD-provided tools on Crusher for profiling of the host side.
However, running failed and I am trying again today to document the status of where it is failing.
To reproduce, first load modules for the dependencies (starting off here from the one for Crusher from the WarpX repository since ImpactX and WarpX share the same dependencies).
https://warpx.readthedocs.io/en/latest/install/hpc/crusher.html#id1
For convenience, download and save under $HOME/crusher_impactx.profile
.
Commands used to build the most recent development branch
source $HOME/crusher_impactx.profile
mkdir -p $HOME/src
# download defaults to development branch
git clone https://github.com/ECP-WarpX/impactx.git $HOME/src/impactx
cd $HOME/src/impactx
cmake -S . -B build -DImpactX_COMPUTE=NOACC -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build -j 16
Running the FODO example works with this.
However, using Omnitrace on this example did not (back during last year)
On the other hand, building for HIP did not work and the install failed
rm -rf build
cmake -S . -B build -DImpactX_COMPUTE=HIP -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build -j 16
See the build and error output by expanding the following details section
[[email protected] impactx]$ cmake -S . -B build -DImpactX_COMPUTE=NOACC -DCMAKE_BUILD_TYPE=RelWithDebInfo
-- Cray Programming Environment 2.7.17 C
-- Cray Programming Environment 2.7.17 CXX
-- Found CCache: /sw/crusher/spack-envs/base/opt/linux-sles15-x86_64/gcc-7.5.0/ccache-4.5.1-gq5xkxxgeqkxopbjy6gacidd36xglhqc/bin/ccache
-- Downloading ABLASTR ...
-- ABLASTR repository: https://github.com/ECP-WarpX/WarpX.git (23.01)
-- Found CCache: /sw/crusher/spack-envs/base/opt/linux-sles15-x86_64/gcc-7.5.0/ccache-4.5.1-gq5xkxxgeqkxopbjy6gacidd36xglhqc/bin/ccache
-- Downloading AMReX ...
-- AMReX repository: https://github.com/AMReX-Codes/amrex.git (23.01)
-- CMake version: 3.23.2
-- AMReX installation directory: /usr/local
-- Build type set by user to 'RelWithDebInfo'.
-- Building AMReX with AMReX_SPACEDIM = 3
-- Configuring AMReX with the following options enabled:
-- AMReX_BUILD_SHARED_LIBS
-- AMReX_PRECISION = DOUBLE
-- AMReX_MPI
-- AMReX_MPI_THREAD_MULTIPLE
-- AMReX_LINEAR_SOLVERS
-- AMReX_PARTICLES
-- AMReX_PARTICLES_PRECISION = DOUBLE
-- AMReX_PIC
-- AMReX_TINY_PROFILE
-- AMReX configuration summary:
-- Build type = RelWithDebInfo
-- Install directory = /usr/local
-- C++ compiler = /opt/cray/pe/craype/2.7.17/bin/CC
-- C++ defines =
-- C++ flags = -O2 -g -DNDEBUG -I/opt/rocm-5.2.0/include
-- C++ include paths = -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base/Parser -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Boundary -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/AmrCore -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/OpenBC -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Particle
-- Link line =
-- AMReX: Using version '23.01' (23.01)
WarpX build configuration:
Version: 23.01 (23.01)
C++ Compiler: Clang 14.0.6 CrayPrgEnv
/opt/cray/pe/craype/2.7.17/bin/CC
Installation prefix: /usr/local
bin: bin
lib: lib64
include: include
cmake: lib64/cmake/WarpX
Build type: RelWithDebInfo
Build options:
APP: OFF
ASCENT: OFF
COMPUTE: NOACC
DIMS: 3
Embedded Boundary: OFF
GPU clock timers: OFF
IPO/LTO: OFF
LIB: OFF
MPI: ON
PSATD: OFF
PRECISION: DOUBLE
PARTICLE PRECISION: DOUBLE
OPENPMD: OFF
QED: OFF
QED table generation: OFF
SENSEI: OFF
-- ABLASTR: Using version '' (23.01)
-- Downloading pybind11 ...
-- pybind11 repository: https://github.com/pybind/pybind11.git (v2.10.1)
-- pybind11 v2.10.1
-- Downloading pyAMReX ...
-- pyAMReX repository: https://github.com/AMReX-Codes/pyamrex.git (f3ba017531e2d1569e63428919632cf908435e56)
-- AMReX::amrex target already imported
-- pybind11::module target already imported
pyAMReX build configuration:
Version: 23.01
C++ Compiler: Clang 14.0.6 CrayPrgEnv
/opt/cray/pe/craype/2.7.17/bin/CC
Installation prefix: /usr/local
bin: bin
lib: lib64
include: include
cmake: lib64/cmake/pyAMReX
python: lib64/python3.9/site-packages
Build type: RelWithDebInfo
ImpactX build configuration:
Version: 23.02 (23.02-1-g1a134a03ef79)
C++ Compiler: Clang 14.0.6 CrayPrgEnv
/opt/cray/pe/craype/2.7.17/bin/CC
Installation prefix: /usr/local
bin: bin
lib: lib64
include: include
cmake: lib64/cmake/ImpactX
python: lib64/python3.9/site-packages
Build type: RelWithDebInfo
Build options:
APP: ON
COMPUTE: NOACC
IPO/LTO: OFF
LIB: ON (static)
MPI: ON
PRECISION: DOUBLE
PYTHON: ON
OPENPMD: OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /ccs/home/mgarten/src/impactx/build
[[email protected] impactx]$ rm -rf build
[[email protected] impactx]$ cmake -S . -B build -DImpactX_COMPUTE=HIP -DCMAKE_BUILD_TYPE=RelWithDebInfo
-- The C compiler identification is Clang 14.0.6
-- The CXX compiler identification is Clang 14.0.6
-- Cray Programming Environment 2.7.17 C
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/cray/pe/craype/2.7.17/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Cray Programming Environment 2.7.17 CXX
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/cray/pe/craype/2.7.17/bin/CC - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found CCache: /sw/crusher/spack-envs/base/opt/linux-sles15-x86_64/gcc-7.5.0/ccache-4.5.1-gq5xkxxgeqkxopbjy6gacidd36xglhqc/bin/ccache
-- Downloading ABLASTR ...
-- ABLASTR repository: https://github.com/ECP-WarpX/WarpX.git (23.02)
-- Found CCache: /sw/crusher/spack-envs/base/opt/linux-sles15-x86_64/gcc-7.5.0/ccache-4.5.1-gq5xkxxgeqkxopbjy6gacidd36xglhqc/bin/ccache
-- Downloading AMReX ...
-- AMReX repository: https://github.com/AMReX-Codes/amrex.git (23.02)
-- CMake version: 3.23.2
-- AMReX installation directory: /usr/local
-- Build type set by user to 'RelWithDebInfo'.
-- Building AMReX with AMReX_SPACEDIM = 3
-- Configuring AMReX with the following options enabled:
-- AMReX_PRECISION = DOUBLE
-- AMReX_GPU_BACKEND = HIP
-- AMReX_MPI
-- AMReX_MPI_THREAD_MULTIPLE
-- AMReX_LINEAR_SOLVERS
-- AMReX_PARTICLES
-- AMReX_PARTICLES_PRECISION = DOUBLE
-- AMReX_TINY_PROFILE
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found MPI_C: /opt/cray/pe/craype/2.7.17/bin/cc (found version "3.1")
-- Found MPI_CXX: /opt/cray/pe/craype/2.7.17/bin/CC (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: C CXX
-- hip::amdhip64 is SHARED_LIBRARY
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Failed
-- Found HIP:
-- HIP: Runtime=rocclr Compiler=clang Path=/opt/rocm-5.2.0/hip
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- hip::amdhip64 is SHARED_LIBRARY
-- Performing Test LINKER_HAS_WHOLE_ARCHIVE_OFFLOAD
-- Performing Test LINKER_HAS_WHOLE_ARCHIVE_OFFLOAD - Failed
-- AMReX configuration summary:
-- Build type = RelWithDebInfo
-- Install directory = /usr/local
-- C++ compiler = /opt/cray/pe/craype/2.7.17/bin/CC
-- C++ defines = -D__HIP_PLATFORM_HCC__=1 -D__HIP_PLATFORM_AMD__=1
-- C++ flags = -O2 -g -DNDEBUG -I/opt/rocm-5.2.0/include -mllvm -amdgpu-early-inline-all=true -amdgpu-function-calls=false -m64 -munsafe-fp-atomics -x hip --offload-arch=gfx90a
-- C++ include paths = -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base/Parser -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Boundary -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/AmrCore -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/OpenBC -I/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Particle -I/opt/rocm-5.2.0/hip/include -I/opt/rocm-5.2.0/llvm/lib/clang/14.0.0/include/.. -I/opt/rocm-5.2.0/include -I/opt/rocm-5.2.0/include/hiprand -I/opt/rocm-5.2.0/include/rocrand -I/opt/rocm-5.2.0/include/
-- Link line = /opt/rocm-5.2.0/llvm/lib/clang/14.0.0/lib/linux/libclang_rt.builtins-x86_64.a
-- AMReX: Using version '23.02' (23.02)
-- Found Git: /usr/bin/git (found version "2.35.3")
WarpX build configuration:
Version: 23.02 (23.02)
C++ Compiler: Clang 14.0.6 CrayPrgEnv
/opt/cray/pe/craype/2.7.17/bin/CC
Installation prefix: /usr/local
bin: bin
lib: lib64
include: include
cmake: lib64/cmake/WarpX
Build type: RelWithDebInfo
Build options:
APP: OFF
ASCENT: OFF
COMPUTE: HIP
DIMS: 3
Embedded Boundary: OFF
GPU clock timers: ON
IPO/LTO: OFF
LIB: OFF
MPI: ON
PSATD: OFF
PRECISION: DOUBLE
PARTICLE PRECISION: DOUBLE
OPENPMD: OFF
QED: OFF
QED table generation: OFF
SENSEI: OFF
-- ABLASTR: Using version '' (23.02)
ImpactX build configuration:
Version: 23.02 (23.02-1-g1a134a03ef79)
C++ Compiler: Clang 14.0.6 CrayPrgEnv
/opt/cray/pe/craype/2.7.17/bin/CC
Installation prefix: /usr/local
bin: bin
lib: lib64
include: include
cmake: lib64/cmake/ImpactX
Build type: RelWithDebInfo
Build options:
APP: ON
COMPUTE: HIP
IPO/LTO: OFF
LIB: OFF
MPI: ON
PRECISION: DOUBLE
PYTHON: OFF
OPENPMD: OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /ccs/home/mgarten/src/impactx/build
[[email protected] impactx]$ cmake --build build -j 16
[ 0%] Building CXX object CMakeFiles/buildInfoapp.dir/app/AMReX_buildInfo.cpp.o
[ 0%] Building CXX object _deps/fetchedablastr-build/CMakeFiles/buildInfoablastr.dir/ablastr/AMReX_buildInfo.cpp.o
[ 1%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BlockMutex.cpp.o
[ 1%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX.cpp.o
[ 1%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_error_fi.cpp.o
[ 3%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Version.cpp.o
[ 5%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_ParmParse.cpp.o
[ 5%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_parmparse_fi.cpp.o
[ 5%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Utility.cpp.o
[ 5%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_FileSystem.cpp.o
[ 7%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_DistributionMapping.cpp.o
[ 7%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Random.cpp.o
[ 7%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_ParallelDescriptor.cpp.o
[ 9%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_ParallelContext.cpp.o
[ 9%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_ForkJoin.cpp.o
[ 9%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_VisMF.cpp.o
[ 11%] Linking CXX static library lib/libbuildInfoapp.a
[ 11%] Built target buildInfoapp
[ 13%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_AsyncOut.cpp.o
[ 15%] Linking CXX static library ../../lib/libbuildInfoablastr.a
[ 15%] Built target buildInfoablastr
[ 15%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BackgroundThread.cpp.o
[ 17%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Arena.cpp.o
[ 17%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BArena.cpp.o
[ 17%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_CArena.cpp.o
[ 19%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_PArena.cpp.o
[ 19%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BLBackTrace.cpp.o
[ 19%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_NFiles.cpp.o
[ 21%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_parstream.cpp.o
[ 21%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_FabConv.cpp.o
[ 21%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_FPC.cpp.o
[ 23%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_VectorIO.cpp.o
[ 23%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_IntConv.cpp.o
[ 25%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Box.cpp.o
[ 25%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BoxIterator.cpp.o
[ 25%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_IntVect.cpp.o
[ 26%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_IndexType.cpp.o
[ 26%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Orientation.cpp.o
[ 26%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Periodicity.cpp.o
[ 28%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_RealBox.cpp.o
[ 28%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_RealVect.cpp.o
[ 28%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BoxList.cpp.o
[ 30%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BoxArray.cpp.o
[ 30%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BoxDomain.cpp.o
[ 32%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_FArrayBox.cpp.o
[ 32%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_IArrayBox.cpp.o
[ 32%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BaseFab.cpp.o
[ 34%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_MultiFab.cpp.o
[ 34%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_MFCopyDescriptor.cpp.o
[ 34%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_iMultiFab.cpp.o
[ 36%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_FabArrayBase.cpp.o
[ 36%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_MFIter.cpp.o
[ 36%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_CoordSys.cpp.o
[ 38%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Geometry.cpp.o
[ 38%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_MultiFabUtil.cpp.o
[ 40%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BCRec.cpp.o
[ 40%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_PhysBCFunct.cpp.o
[ 40%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BCUtil.cpp.o
[ 42%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_FilCC_C.cpp.o
[ 42%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_FilFC_C.cpp.o
[ 42%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_FilND_C.cpp.o
[ 44%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_NonLocalBC.cpp.o
[ 44%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_PlotFileUtil.cpp.o
[ 44%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_PlotFileDataImpl.cpp.o
[ 46%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_GpuControl.cpp.o
[ 46%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_GpuDevice.cpp.o
[ 48%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_GpuUtility.cpp.o
[ 48%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_GpuAsyncArray.cpp.o
[ 48%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_GpuElixir.cpp.o
[ 50%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_Machine.cpp.o
[ 50%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_MemPool.cpp.o
[ 50%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/AMReX_Parser.cpp.o
[ 51%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/AMReX_Parser_Exe.cpp.o
[ 51%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/AMReX_Parser_Y.cpp.o
[ 51%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/amrex_parser.lex.cpp.o
[ 53%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/amrex_parser.tab.cpp.o
[ 53%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/AMReX_IParser.cpp.o
[ 55%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/AMReX_IParser_Exe.cpp.o
[ 55%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/AMReX_IParser_Y.cpp.o
[ 55%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/amrex_iparser.lex.cpp.o
[ 57%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/Parser/amrex_iparser.tab.cpp.o
[ 57%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_BLProfiler.cpp.o
[ 57%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_TinyProfiler.cpp.o
[ 59%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Base/AMReX_MPMD.cpp.o
[ 59%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Boundary/AMReX_Mask.cpp.o
[ 59%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Boundary/AMReX_MultiMask.cpp.o
[ 61%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Boundary/AMReX_LO_BCTYPES.cpp.o
[ 61%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_AmrCore.cpp.o
[ 63%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_Cluster.cpp.o
[ 63%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_ErrorList.cpp.o
[ 63%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_FillPatchUtil.cpp.o
[ 65%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_FluxRegister.cpp.o
[ 65%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_InterpBase.cpp.o
[ 65%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_MFInterpolater.cpp.o
[ 67%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_Interpolater.cpp.o
[ 67%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_TagBox.cpp.o
[ 67%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_AmrMesh.cpp.o
[ 69%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/AmrCore/AMReX_InterpFaceRegister.cpp.o
[ 69%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLMG.cpp.o
[ 71%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp.o
[ 71%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp.o
[ 71%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sync.cpp.o
[ 73%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sten.cpp.o
[ 73%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_misc.cpp.o
[ 73%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.cpp.o
[ 75%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLTensorOp.cpp.o
[ 75%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLTensorOp_grad.cpp.o
[ 75%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLEBNodeFDLaplacian.cpp.o
[ 76%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/OpenBC/AMReX_OpenBC.cpp.o
[ 76%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Particle/AMReX_TracerParticles.cpp.o
[ 78%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Particle/AMReX_ParticleMPIUtil.cpp.o
[ 78%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/Particle/AMReX_ParticleUtil.cpp.o
clang-14: /home/jenkins/llvm/lib/Target/AMDGPU/SIOptimizeVGPRLiveRange.cpp:497: void {anonymous}::SIOptimizeVGPRLiveRange::optimizeLiveRange(llvm::Register, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::SmallSetVector<llvm::MachineBasicBlock*, 16>&) const: Assertion `UseMI->isPHI() && "Uses should be PHI in Endif block"' failed.
PLEASE submit a bug report to Cray and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0. Program arguments: /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14 -cc1 -triple amdgcn-amd-amdhsa -mllvm -cray-omp-opt-fork-call -mllvm -cray-omp-parallel-opt-call -mllvm -cray-openmp-rename-outlined -fcray-gpu -flocal-restrict -mllvm -cray-enhanced-asm=1 -fenhanced-asm=1 -mllvm -cray-enhanced-ir=1 -fenhanced-ir=1 -fomp-local-offload-table -ffortran -aux-triple x86_64-unknown-linux-gnu -emit-obj --mrelax-relocations -disable-free -clear-ast-before-backend -main-file-name AMReX_MLNodeLaplacian_sync.cpp -mrelocation-model pic -pic-level 1 -fhalf-no-semantic-interposition -mframe-pointer=none -fno-rounding-math -mconstructor-aliases -aux-target-cpu x86-64 -fcuda-is-device -mllvm -amdgpu-internalize-symbols -fcuda-allow-variadic-functions -fvisibility hidden -fapply-global-visibility-to-externs -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/hip.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/ocml.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/ockl.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_daz_opt_off.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_unsafe_math_off.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_finite_only_off.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_correctly_rounded_sqrt_on.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_wavefrontsize64_on.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_isa_version_90a.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_abi_version_400.bc -target-cpu gfx90a -mllvm -treat-scalable-fixed-error-as-warning -debug-info-kind=constructor -dwarf-version=5 -debugger-tuning=gdb -resource-dir /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6 -dependency-file CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sync.cpp.o.d -MT _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sync.cpp.o -sys-header-deps -internal-isystem /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6/include/cuda_wrappers -idirafter /opt/rocm-5.2.0/include -include __clang_hip_runtime_wrapper.h -isystem /opt/rocm-5.2.0/hip/include -isystem /opt/rocm-5.2.0/llvm/lib/clang/14.0.0/include/.. -isystem /opt/rocm-5.2.0/include/hiprand -isystem /opt/rocm-5.2.0/include/rocrand -isystem /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6/include -isystem /opt/cray/pe/cce/14.0.2/cce/x86_64/include/craylibs -D __CRAY_X86_TRENTO -D __CRAY_AMD_GFX90A -D __CRAYXT_COMPUTE_LINUX_TARGET -D __HIP_PLATFORM_AMD__=1 -D __HIP_PLATFORM_HCC__=1 -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base/Parser -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Boundary -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/AmrCore -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/OpenBC -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Particle -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-build -I /opt/rocm-5.2.0/include -D NDEBUG -I /opt/cray/pe/mpich/8.1.18/ofi/cray/10.0/include -I /opt/cray/pe/libsci/22.08.1.1/CRAY/9.0/x86_64/include -I /opt/cray/pe/hdf5-parallel/1.12.1.5/crayclang/14.0/include -I /opt/cray/pe/pmi/6.1.3/include -I /opt/cray/pe/dsmml/0.2.2/dsmml//include -I /opt/cray/xpmem/2.4.4-2.3_11.2__gff0e1d9.shasta/include -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++ -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++/x86_64-suse-linux -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++/backward -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++ -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++/x86_64-suse-linux -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++/backward -internal-isystem /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6/include -internal-isystem /usr/local/include -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../x86_64-suse-linux/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6/include -internal-isystem /usr/local/include -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../x86_64-suse-linux/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -O2 -std=c++17 -fdeprecated-macro -fno-autolink -fdebug-compilation-dir=/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-build/Src -ferror-limit 19 -fhip-new-launch-api -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -cuid=a1db559448e604e2 -fcuda-allow-variadic-functions -munsafe-fp-atomics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/AMReX_MLNodeLaplacian_sync-f6a335/AMReX_MLNodeLaplacian_sync-gfx90a.o -x hip /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sync.cpp
1. <eof> parser at end of file
2. Code generation
3. Running pass 'CallGraph Pass Manager' on module '/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sync.cpp'.
4. Running pass 'SI Optimize VGPR LiveRange' on function '@_ZN5amrex13launch_globalILi256EZNS_11ParallelForILi256EZNKS_15MLNodeLaplacian6refluxEiRNS_8MultiFabERKS3_S6_S4_S4_S6_EUliiiE3_EENSt9enable_ifIXsr19MaybeDeviceRunnableIT0_EE5valueEvE4typeERKNS_3Gpu10KernelInfoERKNS_3BoxEOS9_EUlvE_EEvS9_'
#0 0x00000000038258cf PrintStackTraceSignalHandler(void*) Signals.cpp:0:0
#1 0x00000000038229ed SignalHandler(int) Signals.cpp:0:0
#2 0x00007fbdc691a8bf (/lib64/libpthread.so.0+0x168bf)
#3 0x00007fbdc55fecda raise (/lib64/libc.so.6+0x4acda)
#4 0x00007fbdc5600374 abort (/lib64/libc.so.6+0x4c374)
#5 0x00007fbdc55f6cd9 __assert_fail_base (/lib64/libc.so.6+0x42cd9)
#6 0x00007fbdc55f6d61 __assert_fail (/lib64/libc.so.6+0x42d61)
#7 0x000000000126ee3e (anonymous namespace)::SIOptimizeVGPRLiveRange::optimizeLiveRange(llvm::Register, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::SmallSetVector<llvm::MachineBasicBlock*, 16u>&) const SIOptimizeVGPRLiveRange.cpp:0:0
#8 0x0000000001272e93 (anonymous namespace)::SIOptimizeVGPRLiveRange::runOnMachineFunction(llvm::MachineFunction&) SIOptimizeVGPRLiveRange.cpp:0:0
#9 0x00000000028bdacf llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x28bdacf)
#10 0x0000000002dea470 llvm::FPPassManager::runOnFunction(llvm::Function&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x2dea470)
#11 0x00000000024c7319 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) CallGraphSCCPass.cpp:0:0
#12 0x0000000002dec0c6 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x2dec0c6)
#13 0x0000000003badbea clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, clang::FrontendOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x3badbea)
#14 0x0000000004a84d55 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x4a84d55)
#15 0x0000000005601088 clang::ParseAST(clang::Sema&, bool, bool) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x5601088)
#16 0x000000000436c7b8 clang::FrontendAction::Execute() (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x436c7b8)
#17 0x00000000042ec625 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x42ec625)
#18 0x0000000004433a8a clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x4433a8a)
#19 0x0000000000df2365 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0xdf2365)
#20 0x0000000000defa6b ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&) driver.cpp:0:0
#21 0x0000000000d2eb86 main (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0xd2eb86)
#22 0x00007fbdc55e92bc __libc_start_main (/lib64/libc.so.6+0x352bc)
#23 0x0000000000ded0f9 _start /home/abuild/rpmbuild/BUILD/glibc-2.26/csu/../sysdeps/x86_64/start.S:120:0
clang-14: error: unable to execute command: Aborted (core dumped)
clang-14: error: clang frontend command failed due to signal (use -v to see invocation)
Cray clang version 14.0.2 (ecfd9ef4dfd5696cd449133c0da0293d503c2f21)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/share/../bin
clang-14: note: diagnostic msg: Error generating preprocessed source(s).
gmake[2]: *** [_deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/build.make:1406: _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sync.cpp.o] Error 254
gmake[2]: *** Waiting for unfinished jobs....
clang-14: /home/jenkins/llvm/lib/Target/AMDGPU/SIOptimizeVGPRLiveRange.cpp:497: void {anonymous}::SIOptimizeVGPRLiveRange::optimizeLiveRange(llvm::Register, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::SmallSetVector<llvm::MachineBasicBlock*, 16>&) const: Assertion `UseMI->isPHI() && "Uses should be PHI in Endif block"' failed.
PLEASE submit a bug report to Cray and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0. Program arguments: /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14 -cc1 -triple amdgcn-amd-amdhsa -mllvm -cray-omp-opt-fork-call -mllvm -cray-omp-parallel-opt-call -mllvm -cray-openmp-rename-outlined -fcray-gpu -flocal-restrict -mllvm -cray-enhanced-asm=1 -fenhanced-asm=1 -mllvm -cray-enhanced-ir=1 -fenhanced-ir=1 -fomp-local-offload-table -ffortran -aux-triple x86_64-unknown-linux-gnu -emit-obj --mrelax-relocations -disable-free -clear-ast-before-backend -main-file-name AMReX_MLNodeLaplacian.cpp -mrelocation-model pic -pic-level 1 -fhalf-no-semantic-interposition -mframe-pointer=none -fno-rounding-math -mconstructor-aliases -aux-target-cpu x86-64 -fcuda-is-device -mllvm -amdgpu-internalize-symbols -fcuda-allow-variadic-functions -fvisibility hidden -fapply-global-visibility-to-externs -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/hip.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/ocml.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/ockl.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_daz_opt_off.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_unsafe_math_off.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_finite_only_off.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_correctly_rounded_sqrt_on.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_wavefrontsize64_on.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_isa_version_90a.bc -mlink-builtin-bitcode /opt/rocm-5.2.0/amdgcn/bitcode/oclc_abi_version_400.bc -target-cpu gfx90a -mllvm -treat-scalable-fixed-error-as-warning -debug-info-kind=constructor -dwarf-version=5 -debugger-tuning=gdb -resource-dir /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6 -dependency-file CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp.o.d -MT _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp.o -sys-header-deps -internal-isystem /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6/include/cuda_wrappers -idirafter /opt/rocm-5.2.0/include -include __clang_hip_runtime_wrapper.h -isystem /opt/rocm-5.2.0/hip/include -isystem /opt/rocm-5.2.0/llvm/lib/clang/14.0.0/include/.. -isystem /opt/rocm-5.2.0/include/hiprand -isystem /opt/rocm-5.2.0/include/rocrand -isystem /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6/include -isystem /opt/cray/pe/cce/14.0.2/cce/x86_64/include/craylibs -D __CRAY_X86_TRENTO -D __CRAY_AMD_GFX90A -D __CRAYXT_COMPUTE_LINUX_TARGET -D __HIP_PLATFORM_AMD__=1 -D __HIP_PLATFORM_HCC__=1 -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Base/Parser -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Boundary -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/AmrCore -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/OpenBC -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/Particle -I /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-build -I /opt/rocm-5.2.0/include -D NDEBUG -I /opt/cray/pe/mpich/8.1.18/ofi/cray/10.0/include -I /opt/cray/pe/libsci/22.08.1.1/CRAY/9.0/x86_64/include -I /opt/cray/pe/hdf5-parallel/1.12.1.5/crayclang/14.0/include -I /opt/cray/pe/pmi/6.1.3/include -I /opt/cray/pe/dsmml/0.2.2/dsmml//include -I /opt/cray/xpmem/2.4.4-2.3_11.2__gff0e1d9.shasta/include -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++ -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++/x86_64-suse-linux -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++/backward -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++ -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++/x86_64-suse-linux -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../include/g++/backward -internal-isystem /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6/include -internal-isystem /usr/local/include -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../x86_64-suse-linux/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/lib/clang/14.0.6/include -internal-isystem /usr/local/include -internal-isystem /opt/cray/pe/gcc/10.3.0/snos/lib/gcc/x86_64-suse-linux/10.3.0/../../../../x86_64-suse-linux/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -O2 -std=c++17 -fdeprecated-macro -fno-autolink -fdebug-compilation-dir=/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-build/Src -ferror-limit 19 -fhip-new-launch-api -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -cuid=2f96d5e7b873b0d -fcuda-allow-variadic-functions -munsafe-fp-atomics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/AMReX_MLNodeLaplacian-3587ac/AMReX_MLNodeLaplacian-gfx90a.o -x hip /ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp
1. <eof> parser at end of file
2. Code generation
3. Running pass 'CallGraph Pass Manager' on module '/ccs/home/mgarten/src/impactx/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp'.
4. Running pass 'SI Optimize VGPR LiveRange' on function '@_ZN5amrex13launch_globalILi256EZNS_11ParallelForILi256EZNKS_15MLNodeLaplacian21restrictInteriorNodesEiRNS_8MultiFabES4_EUliiiE3_EENSt9enable_ifIXsr19MaybeDeviceRunnableIT0_EE5valueEvE4typeERKNS_3Gpu10KernelInfoERKNS_3BoxEOS7_EUlvE_EEvS7_'
#0 0x00000000038258cf PrintStackTraceSignalHandler(void*) Signals.cpp:0:0
#1 0x00000000038229ed SignalHandler(int) Signals.cpp:0:0
#2 0x00007fba2efcd8bf (/lib64/libpthread.so.0+0x168bf)
#3 0x00007fba2dcb1cda raise (/lib64/libc.so.6+0x4acda)
#4 0x00007fba2dcb3374 abort (/lib64/libc.so.6+0x4c374)
#5 0x00007fba2dca9cd9 __assert_fail_base (/lib64/libc.so.6+0x42cd9)
#6 0x00007fba2dca9d61 __assert_fail (/lib64/libc.so.6+0x42d61)
#7 0x000000000126ee3e (anonymous namespace)::SIOptimizeVGPRLiveRange::optimizeLiveRange(llvm::Register, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::SmallSetVector<llvm::MachineBasicBlock*, 16u>&) const SIOptimizeVGPRLiveRange.cpp:0:0
#8 0x0000000001272e93 (anonymous namespace)::SIOptimizeVGPRLiveRange::runOnMachineFunction(llvm::MachineFunction&) SIOptimizeVGPRLiveRange.cpp:0:0
#9 0x00000000028bdacf llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x28bdacf)
#10 0x0000000002dea470 llvm::FPPassManager::runOnFunction(llvm::Function&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x2dea470)
#11 0x00000000024c7319 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) CallGraphSCCPass.cpp:0:0
#12 0x0000000002dec0c6 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x2dec0c6)
#13 0x0000000003badbea clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, clang::FrontendOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x3badbea)
#14 0x0000000004a84d55 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x4a84d55)
#15 0x0000000005601088 clang::ParseAST(clang::Sema&, bool, bool) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x5601088)
#16 0x000000000436c7b8 clang::FrontendAction::Execute() (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x436c7b8)
#17 0x00000000042ec625 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x42ec625)
#18 0x0000000004433a8a clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0x4433a8a)
#19 0x0000000000df2365 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0xdf2365)
#20 0x0000000000defa6b ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&) driver.cpp:0:0
#21 0x0000000000d2eb86 main (/opt/cray/pe/cce/14.0.2/cce-clang/x86_64/bin/clang-14+0xd2eb86)
#22 0x00007fba2dc9c2bc __libc_start_main (/lib64/libc.so.6+0x352bc)
#23 0x0000000000ded0f9 _start /home/abuild/rpmbuild/BUILD/glibc-2.26/csu/../sysdeps/x86_64/start.S:120:0
clang-14: error: unable to execute command: Aborted
clang-14: error: clang frontend command failed due to signal (use -v to see invocation)
Cray clang version 14.0.2 (ecfd9ef4dfd5696cd449133c0da0293d503c2f21)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/cray/pe/cce/14.0.2/cce-clang/x86_64/share/../bin
clang-14: note: diagnostic msg: Error generating preprocessed source(s).
gmake[2]: *** [_deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/build.make:1392: _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp.o] Error 254
gmake[1]: *** [CMakeFiles/Makefile2:1978: _deps/fetchedamrex-build/Src/CMakeFiles/amrex.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
[[email protected] impactx]$
Edit 2023/02/08: removed -DImpactX_PYTHON=ON
in description here to make it simpler; add download instructions
Add a "repeat" option to Line
to make building of channels with many periods easy.
Perform the benchmarks described in @ChristopherMayes et al., "3D SPACE CHARGE IN BMAD", IPAC2018 (2018)
DOI:10.18429/JACoW-IPAC2018-THPAK085
and also evaluate performance.
Potentially add as CI test for correctness (note: the paper uses 10^6 particles; for CI purposes we might want to go lower because of constrained resources).
Draft in #257
We should check if our beam particles actually moved, instead of only checking 2nd moments.
We often check stationary and periodic solutions. If we only check the 2nd moments, we will not notice if particles did not move at all. For instance, in the development of #214 we accidentally did a zero-element simulation which passed all tests and only failed during plotting.
Thus, we should check:
pt
(and/or pz
) of the reference particlecc @cemitch99 @n01r and me
Start like this:
N
boxes per device; currently, N
is hard-coded to 1Based on #102, implement adaptive, multi-level mesh-refinement
Decide on refinement criteria, e.g., gradients of charge density.
Implement field gather functions beyond first order particle shapes.
Follow-up to #162 and ECP-WarpX/WarpX#3357
As in WarpX, we want to add warnings after step 1 (and at the end) about unused parameters from inputs files.
This helps users to spot typos and logic issues quickly.
We also need an "abort on warnings" options for our CI (unsupervised, automated tests), so that our inputs files always work.
First seen in #118
Add automated docker containers:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.