LLVM 15 toolchain cant find omp.h about hpc-container-maker

[marcomontevechi1]

I also noticed that by installing libc++-dev and adding -stdlib=libc++ in the makefile, then the compilation will complain that it cant find cassert, so i figure there is something weird going on here.

[samcmill]

The llvm building block installs support for OpenMP by default. You would need to pass openmp=False to disable it, and that's not he case here.

I see the header is installed:

root@9fb82817a0b9:/# find / -name omp.h -print
/usr/lib/gcc/x86_64-linux-gnu/11/include/omp.h
/usr/lib/llvm-15/lib/clang/15.0.7/include/omp.h

I'm able to compile a dummy program that just includes that header without any issues.

I suspect your Makefile may be setting CPATH or something similar that is causing the headers not to be found.

[marcomontevechi1]

@samcmill thanks for the reply. Can you please share an example?

I tried compiling a simple test.cpp file with:

#include <omp.h>

int main() {;}

I tried:

mpicxx ./test.cpp
clang-15 ./test.cpp
mpicxx -fopenmp ./test.cpp
clang-15 -fopenmp ./test.cpp

But all got the same error:

./test.cpp:1:10: fatal error: 'omp.h' file not found
#include <omp.h>
         ^~~~~~~
1 error generated.

The Makefile i was using was adapted from miniVite page and does not set CPATH:

CXX = mpicxx

USE_TAUPROF=0
ifeq ($(USE_TAUPROF),1)
TAU=/soft/perftools/tau/tau-2.29/craycnl/lib
CXX = tau_cxx.sh -tau_makefile=$(TAU)/Makefile.tau-intel-papi-mpi-pdt 
endif
# use -xmic-avx512 instead of -xHost for Intel Xeon Phi platforms
OPTFLAGS = -O3 -fopenmp -DPRINT_DIST_STATS #-DPRINT_EXTRA_NEDGES #-DUSE_MPI_RMA -DUSE_MPI_ACCUMULATE #-DUSE_32_BIT_GRAPH #-DDEBUG_PRINTF #-DUSE_MPI_RMA #-DPRINT_LCG_DOUBLE_LOHI_RANDOM_NUMBERS#-DUSE_MPI_RMA #-DPRINT_LCG_DOUBLE_RANDOM_NUMBERS #-DPRINT_RANDOM_XY_COORD
#-DUSE_MPI_SENDRECV
#-DUSE_MPI_COLLECTIVES
# use export ASAN_OPTIONS=verbosity=1 to check ASAN output
SNTFLAGS = -std=c++11 -fopenmp -fsanitize=address -O1 -fno-omit-frame-pointer
CXXFLAGS = -std=c++11 -g $(OPTFLAGS)

OBJ = main.o
TARGET = miniVite

all: $(TARGET)

%.o: %.cpp
	$(CXX) $(CXXFLAGS) -c -o $@ $^

$(TARGET):  $(OBJ)
	$(CXX) $^ $(OPTFLAGS) -o $@

.PHONY: clean

clean:
	rm -rf *~ $(OBJ) $(TARGET)

Im sorry if there is any trivial mistake but i still lack some experience with clang.
Thanks for your help.

[samcmill]

Sorry, I'm unable to reproduce this. I took your recipe up to the point where it installed miniVite.

root@67236b9c6acd:/# cat test.cpp 
#include <omp.h>

int main() {}
root@67236b9c6acd:/# mpicxx test.cpp 
root@67236b9c6acd:/# clang test.cpp 
root@67236b9c6acd:/# clang++ test.cpp 
root@67236b9c6acd:/# clang++-15 test.cpp 
root@67236b9c6acd:/# mpicxx --version
Ubuntu clang version 15.0.7
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Compare this to the corresponding output in your container:

root@67236b9c6acd:/# clang++ --verbose test.cpp 
Ubuntu clang version 15.0.7
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/lib/llvm-15/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/ -resource-dir /usr/lib/llvm-15/lib/clang/15.0.7 -I/usr/local/ucx/include -I/usr/local/xpmem/include -I/usr/local/knem/include -I/usr/local/gdrcopy/include -I. -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/x86_64-linux-gnu/c++/11 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/backward -internal-isystem /usr/lib/llvm-15/lib/clang/15.0.7/include -internal-isystem /usr/local/include -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/ -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/test-694e2b.o -x c++ test.cpp
clang -cc1 version 15.0.7 based upon LLVM 15.0.7 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/ucx/include
 /usr/local/xpmem/include
 /usr/local/knem/include
 /usr/local/gdrcopy/include
 .
 /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11
 /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/x86_64-linux-gnu/c++/11
 /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/backward
 /usr/lib/llvm-15/lib/clang/15.0.7/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/usr/bin/ld" -pie -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /lib/x86_64-linux-gnu/Scrt1.o /lib/x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/11/crtbeginS.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/11 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/lib -L/usr/lib -L/usr/local/ucx/lib -L/usr/local/xpmem/lib -L/usr/local/gdrcopy/lib -L. /tmp/test-694e2b.o -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/bin/../lib/gcc/x86_64-linux-gnu/11/crtendS.o /lib/x86_64-linux-gnu/crtn.o

[samcmill]

Ah - I think I see what's going on.

It's this: Stage0 += apt_get(ospackages=['make', 'wget', 'tar', 'libomp-dev']), specifically libomp-dev.

That's installing libomp-dev from the distro and it conflicts with the version installed from the llvm building block.

root@67236b9c6acd:/# apt-get install -y libomp-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libllvm14 libomp-14-dev libomp5-14
Suggested packages:
  libomp-14-doc
The following packages will be REMOVED:
  libomp-15-dev libomp5-15
The following NEW packages will be installed:
  libllvm14 libomp-14-dev libomp-dev libomp5-14
0 upgraded, 4 newly installed, 2 to remove and 23 not upgraded.

Try removing libomp-dev from that line - I'm not sure why it's there in the first place.

[marcomontevechi1]

Thanks! It worked... kinda

Now i get:

clang-15 -std=c++11 -g -O3 -fopenmp -DPRINT_DIST_STATS  -c -o main.o main.cpp
main.cpp:54:10: fatal error: 'mpi.h' file not found
#include <mpi.h>
         ^~~~~~~
1 error generated.
make: *** [Makefile:23: main.o] Error 1

And

locate /mpi.h

gets me:
/usr/local/openmpi/include/mpi.h

[samcmill]

For that you should use the MPI compiler wrapper, mpicxx (or mpicc, as the case may be).

[marcomontevechi1]

Oops, i didnt see i was using clang-15. Things worked! Thanks for the help :)