Build into container that already has spack about serac

[jamiebramwell]

This is the PR where @chapman39 is using the environment-ized uberenv: #882 . Hopefully this helps!

[vsoch]

okay success! I'll have an automated build with core serac and the provided examples deployed soon. I'd like to test it out in the Flux Operator so my next (and last question, don't worry!) is about workflows. Do you have any example projects that do something cool with serac with parameters we could potentially scale, or any kind of extra complexity or services?

[jamiebramwell]

Passing the buck again! Our biggest users these days are @barreracruz and @kswartz92 . Do either of you have a pure serac problem that @vsoch can use for a scalability study or parameter sweep?

[jamiebramwell]

What about the re-entrant honeycomb problem that I shared with you a few days ago, @jamiebramwell? The one you use to test the new nonlinear solvers. I can generate a mesh with more elements through thickness if refining it is not enough for a scalability study.

Ooo! Good idea! I should be able to package that up for @vsoch . Thanks!

[jamiebramwell]

Hi @vsoch! I unfortunately don't know much about uberenv other than how to use it. @white238 (or maybe @chapman39 ) can give better insight. I do know @chapman39 is currently updating uberenv to use Spack environments directly.

[vsoch]

This is what I'm trying now - although I suspect I'll run into issues with needing particular versions, etc. I'll post an update when this fails! 😆

[vsoch]

Hiya! So I'm stepping back and trying the suggested build, here is my Dockerfile:

ARG tag=gcc-11_latest
FROM seracllnl/tpls:${tag}

# docker build -t serac .
# docker exec -it serac bash

RUN git clone --depth 1 https://github.com/LLNL/serac /home/serac/serac && \
    cd /home/serac/serac && \
    git submodule init && \
    git submodule update
WORKDIR /home/serac/serac
RUN python3 ./config-build.py  -hc host-configs/docker/[email protected] -bp ../build -ip ../install #&& \
    cd ../build && \
    make && \
    make test

(also note the instruction references python but it's python3 that we have in the container.

However, it looks like the MFEM spack directory that is present doesn't match what the python build script needs.

-- Using installed MFEM
-- Looking for MFEM using MFEM_DIR = /home/serac/serac_tpls/gcc-11.1.0/mfem-4.5.2.1-dcl2shc7i2wwfpuf7kq5as3k3jzc6uk2
CMake Error at cmake/SeracMacros.cmake:115 (message):
  Given MFEM_DIR does not exist:
  /home/serac/serac_tpls/gcc-11.1.0/mfem-4.5.2.1-dcl2shc7i2wwfpuf7kq5as3k3jzc6uk2
Call Stack (most recent call first):
  cmake/thirdparty/FindMFEM.cmake:20 (serac_assert_is_directory)
  cmake/thirdparty/SetupSeracThirdParty.cmake:184 (include)
  CMakeLists.txt:126 (include)


-- Configuring incomplete, errors occurred!
See also "/home/serac/build/CMakeFiles/CMakeOutput.log".
Error: CMake command failed with return code: 1
serac@a350f33fe930:~/serac$ ls ../serac_tpls/
bin  gcc-11.1.0  spack
serac@a350f33fe930:~/serac$ ls ../serac_tpls/gcc-11.1.0/
axom-0.7.0.4-lxxz2lzq5kmkwohmjtqpjooewid2bfbm/       ncurses-6.3-7ryidwb4sgpq2dkry3bu7oxyfl4qep4l/
blt-0.5.2-hh6st6lqzpj526jgiwa5zyrydrts7xew/          netcdf-c-4.7.4-ote37ajjhrkqu7ntxkqxryn2g5pc2apx/
camp-2022.03.2-ecxxjszodysrw4tiujtia5kma7whjszv/     parmetis-4.0.3-lxhbfjt6ldkee4qychk54v64kklppzlt/
conduit-0.8.4-pfqoghwltlucmgrplqq364e5o6yhq7jr/      raja-2022.03.0-raeo25xb7cumpvjlhqmrx6fncfuwef55/
hdf5-1.8.21-h5dpyqnqdcwv6rcpdz6ywjp53yeqst7a/        readline-8.1.2-ihaiti2r7b5asve4o7jvpgcjhwftwrs2/
hypre-2.26.0-jm3f5gzyuocir3vkfoh7mmxnl34lfkkx/       superlu-dist-6.1.1-jke7rcnlsep6bbtvimfqkl3sgmgrjpzr/
lua-5.4.4-ve6ws2zzw7qkrzupei6mnhtwdcxxz6jf/          umpire-2022.03.1-pieu6hp5juhqvvm6dp645xxlea4igxj3/
metis-5.1.0-rlnws7c5nyehxcnegqsto5rgtwhho7ru/        unzip-6.0-il72j5gpvxopfg7spcbn5dnt6yu4lexx/
mfem-4.5.3.1-aona2r6vgnjhv7xe5l2ymspxxwmkirwe/       zlib-1.2.13-zfwba3stno36xdnevigxdscpu5kozrmg/
serac@a350f33fe930:~/serac$ ls ../serac_tpls/gcc-11.1.0/m

It's just not using the right hash. Was there a specific branch I should have cloned? I'm following instructions here: https://serac.readthedocs.io/en/latest/sphinx/dev_guide/docker_env.html#docker-label. Thank you!

[white238]

Unfortunately it looks like our latest container tag is out of sync with our develop branch host-configs. We are currently shuffling a lot of TPL build processes.

This one is the date stamped tag seracllnl/tpls:gcc-11_02-16-23_00h-14m. Found here:

https://github.com/LLNL/serac/blob/develop/azure-pipelines.yml#L11

Sorry for the confusion!

[vsoch]

Totally ok! I'll give that one a try.

[white238]

Awesome! @jamiebramwell is the best person to answer that question.

[barreracruz]

What about the re-entrant honeycomb lattice problem that I shared with you a few days ago, @jamiebramwell? The one you use to test the new nonlinear solvers. I can generate a mesh with more elements through thickness if refining it is not enough for a scalability study.

[vsoch]

Awesome! Thank you! 🙏