GithubHelp home page GithubHelp logo

cgal / cgal-testsuite-dockerfiles Goto Github PK

View Code? Open in Web Editor NEW
7.0 5.0 9.0 499 KB

Docker files for the automated CGAL test suite

License: GNU General Public License v3.0

Shell 14.84% Python 37.71% CMake 16.13% Dockerfile 31.32%

cgal-testsuite-dockerfiles's Introduction

cgal-testsuite-dockerfiles

Dockerfiles and tools to run the CGAL testsuite inside containers.

It is recommended to get the images directly from Docker Hub instead of building them locally:

docker pull --all-tags cgal/testsuite-docker # get all images
docker pull cgal/testsuite-docker:TAG # get a specific image by replacing TAG with some tag

A list of available tags can be found here.

Building Images locally

To build images locally see the help of build_images.py.

Running the testsuite

To run the testsuite using this image:

./test_cgal.py --user **** --passwd **** --images cgal-testsuite/centos6

If you would like to use an already extraced internal release:

./test_cgal.py --use-local --testsuite /path/to/release --images cgal-testsuite/centos6

It is also possible to only test specific packages, but keep in mind that this will alter the release:

./test_cgal.py --use-local --testsuite /path/to/release --images cgal-testsuite/centos6 \
               --packages Core Mesh_2 Mesh_3

Running the testsuite for one container

Use a command similar to that one:

docker run --rm -t -i -v $PWD/run-testsuite.sh:/mnt/run-testsuite.sh:z -v $HOME/Git/cgal-master:/mnt/testsuite:ro,z -v $PWD/testresults:/mnt/testresults:z cgal/testsuite-docker:debian-testing bash -x /mnt/run-testsuite.sh

If you want a limited testsuite, you can have a file testresults/list_test_packages containing something like:

echo Mesh_2
echo Triangulation_2

Default Arguments

Default arguments can be provided through a test_cgal_rc file in $XDG_CONFIG_HOME (typically $HOME/.config) or the config directory of the resource CGAL.

Cron Job

To set up a machine for submission of the whole testsuite via a cronjob the following configuration could be used:

--user xxxxxxx
--passwd xxxxxx
--force-rm
--upload-results
--tester-name="My Name"
--tester-address [email protected]
--testsuite /path/to/testsuite
--testresults /path/to/testresults
--images cgal/testsuite-docker:centos5
         cgal/testsuite-docker:centos6
         cgal/testsuite-docker:archlinux

The names of the images depend on how they have been obtained. The example shows the names that will typically be used when obtaining the images through Docker Hub.

Modifying the build environment

To run a testsuite in a specific environment (specific version of some library), set up a Docker image that provides that environment. Don't forget to make sure your installed library is preferred by CMake over the system library by setting appropriate environment variables.

To control how the testsuite is being build use the environment variable CGAL_CMAKE_FLAGS to change the CMake invocation. This variable must look like an declaration where each array element contains one command line argument of the cmake call, e.g.

ENV CGAL_CMAKE_FLAGS="(\"-DCGAL_CXX_FLAGS=-std=c++11 -Wextra\" \"-DCMAKE_CXX_FLAGS=-fno-asm\")"

You can also change default build variables through the classic environment variables (e.g. LD_FLAGS, CXX_FLAGS).

Required Non-Standard Python Packages

The code requires several non-standard python2 packages, which are available in all common distributions.

  • docker
  • xdg

They can be installed using pip:

pip install docker xdg

SELinux issues

On Linux system using SELinux (such as the default setting for the recent versions of Fedora, RHEL, and CentOS), you might need to relabel the host files and directories used as volumes by the containers:

chcon -Rt svirt_sandbox_file_t ./docker-entrypoint.sh ./testsuite ./testresults

If you use the options --testsuite /path/to/testsuite or --testresults /path/to/testresults, then the pointed directories must also be relabeled with svirt_sandbox_file_t:

chcon -Rt svirt_sandbox_file_t /path/to/testresults /path/to/testresults

cgal-testsuite-dockerfiles's People

Contributors

afabri avatar akobel avatar bo0ts avatar dependabot[bot] avatar lrineau avatar maxgimeno avatar saillantnicolas avatar sloriot avatar soesau avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

cgal-testsuite-dockerfiles's Issues

ArchLinux clang-svn failure

The build of the clang-svn container (based on ArchLinux) fails on:

error: In-source builds are not allowed. Please configure from a separate build directory!

See for example that build failure on the Docker Hub.

Is not there a special package from ArchLinux that already packages a clang version for far from being the up-to-date svn version?

Error during parsing of test_cgal_rc

I have a file /home/lrineau/.config/CGAL/test_cgal_rc that contains:

--user member1
--passwd cg4any1
--tester lrineau
--tester-address [email protected]

If I add the following line to it:

--tester-name "Laurent Rineau"

then the script display its usage and says:

test_cgal.py: error: unrecognized arguments: Rineau"

In my opinion, it does not like either the spaces in the option. The quote is not taken into account.

If I use the following line without quotes:

--tester-name Laurent Rineau

it displays almost the same:

test_cgal.py: error: unrecognized arguments: Rineau

In do not see how I can fill an option with a value that contains a space. What does the default parser ArgumentParser.convert_arg_line_to_args?

Testsuite for our containers

I think there should be a minimalist testsuite for the built containers.

It could be as simple as:

docker run --rm $image ls /usr/include/QGLViewer/qglviewer.h \
                          /usr/include/GL/glew.h \
                          /usr/include/eigen3/Eigen/Eigen

or similar...

Error in the Fedora-Rawhide Dockerfile: need copr

https://registry.hub.docker.com/u/cgal/testsuite-docker/build_id/67256/code/b7vx6dklukyipt4u6hpmzm7/

Step 4 : RUN rpm --import https://copr-be.cloud.fedoraproject.org/results/rineau/libQGLViewer-qt5/pubkey.gpg &&     dnf -y copr enable rineau/libQGLViewer-qt5 && dnf -y install libQGLViewer-qt5-devel.x86_64
No such command: copr. Please use /usr/bin/dnf --help

It could be a DNF plugin command, try: "dnf install 'dnf-command(copr)'"

You need to add the following command:

dnf install 'dnf-command(copr)' && dnf clean all

cgal.geomtryfactory.com was overloaded

I just ran:

sudo systemctl stop docker

on cgal.geometryfactory.com because the machine was overloaded (the load was 90, where there are only six processors with hyperthreading, thus 12 threads).

The problem seems to be that the Docker testsuite of one day did not finish in less than a day, and was superimposed with the testsuite of the next day. Once that happens, that is even worst for the following day.

I do not know exactly why that happened, but we cannot allow ourself to overload that machine. I suggest that once a testsuite is started, it kills all the testsuite containers that may remain. And there must be also something in the controlling Python script, that detects that a new testsuite has been launched, and prevent to launch new containers for the old testsuite.

Cc: @sloriot @bo0ts

Sébastien, restart Docker once you have figured out a way that it no longer happens.

Run testsuite as non-root

docker-entrypoint.sh should create a new user, set the necessary privileges and execute the testsuite as that user.

image_default() does not work

I have the image gal-testsuite/archlinux. It is tagged latest implicitly by Docker, but that tag does not seem to be implicit when one query images:

$ docker images cgal-testsuite/archlinux:latest
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE

$ docker images cgal-testsuite/archlinux       
REPOSITORY                 TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
cgal-testsuite/archlinux   latest              d2201388444f        40 minutes ago      1.376 GB

As a consequence, the function image_default() (which should be named images_default() actually) does not work:

$ ./test_cgal.py 
Could not find image: cgal-testsuite/archlinux:latest
Traceback (most recent call last):
  File "/home/lrineau/Git/CGAL-testsuite-dockerfiles/test_cgal.py", line 202, in <module>
    main()
  File "/home/lrineau/Git/CGAL-testsuite-dockerfiles/test_cgal.py", line 152, in main
    assert do_images_exist(args.images), 'Specified images could not be found.'
AssertionError: Specified images could not be found.
zsh: exit 1     ~/Git/CGAL-testsuite-dockerfiles/test_cgal.py
zsh: exit 1

We need a new image, with -fsanitize=address

@maxGimeno

Can you please add a new image, derived from Fedora-Rawhide, with the compilation flags -Og -g -fsanitize=address?

I suggest you first try to add it locally to your local Docker testsuite, and then push it to a pull-request.

Tests of other third-party libraries

In my CMakeCache.txt, I can read:

CGAL_SUPPORTING_3RD_PARTY_LIBRARIES:INTERNAL=GMP;GMPXX;MPFR;ZLIB;OpenGL;LEDA;MPFI;RS;RS3;OpenNL;Eigen3;BLAS;LAPACK;QGLViewer;ESBTL;Coin3D;NTL;IPE;TAUCS

I wonder if GMPXX is tested.

The Docker testsuite is supposed to replace the legacy testsuite, and I think that the following should be tested, at least on some platforms:

  • MPFI, with RS
  • ESBTL (headers only)
  • NTL
  • ipe (there are packages for Fedora, at least).

TAUCS should no longer be tested.

ArchLinux build failure: gpg: keyserver timed out

https://hub.docker.com/r/cgal/testsuite-docker/builds/bn3aymnvipckfbwrj7mnqi8/

RUN pacman -Sy --noconfirm &&     pacman-key --refresh-keys &&     pacman -Su --noconfirm &&     pacman-db-upgrade
:: Synchronizing package databases...
downloading core.db...
downloading core.db...
downloading core.db...
downloading core.db...
downloading core.db...
downloading extra.db...
downloading community.db...
gpg: refreshing 76 keys from hkp://pool.sks-keyservers.net

gpg: requesting key B005D894 from hkp server pool.sks-keyservers.net

gpg: requesting key FFF979E7 from hkp server pool.sks-keyservers.net

gpg: requesting key CDFD6BB0 from hkp server pool.sks-keyservers.net

gpg: requesting key 4C7EA887 from hkp server pool.sks-keyservers.net

gpg: requesting key 6AC6A4C2 from hkp server pool.sks-keyservers.net

gpg: requesting key 824B18E8 from hkp server pool.sks-keyservers.net

gpg: requesting key 31496106 from hkp server pool.sks-keyservers.net
gpg: requesting key EAE999BD from hkp server pool.sks-keyservers.net
gpg: requesting key 98BC6FF5 from hkp server pool.sks-keyservers.net
gpg: requesting key 753E0F1F from hkp server pool.sks-keyservers.net
gpg: requesting key D30DB0AD from hkp server pool.sks-keyservers.net
gpg: requesting key 0F2A092B from hkp server pool.sks-keyservers.net
gpg: requesting key EC133BAD from hkp server pool.sks-keyservers.net
gpg: requesting key 31361F01 from hkp server pool.sks-keyservers.net
gpg: requesting key 5CF9C8D4 from hkp server pool.sks-keyservers.net
gpg: requesting key 0901C163 from hkp server pool.sks-keyservers.net
gpg: requesting key 00F0D0F0 from hkp server pool.sks-keyservers.net
gpg: requesting key 4CE1C13E from hkp server pool.sks-keyservers.net
gpg: requesting key 487328A9 from hkp server pool.sks-keyservers.net
gpg: requesting key B47A0DAB from hkp server pool.sks-keyservers.net
gpg: requesting key F15447D5 from hkp server pool.sks-keyservers.net
gpg: requesting key F53A76ED from hkp server pool.sks-keyservers.net
gpg: requesting key B4440678 from hkp server pool.sks-keyservers.net
gpg: requesting key BA06C6A9 from hkp server pool.sks-keyservers.net
gpg: requesting key 5BD5C938 from hkp server pool.sks-keyservers.net
gpg: requesting key F56C0C53 from hkp server pool.sks-keyservers.net
gpg: requesting key 4F010D48 from hkp server pool.sks-keyservers.net
gpg: requesting key 9205AC90 from hkp server pool.sks-keyservers.net
gpg: requesting key 2FA915EC from hkp server pool.sks-keyservers.net
gpg: requesting key 0864983E from hkp server pool.sks-keyservers.net
gpg: requesting key A9999C34 from hkp server pool.sks-keyservers.net
gpg: requesting key 30D7CB92 from hkp server pool.sks-keyservers.net
gpg: requesting key F04569AE from hkp server pool.sks-keyservers.net
gpg: requesting key 215B37AD from hkp server pool.sks-keyservers.net
gpg: requesting key 4FA415FA from hkp server pool.sks-keyservers.net
gpg: requesting key 615137BC from hkp server pool.sks-keyservers.net
gpg: requesting key 796CA067 from hkp server pool.sks-keyservers.net
gpg: requesting key 7C50773E from hkp server pool.sks-keyservers.net
gpg: requesting key 3B94FA10 from hkp server pool.sks-keyservers.net
gpg: requesting key 013C2580 from hkp server pool.sks-keyservers.net
gpg: requesting key F40D2072 from hkp server pool.sks-keyservers.net
gpg: requesting key 589874AB from hkp server pool.sks-keyservers.net
gpg: requesting key BAB142C1 from hkp server pool.sks-keyservers.net
gpg: requesting key D1CEDDAC from hkp server pool.sks-keyservers.net
gpg: requesting key 9326B440 from hkp server pool.sks-keyservers.net
gpg: requesting key DA2EE423 from hkp server pool.sks-keyservers.net
gpg: requesting key 9741E8AC from hkp server pool.sks-keyservers.net
gpg: requesting key E19DAA50 from hkp server pool.sks-keyservers.net
gpg: requesting key 2D1493D2 from hkp server pool.sks-keyservers.net
gpg: requesting key 8406FFF3 from hkp server pool.sks-keyservers.net
gpg: requesting key C0711BF1 from hkp server pool.sks-keyservers.net
gpg: requesting key 91B842AE from hkp server pool.sks-keyservers.net

gpg: requesting key 2072D77A from hkp server pool.sks-keyservers.net
gpg: requesting key F27FB7DA from hkp server pool.sks-keyservers.net
gpg: requesting key EA433FC7 from hkp server pool.sks-keyservers.net
gpg: requesting key F1D357C1 from hkp server pool.sks-keyservers.net
gpg: requesting key E62EB915 from hkp server pool.sks-keyservers.net
gpg: requesting key 0C84C0A5 from hkp server pool.sks-keyservers.net
gpg: requesting key 9AF5F22A from hkp server pool.sks-keyservers.net
gpg: requesting key 8E4B1A25 from hkp server pool.sks-keyservers.net
gpg: requesting key 06361833 from hkp server pool.sks-keyservers.net
gpg: requesting key 7EDF681F from hkp server pool.sks-keyservers.net
gpg: requesting key 3C4F88BC from hkp server pool.sks-keyservers.net
gpg: requesting key 295AFBF4 from hkp server pool.sks-keyservers.net
gpg: requesting key 437520BD from hkp server pool.sks-keyservers.net
gpg: requesting key 8F173680 from hkp server pool.sks-keyservers.net
gpg: requesting key 6521E06D from hkp server pool.sks-keyservers.net
gpg: requesting key C6C11350 from hkp server pool.sks-keyservers.net
gpg: requesting key 99AD6E84 from hkp server pool.sks-keyservers.net
gpg: requesting key 1985A992 from hkp server pool.sks-keyservers.net
gpg: requesting key 3CFD4BB6 from hkp server pool.sks-keyservers.net
gpg: requesting key 983D4366 from hkp server pool.sks-keyservers.net
gpg: requesting key D21E1A55 from hkp server pool.sks-keyservers.net
gpg: requesting key 70E80477 from hkp server pool.sks-keyservers.net
gpg: requesting key AB441196 from hkp server pool.sks-keyservers.net
gpg: requesting key C2E5C0D2 from hkp server pool.sks-keyservers.net

gpgkeys: key A96E87B54F747FAA816C03BC72F05542B005D894 can't be retrieved
gpgkeys: key AB19265E5D7D20687D303246BA1DFB64FFF979E7 can't be retrieved
gpgkeys: key 27FFC4769E19F096D41D9265A04F9397CDFD6BB0 can't be retrieved
gpgkeys: key 44D4A033AC140143927397D47EFD567D4C7EA887 can't be retrieved
gpgkeys: key 0E8B644079F599DFC1DDC3973348882F6AC6A4C2 can't be retrieved
gpgkeys: key 684148BB25B49E986A4944C55184252D824B18E8 can't be retrieved
gpgkeys: key 717026A9D4779FC53940726640F557B731496106 can't be retrieved
gpgkeys: key 6645B0A8C7005E78DB1D7864F99FFE0FEAE999BD can't be retrieved
gpgkeys: key 9437DD3815A7A9169E3D3946AFF5D95098BC6FF5 can't be retrieved
gpgkeys: key 8E1992167465DB5FB045557CB02854ED753E0F1F can't be retrieved
gpgkeys: key 4FCF887689C41B09506BE8D5F3E1D5C5D30DB0AD can't be retrieved
gpgkeys: key ADC8A1FCC15E01D45310419E94657AB20F2A092B can't be retrieved
gpgkeys: key 40776A5221EF5AD468A4906D42A1DB15EC133BAD can't be retrieved
gpgkeys: key 779CD2942629B7FA04AB8F172E89012331361F01 can't be retrieved
gpgkeys: key 962855F072C7A01846405864FCF3C8CB5CF9C8D4 can't be retrieved
gpgkeys: key CE0BDE71A759A87F23F0F7D8B61DBCE10901C163 can't be retrieved
gpgkeys: key 5A2257D19FF7E1E0E415968CE62F853100F0D0F0 can't be retrieved
gpgkeys: key CFA6AF15E5C74149FC1D8C086D1655C14CE1C13E can't be retrieved
gpgkeys: key F3691687D867B81B51CE07D9BBE43771487328A9 can't be retrieved
gpgkeys: key 6EA3F3F3B9082632A9CBE931D53A0445B47A0DAB can't be retrieved
gpgkeys: key 9515D8A8EAB88E49BB65EDBCE6B456CAF15447D5 can't be retrieved
gpgkeys: key A5CA9D5515DC2CA73DF748CA5C2E46A0F53A76ED can't be retrieved
gpgkeys: key 5696C003B0854206450C8E5BE613C09CB4440678 can't be retrieved
gpgkeys: key 40440DC037C05620984379A6761FAD69BA06C6A9 can't be retrieved
gpgkeys: key 0F334D8698881578F65D2AE55ED514A45BD5C938 can't be retrieved
gpgkeys: key 487EACC08557AD082088DABA1EB2638FF56C0C53 can't be retrieved
gpgkeys: key 5559BC1A32B8F76B3FCCD9555FA5E5544F010D48 can't be retrieved
gpgkeys: key 5357F3B111688D88C1D88119FCF2CB179205AC90 can't be retrieved
gpgkeys: key 07DFD3A0BC213FA12EDC217559B3122E2FA915EC can't be retrieved
gpgkeys: key 5E7585ADFF106BFFBBA319DC654B877A0864983E can't be retrieved
gpgkeys: key 86CFFCA918CF3AF47147588051E8B148A9999C34 can't be retrieved
gpgkeys: key B5971F2C5C10A9A08C60030F786C63F330D7CB92 can't be retrieved
gpgkeys: key F648622B1715468FD654F45CB7310AE5F04569AE can't be retrieved
gpgkeys: key 4D913AECD81726D9A6C74F0ADA6426DD215B37AD can't be retrieved
gpgkeys: key 8218F88849AAC522E94CF470A5E9288C4FA415FA can't be retrieved
gpgkeys: key F4DDD6DDCEC320B665F502AAE8F18BA1615137BC can't be retrieved
gpgkeys: key DB2277BCD500AA3825610BDDDB323392796CA067 can't be retrieved
gpgkeys: key E499C79F53C96A54E572FEE1C06086337C50773E can't be retrieved
gpg: no valid OpenPGP data found.

gpg: Total number processed: 0

gpg: keyserver timed out

gpg: keyserver refresh failed: Keyserver error

==> ERROR: A specified local key could not be updated from a keyserver.


Docker Pull Command
Owner
cgal
Source Repository
CGAL/cgal-testsuite-dockerfiles

Missing dependencies versions, in test results

In a result text file like results_lrineau_x86-64_Linux-Fedora19_g++-4.8_m32.txt, generated by autotest_cgal, there are lines starting with -- USING. For example:

CGAL_VERSION 4.7-Ic-9
TESTER lrineau
TESTER_NAME renoir (GF)
TESTER_ADDRESS [email protected]
CGAL_TEST_PLATFORM x86-64_Linux-Fedora19_g++-4.8_m32
General installation log file: ../../installation.log
Host-specific installation log file: ../installation.log
-- USING CMake version: 2.8.12
-- USING GCC_VERSION = '4.8.3 20140911 (Red Hat 4.8.3-7)'
-- USING CXXFLAGS = '-m32 -frounding-math -Wall -O3'
-- USING LDFLAGS = ' '
-- USING GMP_VERSION = '5.1.2'
-- USING MPFR_VERSION = '3.1.2'
-- USING BOOST_VERSION = '1.54.0'
-- USING QT_VERSION = '3.3.8b'
-- USING OPENGL_VERSION = '2.1'
-- USING QT4_VERSION = '4.8.6'
-- USING ZLIB_VERSION = '1.2.8'

If I remember well, those lines are just a result of grep USING over the CMake log. These lines are important, because they are used to fill the first table in the test results.

See https://cgal.geometryfactory.com/CGAL/Members/testsuite/results-4.7-Ic-24.shtml

Cleanup of the testsuite/ and testresults/ directories

With the current version of the script, the sizes of the directories testsuite/ and testresults/ are increasing each day, until the partition is full.

  • upload_results() should remove the file, once it is uploaded.
  • the destructor of Release should remove the file that was downloaded, as well as the untarred release directory.

Install newer versions of CMake on our test images

@maxGimeno

For CGAL/cgal#1436, our testsuite will require:

  • CMake>=3.1 for gcc and clang on Linux,
  • CMake>=3.2 for our usual platforms,
  • CMake>=3.6 for Intel Compiler.

The following should be done:

  • Install CMake-3.1 on Debian Stable
  • Install CMake-3.1 on CentOS-6 base image
  • Install CMake-3.6 on CentOS-7 base image (that also tests the Intel Compiler).

test_cgal starts all containers at once

I have tried test_cgal without arguments (--user and --passwd were in the rc file). The script launched five containers, and they are all running at the same time. Hopefully, they compile without the -j option.

ArchLinux image build issue

@akobel By the way, are you using ArchLinux as your Linux distribution? I don't, and I don't know how to maintain those docker files: I use Fedora/CentOS, and I also know Debian/Ubuntu tools.

Would you volunteer to maintain our ArchLinux images? They were created by Philipp when he was there. I have a build issue with the main ArchLinux image: see this build log on the Docker Hub.

The command is:

pacman -Sy --noconfirm && pacman-key --refresh-keys && pacman -Su --noconfirm && pacman-db-upgrade 

and the error message is:

[...]
downloading which-2.21-2-x86_64.pkg.tar.xz...
checking keyring...
downloading required keys...
:: Import PGP key 2048R/, "Christian Hesse (Arch Linux Package Signing) <[email protected]>", created: 2011-08-12? [Y/n] 
error: key "Christian Hesse (Arch Linux Package Signing) <[email protected]>" could not be imported
:: Import PGP key 4096R/, "Antonio Rojas <[email protected]>", created: 2014-10-21? [Y/n] 
error: key "Antonio Rojas <[email protected]>" could not be imported 
error: required key missing from keyring
error: failed to commit transaction (unexpected error)
Errors occurred, no packages were upgraded. 

See the build log on the Docker Hub for more details.

500 Server Error: Internal Server Error ("invalid mode for volumes-from: rwz") with docker-1.7.1-108.el7.centos.x86_64

The cgal.geometryfactory.com has upgraded Docker to latest version 1.7.1, from RHEL/CentOS.

The script quits as soon as it tries to launch a container:

INFO:root:Extracted release 4.7-Ic-120 is at /path/to/testsuite/CGAL-4.7-Ic-120
INFO:root:Running a maximum of 2 containers in parallel each using 5 CPUs and using 6 jobs
INFO:root:Using /opt/intel/licenses as intel license directory
INFO:root:A non-Exited container with name CGAL-archlinux-testsuite already exists. Forcing exit and removal.
Traceback (most recent call last):
  File "./test_cgal.py", line 299, in <module>
    main()
  File "./test_cgal.py", line 237, in main
    launch_result = scheduler.launch()
  File "/path/to/cgal_docker.py", line 168, in launch
    cont_id = self.runner.run(image_to_launch, self.available_cpusets[-1])
  File "/path/to/cgal_docker.py", line 102, in run
    container_id = self._create_container(image, cpuset)
  File "/path/to/cgal_docker.py", line 140, in _create_container
    host_config=self.host_config
  File "/usr/lib/python2.7/site-packages/docker/client.py", line 239, in create_container
    return self.create_container_from_config(config, name)
  File "/usr/lib/python2.7/site-packages/docker/client.py", line 247, in create_container_from_config
    return self._result(res, True)
  File "/usr/lib/python2.7/site-packages/docker/clientbase.py", line 106, in _result
    self._raise_for_status(response)
  File "/usr/lib/python2.7/site-packages/docker/clientbase.py", line 102, in _raise_for_status
    raise errors.APIError(e, response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("invalid mode for volumes-from: rwz")

It seems that the new syntax is "rw,z". I am investigating...

Add Travis CI support

@maxGimeno

Problems with Docker Hub automatic build

For the moment, for https://github.com/CGAL/cgal-testsuite-dockerfiles we use the Docker service integration, but it is pretty bad.

  • All images are rebuild sequential, but in an order that we do not control. We have images that depend on other images, such as the image build with ArchLinux-CXX14/Dockerfile that depends on the image build with ArchLinux/Dockerfile.

  • Docker service integration does not test pull requests.

  • Docker service integration does not give status feedback to Github.


Solution: Travis CI

That is why I would like that we try Travis CI for CGAL/cgal-testsuite-dockerfiles. Using a [build matrix], we could build several images in parallel. However, a chain of dependent images will have to be build sequentially in the same job.
[build matrix]: https://docs.travis-ci.com/user/customizing-the-build/#Build-Matrix

Additional:

  • Using encrypted secret files to store authentification tokens, we could also deploy the images to the Docker Registry.
  • Using Travis CI, we could even add tests to our building of images. But I do not know what tests we could do.

Update CentOS7-ICC to version Intel® Parallel Studio XE 2017 Update 4

@maxGimeno,

Currently, CentOS-7-ICC/Dockerfile installs Parallel Studio XE 2015 (update 3).

We have decided with Andreas that we will now only test and support the latest version of the Intel Compiler.

The task is to install Parallel Studio XE 2017 Update 4 (and restrict the installation to the compiler tools).

You can get a try license at https://software.intel.com/en-us/intel-parallel-studio-xe/try-buy

There is an installation guide that explains how to install on a machine, and then duplicate the installation on another one using a script:

  • first use on your local machine the --duplicate <file> to store the choice in a text file,
  • then in the Dockerfile, use --silent <file> to re-run the same choices during the Docker build.

Error in the Fedora-Rawhide Dockerfile

Step 2 : RUN yum -y install dnf && yum clean &&     dnf -y update && dnf clean all
Yum command has been deprecated, redirecting to '/usr/bin/dnf -y install dnf'.

See 'man dnf' and 'man yum2dnf' for more information.

To transfer transaction metadata from yum to DNF, run:

'dnf install python-dnf-plugins-extras-migrate && dnf-2 migrate'



Last metadata expiration check performed 0:04:43 ago on Tue Aug  4 08:27:05 2015.
Package dnf-1.0.1-3.fc23.noarch is already installed, skipping.

Dependencies resolved.
Nothing to do.
Complete!
Yum command has been deprecated, redirecting to '/usr/bin/dnf clean'.
See 'man dnf' and 'man yum2dnf' for more information.
To transfer transaction metadata from yum to DNF, run:
'dnf install python-dnf-plugins-extras-migrate && dnf-2 migrate'

The line should be with yum clean **all**.

RUN yum -y install dnf && yum clean all &&     dnf -y update && dnf clean all

Allow to read options from a file

To avoid passing user and password on the command line, I would like that the script can read default options from a config file. argparse can read from file.

http://tricksntweaks.blogspot.be/2013/05/advance-argument-parsing-in-python.html

I suggest that:

  • The script loads and parse a default config file. Its path would be discovered using XDG: something like xdg.BaseDirectory.load_first_config("CGAL") + "/test_cgal_rc" (or something more pythonic to join paths).
  • The script also allow to load config files using the @ prefix.

Another error during parsing of test_cgal_rc

In my test_cgal_rc, if I have --tester lrineau, then the results files are named:

results_['lrineau']_ArchLinux.tar.gz
results_['lrineau']_ArchLinux.txt
results_['lrineau']_CentOS6-32.tar.gz
results_['lrineau']_CentOS6-32.txt
results_['lrineau']_CentOS6.tar.gz
results_['lrineau']_CentOS6.txt
results_['lrineau']_Debian-Stable.tar.gz
results_['lrineau']_Debian-Stable.txt
results_['lrineau']_Debian-Testing.tar.gz
results_['lrineau']_Debian-Testing.txt

The bug is the extra [' and '].

Kill all containers before starting

The test suite process should have an option to kill all existing containers that it has launched. That will be used to ensure that the testsuite of the previous release (of the day before) no longer runs.

The details are to be discussed.

Fancy names for containers

I would like containers to have names, instead of the random-generated one.

I suggest name like: CGAL-version-image (eg: CGAL-4.7-I-16-ArchLinux). Let's discuss about other names: maybe without the CGAL version, as there should not be two versions tested at the same time.

Use scripts to factorize the installation of deps between different Dockerfile

Instead of copy-pasting a long list of yum install ... in all CentOS/Fedora containers, the RUN command could use a script to do the same job. That script could have the logic to detect x86_64/i686, and adapt the yum/dnf command line. That would help the factorization of Dockerfiles.

And same for Ubuntu/Debian, and the various ArchLinux Dockerfiles we may have.

Eventually we could even contribute those scripts to CGAL in a contrib/ sub-directory (or auxliary/?).

Update the ArchLinux base image, to that the last dev branch of CMake is already used

In ArchLinux, they update Boost as soon as it is released. If one uses the CMake script FindBoost with a version of Boost that is not yet handled by CMake, there is a warning:

CMake Warning at /usr/share/cmake-3.7/Modules/FindBoost.cmake:744 (message):
  Imported targets not available for Boost version 106300
Call Stack (most recent call first):
  /usr/share/cmake-3.7/Modules/FindBoost.cmake:848 (_Boost_COMPONENT_DEPENDENCIES)
  /usr/share/cmake-3.7/Modules/FindBoost.cmake:1435 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:31 (find_package)

The task is too:

  1. revert the commits c3d0cef, 2ebf788, and cf02e3b.
  2. make sure the image ArchLinux always use the last available Git version of CMake, from its next branch.

That should ensure that CMake is able to use the version of Boost packaged by Archlinux without any warning... and that will test our CMake scripts with the development version of CMake.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.