dev-cafe / autocmake Goto Github PK

View Code? Open in Web Editor NEW

41.0 41.0 18.0 612 KB

CMake plugin composer.

Home Page: http://autocmake.org

License: BSD 3-Clause "New" or "Revised" License

Python 35.33% CMake 60.02% C++ 1.20% Fortran 2.80% Nix 0.65%

cmake

autocmake's People

Contributors

Stargazers

Watchers

Forkers

miroi robertodr susannebenze arnfinn bast brijesh-upadhaya tenomoto externalrepositories stknecht bingbingsuo pmpleader yodamaster wieqli tarotene anotherfoxguy yongganzhao h0nza

autocmake's Issues

Add support for CFLAGS and CXXFLAGS

Better comprehensive testing script...

Hi,
I opening issue concerning the need of more flexible testing script.

i) For all tests, I would need option for selecting compiler(s), math library, integer-mode, linking options (static vs dynamic).

ii) The test script should be able to pick up selected tests only.

iii) Also it would be good to have ctest/CDash option to display results of tests. The CI services are good, though the provide only GNU software. Contrary many machines we are operating have commercial compilers/libraries available.

Even the PR into autocmake should be checked against CDash web results employing commercial compilers/math libraries. Who knows is some good intented change into math_libs.cmake does not hurt my favourite pgf90+MKL-static tests ?

How to deal with autocmake files in user's project ?

Hi,
finally I adapted the autocmake for building my simple (github) project.

Should autocmake-user include all autocmake's files into his project's repository ? What is the recommended approach ? The "cmake/" directory and the setup.py script are more-less stable (as "pure" autocmake stuff), only the CMakeLists.txt undergoes heavy changes.

Because all important files are in the "cmake/" directory, maybe on should version only the "cmake/" folder ?

[email protected]:~/Work/qch/software/autocmake_devel/sec_quant_F90_autocmake/.git status 
# On branch autocmake
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   CMakeLists.txt
#   cmake/
#   setup.py
nothing added to commit but untracked files present (use "git add" to track)
[email protected]:~/Work/qch/software/autocmake_devel/sec_quant_F90_autocmake/.

internal tests

Hi Rado,

please how to make internal tests compiled and ran ? The test.py script somehow does not this. Some documentation would help.

[email protected]:~/Work/qch/software/autocmake_devel/autocmake_miroi_fork/test/.python test.py -h        
[email protected]:~/Work/qch/software/autocmake_devel/autocmake_miroi_fork/test/.python test.py --help
[email protected]:~/Work/qch/software/autocmake_devel/autocmake_miroi_fork/test/.

[email protected]:~/Work/qch/qch_software/autocmake_devel/autocmake/test/.python test.py 
[email protected]:~/Work/qch/qch_software/autocmake_devel/autocmake/test/.ls cxx/
cmake/  src/
[email protected]:~/Work/qch/qch_software/autocmake_devel/autocmake/test/.ls cxx/cmake/
autocmake.cfg

More (free) CI services employed for testing ?

Hi again,

there are some more free CI services we are currently using in DIRAC;

The issue is to have more (different) machines employed for testing of autocmake's content.

Machines with older vs newer architecture, modern vs old-fashioned, virtual vs native, different Linux distros etc. - all these factors will have some impact on the testing commands:

sudo apt-get install g++ cmake gfortran libblas-dev
sudo pip install pytest pep8

Various (different) CI servers are equipped with different software versions. Let us try them.

Comments please ?

Missing documentation...

I am just missing documentation
https://autocmake.readthedocs.org/en/latest/developers/configuration.html
according to comment in autocmake.cfg:

# CMakeLists.txt and setup.py will be generated from this file
# see: http://autocmake.readthedocs.org/en/latest/developers/configuration.html

If setup.py fails to configure, it should print the reason to stderr

Example: https://travis-ci.org/miroi/autocmake/builds/70649300

Problem with quotes in test.py

Hi,

for the thorough testing on CI I need to have possibility to switch GNU math libraries order.

In the case below there are THREEfold quotes. Any help ?

configure_build_and_exe('fc_lapack', 'python setup.py --fc=gfortran --static --cmake-options="-DMATH_LIB_SEARCH_ORDER='ATLAS,SYSTEM_NATIVE,OPENBLAS'"')

Other tests for MPI, OpenMP, GPU...

Would be good to add some simple tests checking OpenMPI, OpenMP, GPU stuff.

Wanted comprehensive stderr catcher in test.py

According to the permission, #33 (comment) ,
we are in need of comprehensive (complex) stderr catcher in test.py which shows up all errors stream - from the test executable, from the setup.py script and also from the test.py script.

Currently the test.py's assert Python function does not point the exact source of failure. This is wanted especially with CI services, where we have no direct access to testing machines and we are left with reading available stderr stream.

(Probably this function can be helpfull, https://pytest.org/latest/capture.html , but one has to check it.)

Windows wget alternative

There is no wget command on MS Windows. Fortunately, there the https://pypi.python.org/pypi/wget project, so it become easy to download file:

pip install wget
python -m wget https://github.com/miroi/autocmake/raw/master/update.py

Would it be OK to use this scheme for autocmake on MS Windows ?

Tests covering more functionalities - fc_openblas test

Greetings,

while adapting autocmake for Windows I see need to have more complex tests, what means tests covering all possible Python/CMake functionalities of the autocmake project.

For example, I would welcome all possible entries into test/fc and test/cc autocmake.cfg files. This is closely related to modifying tests to include math libs (blas/lapack), parallel compilation, Fortran-C-C++ languge mixing etc.

The more autocmake's functionalities we test, the better for us.

Simplify default path in appveyor

Hi,

I am assigning this issue to myself :) . In the appveyor.yml file I have to simplify the %path% setting.

As Ivan suggested, there can be this simple structure below. I have to play to determine which path-segments are necessary for our autocmake appveyor-CI tests.

version: 1.0.{build}
init:
- ps: >-
    cd C:\projects\autocmake
environment:
  line_1: First;Line
  line_2: Second;Line
  line_x: X;Line
  path: 'C:\Program Files (x86)\Git\cmd;%line_1%;%line_2%;%line_x%'
  default: '%path%'
build_script:
- echo %path%
- set path=%path%;next++
- echo %path%
- echo %default%

write out wrong keywords in setup.py

Hi again,

would it be please possible to print out wrong flags for the setup.py script ? In this case below the "--type=Release" is wrong, and I didn't know that. (BTW, what flag is for the release buildup ?)

[email protected]:~/Work/qch/software/autocmake_devel/sec_quant_F90_autocmake/.python setup.py --fc=ifort --mkl=sequential --type=Release --int64 build_intel_i8
Usage:
  ./setup.py [options] [<builddir>]
  ./setup.py (-h | --help)

Unknown CMake command "find_package_handle_standard_args"

This problem with fresh autocmake:

[email protected]:~/Work/qch/software/autocmake_devel/sec_quant_F90_autocmake/.python setup.py --fc=ifort   --type=debug build_ifort_dbg
CC=gcc CXX=g++ FC=ifort cmake -DEXTRA_CFLAGS="''" -DENABLE_CODE_COVERAGE=False -DEXTRA_CXXFLAGS="''" -DEXTRA_FCFLAGS="''" -DENABLE_64BIT_INTEGERS=False -DENABLE_BLAS=auto -DENABLE_LAPACK=auto -DMKL_FLAG=off -DMATH_LIB_SEARCH_ORDER="MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE" -DBLAS_LANG=Fortran -DLAPACK_LANG=Fortran -DENABLE_STATIC_LINKING=False -DCMAKE_BUILD_TYPE=debug /home/milias/Work/qch/software/autocmake_devel/sec_quant_F90_autocmake

-- BLAS will be searched for based on MKLROOT=/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl
CMake Error at cmake/modules/autocmake_math_libs.cmake:284 (find_package_handle_standard_args):
  Unknown CMake command "find_package_handle_standard_args".
Call Stack (most recent call first):
  cmake/modules/autocmake_math_libs.cmake:427 (cache_math_result)
  cmake/modules/autocmake_math_libs.cmake:369 (find_service)
  cmake/modules/autocmake_math_libs.cmake:463 (config_math_service)
  CMakeLists.txt:26 (include)


-- The C compiler identification is GNU 4.4.7
-- The CXX compiler identification is GNU 4.4.7
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- The Fortran compiler identification is Intel
-- Check for working Fortran compiler: /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort
-- Check for working Fortran compiler: /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort supports Fortran 90
-- Checking whether /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort supports Fortran 90 -- yes
-- Math lib search order is MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE
-- Configuring incomplete, errors occurred!
See also "/home/milias/Work/qch/software/autocmake_devel/sec_quant_F90_autocmake/build_ifort_dbg/CMakeFiles/CMakeOutput.log".

Adapting and parsing autocmake.cfg in my new project

Hi,

in my new project, https://github.com/miroi/fortran_input_reader

the autocmake.cfg is
https://github.com/miroi/fortran_input_reader/blob/master/cmake/autocmake.cfg

What is wrong with the parsing ?

[email protected]:~/Work/qch/software/software_projects/fortran_input_reader_devel/fortran_input_reader/cmake/.python update.py ..
- parsing autocmake.cfg
- assembling modules: [                              ] (0/10)Traceback (most recent call last):
  File "update.py", line 464, in <module>
    main(sys.argv)
  File "update.py", line 355, in main
    modules = fetch_modules(config, relative_path)
  File "update.py", line 273, in fetch_modules
    config_docopt, config_define, config_export, config_fetch = parse_cmake_module(f.read())
  File "update.py", line 396, in parse_cmake_module
    autocmake_entry = '\n'.join(s_out).split('autocmake.cfg configuration::')[1]
IndexError: list index out of range

Can I implement a new test, fc_lapack ?

In order to test linking of the lapack library, hereby I am asking for permission to write simple test of linking the lapack library with Fortran program.

Removing .gitignore ?

Hi again,

with the aim to have 'clear place' for repeated autocmake testing would it be possible to remove the .gitignore file ?

The commands git clean -f and git pull would clean the table in existing installation for testing. Otherwise one has to remove the entire directory and do git clone to download the 'virgin' autocmake from repo.

One test for checking math libs internal paralelization

Hi,
hereby I advocate for "bigger" fc_lapack_blas test for checking/employing/measuring the internal OpenMP paralelization of selected math libraries (MKL, ESSL, OpenBLAS, Atlas?, ACML?).

The test's executable would accept one parameter (let say n - size of the matrix problem), the user would set up OpenMP environmental variables for the math library and lauch the process on his machine. Based on measured (printed) timings he would see how good is the math lib parallelization. (In DIRAC we have this with MKL library).

py.test: "Error evaluating 'skipif' expression"

Hi,

probably Python 2.6.6, which is on Slovakia clusters, is not equipped with the 'skipif' feature. Any help, please ?

[email protected]:/shared/home/ilias/Work/programming_stuff/autocmake/.py.test -v test/test.py 
================================================================= test session starts =================================================================
platform linux2 -- Python 2.6.6 -- pytest-2.3.5 -- /usr/bin/python
collected 10 items 

test/test.py:120: test_cxx_custom PASSED
test/test.py:126: test_extra_cmake_options PASSED
test/test.py:132: test_cxx PASSED
test/test.py:138: test_fc PASSED
test/test.py:144: test_fc_git_info PASSED
test/test.py:150: test_fc_int64 PASSED
test/test.py:156: test_fc_mpi ERROR
test/test.py:163: test_fc_omp ERROR
test/test.py:171: test_fc_blas ERROR
test/test.py:178: test_fc_lapack ERROR

======================================================================= ERRORS ========================================================================
____________________________________________________________ ERROR at setup of test_fc_mpi ____________________________________________________________
Error evaluating 'skipif' expression
    False
Failed: expression is not a string

____________________________________________________________ ERROR at setup of test_fc_omp ____________________________________________________________
Error evaluating 'skipif' expression
    False
Failed: expression is not a string

___________________________________________________________ ERROR at setup of test_fc_blas ____________________________________________________________
Error evaluating 'skipif' expression
    False
Failed: expression is not a string

__________________________________________________________ ERROR at setup of test_fc_lapack ___________________________________________________________
Error evaluating 'skipif' expression
    False
Failed: expression is not a string

========================================================= 6 passed, 4 error in 29.38 seconds ==========================================================
[email protected]:/shared/home/ilias/Work/programming_stuff/autocmake/.

Remove "-rdynamic" linking flag for PGI

With Portral Group Fortram, pgf90, autocmake is linking with "-rdynamic" option, which is unwanted:

Linking Fortran executable ../bin/example
cd /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc/build_pgf90/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/example.dir/link.txt --verbose=1
/mnt/apps/pgi/linux86-64/13.10/bin/pgf90       -Mpreprocess -Kieee  -fast -O3 -Mipa=fast CMakeFiles/example.dir/example.f90.o CMakeFiles/example.dir/module.f90.o  -o ../bin/example -rdynamic 
pgf90-Error-Unknown switch: -rdynamic

There is a lot of intenet posts about removing it from cmake, see for example http://www.cmake.org/pipermail/cmake/2009-October/032948.html .

deleting of tests build/ directories

Hi,

when repeating the tests

py.test -v test/test.py

developers would be glad to have build/ directories deleted so that each run of test.py creates fresh builds.

"python update.py .." is overwriting ../CMakeLists.txt

Hi,

the "python update.py .." command is overwriting existing ../CMakeLists.txt buildup receipe.
When I modify autoconf.py and run "update.py ..", I lost the CMakeLists.txt. Could there be this file's backup before updating it ?

better Python version verification in update.py

This is associated with 'older' Python versions which are sometimes default part of Linux installations.

For example, on out IBM Linux cluster we have this problem:

[email protected]:~/work/qch/software/autocmake_development/autocmake/test/fc/cmake/.update.py                    
Traceback (most recent call last):
  File "./update.py", line 5, in <module>
    from collections import OrderedDict, namedtuple
ImportError: cannot import name OrderedDict
[email protected]:~/work/qch/software/autocmake_development/autocmake/test/fc/cmake/.python -V 
Python 2.6.8
[email protected]:~/work/qch/software/autocmake_development/autocmake/test/fc/cmake/.

Could be fix for 2.6.8 Python ?

Static linking

Hereby I am opening this issue to discuss adaptation of autocmake for static_linking, which is crucial for grid-computing.

All autocmake tests should work with this option, IMO. Going to adapt them (they are simpler than DIRAC).

Even I suggest to repeat one-two tests with the "--static" option to check this wanted feature.

Test fc_blas fails

I doubt it ever properly worked on Travis.
Previously it passed trivially until 4d0784d.

Python 2.6.2 and update.py on IBM AIX

Hi,

would it be please possible to adapt update.py for Python version 2.6.2 ? This is IBM AIX machine, and if not, I would have to disturb admin concerning low Python version.

[email protected]:~/work/qch/software/mathlibs-tester/cmake/.uname -a
AIX f01c10n02 1 7 00CFA0664C00 powerpc unknown AIX
[email protected]:~/work/qch/software/mathlibs-tester/cmake/.python -V               
Python 2.6.2
[email protected]:~/work/qch/software/mathlibs-tester/cmake/.python update.py  --self
Traceback (most recent call last):
  File "update.py", line 5, in <module>
    from collections import OrderedDict, namedtuple
ImportError: cannot import name OrderedDict
[email protected]:~/work/qch/software/mathlibs-tester/cmake/.python update.py ..     
Traceback (most recent call last):
  File "update.py", line 5, in <module>
    from collections import OrderedDict, namedtuple
ImportError: cannot import name OrderedDict
[email protected]:~/work/qch/software/mathlibs-tester/cmake/.

More output from cmake

Hi,

would it be possible to have more output from autocmake (through verbose flag) ? I am used to the DIRAC cmake output, containing compiler flags, compiler versions, extra libraries...

[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake/test/fc_blas/.python setup.py --fc=gfortran --blas=auto
FC=gfortran cmake -DEXTRA_FCFLAGS="''" -DENABLE_64BIT_INTEGERS=False -DENABLE_BLAS=auto -DENABLE_LAPACK=auto -DMKL_FLAG=off -DMATH_LIB_SEARCH_ORDER="MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE" -DBLAS_LANG=Fortran -DLAPACK_LANG=Fortran -DCMAKE_BUILD_TYPE=release -G "Unix Makefiles" None /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake/test/fc_blas

-- The C compiler identification is GNU 4.4.7
-- The CXX compiler identification is GNU 4.4.7
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- The Fortran compiler identification is GNU
-- Check for working Fortran compiler: /usr/bin/gfortran
-- Check for working Fortran compiler: /usr/bin/gfortran  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 -- yes
-- BLAS will be searched for based on MKLROOT=/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl
-- Searching for BLAS using search order MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE
-- Found BLAS: MKL (-Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_gf_lp64.so;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_gnu_thread.so;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_core.so;/usr/lib64/libpthread.so;/usr/lib64/libm.so;-fopenmp;-Wl,--end-group)
-- LAPACK will be searched for based on MKLROOT=/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl
-- Searching for LAPACK using search order MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE
-- Found LAPACK: MKL (-Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_lapack95_lp64.a;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_gf_lp64.so;-fopenmp;-Wl,--end-group)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake/test/fc_blas/build

   configure step is done
   now you need to compile the sources:
   $ cd build
   $ make
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake/test/fc_blas/.

Autocmake on Microsoft Windows

Would it be please possible to add the "--generator" flag for the setup.py script ?

With the DIRAC software, we are using the 'setup --generator="MinGW Makefiles" ....' on MS Windows platform together with the (GNU) MinGW64 compilers suite.

Best, M.

URL of update.py in one place ?

Greetings,

the URL for downloading the update.py , https://github.com/scisoft/autocmake/raw/master/update.py , are at four different places in autocmake:

update.py
test/test.py
test/fc/cmake/autocmake.cfg
test/cxx/cmake/autocmake.cfg

All these URLs have to be modified in when switching to autocmake forks, for example to https://github.com/miroi/autocmake/raw/master/update.py .

Could we have only one place in the code with this important URL ? The best solution would be separate file (I don't know if it would work for autocmake.cfg).

IBM Linux: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

On the "Linux login 3.0.76-0.11-ppc64 #1 SMP Fri Jun 14 08:21:43 UTC 2013 (ccab990) ppc64 ppc64 ppc64 GNU/Linux" , what is IBM machine with Linux system, there is something wrong with the connection (see below).

I am opening issue for that as we want to have autocmake running on all platforms, including IBM.

[email protected]:~/work/programming/autocmake_devel/autocmake/.py.test -v test/test.py 
========================================================== test session starts ===========================================================
platform linux2 -- Python 2.7.9 -- py-1.4.30 -- pytest-2.7.2 -- /gpfs/home/utils/LINUX/python-2.7.9/bin/python
rootdir: /gpfs/home/ilias/work/programming/autocmake_devel/autocmake/test, inifile: 
collected 4 items 

test/test.py::test_cxx FAILED
test/test.py::test_cxx_auto FAILED
test/test.py::test_fc FAILED
test/test.py::test_fc_blas FAILED

================================================================ FAILURES ================================================================
________________________________________________________________ test_cxx ________________________________________________________________

    def test_cxx():
>       stdout, stderr = configure_build_and_exe('cxx', 'python setup.py --cxx=g++')

../../test.py:111: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../test.py:85: in configure_build_and_exe
    dst='lib/docopt.py')
../../test.py:45: in fetch_url
    opener.retrieve(src, dst)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:245: in retrieve
    fp = self.open(url, data)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:213: in open
    return getattr(self, name)(url)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:443: in open_https
    h.endheaders(data)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:997: in endheaders
    self._send_output(message_body)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:850: in _send_output
    self.send(msg)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:812: in send
    self.connect()
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:1212: in connect
    server_hostname=server_hostname)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:350: in wrap_socket
    _context=self)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:566: in __init__
    self.do_handshake()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ssl.SSLSocket object at 0xfffae348758>, block = False

    def do_handshake(self, block=False):
        """Perform a TLS/SSL handshake."""
        self._check_connected()
        timeout = self.gettimeout()
        try:
            if timeout == 0.0 and block:
                self.settimeout(None)
>           self._sslobj.do_handshake()
E           IOError: [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:788: IOError
_____________________________________________________________ test_cxx_auto ______________________________________________________________

    def test_cxx_auto():
>       stdout, stderr = configure_build_and_exe('cxx_auto', 'python setup.py --cxx=g++')

../../test.py:118: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../test.py:85: in configure_build_and_exe
    dst='lib/docopt.py')
../../test.py:45: in fetch_url
    opener.retrieve(src, dst)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:245: in retrieve
    fp = self.open(url, data)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:213: in open
    return getattr(self, name)(url)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:443: in open_https
    h.endheaders(data)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:997: in endheaders
    self._send_output(message_body)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:850: in _send_output
    self.send(msg)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:812: in send
    self.connect()
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:1212: in connect
    server_hostname=server_hostname)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:350: in wrap_socket
    _context=self)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:566: in __init__
    self.do_handshake()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ssl.SSLSocket object at 0xfffae0edc08>, block = False

    def do_handshake(self, block=False):
        """Perform a TLS/SSL handshake."""
        self._check_connected()
        timeout = self.gettimeout()
        try:
            if timeout == 0.0 and block:
                self.settimeout(None)
>           self._sslobj.do_handshake()
E           IOError: [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:788: IOError
________________________________________________________________ test_fc _________________________________________________________________

    def test_fc():
>       stdout, stderr = configure_build_and_exe('fc', 'python setup.py --fc=gfortran')

../../test.py:125: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../test.py:85: in configure_build_and_exe
    dst='lib/docopt.py')
../../test.py:45: in fetch_url
    opener.retrieve(src, dst)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:245: in retrieve
    fp = self.open(url, data)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:213: in open
    return getattr(self, name)(url)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:443: in open_https
    h.endheaders(data)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:997: in endheaders
    self._send_output(message_body)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:850: in _send_output
    self.send(msg)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:812: in send
    self.connect()
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:1212: in connect
    server_hostname=server_hostname)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:350: in wrap_socket
    _context=self)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:566: in __init__
    self.do_handshake()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ssl.SSLSocket object at 0xfffae0edde8>, block = False

    def do_handshake(self, block=False):
        """Perform a TLS/SSL handshake."""
        self._check_connected()
        timeout = self.gettimeout()
        try:
            if timeout == 0.0 and block:
                self.settimeout(None)
>           self._sslobj.do_handshake()
E           IOError: [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:788: IOError
______________________________________________________________ test_fc_blas ______________________________________________________________

    def test_fc_blas():
        if sys.platform != 'win32':
>           stdout, stderr = configure_build_and_exe('fc_blas', 'python setup.py --fc=gfortran --blas=auto')

../../test.py:133: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../test.py:85: in configure_build_and_exe
    dst='lib/docopt.py')
../../test.py:45: in fetch_url
    opener.retrieve(src, dst)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:245: in retrieve
    fp = self.open(url, data)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:213: in open
    return getattr(self, name)(url)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/urllib.py:443: in open_https
    h.endheaders(data)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:997: in endheaders
    self._send_output(message_body)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:850: in _send_output
    self.send(msg)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:812: in send
    self.connect()
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/httplib.py:1212: in connect
    server_hostname=server_hostname)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:350: in wrap_socket
    _context=self)
/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:566: in __init__
    self.do_handshake()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ssl.SSLSocket object at 0xfffae0edd70>, block = False

    def do_handshake(self, block=False):
        """Perform a TLS/SSL handshake."""
        self._check_connected()
        timeout = self.gettimeout()
        try:
            if timeout == 0.0 and block:
                self.settimeout(None)
>           self._sslobj.do_handshake()
E           IOError: [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

/gpfs/home/utils/LINUX/python-2.7.9/lib/python2.7/ssl.py:788: IOError
======================================================== 4 failed in 2.71 seconds ========================================================
[email protected]:~/work/programming/autocmake_devel/autocmake/.

Module build_info.cmake is untested and most probably broken

Minimal Python version for update.py

Hi,

Ad: https://github.com/scisoft/autocmake/blob/master/update.py#L8

would it be please possible to add the detection of a minimal Python version ?

[email protected]:/Work/qch/software/relat_rad_atomic_ihfscc/automake_ihfscc/cmake/python update.py --self
Traceback (most recent call last):
File "update.py", line 8, in
if sys.version_info.major > 2:
AttributeError: 'tuple' object has no attribute 'major'
[email protected]:/Work/qch/software/relat_rad_atomic_ihfscc/automake_ihfscc/cmake/.python --version
Python 2.6.6

TUTORIAL: Adapting MS-MPI for MinGW64

MS-MPI with the MinGW64 gfortran

Hereby I am creating this thread in order to publicly demonstrate the adaptation of the MS-MPI suite for the MinGW64 gfortran.

Tested on 64-bit MS Windows 8.0; MinGW-W64 project 4.9.0.

Download and unpack the MS-MPI package (this is v4) . Do not install it otherwise you may face error as this one. All necessary files are unpacked from the mpi_x64.Msi archive and are in the corresponding directory !

C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>"C:\Program Files (x86)\7-Zip\7z.exe" x mpi_x64.Msi

7-Zip 9.22 beta  Copyright (c) 1999-2011 Igor Pavlov  2011-04-18

Processing archive: mpi_x64.Msi

Extracting  etl2clog.exe
Extracting  etl2otf.exe
Extracting  mpi.f90
Extracting  mpi.h
Extracting  mpicsync.exe
Extracting  mpiexec.exe
Extracting  mpif.h
Extracting  mpio.h
Extracting  mpitrace.man
Extracting  msmpi.dll
Extracting  msmpi.lib
Extracting  msmpi64.dll
Extracting  msmpi64.lib
Extracting  msmpifec.lib
Extracting  msmpifec64.lib
Extracting  msmpifes.lib
Extracting  msmpifmc.lib
Extracting  msmpifmc64.lib
Extracting  msmpifms.lib
Extracting  msmpires.dll
Extracting  msmpires64.dll
Extracting  note_mpi.txt
Extracting  pmidbg.h
Extracting  smpd.exe

Everything is Ok

Files: 24
Size:       9720065
Compressed: 3613184

In the MS-MPI directory, create the libmsmpi64.a library with the MinGW64 tools gendef and dlltool:

C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>gendef msmpi64.dll
 * [msmpi64.dll] Found PE image
C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>dlltool -d msmpi64.def -l libmsmpi64.a -D msmpi64.dll

In the MS-MPI folder, rename mpi.f90 to mpi.F90 and modify its source code so that the gfortran compiler can accept traditional (C) preprocessor directives (changes are starting from line 344, see also Attachement 1) :
generate the Fortran90 mpi.mod file - you must use appropriate preprocessor statements and no-range checking (-D_WIN64 -D INT_PTR_KIND()=8 -fno-range-check)

C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>gfortran  -c  -D_WIN64  -D INT_PTR_KIND()=8 -fno-range-check   mpi.F90

compile and run the MPI example.F90 - see also the Attachement 3 - using the modified Fortran90 module mpi:

C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>gfortran  -o example.exe  -D USE_MPI_MODULE example.F90 libmsmpi64.a
C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>mpiexec.exe -n 2 example.exe
 PASSED

In the MS-MPI folder, modify the mpif.h include file so that Intel preprocessor directives are replaced by the traditional ones (changes starting from line 341), see the Attachement 2
now compile and run the MPI example.F90 file - see also the Attachement 3 - by using the modified include file "mpif.h" (use -D_WIN64 -D INT_PTR_KIND()=8 -fno-range-check flags as in the previous case):

C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>gfortran -o example.exe  -D_WIN64  -D INT_PTR_KIND()=8  -fno-range-check  example.F90   libmsmpi64.a
C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>mpiexec -n 2 example.exe
 PASSED

Note that you have to link the above generated libmsmpi64.a library. If you would link against the ms-mpi provided dynamical library, msmpi64.dll , you would get the error message of Fatal protocol error: check version between Mpiexec.exe, Msmpi.dll, and Smpd.exe.

Likewise I stress that linking against the delivered lib-file msmpi64.lib does not work, you must have your own prepared libmsmpi64.a library, as explained above:

C:\Users\milias\Documents\Dirac\software\autocmake-devel\ms-mpi>gfortran  -o example.exe  -D USE_MPI_MODULE example.F90 msmpi64.lib
C:\Users\milias\AppData\Local\Temp\ccmTxg9d.o:example.F90:(.text+0x1b): undefined reference to `mpi_init_'
C:\Users\milias\AppData\Local\Temp\ccmTxg9d.o:example.F90:(.text+0x3b): undefined reference to `mpi_comm_rank_'
C:\Users\milias\AppData\Local\Temp\ccmTxg9d.o:example.F90:(.text+0x5b): undefined reference to `mpi_comm_size_'
C:\Users\milias\AppData\Local\Temp\ccmTxg9d.o:example.F90:(.text+0x6a): undefined reference to `mpi_finalize_'
C:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\milias\AppData\Local\Temp\ccmTxg9d.o: bad reloc address 0x0 in section `.pdata'
collect2.exe: error: ld returned 1 exit status

Attachement 1 - (mpi.f90 -> mpi.F90), modified lines

Attachement 2 - mpif.h, modified lines

Attachement 3 - example.F90

Missing working ctest example

As I am trying to adapt tests for https://github.com/miroi/mathlibs-tester, I am missing working ctest-autocmake example.

Is it worth for consideration ?

More tests combinations

Hi,
would it be worth to set up more blas/lapack tests combinations for CI service(s) ?
This is gfortran <> system_native/atlas/openblas/acml <> dynamic/static (the free of charge acml library is available after downloading by the yml-script).

Here I have tests combinations in my new project, which are also on CDash:
https://github.com/miroi/mathlibs-tester/blob/master/cdash_scripts/hpcc_umb_runtest.bash

This task would require IMO adding next CI service into autocmake. The best one, as I checked, seems to be the circleci, which is also very fast. The main reason is that each CI service is operating on different virtual Linux architecture, and there might be some conflicts in packages on older architectures (like I found with travis-ci #51 (comment) ). In such a case to enable to pass tests, the defined MATH_LIB_SEARCH_ORDER would ensure working library linked. For example, for the MATH_LIB_SEARCH_ORDER='OPENBLAS;ATLAS;SYSTEM_NATIVE;ACML' setting the tests will pass at least with one math-library package installed.

One would have to play with Linux packages for (OPENBLAS;ATLAS;SYSTEM_NATIVE;ACML)-dynamic/static linking combinations to settle all lapack/blas tests properly in travis-ci and circle-ci. The number of unique compiler-library-linking tests should be 16 (8 for blas,8 for lapack), without integer*8. However, if some modern free library would provide ILP64 data model, then the total number of tests will be higher.

PS1: The ACML library has 64-bit integer mode, see http://developer.amd.com/tools-and-sdks/archive/amd-core-math-library-acml/acml-downloads-resources/ . This means 4 more combinations.

PS2: I am thinking now(17.8.2015) of more CI tests which would skip the py.test test/test.py layer and do own buildup on selected tests, mainly math tests with various libraries. But if all such above mentioned combinations could be done safely through the test.py script, than there is no need for such solution.

Test fc_blas not linking MKL library

On my Linux machine with Intel/MKL installed it founds MKL as the first library:

[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/.setup.py --fc=ifort --int64  build_ifort_i8_mkl
FC=ifort cmake -DEXTRA_FCFLAGS="''" -DENABLE_64BIT_INTEGERS=True -DENABLE_BLAS=auto -DENABLE_LAPACK=auto -DMKL_FLAG=off -DMATH_LIB_SEARCH_ORDER="MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE" -DBLAS_LANG=Fortran -DLAPACK_LANG=Fortran -DCMAKE_BUILD_TYPE=release -G "Unix Makefiles" None /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas

-- The C compiler identification is GNU 4.4.7
-- The CXX compiler identification is GNU 4.4.7
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- The Fortran compiler identification is Intel
-- Check for working Fortran compiler: /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort
-- Check for working Fortran compiler: /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort supports Fortran 90
-- Checking whether /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort supports Fortran 90 -- yes
-- BLAS will be searched for based on MKLROOT=/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl
-- Searching for BLAS using search order MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE
-- Found BLAS: MKL (-Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_ilp64.so;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_thread.so;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_core.so;/usr/lib64/libpthread.so;/usr/lib64/libm.so;-openmp;-Wl,--end-group)
-- LAPACK will be searched for based on MKLROOT=/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl
-- Searching for LAPACK using search order MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE
-- Found LAPACK: MKL (-Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_lapack95_ilp64.a;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_ilp64.so;-openmp;-Wl,--end-group)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl

   configure step is done
   now you need to compile the sources:
   $ cd build_ifort_i8_mkl
   $ make

but it does not link it:

[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/.make VERBOSE=1
/usr/bin/cmake -H/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas -B/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/CMakeFiles /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory `/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl'
make -f src/CMakeFiles/example.dir/build.make src/CMakeFiles/example.dir/depend
make[2]: Entering directory `/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl'
cd /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/src /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/src /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/src/CMakeFiles/example.dir/DependInfo.cmake --color=
make[2]: Leaving directory `/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl'
make -f src/CMakeFiles/example.dir/build.make src/CMakeFiles/example.dir/requires
make[2]: Entering directory `/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl'
make[2]: Nothing to be done for `src/CMakeFiles/example.dir/requires'.
make[2]: Leaving directory `/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl'
make -f src/CMakeFiles/example.dir/build.make src/CMakeFiles/example.dir/build
make[2]: Entering directory `/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl'
/usr/bin/cmake -E cmake_progress_report /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/CMakeFiles 1
[100%] Building Fortran object src/CMakeFiles/example.dir/example.f90.o
cd /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/src && /mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort  -DHAVE_MKL_BLAS -DHAVE_MKL_LAPACK -i8 -O3 -module ../modules -I/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/modules    -c /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/src/example.f90 -o CMakeFiles/example.dir/example.f90.o
Linking Fortran executable ../bin/example
cd /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/example.dir/link.txt --verbose=1
/mnt/apps/intel/composer_xe_2013_sp1.1.106/bin/intel64/ifort       -i8 -O3 CMakeFiles/example.dir/example.f90.o  -o ../bin/example -i_dynamic -lblas 
make[2]: Leaving directory `/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl'
/usr/bin/cmake -E cmake_progress_report /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/CMakeFiles  1
[100%] Built target example
make[1]: Leaving directory `/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl'
/usr/bin/cmake -E cmake_progress_start /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/CMakeFiles 0
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/.

what is linked is the GNU blas (-lblas ), see also

[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/.ldd bin/example 
        linux-vdso.so.1 =>  (0x00007fff2d72f000)
        libblas.so.3 => /usr/lib64/libblas.so.3 (0x00007ffd9037f000)
        libifport.so.5 => /mnt/apps/intel/composer_xe_2013_sp1.1.106/compiler/lib/intel64/libifport.so.5 (0x00007ffd9014f000)
        libifcore.so.5 => /mnt/apps/intel/composer_xe_2013_sp1.1.106/compiler/lib/intel64/libifcore.so.5 (0x00007ffd8fe0f000)
        libimf.so => /mnt/apps/intel/composer_xe_2013_sp1.1.106/compiler/lib/intel64/libimf.so (0x00007ffd8f948000)
        libsvml.so => /mnt/apps/intel/composer_xe_2013_sp1.1.106/compiler/lib/intel64/libsvml.so (0x00007ffd8ed50000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ffd8eacc000)
        libintlc.so.5 => /mnt/apps/intel/composer_xe_2013_sp1.1.106/compiler/lib/intel64/libintlc.so.5 (0x00007ffd8e876000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ffd8e658000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ffd8e2c4000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ffd8e0ae000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007ffd8dea9000)
        libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007ffd8dbb7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ffd905ef000)
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build_ifort_i8_mkl/.

Detection of openblas on both Linux and MS Windows (as libopenblas.dll)

Hi,

concerning the current openblas support we have:
https://github.com/scisoft/autocmake/blob/master/modules/math_libs.cmake#L115

we are missing the Windows OpenBLAS support, which is described here: OpenMathLib/OpenBLAS#461 (comment)

According to this thread it would be good to incorporate OpenBLASConfig.cmake commands into math_libs.cmake to enable linking against "libopenblas.dll".

For example, passing (and linking) of openblas library works for DIRAC on MS Windows (https://gitlab.com/dirac/dirac/blob/master/maintenance/cdash/cdash.WinS12-Miro.bat#L67, see also https://gitlab.com/dirac/dirac/blob/master/cmake/math/MathLibs.cmake#L80 ).

Windows OpenBLAS version (libopenblas.dll) is needed for the new fc_openblas test I am currently working on.

Avoid boilerplate in autocmake.cfg

For many projects also autocmake.cfg contains
boilerplate. Consider implementing shortcuts with the option
to customize rather than forcing to handcraft the entire file.

Consider splitting math_libs.cmake into modules supporting one library only.

The module math_libs.cmake is monolithic and relatively convenient but rather complex,
difficult to extend and understand, in particular for non-experts in CMake.
I would like to discuss here the possibility to split it into separate modules (one for system native BLAS/LAPACK, one for Atlas, one for MKL, one for ESSL, one for OpenBLAS, etc.).
This could simplify the CMake code and make the modules behave more predictably.
Currently the behavior can be a bit opaque.

switch off email notification in travis-ci

Could you please switch off email notification for travis-ci ? I am flooded with email, it's sufficient to check tests status on the web.

http://docs.travis-ci.com/user/notifications/#Email-notifications

Few modules are undocumented

Close this issue once all modules carry their documentation
which is automatically aggregated in the module reference.

MS Windows Server 2012: IOError: [Errno socket error] [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:590)

Hi,

I am opening separate issue for this task of adapting the core python script - update.py - for the MS Windows platform.

Currently it is throwing this error below. Any help is appreciated.

C:\Users\milias\Documents\work\software\autocmake\autocmake\test\fc\cmake>python update.py  --self
- fetching lib/config.py
Traceback (most recent call last):
  File "update.py", line 308, in <module>
    main(sys.argv)
  File "update.py", line 264, in main
    dst='lib/config.py'
  File "update.py", line 42, in fetch_url
    opener.retrieve(src, dst)
  File "C:\Python27\lib\urllib.py", line 245, in retrieve
    fp = self.open(url, data)
  File "C:\Python27\lib\urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "C:\Python27\lib\urllib.py", line 443, in open_https
    h.endheaders(data)
  File "C:\Python27\lib\httplib.py", line 997, in endheaders
    self._send_output(message_body)
  File "C:\Python27\lib\httplib.py", line 850, in _send_output
    self.send(msg)
  File "C:\Python27\lib\httplib.py", line 812, in send
    self.connect()
  File "C:\Python27\lib\httplib.py", line 1216, in connect
    server_hostname=server_hostname)
  File "C:\Python27\lib\ssl.py", line 350, in wrap_socket
    _context=self)
  File "C:\Python27\lib\ssl.py", line 566, in __init__
    self.do_handshake()
  File "C:\Python27\lib\ssl.py", line 788, in do_handshake
    self._sslobj.do_handshake()
IOError: [Errno socket error] [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:581)

C:\Users\milias\Documents\work\software\autocmake\autocmake\test\fc\cmake>

Suggest to create or fetch .gitignore that ignores *.pyc under cmake/

Otherwise codes which do not ingore *.pyc might start tracking it (have seen this happen).

How to assign user-selected math library ? Test for EXPLICIT_LIBS ?

On my system I have GNU blas as well as MKL-blas. By default is chooses MKL, but I want to force it to use other library.

The -blas="/usr/lib64/libblas.a" is not working. I haven't found in the manual how to do it. Any help ?

[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake/test/fc_blas/.python setup.py --fc=gfortran --blas="/u
sr/lib64/libblas.a"   build_gfortran2
FC=gfortran cmake -DEXTRA_FCFLAGS="''" -DENABLE_64BIT_INTEGERS=False -DENABLE_BLAS=/usr/lib64/libblas.a -DENABLE_LAPACK=auto -DMKL_FLAG=off -DMATH_LIB_SEARCH_ORDER="MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE" -DBLAS_LANG=Fortran -DLAPACK_LANG=Fortran -DCMAKE_BUILD_TYPE=release -G "Unix Makefiles" None /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake/test/fc_blas

CMake Error at src/CMakeLists.txt:5 (message):
  BLAS library not found for the test fc_blas!


-- The C compiler identification is GNU 4.4.7
-- The CXX compiler identification is GNU 4.4.7
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- The Fortran compiler identification is GNU
-- Check for working Fortran compiler: /usr/bin/gfortran
-- Check for working Fortran compiler: /usr/bin/gfortran  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 -- yes
-- LAPACK will be searched for based on MKLROOT=/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl
-- Searching for LAPACK using search order MKL;ESSL;ATLAS;ACML;SYSTEM_NATIVE
-- Found LAPACK: MKL (-Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_lapack95_lp64.a;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_gf_lp64.so;-fopenmp;-Wl,--end-group)
-- Configuring incomplete, errors occurred!
See also "/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake/test/fc_blas/build_gfortran2/CMakeFiles/CMakeOutput.log".

[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake/test/fc_blas/.~~~

Missing set the cmake command flag

Hi,

on one of our SIVVP.sk clusters (more precisely, it's hpc.uniza.sk) we have cmake 2.6 as the default (/usr/bin/cmake); and the newer cmake is called as /usr/bin/cmake28.

Autocmake can not take the /usr/bin/cmake28, and the default cmake of version 2.6 is not acceptable.

I am missing this flag which is present in DIRAC setup:

  --cmake STRING        set the cmake command [default: cmake; e.g. --cmake
                        cmake28]

Any help, plase ?

Avoid code duplication with local CMake modules

Currently all CMake modules (local or remote) are downloaded or copied to the modules/ folder.
For local modules this leads to code duplication and possible confusion about which to edit.
It also requires to rerun update.py after editing local modules. Therefore consider to not copy local modules but rather source them directly.

--int64 ?

Should we include testing of 8bytes integers in relevant tests?

cmake .. returns error of not found libraries

After succesfull tests it fc_blas/fc_lapack the cmake .. returns error of not found libraries:

[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/.py.test -v test/test.py 
============================================================== test session starts ===============================================================
platform linux2 -- Python 2.6.6 -- py-1.4.28 -- pytest-2.7.1 -- /usr/bin/python
rootdir: /home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test, inifile: 
collected 14 items 

test/test.py::test_cxx_custom PASSED
test/test.py::test_extra_cmake_options PASSED
test/test.py::test_cxx PASSED
test/test.py::test_cxx_static PASSED
test/test.py::test_fc PASSED
test/test.py::test_fc_git_info PASSED
test/test.py::test_fc_int64 PASSED
test/test.py::test_fc_mpi PASSED
test/test.py::test_fc_omp PASSED
test/test.py::test_fc_static PASSED
test/test.py::test_fc_blas PASSED
test/test.py::test_fc_blas_static PASSED
test/test.py::test_fc_lapack PASSED
test/test.py::test_fc_lapack_static PASSED

=========================================================== 14 passed in 70.00 seconds ===========================================================
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/.cd test/fc_blas/
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/.ls
CMakeLists.txt  build-2015-08-10-17-26-38/  build-2015-08-10-17-26-43/  cmake/  setup.py*  src/
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/.cd build-2015-08-10-17-26-43/
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build-2015-08-10-17-26-43/.m
[100%] Built target example
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build-2015-08-10-17-26-43/.cmake ..
CMake Error at src/CMakeLists.txt:16 (message):
  BLAS library not found for the test fc_blas!


-- Configuring incomplete, errors occurred!
See also "/home/milias/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build-2015-08-10-17-26-43/CMakeFiles/CMakeOutput.log".
[email protected]:~/Work/qch/software/software_projects/autocmake_devel/autocmake_miroi/test/fc_blas/build-2015-08-10-17-26-43/.

setup,py script should be executable

This is because many people are used to execute ./setup [--flags] in Dalton and DIRAC projects.

dev-cafe / autocmake Goto Github PK

autocmake's People

Contributors

Stargazers

Watchers

Forkers

autocmake's Issues

MS-MPI with the MinGW64 gfortran

Attachement 1 - (mpi.f90 -> mpi.F90), modified lines

Attachement 2 - mpif.h, modified lines

Attachement 3 - example.F90

Recommend Projects

Recommend Topics

Recommend Org

Jobs