cmake-basis / legacy Goto Github PK

A graphical interface which allows the configuration of the automated tests which are implemented by the basistest tools would make the management and configuration of automated tests easier.
Reporter: Andreas Schuh
Begin: 2012-01-10
Completed: 0

These memory leaks are reported by valgrind for the tests parseargs-help, parseargs-helpshort, and parseargs-version, i.e., where the execution is prematurely terminated by TCLAP.

==18575== 8 bytes in 1 blocks are still reachable in loss record 1 of 108
==18575== at 0x4A0695E: operator new(unsigned long) (vg_replace_malloc.c:220)
==18575== by 0x418983: sbia::basis::CmdLine::setup() (in /sbia/comp_space/swtest/testing/Nightly/sbia-cluster-centos5/BASIS-Linux/trunk/Testing/bin/parseargs)
==18575== by 0x419CAF: sbia::basis::CmdLine::CmdLine(std::string const&, std::string const&, std::string const&, std::vector<std::string, std::allocatorstd::string > const&, std::string const&, std::string const&, std::string const&, std::string const&) (in /sbia/comp_space/swtest/testing/Nightly/sbia-cluster-centos5/BASIS-Linux/trunk/Testing/bin/parseargs)
==18575== by 0x408931: main (in /sbia/comp_space/swtest/testing/Nightly/sbia-cluster-centos5/BASIS-Linux/trunk/Testing/bin/parseargs)
==18575==
Reporter: Site Admin
Begin: 2012-01-11
Completed: 0

The basistest-mater script which manages the execution of the automated tests of BASIS projects does not consider the dependencies among the projects which are tested periodically.

How can the dependencies be considered for the automated testing?
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-10
Completed: 0

It can be helpful for debugging only the tests of a particular module if one could enable only the tests of selected modules, not necessarily all enabled modules. Here, the BUILD_TESTING option would act as a "BUILD_ALL_TESTS" option similar to the BUILD_ALL_MODULES. Then, there could be a TEST_ option for each module.
Reporter: Andreas Schuh
Begin: 2012-01-11
Completed: 0

When public header files are configured, the CMake variables as defined at the beginning of the top-level project are dumped and considered only. Instead, when a module's public headers are configured, the CMake variables which are defined after the import of its dependencies and the module's Settings.cmake file should also be considered. This requires the call of basis_configure_public_headers() in each module's basis_project_impl().
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-24
Completed: 0

After finishing the command-line parsing libraries for C++, Python, and BASH, implement one also for Perl development.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-19
Completed: 0

The GeSHi extension for MediaWiki has to be updated. In particular, the lists of BASIS commands should be completed.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-02
Completed: 0

The shflags module is outdated and needs to revised. In particular, the documentation of the file should be adjusted such that the Doxygen filter for shell scripts can process it.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-11
Completed: 0

An implementation of this tool in Python would enable its use on Windows. The current implementation is in BASH and thus restricted to Unix systems.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-02-08
Completed: 0

Especially at the lab where we do have only one license for the MATLAB Compiler, when more than one build is running at the same time, only one may succeed and the others fail because the license is occupied by one build, even if only for a short time.

To avoid the failure of the other build jobs, modify the custom build target to sleep for a short time and then retry building the MCC target. The number of retries shall be configurable via a cached advanced CMake variable.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-22
Completed: 0

Similar to the modules of a BASIS project, should it be possible to package several BASIS projects together and build them in one step using the super-build concept using the ExternalProject.cmake module. The implementation of this super-build shall support the following:

• If the an archive file or a subdirectory with a package's sources is included with the package, use it if not otherwise specified by the user.
• Otherwise, try to find a system installation of the package, in particular if the USE_SYSTEM_ option is set to ON.
• If a package was neither included with the package nor found in the system, download it if a download URL has been specified by the package provider.

In particular, at SBIA the download URLs will point to the lab's web server. This base URL of such downloads shall be the default used by BASIS.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-12
Completed: 0

If the doc target is build by calling ctest_build() a second time in basistest.ctest, the Build.xml file with the output of the actual build step of the software is overwritten! This file would need to be submitted to CDash before.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-27
Completed: 0

The C++ command-line parsing library (based on "TCLAP":http://tclap.sourceforge.net/) is yet only partly implemented.

One of the major features missing is the output of an XML description of the command-line interface.

It must further be revised if all common argument types are supported properly, including an argument with a flag which can either have an argument value or not. For example, "--bias-correct[=suffix]" as used by MICO 1.0.

The use of the library for common programs should be as intuitive as possible.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-13
Completed: 0

The MATLAB Compiler targets are rebuild with every run of the build tool even if the corresponding sources have not been modified since the last build.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-24
Completed: 0

The creation of directories is often needed before output files can be written by a program. In C/C++, this is not trivial to do in a platform independent manner. Hence, add this basic functionality to the path.h module of BASIS.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-17
Completed: 0

Similar to the domains concept of Sphinx, the documentation generator for Python, use domains, i.e., top-level namespaces, for different programming languages such that objects implemented in different languages with the same name can be distinguished. The default domain for Doxygen should be clearly C++.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-13
Completed: 0

The basis_add_test() function should understand generator expressions such as $<TARGET_FILE:tgtname> (see documentation of "add_test":http://www.cmake.org/cmake/help/cmake-2-8-docs.html#command:add_test). As basis_add_test() is using add_test() under the hood, it probably already supports these generator expressions. However, the target names given as argument need to be mapped to target UIDs as used by BASIS.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-25
Completed: 0

At the moment, BASIS supports the addition of pages to the API documentation generated by Doxygen by placing @.dox@ files in the @doc/@ directory. However, Doxygen is already older and the syntax is not as intuitive and powerful as lightweight markup languages such as in particular "Markdown":http://daringfireball.net/projects/markdown/ and "reStructuredText":http://docutils.sourceforge.net/rst.html.

I propose the following feature. Documentation files in the @doc/@ directory in the Markdown format shall be converted to reStructuredText as the Markdown is easier, but less powerful than reStructuredText. On the other side, as long as Markdown is enough for a developer, they may not need to use the a bit more complicated reStructuredText markup at all.

• Files which are in the @doc/user/@ directory shall be converted to a user manual in PDF format.
• Files which are in the @doc/page/@ directory shall be converted to a project web page in HTML.

It shall be possible to easily link the user manual from the project page, maybe even done entirely automatic. The project page can then be uploaded either to our own public web server or a project page on SourceForge (or others). This upload shall be supported by a build target generated by BASIS which can be configured by the developer. Care must be taken regarding authentication at the server, though. This shall be covered by a separate feature request.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-07
Completed: 0

Currently, coverage analysis using the Coverage build configuration is only available for Unix-like systems.
Reporter: Andreas Schuh
Begin: 2012-01-12
Completed: 0

According to a comment of David Cole (6/3/2011) on the CMake mailing list, the compiler flags for coverage builds should be put in the CTest script rather than the CMakeLists.txt files. In this case, consider to change the basistest CTest script to avoid submission to the dashboard to allow developers to still only get local coverage results w/o submission to the dashboard.

This should apply similar to the MemCheck build configuration, which will also allow to choose the best settings for the actually used memory checker.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-23
Completed: 0

Add documentation which describes the purpose and content of the different directories and files of the project template.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-19
Completed: 0

The basistest.ctest script executes the tests even when the build failed.

See for example the Experimental build at https://sbia-portal.uphs.upenn.edu/cdash/index.php?project=MICO&date=2012-01-10.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-10
Completed: 100

This tool would then be used by BASIS to automatically add the full help of the executables as appendix to the user manual. The help of auxiliary executables, on the other side, shall be added to the developer manual (Wiki?).

Alternatively, reStructuredText may be used as output format.
Reporter: Andreas Schuh
Begin: 2012-01-09
Completed: 0

The FindBoost.cmake module which comes with CMake sets the advanced Boost_INCLUDE_DIR and Boost_LIBRARY_DIR variables. However, for the user it would be helpful to show the variable Boost_DIR which is set to the installation prefix of Boost to the CMake cache such that it shows up in the GUI. See FindMOSEK.cmake module of BASIS for reference.
Reporter: Andreas Schuh
Begin: 2012-01-11
Completed: 0

If we decide to implement either Redmine or ChiliProject at SBIA for software project management, consider the implementation of a Redmine/ChiliProject plugin which uses the documentation files of the project in the repository to render HTML pages similar to the gh-pages of GitHub. This feature would be similar to editing the Wiki pages locally and committing them to the repository in order to update the content of the project documentation in Redmine/ChiliProject.
Reporter: Andreas Schuh
Begin: 2012-01-08
Completed: 0

Such tool would enable the execution of the tests in the same way they are invoked by the automated tests, i.e., the @basistest-cron@.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-11
Completed: 0

When embedding the Doxygen documentation using the @Embedded@ plugin of Redmine, the layout of the Doxygen pages is a bit messed up. In particular, lists of hierarchies and the sub-menu items at the top of some pages are not displayed with the right layout.

• Add the ability to specify an additional CSS include file to the header of embedded documents.
• Write a CSS file which fixes the layout of the embedded Doxygen pages.
  Reporter: Andreas Schuh
  Begin: 2012-01-09
  Completed: 0

The basisproject tool uses three-way file merge to merge changes of the project files with those of the possibly newer template files. Therefore, it needs to keep a copy of the original template file in the .basis/ subdirectory of the project. If these files are missing, the basisproject tool should try to recover them if necessary, for example, by simply assuming that the current template file has been the original template file. This would not give the right merge, but at least the update should be feasible for future changes of the template again.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-02-09
Completed: 0

For example, the functions of the path.h module are not assigned to any Doxygen group. They should be part of the C++ utilities group of the BASIS API documentation.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-11
Completed: 0

Currently, CMake uses the default name "Project" at the beginning and writes it also to the cache as the value of CMAKE_PROJECT_NAME.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-11
Completed: 0

Doxygen generates a To-Do page from @todo tags found in the source code documentation.

This feature would add a new issue for each such to-do item if no corresponding issue was found. Otherwise, it does not add another issue. If a to-do item was removed from the Doxygen To-Do page by removing the @todo tag from the source code, the issue related to it shall be set to either resolved or closed. There must be a reliable way to connect issues with items on Doxygen's To-Do page, otherwise it will be impossible to notice that such issue was resolved.

The test if an open issue corresponds to a resolved to-do item must be run automatically, for example, once every day or whenever a change of the Doxygen documentation on disk is recognized using certain Unix filesystem events. Other possibilities to implement this must be investigated.
Reporter: Andreas Schuh
Begin: 2012-01-09
Completed: 0

Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-02-11
Completed: 0

Only the configured public header files in the build tree include/ directory are installed.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-31
Completed: 0

The CTEST_NOTES_FILES variable gives the ability to attach notes to the test result submissions. Of interest may be in particular the details required to re-run the tests with the very same configuration. Therefore, a notes file with the arguments to the basistest CTest script could be helpful.
Reporter: Andreas Schuh
Begin: 2012-01-23
Completed: 0

In particular, the ProjectModularization.dox file has to be completed.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-02-11
Completed: 0

Even if the model is Nightly or Experimental, do not force the test execution if the build failed.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-10
Completed: 0

Currently, the SVN revision of the entire repository is used in PROJECT_VERSION_AND_REVISION. Instead, the SVN revision when the corresponding branch/directory content has been last modified should be used.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-25
Completed: 0

The how-to which is part of the Doxygen generated documentation should be extended to include:

• Details on how to create a project with modules.
• Information on how to resolve merge conflicts.
• Have a reference to the documentation which describes the purpose of the template files.
  Reporter: Andreas Schuh
  Assigned to: Andreas Schuh
  Begin: 2012-02-10
  Completed: 0

The copyright year in the template files cannot be preset. It must be a placeholder which gets replaced by the current year when the project is created using basisproject.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-26
Completed: 0

In case of the Nightly model, even if no files were updated, the tests should be run. I cannot see how this is considered here.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-10
Completed: 0

Here the excerpt from the discussion with Luke (his comments are quoted here) regarding this feature on the "SBIA eLog":https://sbia-infr-vweb.uphs.upenn.edu/elog/Bug+Tracking+and+Feature+Requests+for+Development+Projects/89:

Andreas Schuh replied on Tue Jan 3 11:44:24 2012

1. this one is somewhat minor. Over break I did an svn update on my project then tried to build it. gcc gave me
   the line number and file of the error, I grabbed the filepath from the error message and fixed the problem
   recompiled and everything was fine. The next day i was working on the same project and the error was suddenly
   back, because I had been editing the file in the build tree not the src tree. This is somewhat minor as if i had
   been really thinking I would have noticed it, however it is somewhat annoying particularly for someone who
   doesn't know the project.

I understand this can happen, but this is common for all files in CMake that get configured into the build tree. Of course, when the original file has the
.in suffix and actually changes, it would be more obvious. Still it would not prevent this mistake from happening other than putting a comment at the
top of a .in file stating that this file was generated from a .in file... certainly we don't want to put this comment in a .h file which actually didn't change
and was copied only, though.

2. Module dependency: When you turn of a module, that modules header files remain in the build include, meaning
   that if you've build a module once in a build tree its files are there. this causes some issue with sorting out
   module dependencies. this is also confusing as all the file end up in the final include path
   sbia/prjName/header.h etc.

That is on my to-do list already, to remove the deprecated header files from the build tree again.

3. code coverage: the coverage reports of cdash are linked to the headerfiles in the build tree. thus the
   filenames have no information about what module they are in making finding the file without knowledge of the
   project abit more difficult.

Both, 1) and 3) require a clear understanding of what happens to the public header files in the include/ directory. Once more familiar with BASIS and
given that it is documented and these issues are highlighted in the documentation of BASIS, it should be more obvious.

One way to separate the CDash submissions by module could be to use the subproject feature and the LABELS properties as ITK 4 is doing it on their
Modular Dashboard [1]. Not sure if the support for this by CDash is already matured enough.

Another possibility is to use include paths such as include/sbia/// in the build tree. To get this, you would set the CMake variable
BASIS_USE_MODULE_NAMESPACES to TRUE (see config/Settings.cmake file of the HardiTk).

Overall, I am not feeling to happy about "configuring", i.e., copying, all the header files to the build tree just to have the include path set during
configure time of the project, i.e., to turn include/ into include/sbia//, for example. For files which indeed need to be configured as they use
CMake variables in them and thus use the .in suffix, this is fine and part of the contract with BASIS regarding the meaning of the .in suffix. The nice
thing about it is, that in the project source tree, you don't need to have the as well annoying subdirectory structure include/sbia//. As we
talked about it earlier, there is absolutely no better way than having such include path with only the include/ being in the include search path to avoid
a dependency problem regarding the include statements. In case of the modules, it is even better to not have the subdirectory structure as part of the
module's source tree. If you think about the module as a separate project and would indeed build this module separately, the header files end up in
include/sbia// instead, as now the module is the project. On the other side, even though technically possible, the thing about modules is
that they are tightly coupled to the project they belong to. For the rather loose coupling of projects, we will use the super-build concept. So I guess it
is fine to have a path such as modules/HardiCommon/include/harditk/, i.e., one which includes the name of the project the module belongs to.

All that said, what do you think about having a BASIS CMake variable which can be set in you project's config/Settings.cmake file and is used to specify
which of these implementations a project wants to make use of ? Hence, another way to configure BASIS. For example, let the project developer set
BASIS_USE_SOURCE_INCLUDE_DIRS to either TRUE (no copying of header files to the build tree that don't have the .in suffix) or FALSE (the current
behavior). This would also enable a project to define it's own non-standard include path (although this would violate the standard, but could still be
enforced by checking the path of each header file in the source include/ directory and throwing an error if the standard of include/sbia// is
violated).

[1] http://www.vtk.org/Wiki/ITK_Release_4/Modularization/Modular_Dashboard
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-06
Completed: 0

Currently, memory checks using valgrind can only be run on Unix-like systems.
Reporter: Andreas Schuh
Begin: 2012-01-12
Completed: 0

Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-10
Completed: 0

For example, see "this":https://sbia-portal.uphs.upenn.edu/cdash/viewCoverage.php?buildid=1052&status=2 coverage report of GLISTR. Only a single source file is listed even though this project uses more auxiliary executables written in C++.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-11
Completed: 0

A BASIS project which another BASIS project depends on shall be tested before the other project. Such dependencies are defined in the BasisProject.cmake file. What about making the testing master a bit smarter and have it analyze such dependencies. It can then execute the jobs in order. On the other side, which tests at a particular moment to execute depends also on the current test schedule, i.e., the minimum time that has to pass before the next test job execution. Therefore this approach might be not applicable and the solution implemented as response to #15 might be sufficient.

Other dependencies to deal with are external packages such as Boost, ITK,... in general all packages which are not tested automatically. In this case, the testing master could call a user supplied program with the name of the package and version as arguments. It is then in the responsibility of this script to setup the environment properly. At our lab, this would be the "module load" commands. This would further allow a test to use a different version of a package then the most recent one which has been installed after the last successful test of that package (see #15).

Alternatively (or optionally) add --testcmd option to configuration file which overwrites the default test command given on the command line. This way, it is possible to have for some tests a different test command, e.g., a script that sets up the environment and then executes the default test command. Or, only have a script for the setup of the environment which gets sourced (?). Keep in mind, though, that the testing should also be possible on Windows in the future.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-11
Completed: 0

Before, make sure that the decision for argh is a good one!

Alternatives are the python-gflags library and others. The use of the argh library should, however, be more intuitive.

In any case, provide a command-line parsing solution for Python.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-13
Completed: 0

The arguments which are most often used together with the --args option should become separate options such as

--coverage
--memcheck
--nosubmit
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-25
Completed: 0

After a module was enabled, it's public header files are copied to the binary include directory. Then, when the module is disabled again, these public header files are not removed and hence still available in the build tree, though they should no longer be there.
Reporter: Andreas Schuh
Assigned to: Andreas Schuh
Begin: 2012-01-06
Completed: 0

Currently, Java is not supported directly by BASIS. In particular the basis_add_executable() and basis_add_library() CMake commands should know how to build an executable (executable .jar file, see for example "here":http://www.skylit.com/javamethods/faqs/createjar.html) or library (.jar file) from Java sources.
Reporter: Andreas Schuh
Begin: 2012-01-11
Completed: 0