Add a entry point in setup.py to read ekstazi.json file and show the dependency matrix. The entry point might be ekstazi-depdency-matrix.

Read the results of the last test execution, calculate the hash of dependencies again, check which dependencies were changed and return which test cases have to run.

The project should be created under tests directory.

Requirements:

• Create two test files with some test cases
  • The test cases must cover all files in the project
• The project needs to have 5 files following the below depedency tree. By following that tree, we can test our Ekstazi algorithm in any sort of project structure.
  

Let's define in the comments of this thread, what kind of project we may do.

If the test cases have been updated, the plugin should select them even the source code was not changed.

• Create the files for Python package
• Implement a wrapper for the test case functions to trace function calls, like made in: #3

We need save the last test results (PASS, FAIL, SKIP). Having them we can select the tests cases to run based on previous results.

See if we can do the same as original Ekstazi do in Java, observing the files that are being used in runtime. If we can't do that, our next approach would be read the test files before running the test suite, and check import files and their usage in the test cases.

The next steps depends on the success of the research, we would create a function to get dependecies of each test case in a test file, or create a wrapper to execute the tests and collect their dependencies.

Ekstazi

Ekstazi is a Regression Test Selection algorithm. The best scenario is always run all test cases, but this takes time. This is where Ekstazi comes in, the algorithm selects the test cases based on the last changes on the project.

Overview

See below the main characteristics of Ekstazi:

• Ekstazi is algorithm for non-blackbox tests. So we need the source code to use it.
• Original Ekstazi doesn't use version control system like Git or SVN.
• Ekstazi figures out what project files each test case "touch", in runtime. It saves the depedencies of each test.
• Ekstazi calculates the hash of the content of each project file "touched" in the test cases to identify changes in the files.
• Based on previous testing round and last project changes, Ekstazi selects just test cases that contains features update.
• Original Ekstazi was made in Java. Its approach is change the JVM to figure out the dependencies of the test cases.

TODO: Move this to a Wiki page.

The results include:

• Hash of the content of each dependency file
• The test cases location, and the dependecy matrix. The depedency matrix will tell us what files each test cases covers.

Let's save those informations in a JSON file.

Calculate the hash of the files containing the fixtures used by the test methods

If a new test case is created, the plugin selects all the test cases inside the file that the test has been created.

#2 creates a project to be used for testing. We need to create a test suite to use the plugin on that project.

Run the plugin on a huge Python project with the plugin enabled and disabled. Both test sessions must have the same test cases.

If a test file contains some variable used by some test case in the file, changing the variable doesn't affect the hash value of the test.