easy-software-ufal / mutation-tool-for-annotations

Mutation Tool for Annotations

This a lib to mutate code annotations from Java or attributes from C# source code. From a set of operators, this will create one javaMutant for each change on original code, related to code annotations. For example, consider the following code (Java):

@RequestMapping(value = "/foo")
public String form() {
  return "form";
}

The ADAT (Add Attribute) operator can create the following javaMutants:

@RequestMapping(value = "/foo", method = RequestMethod.POST)
public String form() {
  return "form";
}

@RequestMapping(value = "/foo", method = RequestMethod.GET)
public String form() {
  return "form";
}

With these javaMutants, the developer can evaluate your test suite. The ultimate goal is kill these javaMutants with tests, improving the test suite. This project was based on theory of Mutation Testing. For more details, see An Analysis and Survey of the Development of Mutation Testing and Mutation Operators for Code Annotations.

Operators

This project have a set of 9 operators:

Operator	Description
ADA	Add annotation
ADAT	Add attributes to annotation
CHODR	Change order of annotations
RMA	Remove annotations
RMAT	Remove attributes of annotations
RPA	Replace one annotation for another
RPAT	Replace one attribute for another
RPAV	Replace one attribute value for another
SWTG	Change location of one annotation

To use this project, it is necessary to choose one or more of these operators.

Configuration

You will need the srcML tool: https://www.srcml.org/. Install on your OS.

First, create a Maven project with java.

On pom.xml, Add the Jitpack repository:

<repositories>
  <repository>
    <id>jitpack.io</id>
    <url>https://jitpack.io</url>
  </repository>
</repositories>

And the Dependency:

<dependencies>
  <dependency>
    <groupId>com.github.easy-software-ufal</groupId>
    <artifactId>mutation-tool-for-annotations</artifactId>
    <version>Tag</version>
  </dependency>
</dependencies>

Replace the Tag with target release.

How to use

First you need to fill the annotations.json of the project (located on the config folder). This is a configuration file, with information of annotations. The following code is a example of this file:

{
  "annotations": [
    {
      "annotation": "@org.springframework.web.bind.annotation.RequestMapping",
      "replaceableBy": [],
      "targets": ["type", "method"],
      "attributes": [
        {
          "name": "value",
          "type": "java.lang.String[]",
          "validValues": ["\"/ex/foo\""],
          "default": "true"
        },
        {
          "name": "method",
          "type": "org.springframework.web.bind.annotation.RequestMethod[]",
          "validValues": ["org.springframework.web.bind.annotation.RequestMethod.POST"]
        },
        {
          "name": "headers",
          "type": "java.lang.String[]",
          "validValues": ["\"content-type=text/*\""]
        }
      ]
    },
    {
      "annotation": "@org.springframework.beans.factory.annotation.Autowired",
      "replaceableBy": ["@org.springframework.beans.factory.annotation.Qualifier"],
      "targets": ["field", "method"],
      "attributes": [{}]
    },
    {
      "annotation": "@org.springframework.beans.factory.annotation.Qualifier",
      "replaceableBy": ["@org.springframework.beans.factory.annotation.Autowired"],
      "targets": ["field", "parameter"],
      "attributes": [{}]
    }
  ]
}

For each annotation, we have the following fields:

annotation: the base of annotation, with @, the package and his name. Mandatory.

replaceableBy: list of replacements of the annotation. Each replacement is a base of annotation, such as the annotation field. Optional.

targets: list of targets of the annotation. Targets are locations where the annotations can be placed. valid values: type, method, field, parameter. Mandatory (at least one target).

attributes: list of attributes of annotations. Optional.

For each attribute, we have the following fields:

name: name of the attribute. Mandatory.

type: type of the attribute. Can be any Java type. Mandatory.

validValues: list of valid values. Mandatory.

default: this mark the attribute as default, which means that the annotation will receive only the value without the attribute name. Only works when the annotation have one attribute. Can be true or false. Optional.

This file is quite similar for C#. The name of attribute include the namespace, without the @.

---

After fill the annotations.json, it is necessary to create a Java code to use this project to generate the javaMutants. The following code is an example:

MutationToolConfig config = new MutationToolConfig(new File(SOURCE_PATH));
config.setProjectName(PROJECT_NAME);
config.getOperators().addAll(Arrays.asList(OperatorsEnum.RMA, OperatorsEnum.RMAT));
config.setThreads(2);
config.setLanguage(Language.JAVA) // or Language.C_SHARP
new MutationTool(config).run();

Basic configuration

The constructor of the MutationToolConfig class have the following field:

pathSources: directory of the source code. This doesn't include the test code.

This class have the following methods:

setProjectName: set the name of original project.

setThreads: set the number of threads.

setLanguage: set the language to generate the mutants (Java or C#).

getOperators: get the list of operators.

Setting the operators

The method getOperators of MutationToolConfig class retrieve the list of operators. It is possible to add operators to this list:

Add all operators

config.getOperators().addAll(Arrays.asList(OperatorsEnum.values()));

Add some operators

config.getOperators().addAll(Arrays.asList(OperatorsEnum.RMA, OperatorsEnum.RMAT));

Generate the javaMutants

The last thing is the use the method run of the MutationTool class:

new MutationTool(config).run();

All the javaMutants will be generated in the data/javaMutants folder.

mutation-tool-for-annotations's Issues

Create new JSON config file

Create new JSON configuration file on basis of current JSON configuration files and the new model:

{
    "annotation": "@Consumes",
    "replaceableBy": ["@Produces"],
    "targets": ["type", "method"],
    "attributes": [{
        "name": "value",
        "type": "string[]",
        "validValues": ["image/jpeg", "image/png"]
    }]
}

Create tests for the mutation.tool.context.entity.Entity class

Create tests to 4 types of entity (ClassOrInterfaceDeclaration, MethodDeclaration, FieldDeclaration and Parameter) for the following methods:

• getName
• getAccessModifiers
• getParams
• getReturnType
• getType

Create test for field switchMap in RPA class

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/rpa/RPA.kt

Line 31 in 95aa186

lateinit var switchMap:Map<String, List<String>>

Implement methods of RMA operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/RMA.kt

Lines 6 to 18 in 23ac34a

	/**
	* Remove Annotation Operator
	*/
	class RMA(context: Context) : Operator(context) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant>? {
	TODO("not implemented")
	}
	}

Use the annotationInfos field on related operators

Use the annotationInfos field of MutationToolConfig on related operators (ADA, ADAT, RPA, RPAT, RPAV, SWTG).

Change the folder structure of generated mutants

First level: operators
second level: mutants

Implement methods of RPA operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/RPA.kt

Lines 7 to 19 in 64dee30

	/**
	* Replace an annotation for another
	*/
	class RPA(context: Context, file: File) : Operator(context, file) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant> {
	TODO("not implemented")
	}
	}

add debugOn on config

add debugOn method on MutationToolConfig

Implement methods of ADA Operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/ADA.kt

Lines 7 to 19 in 64dee30

	/**
	* Add Annotation
	*/
	class ADA(context: Context, file:File): Operator(context, file) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant> {
	TODO("not implemented")
	}
	}

Implement methods of RPAV operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/RPAV.kt

Lines 7 to 19 in 64dee30

	/**
	* Replace a code annotation attribute value by another
	*/
	class RPAV(context: Context, file: File) : Operator(context, file) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant> {
	TODO("not implemented")
	}
	}

Implement the AnnotationInfo class

Implement the following AnnotationInfo class (on package mutation.tool.util.json):

AnnotationInfo

• name:String
• replaceableBy:List<String>
• targets:List<InsertionPoint>
• attributes:List<AttributeInfo>

AttributeInfo

• name:String
• type:String
• validValues:List<String>

Implement methods of ADAT operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/ADAT.kt

Lines 7 to 19 in 64dee30

	/**
	* Add valid attribute to annotation
	*/
	class ADAT(context: Context, file:File) : Operator(context, file) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant> {
	TODO("not implemented")
	}
	}

implement test for isSameProp function

mutation-tool-for-annotations/src/main/java/mutation/tool/util/Inspector.kt

Lines 22 to 23 in 0508825

	fun isSameProp(context: Context, field:FieldDeclaration):Boolean = (context.getInsertionPoint() ==
	InsertionPoint.PROPERTY && field.toString() == (context as PropertyContext).entity.toString())

Some operators may identify or catch wrong annotations

Each annotation have a name and a package. Operators who need the configuration file don't identify the package information, that is, they can identify or capture an annotation by name, but not by package.

One possible solution is to consider the name of the annotation and imports in the source file.

Affected operators: ADA, ADAT, RPA, RPAT, RPAV, SWTG.

Implement methods of SWTG operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/SWTG.kt

Lines 7 to 19 in 64dee30

	/**
	* Switch the annotation to another valid target
	*/
	class SWTG(context: Context, file:File, val anotherContexts: List<Context>): Operator(context, file) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant> {
	TODO("not implemented")
	}
	}

implement testMutants

mutation-tool-for-annotations/src/main/java/mutation/tool/MutationTool.kt

Lines 72 to 74 in d21bf86

	private fun testMutants() {
	TODO("not implemented")
	}

implement genMutants

mutation-tool-for-annotations/src/main/java/mutation/tool/MutationTool.kt

Lines 68 to 70 in d21bf86

	private fun genMutants() {
	TODO("not implemented")
	}

add mutation info on info.json file

add the following information:

• operator applied
• before change
• after change

Add AnnotationInfos in MutationToolConfig

Add AnnotationInfos object in MutationToolConfig class. Instantiate this field on constructor. If occur any error, must be null, and related operators must be removed (warning on log).

Implement methods of CHODR operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/CHODR.kt

Lines 7 to 19 in 64dee30

	/**
	* Change order of annotations
	*/
	class CHODR(context: Context, file: File) : Operator(context, file) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant> {
	TODO("not implemented")
	}
	}

Improve the readme file

Write better instructions about how to use the mutation tool.

implement generateMutants

mutation-tool-for-annotations/src/main/java/mutation/tool/mutant/Mutant.kt

Lines 9 to 11 in 64dee30

	fun generateMutants(operators: List<Operator>, javaFile: File, project: Project?) {
	TODO("not implemented")
	}

Fix after string on RMAT

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/RMAT.kt

Line 93 in d098c88

currentMutant?.after = annotation.nameAsString + "()"

Remove old JSON configuration files

Remove old JSON configuration files and related field on MutationToolConfig.

Implement methods of RPAT operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/RPAT.kt

Lines 7 to 19 in 64dee30

	/**
	* Replace a code annotation attribute by another
	*/
	class RPAT(context: Context, file: File) : Operator(context, file) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant> {
	TODO("not implemented")
	}
	}

Improve json file of each mutant

Fix output mutant json for some operators.

implement getAllJavaFiles

mutation-tool-for-annotations/src/main/java/mutation/tool/util/Inspector.kt

Lines 5 to 7 in 500d9fc

	fun getAllJavaFiles(dir:File):List<File> {
	TODO("not implemented")
	}

add constructor to MutationToolConfig

add constructor to MutationToolConfig with one argument (source path)

Update all tests after change JSON configuration file

Update affected tests by changes of JSON configuration file.

implement deleteTempFolder

mutation-tool-for-annotations/src/main/java/mutation/tool/util/FolderOperations.kt

Lines 17 to 20 in ea2b184

	fun deleteTempFolder():Boolean {
	// TODO: not implemented
	return true
	}

Fill the code documentation

Complete the code documentation, after all basic implementations and refactors.

Change Context to Target

Change the class Context to Target. And package context to target. Rename all related instances (maybe Intellij idea will do it automatically).

implement test for isSameMethod function

mutation-tool-for-annotations/src/main/java/mutation/tool/util/Inspector.kt

Lines 19 to 20 in 0508825

	fun isSameMethod(context: Context, methodDeclr:MethodDeclaration): Boolean = (context.getInsertionPoint() ==
	InsertionPoint.METHOD && methodDeclr.toString() == (context as MethodContext).entity.toString())

implement foldersIntersect

mutation-tool-for-annotations/src/main/java/mutation/tool/util/FolderOperations.kt

Lines 24 to 27 in 7c5a3d6

	fun foldersIntersects(folder1:File, folder2:File):Boolean{
	// TODO: not implemented
	return false
	}

rename tests

rename tests to begin with "test". Necessary change to maven catch tests.

Create test for isSameParameter function

mutation-tool-for-annotations/src/main/java/mutation/tool/util/Inspector.kt

Lines 21 to 24 in 68d262d

	fun isSameParameter(context: Context, parameter:Parameter): Boolean =
	(context.getInsertionPoint() == InsertionPoint.PARAMETER && parameter.toString() == context.toString() &&
	context.getRange().begin.line == parameter.range.get().begin.line &&
	context.getRange().begin.column == parameter.range.get().begin.column)

Implement runTests method

mutation-tool-for-annotations/src/main/java/mutation/tool/project/Project.kt

Lines 7 to 9 in 64dee30

	fun runTests(pathTests:File):Boolean {
	TODO("not implemented")
	}

check src and test folders

Verify src and test folders, both settled by user, on method init on class MutationTool.
Basic checks:

• The folders exists?
• Exists intersection between the folders?

implement getListOfAnnotationContext

mutation-tool-for-annotations/src/main/java/mutation/tool/annotation/AnnotationContext.kt

Lines 7 to 9 in 500d9fc

	fun getListOfAnnotationContext(javaFile: File):List<AnnotationContext> {
	TODO("not implemented")
	}

ADA_map.json don't allow two or more types of targets for the same annotation

The following json means: the annotation '@id' can be put in a Field target.

{
  "annotations": [
    {
      "name": "@Id",
      "conditions": [
        {
          "isTarget": "true",
          "contextType": "FIELD"
        }
      ]
    }
  ]
}

But, if this annotation is also applicable to a Class? Maybe the easiest solution is set contextType as an array of strings.

create test for numOfAnnotationAttributes

mutation-tool-for-annotations/src/main/java/mutation/tool/util/Inspector.kt

Lines 27 to 31 in 68d262d

	fun numOfAnnotationAttributes(annotation: AnnotationExpr): Int {
	if (!annotation.toString().contains(Regex("\\((.*?)\\)")))
	return 0
	return annotation.toString().replaceBefore("(", "").split(",").size
	}

create tests for run method of the MutationTool class

mutation-tool-for-annotations/src/main/java/mutation/tool/MutationTool.kt

Line 26 in 56cf43a

fun run():Boolean {

Implement 2 or more operators

Implement 2 or more operators for test purposes

Implement methods of RMAT operator

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/RMAT.kt

Lines 7 to 19 in 64dee30

	/**
	* Removes an attribute from a code annotation
	*/
	class RMAT(context: Context, file: File) : Operator(context, file) {
	override fun checkContext(): Boolean {
	TODO("not implemented")
	}
	override fun mutate(): List<Mutant> {
	TODO("not implemented")
	}
	}

Improve wiki of project

Write instructions for configuration of project

Create log class

Create log class, or use some lib to do that. maybe a report class

implement test for isSameClass function

mutation-tool-for-annotations/src/main/java/mutation/tool/util/Inspector.kt

Lines 16 to 17 in 0508825

	fun isSameClass(context: Context, classDeclr:ClassOrInterfaceDeclaration):Boolean = (context.getInsertionPoint() ==
	InsertionPoint.CLASS && classDeclr.nameAsString == (context as ClassContext).entity.nameAsString)

Add or create resource to test operators

Add or create project with annotations to test implemented operators.

implement getValidOperators

mutation-tool-for-annotations/src/main/java/mutation/tool/operator/Operator.kt

Lines 10 to 12 in 64dee30

	fun getValidOperators(context: Context, javaFile: File):List<Operator> {
	TODO("not implemented")
	}

implement makeRootFolders

mutation-tool-for-annotations/src/main/java/mutation/tool/util/FolderOperations.kt

Lines 12 to 15 in ea2b184

	fun makeRootFolders():Boolean {
	// TODO: not implemented
	return true
	}