feuermagier / autograder Goto Github PK
View Code? Open in Web Editor NEWAutomatic grading of student's Java code
License: MIT License
Automatic grading of student's Java code
License: MIT License
class A {
int a;
String[] b;
void doSomething() {
int a = 0;
String[] b = new String[10];
}
}
In the above example the variable b
in doSomething
is not marked as unused
The initial implementation of #14 proposed in #16, has a number of problems:
Scope
classScanner scanner = new Scanner(System.in);
int index = scanner.nextInt();
int[] newArray = new int[10];
newArray[index] = 2;
newArray[0] = 0; // this assignment can obviously not be removed
int[] newArray = new int[10];
newArray[1] = 2;
newArray[0] = 0; // can be removed becahse index 1 ≠ 2
The restriction has been made to reduce the initial implementation complexity, but could be lifted in the future by keeping better track of which indices have been set to which values/variables (IntelliJ seems to do this through Dataflow Analysis
#22 will be helpful
It is technically possible to make this lint work with normal variables as well.
A common use of this would be in constructors:
class A {
private String string;
A() {
this.string = null; // this assignment might be removed
}
}
Additionally one might generalize the lint to not only detect unnecessary assignments of the default value, but other values as well:
int a;
int b = a;
b = a;
This seems hard to detect, but there might be some good ways through spoon:
short s = 0
is doing an implicit cast, so the rhs is of type int
, which may be problematic for comparison)Example for obvious case:
private static final char[] VALID = {'R', 'x', 'X', '|', '\\', '/', '_', '*', ' '};
false-positives
String weekday = "monday";
switch (weekday) {
case "monday" -> return WeekDay.MONDAY,
case "tuesday" -> return WeekDay.TUESDAY,
case "wednesday" -> return WeekDay.Wednesday,
// ...
default -> throw new IllegalStateException(),
}
if (a) {
doSomething();
} else {
if (b) {
doSomething2();
} else {
doSomethingElse();
}
}
can be written as
if (a) {
doSomething();
} else if (b) {
doSomething2();
} else {
doSomethingElse();
}
This might be implemented already, in this case I have found code where the lint does not trigger.
For example
class A {
private List<String> myList;
public List<String> getList() {
return this.myList;
}
}
instead of
class A {
private List<String> myList;
public List<String> getList() {
return new ArrayList<>(this.myList);
}
}
Same problem with
class A {
private List<String> myList;
public void setList(List<String> myList) {
this.myList = myList;
}
}
Option<bool>
or nullable Boolean
should be avoidedString a;
// ... some code
// then initialized when it is used:
a = "abc";
For null
checks one must use ==
, so this should not be linted.
if (a) {
return true;
} else if (b) {
return false;
}
return false;
return a;
if (a) {
doSomething();
return true;
} else {
return false;
}
boolean result = a;
if (result) {
doSomething();
}
return result;
boolean isTrue(String value) {
if (value.isEmpty()) {
return false;
}
return 0 > value.get(value.size() - 1).getPosition() - VALUE_DISTANCE;
}
The line
this.stringTerrainInputList = new ArrayList<String>();
created the custom comment "Verwende den 'diamond operator'" ("Use the 'diamond operator'"), but should have been "Explizite Typangabe 'String' kann mit <> ersetzt werden" ("Explicit type argument 'String' can be replaced with <>") (text copied from IntelliJ).
The ConcreteCollectionCheck does only check for fields and return values. This may require reimplementing the check with Spoon.
The following warning is generated each time the tests are run:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
I think this can be fixed by adding this to the pom.xml:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>2.0.5</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>2.0.5</version>
</dependency>
(I don't know if the simple one is the right logger)
Sometimes one overwrites equals
, but not hashCode
, which is not okay.
Sometimes in code one can find:
@SupressWarnings("checkstyle:someRule")
class MyClass {}
or
@SupressWarnings("unchecked")
void doesUnsafeCast() {}
Could you deploy the artifacts to Maven Central? Then we could easily integrate them.
An interface
interface MyConstants {
final static String MY_CONSTANT = "Hello World";
final static String MY_ERROR = "Input is wrong";
}
void setArray(String[] array) {
this.array = array;
}
and
String[] getArray() {
return this.array;
}
are not detected. Similarly for multi-dimensional arrays.
e.g. ANSI and UTF-8. The reason is that Spoon doesn't accept files with different encodings, so this code checks the encoding. We could try to rewrite all files in UTF-8 and use e.g. in-memory files for the linters.
protected
or lower visibility. Not public
.StringBuilder builder = new StringBuilder();
builder.append('\n');
Does not trigger the lint.
@author
and @version
should not be in method/attribute javadoc@author
with something that is not a u-shorthandTODO
s in javadocs are not detected:/**
* TODO Write Javadoc
*
* @author uxxxx
* @version 1.0
*/
/**
*
*/
*/
/**
*
/*
*/
*/
/*
*
*/
I am not sure that spoon is capable of inferring variables and constant values.
This capability can be useful for a number of lints, which may not trigger, because constants/variables are not resolved.
For example #14 does not lint this:
int myVariable = 0;
int[] myArray = new int[10];
myArray[0] = myVariable;
boolean a, b, c;
if (a) {
doSomething();
} else if (b) {
doSomething();
} else if (c) {
doSomething();
} else {
doSomethingElse();
}
can be written as
if (a || b || c) {
doSomething();
} else {
doSomethingElse();
}
Similarly, the following is not detected as well:
if (a) {
doSomething1();
return /* terminal statement */;
}
if (b) {
doSomething2();
return /* terminal statement */;
}
doSomething1();
return /* terminal statement */;
if (a || !b) {
doSomething1();
return /* terminal statement */;
}
doSomething2();
return /* terminal statement */;
if (!(a == b)) {
}
can be written as
if (a != b) {
}
There is an issue, which causes the AutoGrader to crash.
Error:
!MESSAGE Autograder failed: java.lang.NullPointerException: Cannot invoke "spoon.reflect.declaration.CtMethod.isPublic()" because the return value of "spoon.reflect.code.CtReturn.getParent(java.lang.Class)" is null
at de.firemage.autograder.core.check.oop.ListGetterSetterCheck$1.process(ListGetterSetterCheck.java:34)
at de.firemage.autograder.core.check.oop.ListGetterSetterCheck$1.process(ListGetterSetterCheck.java:31)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:72)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:149)
at spoon.reflect.visitor.CtScanner.visitCtBlock(CtScanner.java:321)
at spoon.support.reflect.code.CtBlockImpl.accept(CtBlockImpl.java:58)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.visitCtIf(CtScanner.java:505)
at spoon.support.reflect.code.CtIfImpl.accept(CtIfImpl.java:36)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:149)
at spoon.reflect.visitor.CtScanner.visitCtBlock(CtScanner.java:321)
at spoon.support.reflect.code.CtBlockImpl.accept(CtBlockImpl.java:58)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.visitCtConstructor(CtScanner.java:390)
at spoon.support.reflect.declaration.CtConstructorImpl.accept(CtConstructorImpl.java:48)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:149)
at spoon.reflect.visitor.CtScanner.visitCtClass(CtScanner.java:357)
at spoon.support.reflect.declaration.CtClassImpl.accept(CtClassImpl.java:63)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:149)
at spoon.reflect.visitor.CtScanner.visitCtPackage(CtScanner.java:679)
at spoon.support.reflect.declaration.CtPackageImpl.accept(CtPackageImpl.java:82)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:149)
at spoon.reflect.visitor.CtScanner.visitCtPackage(CtScanner.java:678)
at spoon.support.reflect.declaration.CtPackageImpl.accept(CtPackageImpl.java:82)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:149)
at spoon.reflect.visitor.CtScanner.visitCtPackage(CtScanner.java:678)
at spoon.support.reflect.declaration.CtPackageImpl.accept(CtPackageImpl.java:82)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:149)
at spoon.reflect.visitor.CtScanner.visitCtPackage(CtScanner.java:678)
at spoon.support.reflect.declaration.CtPackageImpl.accept(CtPackageImpl.java:82)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:184)
at spoon.reflect.visitor.CtScanner.visitCtModule(CtScanner.java:964)
at spoon.reflect.factory.ModuleFactory$CtUnnamedModule.accept(ModuleFactory.java:96)
at spoon.reflect.visitor.CtScanner.scan(CtScanner.java:194)
at spoon.support.visitor.ProcessingVisitor.scan(ProcessingVisitor.java:68)
at spoon.support.QueueProcessingManager.process(QueueProcessingManager.java:118)
at spoon.reflect.CtModelImpl.processWith(CtModelImpl.java:126)
at de.firemage.autograder.core.integrated.StaticAnalysis.processWith(StaticAnalysis.java:61)
at de.firemage.autograder.core.check.oop.ListGetterSetterCheck.check(ListGetterSetterCheck.java:31)
at de.firemage.autograder.core.integrated.IntegratedCheck.run(IntegratedCheck.java:37)
at de.firemage.autograder.core.integrated.IntegratedAnalysis.lint(IntegratedAnalysis.java:105)
at de.firemage.autograder.core.Linter.checkFile(Linter.java:112)
at de.firemage.autograder.core.Linter.checkFile(Linter.java:54)
at de.firemage.autograder.cmd.Application.call(Application.java:118)
at de.firemage.autograder.cmd.Application.call(Application.java:35)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at de.firemage.autograder.cmd.Application.main(Application.java:67)``
On demand I will provide the code the grader failed on.
throw new IllegalArgumentException();
vs
throw new IllegalArgumentException("the day should be between 1 and 31");
Might be better to ignore exceptions that are declared in the same project (which might already have a descriptive message).
Either completely discourage them or only accept them if they are private inside the class (for example in LinkedList
s which may contain Node
inner classes). Most of the time it is better to use an extra file.
Tests:
Should be reliable to detect non-latin languages in comments.
Did you considered adding an appropriate license to this project. Otherwise, it will be hard to use :)
Currently the lint can detect for example int ZERO = 0
or int ONE = 1
, but only because of a hardcoded list of known problems.
Most commonly problematic constants
String ERROR_STRING = "An error occurred."
, which can be fine, but is redundantHere are some examples where the lint does not trigger at the moment:
class Test {
private static final boolean TRUE = true;
private static final boolean FALSE = false;
// for those constants there are an infinite number of variants,
// which is why I would suggest checking that if the literal content
// is equal to the name with the correct naming convention
//
// e.g. "debug-path".toUpperSnakeCase().equals(attribute.name())
private static final String UP = "up";
private static final String DOWN = "down";
private static final String DEBUG_PATH = "debug-path";
// this is linted?
private static final String LEFT = "left";
// and this is not?
private static final String RIGHT = "right";
// Another example is:
private static final String BIG_X_REGEX_FORMAT = "X"; // hard to check, because single letters are common in words...
public static void doSomething() {
MyUiClass myUiObjectClass = new MyUiClass(); // "My", "Object" and "class" are all redundant words
UserInterface userInterface = new UserInterface(); // much better
}
}
This code should be avoided:
int a = 3;
à = a + 5;
instead, one should write:
int a = 3;
a += 5;
Commutative operations, where a = b <op> a
is a <op>= b
:
&
|
^
*
+
a = a <op> b
is a <op>= b
:%
-
/
<<
>>
The lint should work with a = a <op> b <op> c
and a = a <op1> b <op2> c
as well (e.g. a = a + b - c
)
if (a) {
// do something
return /* terminal */; // applies for continue and break as well
} else {
// normal path
}
if (a) {
// do something
return /* terminal */; // applies for continue and break as well
}
// normal path
Makes the code easier to read.
Currently one declares on which line a lint should trigger. This has the obvious problem that adding new lines will break tests. Especially for larger tests, with hundreds of lines, it is very annoying to readjust the line number for every single error.
This means omitting the generic:
public List getList() {
return myList;
}
The code if (a == true)
, if (a != true)
, if (a != false)
and if (a == false)
should be detected right now. It does not seem to resolve constants, so one can bypass this lint:
private static final boolean TRUE = true;
boolean a;
if (a == TRUE) {}
For example
/**
* Does return a value.
*
* @returns the value
*/
public int get() {
// this is not documented through an `@throws` in the javadoc
throw new IllegalArgumentException();
}
There is an easy implementation possible by
.close()
callThis does not guarantee that the scanner is reliably closed every time and never leaks, but should cover most of the cases.
What one might consider
<E> boolean containsDuplicates(Iterable<E> collection) {
Set<E> visited = new HashSet<>();
for (var e : collection) {
if (visited.contains(e)) {
return true;
}
visited.add(e);
}
return false;
}
vs
<E> boolean containsDuplicates(Iterable<E> collection) {
return new HashSet<>(collection).size() < collection.size();
}
System.arraycopy
/Arrays.copyOf
// Note: I am not 100% sure that this code is correct
<T> T[] copyArray(T[] array) {
T[] newArray = new T[array.length];
for (int i = 0; i < array.length; i++) {
newArray[i] = array[i];
}
return newArray;
}
contains
method:boolean isValid = false;
for (var value : array) {
if (value.equals(other)) {
isValid = true;
break;
}
}
vs
boolean isValid = Arrays.asList(array).contains(other);
A Dockerfile
could be useful as well.
I've had an assignment with the following code example (paraphrased):
public class Test {
private static boolean someVariable = false;
public static void main(String[] args) {
for (int i = 0; i < 50; i++) {
someVariable = someOtherMethodThatReturnsABoolean();
}
}
}
The variable declaration got annotated saying it should be an instance attribute, however, because the static main method uses the variable, it needs to be static from my understanding.
Issue templates are useful to improve the quality of reported issues and they make it easier to create them.
Here are some categories that might benefit from a template:
It would be nice if one could test error messages (especially to see if they span the correct text) and that one did not mistype a localized key.
As a rule of thumb: System.out
calls and Scanner
should only be called in a single class.
Most commonly one calls:
System.out.println
System.out.print
System.out.printf
<- this one has been observed, but did not trigger the lint if it exists?Scanner.nextLine()
Scanner.nextInt()
For example boolean
is by default false
, therefore
boolean[] myArray = new boolean[mySize];
for (int i = 0; i < myArray.length; i++) {
myArray[i] = false;
}
can be written as
boolean[] myArray = new boolean[mySize];
(Note: this would be wrong if the myArray
variable is accessible outside of the class, because the object identity is changed through the assignment)
In both cases, one has no other option, except for catching/rethrowing as a checked exception.
For example
StringBuilder builder = new StringBuilder();
String x;
builder.append("[").append(x).append("]");
or
String x;
String y = "a" + x + "b";
There are an endless number of cases, where String.format
might apply.
"" + ...
will always evaluate to a string. Those could be detected easily and converted to a "".formatted(...)
based on the type of each argument. (Note: this should work with constants as well, because of magic strings...)It is common to add a REGEX
suffix/prefix to the constant. If the constant is not used as a String
, but instead compiled to a Pattern
, it should be compiled to a Pattern
in the constant:
private static final String MATCH_NEW_COMMAND_REGEX = "new \\d,\\d";
public void parseCommand(String command) {
if (Pattern.matches(MATCH_NEW_COMMAND_REGEX, command)) {
System.out.prinln("new command got called");
}
}
vs
private static final Pattern MATCH_NEW_COMMAND_REGEX = Pattern.compile("new \\d,\\d");
public void parseCommand(String command) {
if (MATCH_NEW_COMMAND_REGEX.matcher().matches()) {
System.out.prinln("new command got called");
}
}
For example:
enum WeekDay {
MONDAY,
TUESDAY;
// ...
}
switch (weekday) {
case MONDAY -> 1,
case TUESDAY -> 2,
// ...
}
could be either a Map<WeekDay, Integer>
or an associated attribute of the enum.
With #33 only interfaces are detected. One can do constants classes with enums and classes as well.
I am not sure when this is okay and when not, but it is something that could be considered.
As enums only have one instance, the attributes should be final and never change state.
For example
enum MyEnum {
MY_VARIANT;
private final List<String> strings = new ArrayList<>();
public void addString(String string) {
this.strings.add(string);
}
}
This can result in confusing behavior.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.