A library for grading Java assignments
This is a fork of tkutcher/JGrade, created by Tim Kutcher. Because he has moved on to other projects, I forked it so I could run it with the current (2022) version of Gradescope. I have made minor changes to the JGrade library and significant changes to the Gradescope example. See my changes .
JGrade is a helper tool with various classes designed to assist in course instructors "autograding" an assignment, inspired by the Gradescope Autograder . There are classes that the client can integrate with directly, or use the jar's main method (and provide a class with annotations) that wraps a lot of common functionality ( see examples). It was designed to produce the output needed for Gradescope while being extensible enough to produce different outputs and configure the specific JSON output Gradescope is looking for.
To make use of this, you first need to grab the jar file from the Releases page. This includes many classes you can make use of, as well as a main method for running and producing grading output.
With this, you could have the following setup:
A class that runs some unit tests we want to treat their success as a grade ( these would import student code):
import com.github.tkutcher.jgrade.gradedtest.GradedTest;
import org.junit.Test;
import static com.github.tkutcher.jgrade.gradedtest.GradedTestResult.HIDDEN;
import static com.github.tkutcher.jgrade.gradedtest.GradedTestResult.VISIBLE;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
public class ExampleGradedTests {
@Test
@GradedTest(name = "True is true", points = 2.0, visibility = VISIBLE)
public void trueIsTrue() {
assertTrue(true);
}
@Test
@GradedTest(name = "False is false", number = "2", points = 3.0, visibility = HIDDEN)
public void falseIsFalse() {
assertFalse(false);
}
@Test
@GradedTest(name = "Captures output")
public void capturesOutput() {
System.out.println("hello");
}
@Test
@GradedTest(name = "This test should fail")
public void badTest() {
fail();
}
}
and a main method with some other grading-related non-unit-testing
logic MyGrader.java
:
import com.github.tkutcher.jgrade.BeforeGrading;
import com.github.tkutcher.jgrade.AfterGrading;
import com.github.tkutcher.jgrade.Grade;
import com.github.tkutcher.jgrade.Grader;
import com.github.tkutcher.jgrade.gradedtest.GradedTestResult;
import static com.github.tkutcher.jgrade.gradedtest.GradedTestResult.HIDDEN;
public class BasicGraderExample {
/* All @Grade/@BeforeGrading/@AfterGrading methods must take exactly one parameter
* of type Grader. This parameter is the same grader throughout.
*
* @BeforeGrading methods are run before others.
*/
@BeforeGrading
public void initGrader(Grader grader) {
grader.startTimer();
}
/* You can run unit tests that are annotated with @GradedTest to add
* GradedTestResults to the Grader in this way.
*/
@Grade
public void runGradedUnitTests(Grader grader) {
grader.runJUnitGradedTests(ExampleGradedTests.class);
}
/* You can also manually add GradedTestResults you create to the grader. */
@Grade
public void singleTestResult(Grader grader) {
grader.addGradedTestResult(
new GradedTestResult("manual test", "1", 1.0, HIDDEN)
);
}
/* Grader.startTimer() and Grader.stopTimer() can be used to time the grader */
@Grade
public void loopForTime(Grader grader) {
long startTime = System.currentTimeMillis();
while (System.currentTimeMillis() - startTime < 1000) ;
}
/* @AfterGrading methods are run after all other methods. */
@AfterGrading
public void endGrader(Grader grader) {
grader.stopTimer();
}
}
Then, you could run
java -jar ../lib/jgrade-1.1-all.jar -c MyGrader -o results.json
and get GradeScope-formatted json. See the examples for more complete examples and how to set up a script to work with GradeScope, and expand the usage below to see the arguments you can provide this main program.
Usage
-c,--classname arg the class containing annotated methods to grade
-f,--format output-format specify output, one of 'json' (default) or 'txt'
-h,--help<br>
--no-output don't produce any output (if user overriding)
-o destination save output to another file (if not specified,
prints to standard out)
--pretty-print pretty-print output (when format is json)
-v,--version
The way I used this library is to have a base class for the course (for example,
a _226Grader
) that contains
annotated methods for functionality/grading parts that are consistent across all
assignments. For example, the
@BeforeGrading
method starts a timer and the @AfterGrading
method stops it.
There is a @Grade
method that
does the "grading" of style with checkstyle. Subclasses, for
example Assignment1Grader
(or Assignment0Grader
I suppose ๐), extend this and add @Grade
methods to add
assignment-specific grading.
See the gradescope folder in the examples for a rough example setup.
See the API Docs for more complete documentation.
With the CheckstyleGrader
you can specify grading deductions for checkstyle
errors. This method below, for example,
would check the students files and deduct a point for each checkstyle error
type (missing javadoc, require this, etc.).
@Grade
public void runCheckstyle(Grader grader){
CheckstyleGrader checker=new CheckstyleGrader(5.0,1.0,MY_CHECKSTYLE_JAR,STUDENTFILES);
checker.setConfig(MY_CHECKSTYLE_CONFIG);
GradedTestResult result=checker.runForGradedTestResult();
result.setScore(Math.max(0,5-checker.getErrorTypeCount()));
grader.addGradedTestResult(result);
}
You can use this strategy to make failed tests deduct points from a total. So say in the current assignment there are two parts, A and B, each worth 25 points. If someone fails 30 tests for part B each worth one point, you don't want that to cut in to the assignment A portion:
public class GradeAssignment7 extends Grade226Assignment {
private static final int AVL_POINTS = 30;
private static final int TREAP_POINTS = 20;
@Grade
public void gradeAvlTree(Grader grader) {
grader.setGraderStrategy(new DeductiveGraderStrategy(AVL_POINTS, "AvlTreeMap"));
grader.runJUnitGradedTests(GradeAvlTreeMap.class);
}
@Grade
public void gradeBinaryHeapPQ(Grader grader) {
grader.setGraderStrategy(new DeductiveGraderStrategy(TREAP_POINTS, "TreapMap"));
grader.runJUnitGradedTests(GradeTreapMap.class);
}
}
A class to help wrap testing command line programs. You subclass CLITester
,
then implement
the getInvocation()
method for how the command line program is invoked, then
you can use
runCommand(String)
to get the output in an object that you can test for
expected output.
mvn install
to compilemvn test
to run unit testsmvn checkstyle:checkstyle
to run checkstylemvn javadoc:jar
to generate API docs.
Check out contributing for more.
JGrade is written in Java 8 . Since the library has classes designed to run alongside JUnit, JUnit 4 is a dependency for the entire project (as opposed to just for running the projects own unit tests). The org.json package is used in producing correctly formatted JSON output, and the Apache Commons CLI library is used for reading the command line in the main program.
For simplicity, the main jar (appended with "-all") includes all of these dependencies.
- Feedback for required files
- In our autograder, we built in something that took a list of required files and created a visible test case worth 0 points of what files were missing - this helped students debug.
- Could try and move some of this there.
- Actual Observer pattern
- Allow for people to specify custom handlers whenever things like new graded test results are added
- Old "observer" terminology not really an observer