i-al-istannen / simplecodetester

A web application that checks code against a range of user submitted tests.

SimpleCodeTester

The final exam in some courses is a programming exercise where students create their own programs. Many people are just starting out with programming (and we are only human), so actually testing the code is extremely paramount.

Sadly, most traditional tools, like JUnit, are hard to use for newcomers, do not allow easy sharing of test cases and are time consuming to write. SimpleCodeTester is a website that solves this problem:
Everybody can create checks verifying program output in a simple and intuitive format. The editor for this supports syntax highlighting to make it a bit more enjoyable :)

Finally, users can test their code against all submitted tests. Everybody is free to participate and create tests - everybody will benefit from them instant they are submitted.

Great care was taken to ensure the online runtime environment is secure (it is run with a restrictive security manager and each submission is sandboxed in its own Docker container, without any capabilities or network/disk access). We also conciously decided, that no user submissions should ever be stored - your code is received, passed to the sandbox via stdin, fully compiled in-memory and the last traces of it die with the result response. Feel free to have a look at how it is implemented though!

Screenshots

The following is a submitted check

---

And we also have a dark theme!

---

If you submit your code, it will automatically be tested against all checks. Here is our demo test from earlier, all grown up and failing! In such a case, we try to match the output your program produced with the expected output and provide useful error messages.

---

The logo was artfully crafted by @libaurea 🎉

simplecodetester's Issues

CLI for interacting with the backend server

Hey - less of an issue, more of a question:
I'm concidering to write a commandline-tool which would automate the entire process of zipping code, logging in, selecting the checks and uploading the code, which would normally be done using the frontend. It would basically be a script, calling the same backend api's as the frontend does - just from a command line, instead of the web frontend.

Just wanted to ask you if that's fine with you :)
(Also, amazing job with this tool - saved me some pain while testing ;) )

Test stürzt ab durch Parallelisierung

Hey, mir ist aufgefallen, dass der Codetester komische Fehler ausgibt, wenn auf einen Java 8 Stream .parallel() aufgerufen wird.

Hier ein Auszug aus dem Fehlerbericht:
ExceptionInInitializerErrorjava.lang.ExceptionInInitializerError at java.base/java.util.concurrent.ForkJoinPool$InnocuousForkJoinWorkerThreadFactory$1.run(ForkJoinPool.java:3227) at java.base/java.util.concurrent.ForkJoinPool$InnocuousForkJoinWorkerThreadFactory$1.run(ForkJoinPool.java:3225) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/java.util.concurrent.ForkJoinPool$InnocuousForkJoinWorkerThreadFactory.newThread(ForkJoinPool.java:3224) at java.base/java.util.concurrent.ForkJoinPool.createWorker(ForkJoinPool.java:1328) at java.base/java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1352) at java.base/java.util.concurrent.ForkJoinPool.signalWork(ForkJoinPool.java:1476) at java.base/java.util.concurrent.ForkJoinPool.externalPush(ForkJoinPool.java:1903) at java.base/java.util.concurrent.ForkJoinTask.fork(ForkJoinTask.java:704) at java.base/java.util.stream.AbstractTask.compute(AbstractTask.java:324) at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:408) at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateParallel(ReduceOps.java:919) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)

Wäre cool, wenn du mal drüberschauen könntest,
LG Paul

command called for the x time creates AccessControlException access denied ("java.lang.RuntimePermission" "createClassLoader")

I have a problem with your online code tester, since just copyin all the input and paste it locally on my machine (Oracle JDK 14, no modification made) doesn't result in an AccessControlException access denied ("java.lang.RuntimePermission" "createClassLoader"), whereas in your tester it does.
Funny thing is: the command is called several times before in this test, even with the exact same arguments. Therefore if the code really needs to have the permission "createClassLoader" then it should have failed earlier on and not fail at the x call of the command.

Full error message:

Expected 'C;23'
AccessControlException
access denied ("java.lang.RuntimePermission" "createClassLoader")
java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "createClassLoader")
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at me.ialistannen.simplecodetester.execution.slave.SubmissionSecurityManager.checkPermission(SubmissionSecurityManager.java:57)
	at java.base/java.lang.SecurityManager.checkCreateClassLoader(SecurityManager.java:384)
	at java.base/java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:369)
	at java.base/java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:359)
	at java.base/java.lang.ClassLoader.<init>(ClassLoader.java:456)

Expected behaviour would be to just work since it does so on my local machine.

If this helps for debugging I will give you a copy of my code, problem is, its in a private repo so collaborators have write access, therefore I would prefer hand over a link to a zip or something like this. Open for better ideas.