dmoj / judge-server Goto Github PK
View Code? Open in Web Editor NEWJudging backend server for the DMOJ online judge.
Home Page: https://dmoj.ca
License: GNU Affero General Public License v3.0
Judging backend server for the DMOJ online judge.
Home Page: https://dmoj.ca
License: GNU Affero General Public License v3.0
judge.json
is painful because it's JSON according to @Xyene and I don't disagree.
Would be really nice to be able to measure submission count and average submission judging time for showing on http://status.dmoj.ca/
Currently, we add some new lines on top of the submission, for optimization, with disastrous results since from __future__ import <something>
doesn't work.
We should use an alternative mechanism to make that work.
waterloo first year cs teaches racket exclusively...
A class-based approach is fully modifiable by subclasses instead of making the make_executor
function take more and more arguments.
Some generators take prohibitively long times to generate testdata, and in a contest scenario it would be beneficial to have a cache of generated data so that subsequent submissions run faster. The cache would likely expire after a few minutes, to free resources.
When operating on the child's stdin on Windows, and the child ignores all the input, IOError: [Errno 22] Invalid argument
is caused when the buffer is flushed. This happens on write
or close
, and causes exceptions.
Sample tracebacks:
Traceback (most recent call last):
File "C:\Users\Quantum\Desktop\judge\judge.py", line 281, in _begin_grading
interactive=('grader' in init_data)):
File "C:\Users\Quantum\Desktop\judge\judge.py", line 664, in run
result.proc_output, error = communicate(input_data, outlimit=25165824, errlimit=1048576)
File "C:\Users\Quantum\Desktop\judge\communicate.py", line 53, in safe_communicate
proc.stdin.write(input)
IOError: [Errno 22] Invalid argument
Traceback (most recent call last):
File "C:\Users\Quantum\Desktop\judge\judge.py", line 281, in _begin_grading
interactive=('grader' in init_data)):
File "C:\Users\Quantum\Desktop\judge\judge.py", line 664, in run
result.proc_output, error = communicate(input_data, outlimit=25165824, errlimit=1048576)
File "C:\Users\Quantum\Desktop\judge\communicate.py", line 63, in safe_communicate
proc.stdin.close()
IOError: [Errno 22] Invalid argument
Would prevent awesome havoc when the system language is set to something like zh_CN
. GCC would be more than happy to give translated messages, puzzling everyone.
Note: Herculaneum uses Unicode smart quotes in error messages exactly for this reason.
On powerful computers it may be beneficial to support the running more than one judge without going through the pain of having to copy the judge directory. Simply pass a path to a judge json model as an optional commandline parameter, and default to /data/judge/judge.json
for compatibility with judges that are already set up.
Might be best to move the java_executor to its own repository after all these changes.
Because Dart is just so cool.
The judge servers were written to "chug along", but this has a terrible effect on error messages.
The following traceback is generated when a user misconfigures a problem:
Traceback (most recent call last):
File "/code/site/judge/bridge/judgehandler.py", line 199, in on_bad_problem
raise ValueError('\n\n' + packet['message'])
ValueError:
Traceback (most recent call last):
File "/judge/judge/judge.py", line 185, in _begin_grading
with open(os.path.join(get_problem_root(problem_id), 'init.json'), 'r') as init_file:
File "/usr/lib/python2.7/posixpath.py", line 77, in join
elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'
Intuitive, right? Very helpful to those who are not familiar with the judge's source code.
Don't just silently terminate grading and pretend like things will be fine.
Should be possible to export a C grader to Pascal.
Would by highly useful for problemsetters without installing the DMOJ Django site. Could be written as a lightweight Java applet that implements a minimal grading UI.
It would reduce the amount of effort that needs to go into organizing contests, since data would not have to be uploaded for every try. Would also be useful for allowing this judge to be used as a standalone program for private training (there are more stars on this repo than the site repo, which might be indicative that some potential users are interested in private graders).
PTRACE_O_TRACE*
.SecurePopen
-compatible class to log only only these events. Useful for example to track languages with builtin sandbox without performance penalty.cptbox
to more platforms, most notably linux x32 to use 64-bit powers with only 32-bit pointers.For DMOJ/hephaestus to work, we have to have the option of running all the executors without the sandbox. Ideally this would mean migrating most duplicate executor features into ResourceProxy
and renamed BaseExecutor
. This BaseExecutor
would then be able to expose the basic features as seen fit.
No reason why it should be fixed into the file, as it makes DMOJ/hephaestus rather painful.
To speed up IO in Java, the executor uses an ASCII-encoded stream. This has the effect of making Unicode-dependent problems such as http://www.dmoj.ca/problem/denoun2 painful to solve in Java.
The stream encoding should be configurable per-problem.
Caused by #22, and the traceback now contains <string>
, and python tries to open that as a file and read from it.
Hello, I want to try out this repo of judge on Windows ? How to use it ? Any documentation will be helpful. ๐
Well you see, normal grader takes string to safe_communicate
, checkers also take strings, but these interactive graders, take umm, cStringIO.StringIO
???
Traceback (most recent call last):
File "/var/lib/openshift/532715544382ec40ff00017a/app-root/data/judge/judge.py", line 269, in _begin_grading
short_circuit=short_circuit, source_code=original_source, interactive=('grader' in init_data)):
File "/var/lib/openshift/532715544382ec40ff00017a/app-root/data/judge/judge.py", line 566, in run
topen = self._resolve_open_call(init_data, problem_id, forward_test_cases)
File "/var/lib/openshift/532715544382ec40ff00017a/app-root/data/judge/judge.py", line 356, in _resolve_open_call
generator_launcher = clazz.Executor('%s-generator' % problem_id, generator_source).launch_unsafe
File "/var/lib/openshift/532715544382ec40ff00017a/app-root/data/judge/executors/GCCExecutor.py", line 57, in __init__
raise CompileError(compile_error)
CompileError: cc1plus: error: unrecognized command line option "-std=c++11"
Or something among those lines. TCP packets in async node.js are just too painful.
Generators should automatically stop executing when being aborted, at least stop before spawning the next process to generate the next file. Similar action for compilers would be nice to have.
We can set RLIMIT_FSIZE
to limit the maximum file size a compiler can possibly create. However, setting this limit in the judge process is not a trivial matter. A clean solution requires a script which sets the limit and execve
s the compiler, which would require a utility function to spawn compilers as this only works on *nix. The helper might also need to catch SIGXFSZ
exit and raise compile error as appropriate.
Needless to say, this has no priority.
Should be possible to bind a C grader to a JNI interface and provide callbacks. Will require a lot of machine-generated code.
Same as #19, but for firewall rules.
For linked list, the checker takes longer than the submission itself to check the data.
Some profile data:
name ncall tsub ttot tavg
..judge/checkers/standard.py:1 check 25 20.60029 39.94269 1.597708
The submission itself takes 7.69s. This is clearly not acceptable. I suggest we start porting checkers to C.
We already have a Java executor.
error: [Errno 10054] An existing connection was forcibly closed by the remote host
Importing classes from judge.py
for use in interactive graders/custom checkers is nasty and ultimately depends highly on the current judge version, forcing any future changes to the internals to be signature-compatible.
An API could easily resolve this.
Add all the JVM languages!
I don't think Haskell "A Plus B" program should take 2.5 seconds...
Also, according to this documentation, -O
produced reasonable code at a reasonable speed, while "At the moment, -O2
is unlikely to produce better code than -O
."
Is most confusing for newcoming Java users as it is.
Since the user submission runs in the same process as the judge executor calls to System.exit
are blocked such that the user does not exit the grading JVM.
Should be trivial to handle the exit permission request and kill the submission thread instead of throwing an AccessControlException
.
A certain online judge doesn't require flushing for interactive problems. A wild guess scientific analysis suggests that they present pty
devices for stdin
and stdout
. This can be arranged on *nix through the use of the pty
module's openpty()
in place of os.pipe()
.
Since GCC 5.1 released a while ago, we could allow C++14 features to be used on our judges.
According to some StackOverflow post, Clang is recommended for students due to "extremely clear diagnostics are definitely easier for beginners to interpret". The idea is rather popular to the point that GCC tries to debunk it as a myth.
It might be worth to add clang
as an alternative C/C++ frontend.
Should this be added in place of Rhino? It ships with Java 8.
socket.error: [Errno 10013] An attempt was made to access a socket in a way for idden by its access permissions
After exiting from the issue documented in #5 , Windows judges are not able to connect again.
The Java submission:
public class Clazz {
public static void main(String[] argv) {
}
static {
String x = "x";
for(int i = 0; i < 1000000; i++) x += x;
}
}
Will not produce an MLE status code as expected, but rather an IR. The offending line? https://github.com/DMOJ/judge/blob/master/java_executor/src/ca/dmoj/java/SubmissionThread.java#L27
But why? Clearly the OutOfMemoryError
is handled. The reason is quite illegit. Let's take a look at the docs for java.lang.reflect.Method#invoke
(emphasis mine, cruft removed).
If the underlying method is static, the class that declared the method is initialized if it has not already been initialized.
Throws:
- InvocationTargetException - if the underlying method throws an exception.
- ExceptionInInitializerError - if the initialization provoked by this method fails.
Now, at this time, the class that declared the method is not initialized. So we're not given an InvocationTargetException
when the static block runs out of heap space. Easy though, right? Just wrap the invocation to catch ExceptionInInitializerError
, and check if the cause is an OutOfMemoryError
. No.
In my experience with the Java language and Java documentation, the terms "Exception" and "Error" are used quite interchangeably. InvocationTargetException
states that it will be thrown if "if the underlying method throws an exception", but this is not true. It will be thrown if the underlying method throws any Throwable - indeed, the entire Java executor relies on this. It's therefore only logical to assume that ExceptionInInitializerError
would act the same. It doesn't.
Unlike what you might expect, it only gets thrown if the initialization throws an Exception
. Not a Throwable
, as you might believe - an Exception
. Hence, "correct" code would wrap the invocation call with a try/catch block for OutOfMemoryError
, which, by virtue of being an Error
, will not be wrapped in ExceptionInInitializerError
.
Thanks, Java.
This rant issue took less time to write than this bug took to identify.
When wbox
crashes, its generated accounts are never cleaned. They should be cleaned the next time the judge boots.
C:\Users\Tudor>net user
User accounts for \\BERLIN
-------------------------------------------------------------------------------
Administrator Guest Tudor
----------- wbox0000000000c6afaf wbox0000000001871be0
wbox00000000018a93b8 wbox000000000191d021 wbox0000000001aa8ccc
wbox0000000001b196a0 wbox000000000ace72e2 wbox000000000b23e734
wbox000000000b296475 wbox000000000b324dc7 wbox000000000ba9c9f5
wbox000000000bb804e8 wbox000000000bbb3434
The command completed successfully.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.