codewars / codewars-runner-cli

Old CodeRunner project. See https://github.com/codewars/runner instead.

ARCHIVED

Please use https://github.com/codewars/runner to report any language related issues.

About

This project was previously used by Codewars and Qualified.io to execute small sets of code within various languages, using various testing frameworks. Each time code is run, it is executed within a Docker container in order to secure unsafe code execution.

All execution is done within Docker, with a Node CLI app contained within each container that manages the code execution for that specific language environment and returns the result via stdout.

Contributions

This project has been open-sourced so that the Codewars and Qualified communities can contribute support for new languages and frameworks. See the Language Support Status section for more information regarding which languages are currently supported and where their Codewars/Qualified support has been added.

Basic Usage

Within each Docker image, there is a copy of the Node executable and a run script. That script accepts multiple options for executing code. For example to run a simple javascript script which would output 2:

# this is how you run the Node CLI. For this to work you would have to bash into the correct Docker image
node run -l javascript -c "console.log(1+1)"

Because everything runs inside of Docker, you would normally not run Node directly from your host but instead via a Docker run command. To do this, you would either bash into the correct Docker image like so:

# direct Docker call:
docker run --rm -it --entrypoint bash codewars/node-runner

# alternatively you can use the provided Docker Compose configuration:
docker-compose run node-runner

Or you could choose to execute the code outside of Docker by creating a container that will remove itself after it executes:

# direct Docker call:
docker run --rm codewars/node-runner run -l javascript -c "console.log('I ran inside of Docker using NodeJS')"
docker run --rm codewars/ruby-runner run -l ruby -c "puts 'I ran inside of Docker using Ruby'"

# alternatively you can use the provided Docker Compose configuration:
docker-compose run javascript -c "console.log('I ran inside of Docker using NodeJS')"
docker-compose run ruby -c "puts 'I ran inside of Docker using Ruby'"

Integrated Test Suites

The most significant aspect of this project is that integrated test support is built in to many languages. This is how Codewars and Qualified work, instead of testing STDOUT of a program, the executed code is tested using traditional code testing methods, with unit-tests and language-specific testing frameworks.

Here is a very simple example of running tests using the simplified Codewars testing framework.

# manually running docker
docker run --rm codewars/node-runner run -l javascript -c "var a = 1;" -t cw -f "Test.assertEquals(a, 1)"

# using docker compose
docker-compose run javascript -c "var a = 1;" -t cw -f "Test.assertEquals(a, 1)"

Which would output <PASSED::>Test Passed: Value == 1 to STDOUT.

Language Support Status

Many languages are currently supported in various states of completeness. This list tries to keep track of each.

Legend: !!! = Failing Specs, ??? = Status is unknown, * = Any

Language	Version	Basic Run	Project Mode	Test Integration	Codewars	Qualified	Docker Image	Examples	Notes
Assembly (GAS)		!!!					systems-runner		Travis is failing, tests pass locally
Bash		✓	✓	rspec	rspec	rspec	ruby-runner
Brainf**k	20041219	✓	✓	cw-2	cw-2		esolangs-runner
C	Clang 3.6/C11	✓		criterion	criterion		systems-runner
Clojure	1.8.0	✓		clojure.test	clojure.test	clojure.test	jvm-runner	clojure.test
CoffeeScript	1.10.0	✓		cw-2	cw-2	cw-2	node-runner	cw-2
C++	14	✓		igloo	igloo	igloo	systems-runner
C#	Mono 4.8	✓	✓	nunit	nunit	nunit	dotnet-runner	nunit
Chapel	1.15.0	✓	✓	cw-2	cw-2		chapel-runner	cw-2
Crystal	0.21.1	✓		spec	spec	spec	crystal-runner	spec
Dart	1.23.0	✓		test	test		dart-runner	test
Elixir	1.2.4	✓		exunit	exunit		elixir-runner
Erlang	20.0	✓		eunit			erlang-runner
F#	4.1	✓		fuchu	fuchu		dotnet-runner	Fuchu	Tests should be placed in a module called "Tests", in a Fuchu testList
Go	1.8	✓		ginkgo	ginkgo	ginkgo	go-runner	ginkgo
Groovy	2.4.12	✓		junit4,spock			gradle-runner
Haskell	7.10.3	✓		hspec!!!	hspec	hspec	haskell-runner	hspec	An older version is running on CW & Qualified that is fully functional
Java	1.8.0_91	✓		junit	junit	junit	java-runner	junit
JavaScript	0.10.33	✓	✓	cw-2	cw-2	cw-2,	node-runner	cw-2
JavaScript	0.10.33/Babel	✓	✓	cw-2	cw-2	cw-2,	node-runner	cw-2
JavaScript	6.11.0	✓	✓	cw-2, mocha	cw-2	cw-2, mocha	node-runner	cw-2
JavaScript	6.11.0/Babel	✓	✓	cw-2, mocha	cw-2	cw-2, mocha	node-runner	cw-2
JavaScript	8.1.3	✓	✓	cw-2, mocha	cw-2	cw-2, mocha	node-runner	cw-2
JavaScript	8.1.3/Babel	✓	✓	cw-2, mocha	cw-2	cw-2, mocha	node-runner	cw-2
Julia	0.4.6	✓		factcheck	factcheck
Kotlin	1.1.4	✓		junit4,kotlintest			gradle-runner
Lisp		✓			Kumite Only		func-runner
Lua	5.2	✓		busted	busted		lua-runner
Objective-C	2.0	✓		UnitKit	UnitKit		objc-runner
OCaml	4.02.3	✓		ounit	ounit		ocaml-runner		Tests should be placed in a module called "Tests", in an array of OUnit labels named "suite"
Perl		✓			Kumite Only		*
PHP	7.0	✓		cw-2, phpunit	phpunit	phpunit, cw-2	alt-runner
PowerShell	6.0.0-beta.4	✓		Pester			powershell-runner	Pester
Python	2.7.6	✓	✓	cw-2, unittest	cw-2	cw-2, unittest	python-runner	cw-2
Python	3.4.3	✓	✓	cw-2, unittest		cw-2, unittest	python-runner	cw-2
Python	3.6	✓	✓	cw-2, unittest		cw-2, unittest	python-runner	cw-2
R	3.4.1	✓		testthat			r-runner
Racket		✓			Kumite Only		func-runner
Ruby	2.3.0	✓	✓	cw-2, rspec	cw-2	cw-2, rspec	ruby-runner	cw-2
Rust	1.15.1	✓		rust	rust
Scala	2.12.3	✓		junit4,scalatest			gradle-runner
Scss/Sass		???		???
SQL	SQLite3	✓	✓	rspec
SQL	Postgres 9.6	✓	✓	rspec
Swift	3.1.1	✓		xctest	xctest
TypeScript	2.4	✓		mocha	mocha		node-runner		TypeScript utilizes require instead of concatenating files

Setup

You should have Docker installed, if not do that first. Before you can run any of the code environments you will need to build the proper Docker image. To get started lets work with the node image.

Run make node to build the base and node images. This will take a few minutes. You can speed up the process by first downloading the existing images that you intend to work on, which will allow you to only need to build when you make a change.

For example, if you intend to work on the jvm image, you would do this:

# download existing images first to greatly speed up time
docker pull codewars/base-runner
docker pull codewars/jvm-runner

# now build the jvm image, so that any recent changes that have may not have been pushed yet get added into the image
make jvm

Once you image is downloaded/built, you can create a container to work within it. Doing this means you do not have to worry about having any of the project dependencies loaded directly on your machine.

Run the following command:

docker run \
    -it \
    --rm \
    --entrypoint bash \
    -v $(pwd)/lib:/runner/lib \
    -v $(pwd)/examples:/runner/examples \
    -v $(pwd)/frameworks:/runner/frameworks \
    -v $(pwd)/test:/runner/test \
    codewars/node-runner

This will create a new container and send you into the instance with your project's lib and test directories mounted as volumes. Mounting as a volume allows you to change files on your local machine and have those changes available to you from within the container.

Notice: We did not mount the entire directory because that would overwrite things such as your node_modules directory. If you need to update these you should make {image_you_want_to_update} the image to ensure you are always testing against the correct packages.

Docker Compose

We mentioned before that you also have the option of using Docker Compose to run the CLI tool. We have setup the docker-compose.yml file to provide very useful pre-configured services for making development easier. Instead of having to issue the long command mentioned above, you can simply run docker-compose run node-runner to bash into a fresh container with your local volumes already mounted.

All of the docker compose services are setup to mount volumes to make development easier, so that is the recommended way of interacting with the codebase. You should note though that the compose file is unable to build images due to how the directory structure is laid out, so you have to first make {runner_name} the image before you can run it. Otherwise it will pull down the latest copy from Docker Hub.

Running Tests

Once you are in the Docker image, you can run tests as a part of your development flow. For example:

# inside of container
mocha test/runners/typescript_spec.js

# or from outside the container
docker-compose run typescript_test

Test Suite Output Format

A custom and very basic format is used for sending data out of the CLI tool. All formatted data is returned via STDOUT. If you do nothing but write normal strings to STDOUT, then codewars.com will display each line as you would expect, unformatted.

A small subset of commands is supported that can be used to format output. They are:

• <DESCRIBE::>
• <IT::>
• <PASSED::>
• <FAILED::>
• <ERROR::>
• <COMPLETEDIN::>

Prefixing a new line with these commands will cause that line to be formatted. Since each new STDOUT line is considered a new piece of data, if you wish to format multiple lines as one item (such as a multi line "passed" message), then you must replace all \n line feed characters with the <:LF:> token.

For example, in Ruby, if you wanted to write a multi-line passed message:

def passed(msg)
  puts "<PASSED::>#{msg.gsub("\n", "<:LF:>")}"
end

Nested Describes

Some test frameworks support nested levels of describes. In order for our output to support multiple levels, you must also use the <COMPLETEDIN::> token, which acts as a terminator for the current item. This should be used to terminate both <DESCRIBE::> and <IT::> statements.

The following is a full example of what the output might look like, that supports nested describes:

<DESCRIBE::>Foo
<IT::>It should return a string
<PASSED::>Test Passed
<COMPLETEDIN::>23
<IT::>It should return "foo"
This is some direct output (i.e. console.log("..."))
<FAILED::>Expected "foo" but instead got ""
<COMPLETEDIN::>10
<DESCRIBE::>This is a nested describe
<IT::>Should not be null
<PASSED::>Test Passed
<COMPLETEDIN::>20
<COMPLETEDIN::>22
<COMPLETEDIN::>100

Notice how there are 3 <COMPLETEDIN::> statements at the end. The first one completes the last IT statement, the 2nd completes the nested DESCRIBE and the 3rd completes the top level "Foo" DESCRIBE.

COMPLETEDIN:: Details

The value of COMPLETEDIN should be time spent executing the related statement, in milliseconds. It is not required to support time. <COMPLETEDIN::> is valid on its own, and in that case it is only used to terminate the current statement.

Why the custom format?

Getting different test suites in different languages to all play together with the same format can be tricky. In many cases, customizing the test suite output is very limited (sometimes requiring hacking). Because of this, using formats such as XML and JSON are complicated because its not always possibly to correctly close out the data format when a program raises an exception.

The format chosen was originally done so that at any point in time the program could exit while still having readable data. Other formats, such as TAP (Test Anything Protocol) could also be an option. However another requirement that we had when designing the format was to have it be incredibly simple yet flexible, so that Codewars.com could support more than simply outputting test results. With the current format there is nothing stopping you from outputting HTML, JS canvas code, etc in order to create a rich and even interactive test result output.

How to add a new language

Note: These steps all assume you are adding a completely new language to the project. Many languages are currently in an incomplete state so not all steps may be needed in your case

1. Install the language and its related packages on one of the Docker images. We have grouped many of the Docker images together so if that grouping makes sense then add it there, otherwise you will need to create a new docker image within the docker folder
2. Add the newly created docker file to the Make and docker-compose files (if applicable)
3. Add a new runner script within lib/runners. More details about this script later on.
4. Add a new runner spec within test/runners. You will also need to add the runner spec to the Docker image so that it is tested as a part of the build process.
5. Add a new examples yml file within the examples folder. These are the code examples that are used on Codewars.com when a user clicks the "Insert Example" button within the kata editor. There is also a helper available for running your examples as a part of the test suite.

Runner Script

The runner script is responsible for outputting a run method. This method utilizes the shovel helper which will handle all of the inter-process communication for you. The shovel config accepts strategies to determine how to handle running the code based on the options passed in. There are currently two types of strategies.

• solutionOnly: Code is simply executed and its STDOUT returned. There is no test integration.
• testIntegration: Code is executed within a configurable test suite and the test output is returned via STDOUT.

Each of these strategies is passed in a run method which is used to ultimately execute the final command.

codewars-runner-cli's Issues

Escape character's for user's console.log

If the user writes console.log('<PASSED::>All tests passed') this shouldn't be a new passed test.
By escaping < and >, this can be avoided.

IDL

The preferred data processing language for the working astronomer. It's okay if you haven't heard of it. It's proprietary, but the good news is that there's a free dialect.

Batteries Included

For a lot of languages, there’s not much built into the core, with the exception of Python and maybe Racket.

Here's a list of core libraries/frameworks I can think of for various languages:

• Python
  • pandas
  • numpy
  • scipy
  • django
  • jinja2
  • requests
  • sh
  • matplotlib
  • twisted
  • Niche Stuff
    • scikit-learn
    • pymc
    • PIL
    • OpenCV
    • NLTK
    • PyQuery
• Java
  • Spring
  • h2
  • Apache Commons
  • Tomcat
  • Google Guava
• Javascript
  • JQuery
  • D3.js
  • React
  • 〈Your favorite MVC Framework〉(mine's Angular)
• Clojure
  • ^^^The JAVA Stuff^^^
  • java.jdbc
  • core.async
  • core.logic
  • core.matrix
  • core.match
  • core.unify
  • core.cache
  • prismatic/schema
  • prismatic/plumbing
  • compojure
  • hiccup
  • enlive

More later...

CI For Github

Can we hook into Travis CI? Would be nice...

Full Embedded Environment for ARM Assembly

A little bird told me that the coolest thing to learn in ARM assembly, a black art that is almost impossible to learn online, is interrupt request handling.

Here are a couple of blog posts related on how to do this QEMU:

http://balau82.wordpress.com/2010/02/28/hello-world-for-bare-metal-arm-using-qemu/
http://balau82.wordpress.com/2012/04/15/arm926-interrupts-in-qemu/

Hide Python's `reload` somehow

So a kata writer tried to block itertools.permutations for a little premutations kata in python.

This is really easy to sidestep:

def permutations(*args): 
  return (x for x in reload(__import__('itertools')).permutations(*args))

We need to figure out how to hide reload somehow...

Fix <:LF:> for Line Feeds in Haskell, Clojure, Julia

I could do this, however:

1. I'm not 100% sure what I'm supposed to do exactly, since I don't know how the downstream code that parses the test framework output works
2. I'm siloed as the sole developer for these things. What's worse, I'm not an employee... so I really think someone else should touch this code (I'll review it)

C Threads

Cheating by replacing the global object

After reading the source code here: https://github.com/entrefuse/cdr/blob/master/frameworks/javascript/cw-2.js

I have found a way of cheating that works for almost every javascript kata.
The key to it is hacking the global object, and the trick to it is that although the crucial keywords of global can't be modified because of defineProperty, but global can be REPLACED, and then console.log('<PASSED::>All tests passed')

Erlang Test Framework

This has not been written, so Erlang can't be said to be done...

Find a way to enforce disc limits on a per container basis

Java 8?

Want to use Oracle's Java 8, rather than OpenJDK 6?

Python3 support

Scala support

Javascript Runner Broken

Not sure how this happened, I hope it wasn't one of my commits...

But when I run:

docker build -t codewars/cli-runner . ; docker run -i --entrypoint mocha codewars/cli-runner test/*

I get:

...
  2 failing

  1) javascript runner .run cw-2 error handling should gracefully handle reference errors:
     Uncaught AssertionError: expected '<DESCRIBE::>test\n<IT::>test1\n<ERROR::>ReferenceError: b is not defined<:LF:>        at Test.it<:LF:>        at Test.describe<:LF:>            <:LF:>            \n<COMPLETEDIN::>2\n<IT::>test2\n<PASSED::>Test Passed\n<COMPLETEDIN::>1\n<COMPLETEDIN::>9\n' to include '\\n'
      at /codewars/test/runners/javascript_spec.js:77:50
      at /codewars/lib/shovel.js:16:9
      at ChildProcess.<anonymous> (/codewars/lib/shovel.js:106:9)
      at ChildProcess.emit (events.js:98:17)
      at Process.ChildProcess._handle.onexit (child_process.js:809:12)

  2) javascript runner .run cw-2 error handling should gracefully top level handle reference errors:
     Uncaught AssertionError: expected '<ERROR::>ReferenceError: b is not defined<:LF:>        <:LF:>            \n' to include '\\n'
      at /codewars/test/runners/javascript_spec.js:91:50
      at /codewars/lib/shovel.js:16:9
      at ChildProcess.<anonymous> (/codewars/lib/shovel.js:106:9)
      at ChildProcess.emit (events.js:98:17)
      at Process.ChildProcess._handle.onexit (child_process.js:809:12)

Load runner files in run.js

Most languages don't actually support running with solutionfile.

Upgrade Erlang to R17+

the erlang vm used is R16B03. Please upgrade it to R17+, since the introduction of "Named Funs" makes it far easier to write recursive functions (Maps were also added)

http://joearms.github.io/2014/02/01/big-changes-to-erlang.html.

what TDD test framework does codewars use for javascript?

Elixir

No great reason not to support this if we are supporting Erlang

Java is outputing <:LF:> tags, possibly HTML escaped

Within the Codewars UI, <:LF:> tags are being shown, which probably means they are being escaped before they are able to be translated back into new lines.

See http://www.codewars.com/kumite/53ffdfd2153f95440b0002d6?sel=53ffdfd2153f95440b0002d6

Haskell support

Update documentation

At the moment, it's not really clear what has to be provided to add support for a new language or to upgrade to a newer version of a compiler/runtime. Apparently one needs a docker image (or change the full image?), and a test framework. But this doesn't include building/running strategies for the actual tests, which are essential.

I'm not sure whether those informations are considered internal by you or whether they're simply missing in this repository, but they're required to create support for something new.

go support

Ruby file permissions

require 'yaml/store'
store = YAML::Store.new 'candy.yml'

is failing due to file access error.

c# support

Check out this repo for a version of the CW test framework ported over to C#. This is an older version that writes HTML output directly, so that code needs to be updated.

Coq, HOL-Light, Isabelle/HOL, or Isabelle/ZF

I've always dreamt that mathematics education, in particular formal proof, could largely be automated.

Codewars could be a platform for this... it could be crazy awesome.

Tests Are Broken In Docker Image

When I run:

docker build -t codewars/cli-runner . ; docker run -i --entrypoint mocha codewars/cli-runner test/*

I get:

  1) go runner .run should handle basic code evaluation:
     Uncaught Error: spawn ENOENT
      at errnoException (child_process.js:1000:11)
      at Process.ChildProcess._handle.onexit (child_process.js:791:34)

  2) java runner junit should handle basic junit tests:

      Uncaught AssertionError: expected '' to equal '42\n'
      + expected - actual

      +"42\n"
      -""

      at /codewars/test/runners/go_spec.js:15:42
      at /codewars/lib/shovel.js:14:9
      at exit (/codewars/lib/shovel.js:76:9)
      at null._onTimeout (/codewars/lib/shovel.js:135:13)
      at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)

  3) java runner junit should handle junit tests failing:

      Uncaught AssertionError: expected '<DESCRIBE::>myTestFunction(TestFixture)\n<FAILED::>Failed Message expected:<5> but was:<3>\n' to equal '<DESCRIBE::>myTestFunction(TestFixture)\ntest out\n<PASSED::>Test Passed\n'
      + expected - actual

      +"<DESCRIBE::>myTestFunction(TestFixture)\ntest out\n<PASSED::>Test Passed\n"
      -"<DESCRIBE::>myTestFunction(TestFixture)\n<FAILED::>Failed Message expected:<5> but was:<3>\n"

      at /codewars/test/runners/java_spec.js:42:43
      at /codewars/lib/shovel.js:14:9
      at ChildProcess.<anonymous> (/codewars/lib/shovel.js:124:9)
      at ChildProcess.emit (events.js:98:17)
      at Process.ChildProcess._handle.onexit (child_process.js:809:12)

  4) javascript runner .run cw-2 error handling should gracefully handle custom errors:
     Error: timeout of 2000ms exceeded
      at done (/usr/local/lib/node_modules/mocha/lib/runnable.js:202:67)
      at /usr/local/lib/node_modules/mocha/lib/runnable.js:223:9
      at /codewars/test/runners/java_spec.js:68:17
      at /codewars/lib/shovel.js:14:9
      at ChildProcess.<anonymous> (/codewars/lib/shovel.js:124:9)
      at ChildProcess.emit (events.js:98:17)
      at Process.ChildProcess._handle.onexit (child_process.js:809:12)

  5) javascript runner .run cw-2 error handling should gracefully handle reference errors:
     Uncaught AssertionError: expected '<DESCRIBE::>test\n<IT::>test1\n<ERROR::>ReferenceError: b is not defined<:LF:>        at Test.it<:LF:>        at Test.describe<:LF:>            <:LF:>            \n<COMPLETEDIN::>1\n<IT::>test2\n<PASSED::>Test Passed\n<COMPLETEDIN::>0\n<COMPLETEDIN::>15\n' to include '\\n'
      at /codewars/test/runners/javascript_spec.js:85:50
      at /codewars/lib/shovel.js:14:9
      at ChildProcess.<anonymous> (/codewars/lib/shovel.js:124:9)
      at ChildProcess.emit (events.js:98:17)
      at Process.ChildProcess._handle.onexit (child_process.js:809:12)

  6) typescript runner .run should handle basic code evaluation:

      Uncaught AssertionError: expected '' to equal '42\n'
      + expected - actual

      +"42\n"
      -""

      at /codewars/test/runners/typescript_spec.js:9:42
      at /codewars/lib/shovel.js:14:9
      at ChildProcess.<anonymous> (/codewars/lib/shovel.js:124:9)
      at ChildProcess.emit (events.js:98:17)
      at Process.ChildProcess._handle.onexit (child_process.js:809:12)

JSON Intercommunication or Dependency Support

We have some other issues open regarding restructuring the containers, and I know that this is being planned for.

The idea of having small services that intercommunicate via JSON is cool. I'd like to get started thinking about how to support this.

Notes:

• One difficulty with this problem is: while it is straightforward to wrap a function in a dynamically typed language in something that calls it with arguments read from a JSON, this gets to be more challenging and ambiguous for typed languages and systems languages.
• What happens if two services call one another, and get into a loop? How will this be prevented?
• There's a lot of overhead waiting for a service. Something similar would be if we could let kata/kumite depend on other kata/kumite as libraries. In the case of GCC/LLVM languages, this would be a matter of linking object code together after compilation. In the case of python, this would be a matter of writing out modules into directory trees in a sane/predictable way. And so on for other languages.
  • If we ever decide to support Isabelle or Coq, then this would be a natural mechanism for stacking lemmas and theorems together to prove complex facts.

Erlang Needs a Killswitch Thread

From time to time when I am working on testing erlang, my test crashes and then erlang hangs and can't start another VM.

Erlang needs a thread that always starts at boot and kills the VM after some timeout.

Rspec support

Java support

Julia Fast Fail

Julia should fail fast

Expose the Disassembler in Node.JS

Right now we don't have disassembly enabled for node.js

To illustrate:

http://www.codewars.com/kumite/53f4d3a067dcf4c2a500197b?sel=53f4d3a067dcf4c2a500197b

This would be nice for arguing over micro-optimization...

Daemons / Slow Test Runners

When I was writing the clojure runner, I found its performance was pitiful. I expect scala to be even worse. It would be pretty awful, but not surprising, if hello world in scala took more than 5 seconds.

One solution to the performance problem would be to switch over to a queue, like RabbitMQ Reddis, or SMQ, and have persistent JVMs process jobs off the queue. I recognize this is painful to implement and test, but otherwise you are looking at 1 to 2 seconds of load time you want to do anything with the JVM.

Replace sync code with async code.

Much of the code has been implemented in their sync forms. The exec-sync package was recently added to the project as well. This package is not meant to be used in production code.

All code should be asynchronous to ensure that the server doesn't lock up when dealing with additional requests.

ASM

I have some contacts in industry and aerospace that have expressed that they would like to see this. In particular, people want

• x86 ASM (with MMX?)
• ARM ASM
• PTX

I'm not sure the vagrant box supports MMX extensions...

Also, we need to figure out a test framework for this.

Separate docker files

The single Docker file has gotten out of control. At this point we end up having to rebuild most if not all of the thing too often. This causes space issues as well as upload issues since all of the old stuff can't be re-used. I want to try to group common sets so that we don't have 20 images to manage. Here is what I am thinking:

Base Image:

• ENV config
• setup codewarrior user
• common services (redis, mongo, sqlite, elasticsearch, etc)
• any packages commonly used by installers (python libs, node, etc).

Node Image

• node
• coffeescript
• typescript
• npm packages

Ruby Image

• ruby
• gems
• run ruby tests

Python Image

• python 2
• python 3
• language packages
• run python tests

.NET Image

• c#
• f#
• dlls
• c#/f# language tests

JVM Image

• java
• groovy
• clojure
• scala
• language packages
• JVM runner
• run jvm language tests

Systems image

• c
• c++
• asm/arm
• language packages
• ?
• language specific tests

Lispy Image

• Lisp
• Racket
• OCaml
• Haskell
• language packages
• tests

Alt Languages Image (less commonly used)

• R
• Rust
• Erlang
• Elixir
• Go
• Julia
• Lua
• language packages
• tests

We would deploy the runner to all of the non-base images just as it is today. The only difference is that the server will need to pick the image based off of the language provided. We will also need to update the build, pull and push utilities to do all of the image magic for us.

VHDL

Don't be Root

It's really bad that the runner runs as root. It should probably just run as a set user, who doesn't have permission to do much at all other than write some stuff to /tmp, execute some executables and read some files.

Fortran

Drinks?

I'm in LA for the week, wanna go get drinks?

(I recognize this is the wrong way to ask, sorry...)

F# support

I'd love to help get F# katas running in codewars. AFAICT the .NET runner already supports F#, so I was wondering what else needs to be done to get to the point where we can start porting katas over to F#?

Python2 support

C++ support

Swift Support

Verilog

Rust support

First stable version was released today.
I would love to help, btw.

Write All Temporary Files to Directories in /tmp, Delete them When Done

I'm using node-temp for my haskell runner, since I noticed that the runner was leaking files. Now all the files are deleted right after they are created and run, and they all live in unique directories so they can never collide.

I am of the opinion that all of the runners should be done this way.

Haskell build issue on "docker_files" branch

Run node build haskell to see error. If your first run ever on the new branch then you should run node build base haskell.

Issue is related to codewarrior user installing cabal packages.

Docker Build is failing

Step 51 : RUN echo"Package: racket-docPin: origin ''Pin-Priority: -1" >> /etc/apt/preferences
 ---> Running in 6362be0ef402
/bin/sh: 1: echoPackage: racket-docPin: origin ''Pin-Priority: -1: not found
2014/08/18 02:34:07 The command [/bin/sh -c echo"Package: racket-docPin: origin ''Pin-Priority: -1" >> /etc/apt/preferences] returned a non-zero code: 127