ndejay / ripe Goto Github PK

Controller, WorkerController, Worker and CLI currently share the burden for interfacing with the compute cluster. It would be nice to do as follows:

• Rename Controller to Repo and have it manage only the ripe repository (folders in which .ripe is found.)
• Delegate all job management tasks from Worker to WorkerController.
• It will then be possible to have the CLI properly interface to Controller and WorkerController.

ripe currently supports PBS/TORQUE. Let's make sure that it can work on any stack that contains qsub.

• Remove all dependencies with MOAB or Torque.
• Produce shell scripts instead.

It would be nice to have WorkerController#sync return a list of changed Worker records as was the case prior to v0.2. It currently returns a list of booleans rather than the list of workers.

This is the single most important case of the pipeline and should be generalized into a test.

The current API is exposes too many internals to the end user. It is critical to define a DSL in order to allow the user to easily define workflows and tasks.

By tracking which user submitted jobs, it prevents issues that occur when multiple users are tracking jobs on the compute cluster within the same ripe repo.

It would be nice for the engine to produce a shell file per task that exclusively contains the part of the Worker-level shell script it is involved with.

With all the .sh scripts being written in the .ripe (by default) folder, it's hard to know, for a given sample, which workflow(s) and tasks(s) were applied with what options. It would be nice to have a log file in each sample directory that keeps a trace of what was done.

• Allow users to specify multiple json files with the -c options. Each specified json file needs to be separated by comma.
• The json file(s) will be searched in:
  1. The working directory
  2. The $RIPELIB/config path
• When there are multiple configurations for a given key, issue a warning and use the value defined in the first configuration file where it is described