reusable output node about gmxapi

[peterkasson]

Just checking--this is a workflow node but not one that requires communication between contexts, right? i.e. it could effectively be a map() operation so contexts do their output independently.

[eirrgang]

I see what you mean about a map() operation, but that's not quite what I was getting at here. I've updated the description in the hopes of clarifying.

Map operation is a good thing to bring up, though. The grand plan is to have parallel execution essentially work that way, but right now it is rigidly structured as an array of pipelines with possible operations across the full array (ensemble operations). That is, we have simulation scope and ensemble scope for operations or data flow. I think the first step in allowing something more map() like would be to explicitly expand the parallel edges and nodes in the execution DAG. Right now we have it more as "array operations of a given width can feed array operations of the same width, with the exception of many-to-one and one-to-many". Otherwise, we start dealing with more scheduling logic, which I would rather defer to specific context implementations (and try to pawn it off to third-party data flow graph execution managers). I think some of the necessary metadata will become clearer as we implement full workflow checkpointing.

[eirrgang]

Recording some comments from the pull request pull request that may not be resolved in that pull request:

In current plans for gmxapi 0.1.0, each operation in each identifiably distinct work description is uniquely tagged. In order to implement [more output flexibility], we need to be specific about behavior and artifact handling.

• What indicates whether two MD operations should continue a previous simulation as if continuing from checkpoint?
• What indicates that an MD operation should use another simulation output as input, but start a fresh trajectory and log files?
• For an MD operation that continues by appending to another's output, how do we determine whether to retain the snapshot of the output from the first operation (distinct artifacts) rather than to let the second operation completely take ownership (as if they were the same operation)?

An update should specify behavior in a sufficient level of detail that we can discuss whether the specified behavior as written should be different.