Comments (3)
Just checking--this is a workflow node but not one that requires communication between contexts, right? i.e. it could effectively be a map() operation so contexts do their output independently.
from gmxapi.
I see what you mean about a map() operation, but that's not quite what I was getting at here. I've updated the description in the hopes of clarifying.
Map operation is a good thing to bring up, though. The grand plan is to have parallel execution essentially work that way, but right now it is rigidly structured as an array of pipelines with possible operations across the full array (ensemble operations). That is, we have simulation scope and ensemble scope for operations or data flow. I think the first step in allowing something more map() like would be to explicitly expand the parallel edges and nodes in the execution DAG. Right now we have it more as "array operations of a given width can feed array operations of the same width, with the exception of many-to-one and one-to-many". Otherwise, we start dealing with more scheduling logic, which I would rather defer to specific context implementations (and try to pawn it off to third-party data flow graph execution managers). I think some of the necessary metadata will become clearer as we implement full workflow checkpointing.
from gmxapi.
Recording some comments from the pull request pull request that may not be resolved in that pull request:
In current plans for gmxapi 0.1.0, each operation in each identifiably distinct work description is uniquely tagged. In order to implement [more output flexibility], we need to be specific about behavior and artifact handling.
- What indicates whether two MD operations should continue a previous simulation as if continuing from checkpoint?
- What indicates that an MD operation should use another simulation output as input, but start a fresh trajectory and log files?
- For an MD operation that continues by appending to another's output, how do we determine whether to retain the snapshot of the output from the first operation (distinct artifacts) rather than to let the second operation completely take ownership (as if they were the same operation)?
An update should specify behavior in a sufficient level of detail that we can discuss whether the specified behavior as written should be different.
from gmxapi.
Related Issues (20)
- Inconsistent type for WorkElement.namespace and WorkElement.operation
- docker images for 0.0.7 HOT 2
- Avoid race conditions from unsynchronized ensembles HOT 3
- Invalid attempt to use Context.rank in Context.work setter.
- Logic error when checking for operation implementations
- Specifications for referencing external data sources in serialized work
- Enable readthedocs build for 0.1.0.1
- Enable readthedocs build for >= 0.2a
- Provide long description for gmxapi package
- reposority reorganization makes gmxapi 0.0.7 install docs confusing
- commandline_operation should be able to read from STDIN HOT 10
- Validity checks for WorkSpec updates are not applied consistently.
- Accesing commandline output. HOT 5
- TypeError when importing gmxapi 0.0.8 HOT 9
- How to implement trjcat? HOT 3
- Neural Net potential and py interface HOT 1
- Using interactive gmx commands such as trjconv within gmxapi HOT 1
- How to replace coordinates in the simulation input HOT 1
- How to control the operation of MD by output conformation HOT 1
- `gmxapi.commandline_operation` seems to have memory leaks HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gmxapi.