Comments (6)
RM sounds like an interesting approach to give more insight into the workflow execution status and progress monitoring. Wouldn't this recovery manager also offer new opportunities to investigate and visualise the workflow progress and errors in a more concise way than our current approach?
BTW, what is meant by "transparency of infrastructure failures"? Isn't this tool oriented to the developer's need to debug his workflows, or would it rather be an infrastructure management tool (infrastructure status/health check)?
from knix.
It was envisioned as an infrastructure management tool. Some failures may be due to the underlying infrastructure (e.g., crashed nodes) and cannot be anticipated by the developers (and probably cannot be expressed in their catch and retry descriptions).
I don't think it would provide additional information regarding the actual progress of the workflow execution. If I recall correctly, the current approach was an alternative of the progress tracker for visualization. In fact, it is exactly the same information it would get as the progress log that we use for visualization.
The reason why the recovery manager would also act as a progress tracker is that it requires to know the progress of the workflow in order to take the correct recovery actions; not because it would be managing the progress of the workflow execution in the failure-free case.
from knix.
from knix.
The data layer would still receive the result of the entire workflow (i.e., when publishing to the exit topic), such that the sandbox frontend can find it in the data layer if needed, especially for 'async' execution. However, that's just for the last function in a workflow.
from knix.
The recovery manager implementation should be a separate issue. Changed the title accordingly.
from knix.
@paarijaat @manuelstein @abeckn: Any other comments/thoughts?
from knix.
Related Issues (20)
- CentOS support for ansible deployments HOT 2
- Caching of downloaded function dependencies
- Shutting down sandboxes may hang HOT 5
- Upgrading KNIX breaks the datalayer HOT 1
- workflow logs not available due to elasticsearch sharding problem HOT 1
- test asl_Map hangs with maxconcurrency values != 0 HOT 3
- allow user-supplied data be included in the trigger from TriggersFrontend
- bare metal installation fails due to missing dependencies HOT 22
- error when deploy the knix on a host machine HOT 5
- Does it mean I deploy successful? HOT 1
- error when workflow.excute. requests.exceptions.HTTPError: 405 Client Error: Not Allowed for url: http://XXXX:32777 HOT 10
- knix can not support parallel states. HOT 6
- Error when executing java function with maven dependency of Gson HOT 1
- Handling of client context for remotely handled messages HOT 4
- Provide more detailed error logs when attempting to test functions
- Triggers SDK update (storage, message queues, timers) HOT 1
- Triggers API object update for Java (storage, message queues, timers) HOT 5
- Recovery manager
- KNIX GPU monitoring/accounting capabilities
- Execution log entries get disordered HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from knix.