Below is user feedback via Slack. Posting it here so it doesn't get lost with Slack message rollover
StackStorm's New Workflows
Unit Testing
Currently, in my opinion, the biggest downfall of Mistral is the lack of unit
testing around workflows. Without unit testing we're relying on smoke testing to
ensure these workflows behave properly (not very effective or maintainable).
Things we would like to test in a workflow include:
- Task execution yes/no
- Task execution order
- Error handling
- Publish expressions (Jinja, YAQL)
- Transition expressions (Jinja, YAQL)
- Workflow inputs and outputs
I think this would require some sort of ability to mock action output so that
these things can be tested.
Publish and reuse in same task
Currently, in Mistral, all of the publish
and publish-on-error
statements
occur in bulk meaning i can't reuse one of the variables published in the
current task until the next task is executed.
Example:
task1:
action: core.local
input:
cmd: "echo hello"
publish:
output: "{{ task('task1').result.stdout }}"
# this errors out because output isn't available in the contex yet
hello_world: "{{ _.output + ' world' }}"
Maybe converting the publish
statement to an array would make it easier
to execute in sequence?
task1:
action: core.local
input:
cmd: "echo hello"
publish:
- output: "{{ task('task1').result.stdout }}"
# output should now be available in the current context, so the following
# should work
- hello_world: "{{ _.output + ' world' }}"
When condition on a task level
Sometimes it's necessary to skip a task given a condition. Currently we have to
work around this by adding this "skip" condition into every on-success
statement
in the workflow that may call our task. This is a maintenance burdem.
Ansible example:
- name: install stackstorm pack
shell:
cmd: "st2 pack install {{ st2_pack }}"
when: st2_pack is defined
Mistral example (current implementation):
input:
- servicenow_provision_id: null
task_1:
action: std.noop
publish:
last_task: "task_1"
on-success:
- task_servicenow_update: "{{ _.servicenow_provision_id }}"
- task_2
task_2:
action: std.noop
publish:
last_task: "task_2"
on-success:
- task_servicenow_update: "{{ _.servicenow_provision_id }}"
task_servicenow_update:
action: encore_servicenow.provision_state_update
input:
provision_id: "{{ _.servicenow_provision_id }}"
current_state: "{{ _.last_task }}"
Mistral example using a when
condition on task_servicenow_update
task:
input:
- servicenow_provision_id: null
task_1:
action: std.noop
publish:
last_task: "task_1"
on-success:
- task_servicenow_update
- task_2
task_2:
action: std.noop
publish:
last_task: "task_2"
on-success:
- task_servicenow_update
task_servicenow_update:
action: encore_servicenow.provision_state_update
when: "{{ _.servicenow_provision_id }}"
input:
provision_id: "{{ _.servicenow_provision_id }}"
current_state: "{{ _.last_task }}"
"Main" task / entry-point
There are many occasions on Slack where users of Mistral are confused by its
default execution behavior. People usually forget to tie tasks together with
explicit on-success
, on-error
, or on-complete
statements, causing tasks
to be run in parallel resulting in odd and unpredictable behaviors.
It might be better to explicity define a "main" or "entry-point" in the workflow.
description: My entry-point workflow
input:
- x
- y
- z
# the tasks that should be executed as "roots"
# this can either be a string, or a list so that >1 can be executed in parallel?
entry-point: my_first_task
tasks:
my_first_task:
action: std.noop
on-success:
- some_other_task
some_other_task:
action: std.noop
Linear execution by default
On the same lines as the last point about "main tasks" people new to Mistral are
often confused by the non-linearity as the default operating state of the
workflow engine.
Maybe as an alternative to the "main task" idea above, workflows could be executed
linearly and in parallel/DAG-optimized using an option. I'm going to
suggest a workflow parameter called execution_model
. The execution_model
with a value of linear
means that tasks are executed in order just like an
actionchain or Ansible. Another implementation could be the parallel
execution
model that will switch it over to a Mistral-like parallel execution by creating
a DAG and executing as much in parallel as possible.
description: >
My linear workflow, all of these are executed in order
without the need for on-success.
execution_model: linear
tasks:
task_0:
action: std.noop
task_1:
action: std.noop
task_2:
action: std.noop
description: >
Parallel execution model tries to do as much in parallel as possible, requires
on-success and on-error to construct our DAG.
execution_model: parallel
tasks:
task_0:
action: std.noop
on-success:
- task_1
task_1:
action: std.noop
on-success:
- task_2
- task_3
task_2:
action: std.noop
task_3:
action: std.noop
Keep: defined inputs and output
We really like having the inputs
and outputs
sections explicity defined.
Please keep these!
Keep: allowed temporaries in workflow
We currently use (and maybe abuse) the ability of having input
values defined
in the workflow that are not present in the parameters
of the action itself.
These are mostly used as temporary variables that are local to the workflow