variable instance lost because RuntimeService#setVariable behaves like VariableScope#setVariable(_,_,false) about flowable-engine

[jbarrez]

The fetchVariables being false is the right value in this case, as not all variables should be fetched when only one variable is set.

If the variable is not in the cache and the variableInstances are empty, we're in the 'else' clause here: https://github.com/flowable/flowable-engine/blob/master/modules/flowable-engine/src/main/java/org/activiti/engine/impl/persistence/entity/VariableScopeImpl.java#L686. At that point the getSpecificVariable is called, which only fetched the one variable. The createVariableInstance should not be called.

So my guess is that, since you mention nested workflows, something goes wrong here with the executionId that is being used here. I've tried with a few simple processes here, but was unable to reproduce. It would help a lot if you could post an example process definition / structure where you see this problem.

[QAhell]

Thank you very much for your quick response. Actually, we also depend on RuntimeService#setVariable to create the variable instance. I should have stressed that. We want the call to createVariableInstance to happen because getSpecificVariable returns null. It's just that the new variable instance seems to be limited to a very small scope now.

Unfortunately our workflow definitions are confidential and rather complex and I didn't get the time budget to extract a test case that shows the problem. So these are all the details I can provide, at least for now.

[jbarrez]

Ok, thanks for the clarification around createVariableInstance, that makes it indeed a bit clearer.
How are you calling the runtimeService#setVariable ? Are you calling this from within a JavaDelegate or ActivityBehavior?

In what kind of context is the setVariable called? You mention that the scope is more limited now, from which I infer that you use the variable somehow directly after that and that it's failing now?

No problem about not being able to share the workflow definition, that happens often ;-). But it would help here if you could explain what you mean with 'nested workflows' and how/when the setVariable is called when it's failing.

[QAhell]

With nested workflow I mean that we use a <callActivity> to start a sub-process, but at the moment I don't think that having this kind of nesting is necessary to reproduce the bug.

We set the execution explicitly and set the variables like this:

Task task = taskService.createTaskQuery().taskId(taskId).singleResult();
Execution ex = 
  runtimeService.createExecutionQuery().executionId(task.getProcessInstanceId()).singleResult();
try {
  setExecutionContext((InterpretableExecution) ex);
  runtimeService.setVariable(ex.getId(), "variableName", someValue);

I guess we do this because of an issue in version 5.9. A comment indicates that we would rather use CDI.

After that we complete the task and need the variable during its completion.

  // We need the variable in a TaskListener.
  taskService.complete(taskId);
} finally {
  removeExecutionContext();
}

The funny thing is that it works when we insert the following just between the setVariable and the call to complete:

if (Context.getCommandContext() != null && Context.getCommandContext().getDbSqlSession() != null) {
  Context.getCommandContext().getDbSqlSession().flush();
}

Do we have to flush manually if we set the execution manually? We don't set the command context explicitly, however.

I found out about this flush because our test case runs two completions, one without a command context reuse (CommandContextInterceptor line 65) and one with a command context reuse. The call without the context reuse triggered a flush, the other one didn't.