Allow dedupe statuses when scheduling new orchestration instance about durabletask-dotnet

[davidmrdavid]

Hi @sagimar: thanks for reaching out.
I think I need more context here: when you say that "In the current implementation, the orchestration is actually being triggered twice", do you mean that you're seeing the orchestration being triggered twice one after the other, or do you mean in parallel? Just trying to get clarity around the current behavior you're seeing. Any traces that you may share will help me understand your requirements.

[sagimar]

Hi, thanks for the response!
I'll further explain the issue:
I'm making the following calls (scheduling the same orchestration id twice):

public async Task ScheduleOrchestrations()
{
    await client.ScheduleNewOrchestrationInstanceAsync("orchestrationId", input, options, cancellationToken).ConfigureAwait(false);

    await client.ScheduleNewOrchestrationInstanceAsync("orchestrationId", input, options, cancellationToken).ConfigureAwait(false);
}

The expected behavior: Creation of one orchestration from the first call, and no-op in the second call (idempotence).

The behavior- this is the logs I see from my service:

1. Fetched instance status - RuntimeStatus: NotFound
2. Sending [ExecutionStarted] message, SequenceNumber: 1
3. Updated instances table and set the runtime status to 'Pending'
4. Fetched instance status - RuntimeStatus: Pending
5. Sending [ExecutionStarted] message, SequenceNumber: 2
6. Updated instance table and set the runtime status to 'Pending'

Logs 1-3 are from the first call and 4-6 are from the seconds call. Same happens also if the orchestration was in 'Running' state.

I would expect that after log number (4)- no-op will occur. This could be achieved using dedupeStatuses functionality from azure/durabletask (it will throw an exception instead of sending ExecutionStarted event and updating the instance table).

Thanks!

[davidmrdavid]

Just to confirm, you're saying that you sometimes see an orchestrator already in the Running state get set back to Pending when you call await client.ScheduleNewOrchestrationInstanceAsync?. That seems like a bug to me. The case where Pending is set again to Pending seems safe enough, but the former case is more problematic.

[sagimar]

Yes- this is absolutely what happens (even not sometimes, using storage).
But more than that, I would also want to have the ability to control the idempotence level, so for example also if the state is 'Completed' (also 'Pending' in my opinion)- it won't re-schedule the orchestration.

[violetVo]

@davidmrdavid - any update on this? this is a blocking bug since it breaks the idempotency. As far as I understand from @sagimar comment above, this is a very easy fix on your side (just to expose the extra parameter and call the the method's overload).
Is there an ETA for the fix?

[davidmrdavid]

FYI @jviau - would be good if you could take a look at this one

[violetVo]

@jviau - Any update on this fix? this is a critical bug, and we are blocked on this for a very long time.