After upgrading Orleans8. x, Reminder cannot reliably trigger ReceiveReminder() about orleans

[HermesNew]

I have 20 Reminders. When kubernetes rolls restart the silo node, some reminders may not trigger ReceiveReminder() .

[HermesNew]

When kubernetes rolls to restart the pod, start one first and then close another. After the last node starts and closes, the ReminderGrain activated on that node also closes. In theory, the reminder will be reloaded through the database later, but unfortunately, this did not trigger the reactivation of the reminder.

[ReubenBond]

@HermesNew the reminder never fires again? Are you able to share logs privately?

[ReubenBond]

Could you also provide some info about the reminder dueTime & period and the provider you are using?

[HermesNew]

Yes,the reminder never fires again.
The previous 3. x version did not have this issue, and the same project has the same implementation.

@HermesNew the reminder never fires again? Are you able to share logs privately?

[HermesNew]

dueTime is TimeSpan.FromMinutes(4),period is TimeSpan.FromMinutes(5)

[HermesNew]

We use this AdoNetReminderTable. Reminder database is PostgreSQL 15.0.

[ReubenBond]

@HermesNew are you able to reproduce this with debug logs enabled?

[HermesNew]

Our testing environment does not support logs below warning.
As long as it is a rolling node, there will be some reminders that cannot be fired.
I will enable debug logs in my local environment, reproduce them, and then send out the logs.