Can't make fatigue filter work with proxy entities about sensu-go-fatigue-check-filter

[nixwiz]

Also the sensu documentation says that the annotations are at check.annotations, so I should be missing something.
Aren't the event received by the filter and the event that is received by the handler the same (without any mutator)?

Actually no, they are not. Filters are ran inside an Otto javascript VM within the Sensu backend and the version of the event they are passed has the annotations (and other metadata) at the top level of the check and entity objects. They are moved into the ObjectMeta (metadata) before they are sent to the handler(s).

As for the issue of the filter never letting it pass, let's look at all of the filters you have applied, at least to the one handler definition you provided:

  filters:
  - is_incident
  - not_silenced
  - state_change_only
  - fatigue_check

Let's review these in order, minus the not_silenced filter as it is not coming into play for what you are seeing:

The is_incident filter "allows only warning ("status": 1), critical ("status": 2), and resolution events to be processed."

The state_change_only filter I assume you got from the docs. So that will only pass through events on the first occurrence.

Finally the fatigue_check filter with annotations set to 2 occurrences, 1 hour interval, and no is_resolution events.

So on first failure of a check (occurrences = 1, status = 2) the is_incident filter will pass this as true, the state_change_only filter will pass this is true, but finally the fatigue_check filter will return false because occurrences != 2 and the event is dropped and not sent to the handler.

On the next interval, if still critical (occurrences = 2, status 2), the is_incident filter will pass this as true, however the state_change_only filter will return false since occurrences != 1. The fatigue_check filter is not ran because we've already received a false from the prior filter. This event is dropped and not sent to the handler.

Let's say on the next interval the problem is corrected and we now have (occurrences = 1, status = 0, is_resolution = true). The is_incident filter will pass this as true due to is_resolution being true. state_change_only will pass it as true since occurrences == 1, however since you have allow_resolution set to false in the annotations, the fatigue_check filter would return false and the event would get dropped and not sent to the handler.

Personally, I would suggest dropping the state_change_only filter and you should then start to see events passed to your handlers as you would expect.

Yes! you are right! The state_change_only filter was causing the problem. It's pretty obvious now thanks to your explanation.
Also i haven't heard about the otto VM I will investigate it

You saved me a lot of time, thank you very much!