Allow filter on _source fields about anomaly-detection-dashboards-plugin

[ohltyler]

@prpaluch Thanks for the suggestions! There actually is a new validation API that is being introduced in 1.2.0.0, which will check all of the detector fields and bring up errors if any critical issue is found, including verifying data filters (on both the source data like you've mentioned, as well as custom feature queries).

If using the Dashboards plugin, this validation will run behind-the-scenes and validate your detector configuration (including custom filters) before trying to save the detector. More information can be found in #106. For more information on the API itself, see here.

But, this solution still doesn't provide immediate feedback when writing the filter query, and the validation only runs on a later step in the creation process. Understandably, this may be more useful to validate even earlier as you've suggested.

@amitgalitz as owner of the validation API, do you see the suggestion above as a feasible solution given the implementation of the current API?

[amitgalitz]

Again thanks for the suggestions @prpaluch! Sorry for the late reply but I wanted to update that we released with 1.3 more updates to our Validate API. You should check it out: https://opensearch.org/docs/latest/monitoring-plugins/ad/api/#validate-detector. This API can either ran through the backend or on the last step of creating a detector and it will review the detector configurations. We didn’t implement a check button right next to the data filter as we wanted to take all configurations into account when validating but it will have further validation on the data filter for example on the "review and create" step.



In relation to your first suggestion of having a default example, I will create a separate issue for this for more conversation regarding this enhancement.

 For the last point, I believe supporting something like auto completion for a custom query is out of scope for this plugin. This could be a good feature request for the core Opensearch team to implement. I will be closing this issue and creating a sub issue for your first suggestion and feel free to create a new issue with a specific suggestion for the Validation API as we now have a base for validating configuration which we didn’t have at the time you made this issue.