Is it possible to remove corrupted data automatically rather than panic? about victoriametrics

[zekker6]

Hello @viperstars

While we understand that this behavior is intentional in VictoriaMetrics, the need for manual intervention each time can incur significant costs.

Usually, the recommendation for such cases is to fix analyze and fix the root cause of such events. If handling of such events can incur significant costs I can assume it happens quite frequently, and it is better to investigate the ways to improve the reliability to avoid the root cause.
The reason this hasn't been implemented yet is that usually when there were similar requests raised it was better to improve the reliability of the platform which hosts VictoriaMetrics.

BTW, is this the right approach to recover, including index and block corruption? #5012 (comment)

This can be used to recover from such cases, but this can lead to partial data loss.
It is much better to perform regular backups and use HA setup. Using HA configuration allows losing data on one or more nodes without affecting data availability in general and give time to perform manual restore from the backup.

[viperstars]

Thanks for the reply, I'm closing the issue now.