Add application name to appcontroller logs about argo-cd

[crenshaw-dev]

I'd be very excited to see this PR.

[crenshaw-dev]

We should probably consider switching to JSON by default in 3.0. But the option is already available today.

[andrii-korotkov-verkada]

Turns out the problem is more complicated than I thought. While some logs in appcontroller don't have info about the application, a lot of them include it in the structured data, including the example I've shown. However, when viewing the logs in the Datadog, application's fully qualified name is not available in the parsed event attributes and also not added to the log line (as it probably shouldn't). There's also been an example of logs where fields are added to the log message but in a way that indicate some broken parsing.

So the source of the issue lies in ArgoCD not using JSON format recommended by Datadog for integration, and the custom Grok parser for the ArgoCD logs we have seems to not cover some edge cases (probably / in the unquoted value of the field).

I'd open a discussion about potentially switching to JSON log format or finding other ways to have parsing working better.

[andrii-korotkov-verkada]

Oh, my search for potential usage of JSONFormatter was too naive. After your pointer, I've traced that to util/log/logrus.go.

[andrii-korotkov-verkada]

I'd just run the controller with the corresponding option, though still plan to submit the PR for some logging improvements.

[andrii-korotkov-verkada]

The PR is ready and passes CI. I'm not 100% sure though whether to log an app fully qualified name, or log a regular name and namespace separately. There's no consistency now - audit logger and couple of places choose the later, while multiple other places choose the former. In my PR I gave preference to the qualified name, but didn't go as far as editing audit logger. What do you think?

[andrii-korotkov-verkada]

Thinking more about this, I'll probably switch to using a regular app name and have namespace as a separate parameter, as well as separate field for a qualified name. It's going to be reasonable when querying structured data via Datadog and also supporting just using grep on raw logs.