Adding `grpc_code` label to prometheus `grpc_server_handling_seconds` metric about go-grpc-middleware

[bwplotka]

Yea. I think with native histograms it would be easier decision. Wonder if it makes sense to be early adopter of native histograms here for this reason cc @beorn7

Otherwise we could add opt-in grpc_code, that's also fine.

[jon-whit]

Related to grpc-ecosystem/go-grpc-prometheus#75, but reviving since that repository is now deprecated.

[johanbrandhorst]

Sounds like the consensus in the other thread is that adding grpc_code would be too much, but that having a success/failure signal would be nice. I don't know enough about the prometheus library to say if this is possible, perhaps @bwplotka can weigh in on here too.

[jon-whit]

Sounds like the consensus in the other thread is that adding grpc_code would be too much, but that having a success/failure signal would be nice. I don't know enough about the prometheus library to say if this is possible, perhaps @bwplotka can weigh in on here too.

@johanbrandhorst I'm not sure I agree with that conclusion. I wouldn't say any consensus was yet achieved. There are still community members requesting it and the "consensus" was merely the response of the maintainers. But, in any case, why is it not possible to allow the client to choose whether to include it or not? For me the increase in cardinality is well worth the value I get from it. I should be able to enable this label if I want and understand that there is a tradeoff. For example, I should be able to

var serverMetrics *grpc_prometheus.ServerMetrics

serverMetrics.EnableHandlingTimeHistogram(
    ...,
    WithStatusCodeLabel(),
)  

If the WithStatusCodeLabel option is provided to the handling time histogram, then that label should be emitted and a value provided for it in each observation. The default can be to omit the grpc_code label, which maintains equivalent behavior with what is standard today, but if you want that label and are willing to increase cardinality, then that should be the developer's choice.

This work ☝️ can be in addition to switching to a native histogram so as to further reduce the impact/burden if you want to enable the grpc_code label.

If I submit a PR, will you guys consider merging it? If so, I'd be happy to submit one this week.

[beorn7]

It would be great to offer native histogram support, but I would recommend to also support classic histograms in parallel so that scraper that aren't ready for native histograms yet will still get a classic histogram.