jaegertracing / jaeger-lib Goto Github PK

Requirement - what kind of business use case are you trying to solve?

Improve rate limiter API.

Problem - what in Jaeger blocks you from solving the requirement?

Current rate limiter API does not have a way of checking when next credit will accrue.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Add method DetermineWaitTime to return the approximate time until next credit is accrued.

Currently tags are ignored in metrics/LocalBackend. That will make it impossible to use this local backend in unit tests that are sensitive to the tags. Address this by copying functionality from the metrics implementation in jaeger-client-go.

Requirement - what kind of business use case are you trying to solve?

We've had situations where it would be useful to report histograms of non-timer values, e.g. a distribution of queue lengths, or the span sizes. Backends like Prometheus do support them.

Problem - what in Jaeger blocks you from solving the requirement?

The current metrics.Factory only allows timers that are internally implemented as histograms.

Proposal - what do you suggest to solve the problem or improve the existing situation?

• Replace Factory.Timer() with Factory.Histogram(HistogramOptions{Name, Tags, Description, Buckets})
• Add static factory methods metrics.Timer(TimeOptions{HistogramOptions, Factory})

Any open questions to address

• Breaking change, may want to include in 2.x (#47)
• Decide if we want to use structs for arguments, to ensure better extensibility

Requirement - what kind of business use case are you trying to solve?

By default, the underlying prometheus client uses the following default buckets:

DefBuckets = []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10}

These values are tailored for measuring network latency, but also assume relatively low response times.

It should be possible to specify custom default buckets for the Histogram metric type that fit various use cases.

Problem - what in Jaeger blocks you from solving the requirement?

A function WithBuckets(buckets []float64) is already exposed in the prometheus package to define default buckets for the Factory type.

However, when calling the factory's Histogram(options metrics.HistogramOptions) method to build a new histogram, the default buckets that may have been set earlier are not taken into consideration.

Instead, only the buckets from the provided options argument are used.

Proposal - what do you suggest to solve the problem or improve the existing situation?

The factory Histogram method should determine whether to use the configured default buckets or the buckets from the provided histogram options. If the options.buckets slice is nil, the defaults must be used.

Any open questions to address

Please, let me know if all of this makes sense to you or have any additional questions.

I have also addressed the necessary changes in the code and I'm ready to open a PR if you agree.

See jaegertracing/jaeger#254

There is a pending breaking change (#46). I suggest we take this opportunity to clean up the lib further. Specifically:

• remove metric/go-kit/prometheus package, since we have metrics/prometheus implemented directly.
• move LocalBackend to its own package. Maybe it should be explicitly in testutils, since the functions there already depend on the LocalBackend. Perhaps a better name is metricstest. (#46) (#51)
• require descriptions for metrics (#60)

Requirement - what kind of business use case are you trying to solve?

PR #63 includes support for defining buckets on Timer and Histogram metric types. The buckets can be statically declared with the metrics and parsed on reflection.

Problem - what in Jaeger blocks you from solving the requirement?

Buckets for histogram metrics are being parsed, but currently not for timer durations.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Parse the timer durations.

Any open questions to address

How should the durations be expressed? For readability it would probably be best as a number + unit (e.g. 1s, 10m)

jaeger-lib currently has a constraint on prometheus/client_golang v0.8.0.

Now that prometheus/client_golang 1.0 was released on 2019-06-15,
https://github.com/prometheus/client_golang/releases, we should switch the constraint to 1.0 or newer.

yarpc-go depends on jaeger-client-go, which depends on jaeger-lib. So if jaeger-lib can update the dependency, it would help the other services update their dependency as well.

Support Go mod so that downstream projects can migrate to go mod

Requirement - what kind of business use case are you trying to solve?

Trying to add the jaeger-lib into my project, but it locks the version of go-kit/kit to 0.5.0 in dep 'Gopkg.toml', since we also need the influxstatsd version inside the go-kit/kit which only exists above the version of 0.5.0.
So far cannot dep add the jaeger-lib into the project.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Is that ok just bump the version of go-kit/kit to current 0.9.0? Would the apis be broken?

Thank you very much.

Per jaegertracing/jaeger#409

• Move the repo
• Re-register the repo with Travis
• Re-register the repo with Coveralls/CodeCov
• Update the links to badges in the README
• Add DCO file
• Add sign-off checker bot
• Update CONTRIBUTING instructions to require sign-off
• Update to Apache license
  • Replace LICENSE file
  • Update CONTRIBUTING instructions and references to license in the README
  • Update the existing license headers (keep Uber copyright)
  • Update the check-license script (new files get The Jaeger Authors copyright)

Hi @yurishkuro - this repository doesn't have any releases tagged. Would you mind cutting a release so that our internal tools can pin to a reasonable semver range?

Requirement - what kind of business use case are you trying to solve?

Histogram type has been added in #63 which enables buckets to be defined.

Problem - what in Jaeger blocks you from solving the requirement?

Expvar implementation does not currently respect the defined buckets.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Investigate how expvar impl can support the histogram buckets.

I just noticed this while looking at the dependencies added by this lib:

Investigate using https://github.com/uber-go/tally as the metrics API instead of having jaeger-lib/metrics.

The APIs are similar, and tally supports writing to M3, Prometheus, statsd, and to multiple reporters out of the box.

##Open Questions

• Should we continue to provide an out of the box reporter for expvar?

Requirement - what kind of business use case are you trying to solve?

With #63 it is now possible to define Histograms with buckets - but also define duration based buckets for the Timer metric type.

Problem - what in Jaeger blocks you from solving the requirement?

The buckets are being used with the Tally Histogram type - but it appears the Tally Timer type does not enable buckets to be defined.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Either buckets will have to be ignored for Tally impl (with a warning being produced?), or the underlying representation should be changed to a Tally Histogram with DurationBuckets.

Requirement

We would like to emit gauge updates with floating numbers (e.g. probabilities)

Problem

The gauge interface in jaeger-lib only supports int64

Implementations like tally, prom, go-kit, influx and expvar support float64 gauges, but jaeger-lib casts the int64 to a float64 value. Is there a reason for this?

Proposal

Update Gauge to use float64 instead of int64 in jaeger-lib 3.0; alternatively jaeger-lib can declare new types GaugeV2, FactoryV2, etc and create an adaptor.

Requirement - what kind of business use case are you trying to solve?

Travis CI should use Go 1.11.1.

Problem - what in Jaeger blocks you from solving the requirement?

Travis CI uses Go 1.7 and Go 1.9, which are both outdated.

This will restrict us collect float number when use Gauge

# github.com/uber/jaeger-lib/metrics/prometheus ../../../github.com/uber/jaeger-lib/metrics/prometheus/factory.go:143:12: cannot use hv.WithLabelValues(f.tagsAsLabelValues(labelNames, tags)...) (type prometheus.Observer) as type prometheus.Histogram in field value: prometheus.Observer does not implement prometheus.Histogram (missing Collect method)

Requirement - what kind of business use case are you trying to solve?

Error

app_1              | $ go run ./src/cmd/app/app.go
app_1              | # github.com/eldad87/go-boilerplate/vendor/github.com/uber/jaeger-lib/metrics/prometheus
app_1              | vendor/github.com/uber/jaeger-lib/metrics/prometheus/factory.go:143:3: cannot use hv.WithLabelValues(f.tagsAsLabelValues(labelNames, tags)...) (type prometheus.Observer) as type prometheus.Histogram in field value:
app_1              | 	prometheus.Observer does not implement prometheus.Histogram (missing Collect method)

How to reproduce:

git clone https://github.com/eldad87/go-boilerplate
docker-compose up --build

You'll see the error. Its a result of the following likes:

• https://github.com/eldad87/go-boilerplate/blob/master/src/cmd/app/app.go#L96
• https://github.com/eldad87/go-boilerplate/blob/master/src/cmd/app/app.go#L97

Any open questions to address

Is there a workaround?

Thanks!

Part of jaegertracing/jaeger#2691

• migrate steps from .travis.yml to GHA workflows
• update README build badge
• change required checks to new GH jobs
• remove .travis.yml

Problem

The codahale/hdrhistogram repo has been transferred under the github HdrHstogram umbrella with the help from the original author in Sept 2020 (new repo url https://github.com/HdrHistogram/hdrhistogram-go). The main reasons are to group all implementations under the same roof and to provide more active contribution from the community as the original repository was archived several years ago.

Unfortunately such URL change will break go applications that depend on this library directly or indirectly, as discussed here.

The dependency URL should be modified to point to the new repository URL. The tag "v0.9.0" was applied at the point of transfer and will reflect the exact code that was frozen in the original repository.

If you are using Go modules, you can update to the exact point of transfter using the @v0.9.0 tag in your go get command.

go mod edit -replace github.com/codahale/hdrhistogram=github.com/HdrHistogram/[email protected]

The std lib usually uses MustXXX for panicing functions, so we should fix it now that we're doing a major bump:

func Init() error
func MustInit()

Requirement - what kind of business use case are you trying to solve?

Attempting to enable client side prometheus metrics which blows up application code (See below). We need metrics to track dropped spans, etc.

Jaeger Go Client: v2.14.0
Jaeger Lib: v1.5.0

Problem - what in Jaeger blocks you from solving the requirement?

tl;dr Initializing new tracer with prometheus DefaultRegisterer is panicking with panic: duplicate metrics collector registration attempted.

Upon initializing tracer with the default prometheus registerer, by doing either:

factory := jprom.New(jprom.WithRegisterer(prometheus.DefaultRegisterer))
return cfg.NewTracer(
    jaegercfg.Metrics(factory),
)

or (from example - both should be exactly the same):

factory := jprom.New()
return cfg.NewTracer(
	jaegercfg.Metrics(factory),
)

I'm running into the following panic from the MustRegister call from the metrics cache code which is used to initialize all metrics (counters, gauges, etc) via reflection from metrics defined here.

The error and conditions of the error is described in prometheus docs.

panic: duplicate metrics collector registration attempted

goroutine 1 [running]:
github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister(0xc420083500, 0xc4203f9790, 0x1, 0x1)
	/go/src/github.com/prometheus/client_golang/prometheus/registry.go:404 +0x9e
github.com/uber/jaeger-lib/metrics/prometheus.(*vectorCache).getOrMakeCounterVec(0xc420366300, 0x0, 0x0, 0x0, 0x0, 0xc4203e6640, 0xd, 0xc4203e6640, 0xd, 0x0, ...)
	/go/src/github.com/uber/jaeger-lib/metrics/prometheus/cache.go:50 +0x213
github.com/uber/jaeger-lib/metrics/prometheus.(*Factory).Counter(0xc4203dc780, 0xc4203e6640, 0xd, 0xc4203663f0, 0xc42028b298, 0x2)
	/go/src/github.com/uber/jaeger-lib/metrics/prometheus/factory.go:136 +0x186
github.com/uber/jaeger-lib/metrics.initMetrics(0xb098a0, 0xc420242c00, 0xd86f00, 0xc4203dc780, 0x0, 0x6, 0x6)
	/go/src/github.com/uber/jaeger-lib/metrics/metrics.go:72 +0x43e
github.com/uber/jaeger-lib/metrics.Init(0xb098a0, 0xc420242c00, 0xd86f00, 0xc4203dc780, 0x0)
	/go/src/github.com/uber/jaeger-lib/metrics/metrics.go:25 +0x57
github.com/uber/jaeger-client-go.NewMetrics(0xd86f00, 0xc4203dc730, 0x0, 0xd86f00)
	/go/src/github.com/uber/jaeger-client-go/metrics.go:100 +0xa5
github.com/uber/jaeger-client-go/config.Configuration.NewTracer(0xc42003a074, 0xa, 0x0, 0x0, 0x0, 0x0, 0xc42025bc40, 0xc420366210, 0x0, 0x0, ...)
	/go/src/github.com/uber/jaeger-client-go/config/config.go:178 +0xef

... application stack

Now our application code defines a ton of metrics that are already registered to the DefaultRegisterer at this point but there is no way to tell if an already registered metrics' Desc is clashing with jaeger client metrics or there is a bug in this reflection code which is re-registering one of the metrics twice (there is some hashing going on to create the cache key).

Proposal - what do you suggest to solve the problem or improve the existing situation?

tl;dr: Use prometheus.Register instead of prometheus.MustRegister so errors can be logged or applied to use in business logic.

Couple of proposals:

1. Add test cases either here or in jaegertracing/jaeger-client-go to test for duplicate metrics registration (apologies if this is already done - i havent checked if these tests exist).

2. Change the signature of the functions in cache.go to return error object on failure:

func (c *vectorCache) getOrMakeCounterVec(opts prometheus.CounterOpts, labelNames []string) (*prometheus.CounterVec, error) {
// ...

This can be done by switching from using c.registerer.MustRegister(cv) (docs) to err := c.registerer.Register(cv) (docs) and return the error to calling code. The error should also be logged to give more verbosity to application user. Obviously, repeated changes for getOrMakeGaugeVec and getOrMakeHistogramVec as well.

The reason for this second proposal is because its impossible to find the offending metric definition in hundreds/thousands of metrics defined both in user application and library code (jaeger-client-go). Logging the offending metric would at least let the user more information.

Any open questions to address

Should we skip registering an offending metric if registering the said metric with getOrMake*Vec (Counter, Gauge, Timer) returns an error?