performancecopilot / speed Goto Github PK

View Code? Open in Web Editor NEW

37.0 7.0 6.0 2.23 MB

A Go implementation of the PCP instrumentation API

License: MIT License

Makefile 0.48% Go 99.52%

pcp metrics vector go go-kit performance monitoring observability

speed's People

Contributors

Stargazers

Watchers

Forkers

owenbutler saurvs lzap linus5 natoscott adrianbiro

speed's Issues

implement prometheus style `Must` methods that panic automatically on an error instead of returning them

can be done for writer start and stop to allow a safe defer writer.Stop() and metric registration, as well as writing values in the bytebuffer to avoid all those errcheck linting errors from gometalinter

explore using golang supplemental sys packages for MemoryMappedWriter

https://github.com/golang/sys

built for much more archs than core golang syscall

Add better tests for mmvdump using qas from PCP

Currently there is a basic test using the 'simple output'. Better tests mean better integrity for the package.

use pkg/errors to create errors with contexts

https://github.com/pkg/errors

helps get better stack frames on errors than this

Client name with invalid characters does not work

It does not appear in PCP, I was tracking it down for an hour until I found that "client-123" won't work. Just a reminder for googlers.

explore making instance domains a completely internal concept

we already have shorthands (RegisterString, CounterVector, GaugeVector) that initialize an instance domain alongside an instance metric, so instance domains can probably be made an internal concept and completely removed from the public api

github.com/performancecopilot/speed/bytewriter package does not compile on Windows.

C:\>go version
go version go1.8 windows/amd64

C:\>go get -u -v github.com/performancecopilot/speed/bytewriter
github.com/performancecopilot/speed (download)
github.com/performancecopilot/speed/bytewriter
# github.com/performancecopilot/speed/bytewriter
..\..\performancecopilot\speed\bytewriter\memorymappedwriter.go:46: undefined: syscall.Mmap
..\..\performancecopilot\speed\bytewriter\memorymappedwriter.go:46: undefined: syscall.PROT_READ
..\..\performancecopilot\speed\bytewriter\memorymappedwriter.go:46: undefined: syscall.PROT_WRITE
..\..\performancecopilot\speed\bytewriter\memorymappedwriter.go:46: undefined: syscall.MAP_SHARED
..\..\performancecopilot\speed\bytewriter\memorymappedwriter.go:60: undefined: syscall.Munmap

Is this package still maintained?

As the title says: Is this package still maintained? Do you accept PRs?

I'd like to update the dependencies of this package, particularly github.com/codahale/hdrhistogram as it's been moved to a new place.

Would you accept a PR like that? Or am I better off forking the library?

Thanks!

rename Writer to Client

The fundamental reason for doing this is that the name writer in the golang community implies to most that the type implements io.Writer, and initially it did, but later on all the writing capability was abstracted away into bytebuffer, and the Buffer type does implement io.Writer, but speed.Writer doesn't, and I don't think the name is apt anymore. Looking at the current definition of the interface, I think Client is a more appropriate name, and will probably create less confusion.

Reuse instances for histograms

Hey,

we use Speed to report about a thousand of histograms, this creates about a thousand of min,max,mean,std_dev instances. I am under impression that it should be possible to create those instances just once and then reuse their IDs.

If I am wrong just close the RFE ticket, sorry for the noise :-)

Allow external configuration of logger

I would like to have speed's internal logger under my control, ideally if I am able to provide own logger configuration (or log instance) that would be great. Currently it depends on some external custom formatter which I don't like, I would like to send everything into journald/syslog by default.

add support for expvar as a backend

https://golang.org/pkg/expvar/

one of the original goals of the project was to create generic interfaces that could be implemented for any metrics reporting backend, and expvar should be the simplest to implement

figure out ways to make instances mutable in an InstanceMetric

should be able to atleast add new instances after creation

Expose Go runtime metrics

The Go runtime package exposes metrics related to the host CPU, memory usage and garbage collector.

We can either add a new example demonstrating how to expose those metrics using the existing API, or we can implement a new PCPInstanceMetric called GoRuntime, which exposes those metrics and implements a SetTimeResolution method for periodically updating the metrics.

implement a go port of mmvdump

should help in better testing the writer

add support for measuring quantiles and apdex scores and subsequently implement histograms and summaries

https://prometheus.io/docs/practices/histograms/

PMID hash collisions

Hey,

I am hitting a hard wall of 2^10 maximum metrics and I am getting collisions which are causing pmval: pmGetInDom(70.1560651): Unknown or illegal instance domain identifier when trying to read the values via CLI tool. I see it with just few dozens of metrics:

3 fm_rails_http_request_db_duration.hosts_controller.index
16 fm_rails_activerecord_instances.Location
16 fm_rails_ruby_gc_allocated_objects.environments_controller.index
23 fm_rails_http_request_view_duration.discovered_hosts_controller.index
23 fm_rails_ruby_gc_minor_count.subnets_controller.index
111 fm_rails_ruby_gc_major_count.environments_controller.index
156 fm_rails_activerecord_instances.Host__Managed
156 fm_rails_ruby_gc_count.domains_controller.index
171 fm_rails_http_request_total_duration.hosts_controller.get_power_state
171 fm_rails_ruby_gc_count.hosts_controller.show
340 fm_rails_http_requests.domains_controller.index
340 fm_rails_ruby_gc_freed_objects.compute_resources_controller.index
380 fm_rails_http_request_view_duration.api_v2_bookmarks_controller.index
380 fm_rails_ruby_gc_major_count.compute_resources_controller.index
999 fm_rails_http_requests.notification_recipients_controller.index
999 fm_rails_http_request_total_duration.hosts_controller.runtime

Possible solutions include explicit metric ID assignment instead of hash, that would perhaps require storing the ID in some "cache" file. Alternatively, there is plenty of bits in PMID in "cluster" but I am unsure what this is supposed to be for. In Speed, cluster seems to be bound to the client.

I need to implement support for instances to bring the number of metrics down to about a dozen and hope for no collisions. But I assume many users can be unlucky and symptoms are hard to track.

Edit: For the record here is the utility I generated PMIDs with (pipe through sort -n for best results):

 package main
  
    import (
      "fmt"
      "hash/fnv"
      "bufio"
      "os"
    )
    
    func hash(s string, b uint32) uint32 {
      h := fnv.New32a()
    
      _, err := h.Write([]byte(s))
      if err != nil {
        panic(err)
      }
    
      val := h.Sum32()
      if b == 0 {
        return val
      }
        
      return val & ((1 << b) - 1)
    } 
  
    func main() {
      scanner := bufio.NewScanner(os.Stdin)
      for scanner.Scan() {
          text := scanner.Text()
          fmt.Printf("%d %s\n", hash(text, 10), text)
      }
    }

add support for string data types for metrics

Explain metric registration in documentation

Hello,

I am building an adapter or bridge that will read statsd protocol data and write to PCP using your library, but I don't understand how metrics survive restart of PCP daemon. Protocol statsd is a pretty dynamic environment where clients simply send metrics and in PCP all metrics must be registered at the initialization.

I tried to register metrics dynamically stopping the client first but it did not work well (I was running into issues trying to stop already stopped client - maybe just a race condition). Can you confirm it should be possible to post-register a new metric for already started client (stopping it first of course)? The documentation only mentions the client must be stopped, this could work. Will this approach work with archiving and long-term monitoring?

Thanks

implement custom metric types

Counters, Gauges with sensible types, semantics and units that require much less info for construction than a raw PCPMetric

Add mmv v2 support

Test failures on big-endian system (s390x)

I've recently packaged this library for Debian, and when its tests are run on a big-endian system (s390x), several of the tests fail. My initial guess is that since the mmvdump test files were created on a little-endian system, they are being read improperly on the big-endian system.

	cd _build && go test -vet=off -v -p 10 github.com/performancecopilot/speed github.com/performancecopilot/speed/bytewriter github.com/performancecopilot/speed/mmvdump
error initializing config. maybe PCP isn't installed properly
=== RUN   TestMmvFileLocation
--- PASS: TestMmvFileLocation (0.00s)
=== RUN   TestTocCountAndLength
--- PASS: TestTocCountAndLength (0.00s)
=== RUN   TestMapping
--- PASS: TestMapping (0.00s)
=== RUN   TestWritingSingletonMetric
    client_test.go:373: Incomplete/Partially Written TOC
--- FAIL: TestWritingSingletonMetric (0.03s)
=== RUN   TestUpdatingSingletonMetric
    client_test.go:427: Cannot extract dump from the writer buffer
--- FAIL: TestUpdatingSingletonMetric (0.02s)
=== RUN   TestWritingInstanceMetric
    client_test.go:539: Incomplete/Partially Written TOC
--- FAIL: TestWritingInstanceMetric (0.06s)
=== RUN   TestUpdatingInstanceMetric
    client_test.go:582: cannot get dump, error: Incomplete/Partially Written TOC
    client_test.go:342: expected 1 metrics, got 0
    client_test.go:346: expected 2 values, got 0
    client_test.go:301: expected a metric of name met.1
    client_test.go:486: expected 2 instances, got 0
    client_test.go:493: expected an instance domain of name met
    client_test.go:500: expected an instance domain at 216
    client_test.go:500: expected an instance domain at 136
    client_test.go:612: cannot get dump, error: Incomplete/Partially Written TOC
    client_test.go:342: expected 1 metrics, got 0
    client_test.go:346: expected 2 values, got 0
    client_test.go:301: expected a metric of name met.1
    client_test.go:486: expected 2 instances, got 0
    client_test.go:493: expected an instance domain of name met
    client_test.go:500: expected an instance domain at 136
    client_test.go:500: expected an instance domain at 216
--- FAIL: TestUpdatingInstanceMetric (0.24s)
=== RUN   TestStringValueWriting
    client_test.go:638: Incomplete/Partially Written TOC
--- FAIL: TestStringValueWriting (0.09s)
=== RUN   TestWritingDifferentSemantics
    client_test.go:705: cannot create dump: Incomplete/Partially Written TOC
    client_test.go:342: expected 8 metrics, got 0
    client_test.go:346: expected 12 values, got 0
    client_test.go:284: expected a metric of name m.2
    client_test.go:284: expected a metric of name m.3
    client_test.go:284: expected a metric of name m.4
    client_test.go:301: expected a metric of name m.5
    client_test.go:301: expected a metric of name m.6
    client_test.go:301: expected a metric of name m.7
    client_test.go:301: expected a metric of name m.8
    client_test.go:284: expected a metric of name m.1
    client_test.go:486: expected 2 instances, got 0
    client_test.go:493: expected an instance domain of name m
    client_test.go:500: expected an instance domain at 136
    client_test.go:500: expected an instance domain at 216
--- FAIL: TestWritingDifferentSemantics (0.13s)
=== RUN   TestWritingDifferentUnits
    client_test.go:758: cannot get dump: Incomplete/Partially Written TOC
--- FAIL: TestWritingDifferentUnits (0.13s)
=== RUN   TestWritingDifferentTypes
    client_test.go:794: cannot get dump: Incomplete/Partially Written TOC
--- FAIL: TestWritingDifferentTypes (0.24s)
=== RUN   TestMMV2MetricWriting
    client_test.go:817: cannot create dump, error: Incomplete/Partially Written TOC
--- FAIL: TestMMV2MetricWriting (0.63s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x155e8c]

goroutine 215 [running]:
testing.tRunner.func1.2({0x177be0, 0x2a6190})
	/usr/lib/go-1.19/src/testing/testing.go:1396 +0x2e8
testing.tRunner.func1()
	/usr/lib/go-1.19/src/testing/testing.go:1399 +0x3fc
panic({0x177be0, 0x2a6190})
	/usr/lib/go-1.19/src/runtime/panic.go:884 +0x240
github.com/performancecopilot/speed.TestMMV2MetricWriting(0xc018521a00)
	/tmp/autopkgtest-lxc.jhd0e6dy/downtmp/autopkgtest_tmp/_build/src/github.com/performancecopilot/speed/client_test.go:820 +0x2bc
testing.tRunner(0xc018521a00, 0x1a8d30)
	/usr/lib/go-1.19/src/testing/testing.go:1446 +0x128
created by testing.(*T).Run
	/usr/lib/go-1.19/src/testing/testing.go:1493 +0x448
FAIL	github.com/performancecopilot/speed	1.614s
=== RUN   TestWriteInt32
--- PASS: TestWriteInt32 (0.00s)
=== RUN   TestWriteInt64
--- PASS: TestWriteInt64 (0.00s)
=== RUN   TestWriteString
--- PASS: TestWriteString (0.00s)
=== RUN   TestOffset
--- PASS: TestOffset (0.00s)
=== RUN   TestMemoryMappedWriter
--- PASS: TestMemoryMappedWriter (0.01s)
PASS
ok  	github.com/performancecopilot/speed/bytewriter	0.023s
=== RUN   TestMmvDump1
    mmvdump_test.go:17: Incomplete/Partially Written TOC
--- FAIL: TestMmvDump1 (0.05s)
=== RUN   TestInputs
    mmvdump_test.go:67: Incomplete/Partially Written TOC
--- FAIL: TestInputs (0.03s)
FAIL
FAIL	github.com/performancecopilot/speed/mmvdump	0.098s
FAIL

implement an agent in go to export metrics from the API directly

similar to parfait-agent

install pcp and test visibility on travis

need to completely figure this out but we can install pcp and check actual visibility of metrics on travis

add elapsed type support

shouldn't have to 'go get' vendor packages in travis

https://github.com/performancecopilot/speed/blob/master/.travis.yml#L16-L18

go vendoring works by default for go1.6+, and does work locally, but for some reason, not on travis

mmvdump: add PCP qa tests

match pcp mmvdump qa outputs for the implemented mmvdump package

Histogram percentile support

Hey,

are there plans to give PCPHistorgram a percentile support in a way that when update function is called, it provides the instances? One idea would be to have an array of percentiles user is interested in and those would be added as instances named "perc_99" or "perc_95". For simplicity, just integer percentiles would be fine (50, 90, 95, 99). Would you accept such a patch?

If this is not planned or wanted, what is the best way of "plugging-in" the update function so percentiles gets passed into PCP? Maybe a callback function or similar pattern would do it so I could write my own handler.

Thanks!

performancecopilot / speed Goto Github PK

speed's People

Contributors

Stargazers

Watchers

Forkers

speed's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs