thoas / stats

A Go middleware that stores various information about your web application (response time, status code count, etc.)

Go stats handler

stats is a net/http handler in golang reporting various metrics about your web application.

This middleware has been developed and required for the need of picfit, an image resizing server written in Go.

Compatibility

This handler supports the following frameworks at the moment:

• negroni
• martini
• gocraft/web
• Gin
• Goji
• Beego
• HTTPRouter

We don't support your favorite Go framework? Send me a PR or create a new issue and I will implement it :)

Installation

1. Make sure you have a Go language compiler >= 1.3 (required) and git installed.
2. Make sure you have the following go system dependencies in your $PATH: bzr, svn, hg, git
3. Ensure your GOPATH is properly set.
4. Download it:

go get github.com/thoas/stats

Usage

Basic net/http

To use this handler directly with net/http, you need to call the middleware with the handler itself:

package main

import (
    "net/http"
    "github.com/thoas/stats"
)

func main() {
    h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        w.Write([]byte("{\"hello\": \"world\"}"))
    })

    handler := stats.New().Handler(h)
    http.ListenAndServe(":8080", handler)
}

Negroni

If you are using negroni you can implement the handler as a simple middleware in server.go:

package main

import (
    "net/http"
    "github.com/codegangsta/negroni"
    "github.com/thoas/stats"
    "encoding/json"
)

func main() {
    middleware := stats.New()

    mux := http.NewServeMux()

    mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        w.Write([]byte("{\"hello\": \"world\"}"))
    })

    mux.HandleFunc("/stats", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")

        stats := middleware.Data()

        b, _ := json.Marshal(stats)

        w.Write(b)
    })

    n := negroni.Classic()
    n.Use(middleware)
    n.UseHandler(mux)
    n.Run(":3000")
}

HTTPRouter .......

If you are using HTTPRouter you need to call the middleware with the handler itself:

package main                                                                          

import (
        "encoding/json"
        "github.com/julienschmidt/httprouter"
        "github.com/thoas/stats"
        "net/http"
)

func main() {
        router := httprouter.New()
        s := stats.New()
        router.GET("/stats", func(w http.ResponseWriter, _ *http.Request, _ httprouter.Params) {
                w.Header().Set("Content-Type", "application/json; charset=utf-8")
                s, err := json.Marshal(s.Data())
                if err != nil {
                        http.Error(w, err.Error(), http.StatusInternalServerError)
                }
                w.Write(s)
        })
        http.ListenAndServe(":8080", s.Handler(router))
}

Martini

If you are using martini, you can implement the handler as a wrapper of a Martini.Context in server.go:

package main

import (
    "encoding/json"
    "github.com/go-martini/martini"
    "github.com/thoas/stats"
    "net/http"
)

func main() {
    middleware := stats.New()

    m := martini.Classic()
    m.Get("/", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        w.Write([]byte("{\"hello\": \"world\"}"))
    })
    m.Get("/stats", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")

        stats := middleware.Data()

        b, _ := json.Marshal(stats)

        w.Write(b)
    })

    m.Use(func(c martini.Context, w http.ResponseWriter, r *http.Request) {
        beginning, recorder := middleware.Begin(w)

        c.Next()

        middleware.End(beginning, stats.WithRecorder(recorder))
    })
    m.Run()
}

Run it in a shell:

$ go run server.go

Then in another shell run:

$ curl http://localhost:3000/stats | python -m "json.tool"

Expect the following result:

{
    "total_response_time": "1.907382ms",
    "average_response_time": "86.699\u00b5s",
    "average_response_time_sec": 8.6699e-05,
    "count": 1,
    "pid": 99894,
    "status_code_count": {
        "200": 1
    },
    "time": "2015-03-06 17:23:27.000677896 +0100 CET",
    "total_count": 22,
    "total_response_time_sec": 0.0019073820000000002,
    "total_status_code_count": {
        "200": 22
    },
    "unixtime": 1425659007,
    "uptime": "4m14.502271612s",
    "uptime_sec": 254.502271612
}

See examples to test them.

Inspiration

Antoine Imbert is the original author of this middleware.

Originally developed for go-json-rest, it had been ported as a simple Golang handler by Florent Messa to be used in various frameworks.

This middleware implements a ticker which is launched every seconds to reset requests/sec and will implement new features in a near future :)

stats's Issues

Integration with labstack/echo

It would be appreciated

Negroni middleware records stats usage for the /stats route

It would be good if we could have the middleware not to record stats for the configured route that displays the /stats results.

Presently, viewing the stats counts towards the stats and thus affects the timings and counts.

Latest check in breaks my build

I'm using your fine package in a small app thats a go module, when I did a go get -u I get the error:
options.go:55:4: o.saveResult undefined (type *Options has no field or method saveResult)

While this is not a problem, it would be advantageous if you could tag a release or - even better - use Git flow to ensure we can keep using your package as you develop.

o.saveResult undefined (type *Options has no field or method saveResult

Data method is not thread safe.

Though this method is protected by lock and create a new structure on every call, this structure still use the same instance of ResponseCounts. So it is possible that somebody would read this map in the same moment as EndWithStatus would write into it.
In go 1.6 it will cause a panic.

RecorderResponseWriter does not implement negroni.ResponseWriter

The newly added Before(func(ResponseWriter)) from negroni.ResponseWriter is missing.

beego support

Please add beego support too.

Why reset Response Counts?

https://github.com/thoas/stats/blob/master/stats.go#L33

Why reset the ResponseCounts every second? My team and I were using this value, banking on getting a total of status codes from this, but, after seeing this have switched to using the TotalResponseCounts and diffing.

Example for Gin doesn't work (anymore?)

main.go:35: cannot use c.Writer (type gin.ResponseWriter) as type stats.ResponseWriter in argument to Stats.End:
        gin.ResponseWriter does not implement stats.ResponseWriter (missing Before method)
FATAL: command "build" failed: exit status 2

wrong status code of middleware

the example of gin should be:

// StatMiddleware response time, status code count, etc.
func StatMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        beginning, _ := Stats.Begin(c.Writer)
        c.Next()
        Stats.EndWithStatus(beginning, c.Writer.Status())
    }
}

the example of martini has same issue

Project abandoned?

Seems like @thoas never commented or did anything in this project since april.

Is it abandoned?

If yes, I'll continue it in my fork (https://github.com/caarlos0/stats)...

Any thoughts on how this could integrate with Gorilla Mux?

Possible false response code

if we look at https://github.com/thoas/stats/blob/master/recorder.go#L39 and take following scenario:

func (r *recorderResponseWriter) WriteHeader(code int) {
	r.written = true
	r.ResponseWriter.WriteHeader(code)
	r.status = code
}

1. We write header 200. :: r.status=200, r.ResponseWriter.status=200
2. We write header 200. :: r.status=300, r.ResponseWriter.status=200
   in step 2 internal response writer ignores new status code.

ref:

• go http doc
• source code

if I understood the issue correctly and if it really is a genuine issue, happy to make a PR

Add support for Gin Gonic

It would be great if you add support for Gin Gonic (https://github.com/gin-gonic/gin).

Add example for go-chi

I'd like to see an example of using stats with go-chi.

integrate with expvar

Looks like if you integrate with expvar https://golang.org/pkg/expvar/ (just publish the metrics there) then it would be very cross-compatible with a variety of other metric libraries

Add support for gocraft

Support httprouter

https://github.com/julienschmidt/httprouter

Thanks!

*Options has no field or method saveResult

// Options are stats options.
type Options struct {
	statusCode *int
	size       int
	recorder   ResponseWriter
}

stats/options.go

Line 55 in ec540af

o.saveResult = save

../../thoas/stats/options.go:55:4: o.saveResult undefined (type *Options has no field or method saveResult)

Is prometheus supported?

Hi,

I want to know it the result of stats.Data() has a format supported by Prometheus.

Thanks

use martini status_code_count and total_status_code_count code question,

{
"pid": 27613,
"uptime": "2m7.358545583s",
"uptime_sec": 127.358545583,
"time": "2016-11-10 14:45:35.526270515 +0800 CST",
"unixtime": 1478760335,
"status_code_count": {
"200": 1
},
"total_status_code_count": {
"200": 17
},
"count": 1,
"total_count": 17,
"total_response_time": "78.531534ms",
"total_response_time_sec": 0.078531534,
"average_response_time": "4.619502ms",
"average_response_time_sec": 0.004619502
}

status_code_count and total_status_code_count code always 200, reivew the code , it is hard-coded. so I think the statecode should be depend on w.
func (mw *Stats) Begin(w http.ResponseWriter) (time.Time, ResponseWriter) {
start := time.Now()

writer := NewRecorderResponseWriter(w, 200)

return start, writer

}

Bogus HTTP response status code stats

Hi

I'm observing what looks to be a bogus behavior of your middleware regarding the counting of HTTP responses status code. Given the following implementation with Negroni:

package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"time"

	"github.com/gorilla/mux"
	"github.com/thoas/stats"
	"github.com/urfave/negroni"
)

func main() {
	router := mux.NewRouter()
	httpStats := stats.New()

	router.HandleFunc("/a", handleA).
		Methods("GET", "POST", "PUT")

	router.HandleFunc("/b", handleB).
		Methods("GET")

	router.HandleFunc("/c", handleC).
		Methods("GET")

	router.HandleFunc("/stats", func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/json")
		stats := httpStats.Data()
		b, _ := json.Marshal(stats)
		w.Write(b)
	})

	n := negroni.New()

	n.UseHandler(router)
	n.Use(httpStats)

	http.ListenAndServe(":8000", n)
}

func handleA(rw http.ResponseWriter, r *http.Request) {
	switch r.Method {
	case "POST", "PUT":
		time.Sleep(100 * time.Millisecond)
		rw.WriteHeader(http.StatusCreated)

	case "GET":
		time.Sleep(10 * time.Millisecond)
	}

	fmt.Fprintf(rw, "A\n")
}

func handleB(rw http.ResponseWriter, r *http.Request) {
	time.Sleep(50 * time.Millisecond)
	fmt.Fprintf(rw, "B\n")
}

func handleC(rw http.ResponseWriter, r *http.Request) {
	time.Sleep(10 * time.Millisecond)
	fmt.Fprintf(rw, "C\n")
}

Sending the following HTTP requests...

$ curl -i -X POST localhost:8000/a
HTTP/1.1 201 Created
Date: Mon, 18 Dec 2017 15:16:25 GMT
Content-Length: 2
Content-Type: text/plain; charset=utf-8

A

$ curl -i -X POST localhost:8000/b
HTTP/1.1 405 Method Not Allowed
Date: Mon, 18 Dec 2017 15:16:32 GMT
Content-Length: 0
Content-Type: text/plain; charset=utf-8

$ curl -i localhost:8000/c
HTTP/1.1 200 OK
Date: Mon, 18 Dec 2017 15:16:40 GMT
Content-Length: 2
Content-Type: text/plain; charset=utf-8

C

$ curl -i localhost:8000/z
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Mon, 18 Dec 2017 15:16:42 GMT
Content-Length: 19

404 page not found

...I expected the /stats response total_status_code_count value to be {"200":1,"201":1,"404":1,"405":1}, but it's not:

$ curl localhost:8000/stats | jq .
{
  "pid": 7201,
  "uptime": "46.962268141s",
  "uptime_sec": 46.962268141,
  "time": "2017-12-18 16:16:55.098574 +0100 CET m=+46.963702150",
  "unixtime": 1513610215,
  "status_code_count": {},
  "total_status_code_count": {
    "200": 4
  },
  "count": 0,
  "total_count": 4,
  "total_response_time": "13.083µs",
  "total_response_time_sec": 1.3083e-05,
  "average_response_time": "3.27µs",
  "average_response_time_sec": 3.27e-06
}

Am I doing something wrong?