gogearbox / gearbox Goto Github PK

Instead of creating our own middlewares or forcing users to re-write their middlewares into Gearbox.
So the plan is to start by being able to take fiber context and middlewares and able to use it with gearbox.

We need to validate the max params count & the request URL length avoiding any malicious actions

Users should be able to use Gearbox without the need to interact with FastHTTP at all. giving us the freedom to have our standard API and give us the option to be able to move away to any other HTTP library in the future if we ever want to.

It should be done gradually starting with the most needed functions.

Remove ineffectual assignment according to goreportcard report

Is your feature request related to a problem? Please describe.
When creating an endpoint and try to call it from the browser makes OPTIONS request which fails because of not existing endpoint.

Describe the solution you'd like
Automatic handling for the options call

Add a startup message to be logged in console when service is stopped as we should notify that service has been stopped

Hello. I'm working in a suport for Datadog APM

I have this:

func Datadog(ctx gearbox.Context) {

	method := string(ctx.Context().Method())

	opts := []ddtrace.StartSpanOption{
		tracer.SpanType(ext.SpanTypeWeb),
		//tracer.ServiceName(cfg.serviceName)
		tracer.Tag(ext.HTTPMethod, method),
		tracer.Tag(ext.HTTPURL, ctx.Context().URI().String()),
		tracer.Measured(),
	}

	span, ctxOfSpan := tracer.StartSpanFromContext(context.Background(), "http.request", opts...)
	defer span.Finish()

	ctx.Next()

	span.SetTag(ext.ResourceName, string(ctx.Context().URI().Path()))

	status := ctx.Context().Response.StatusCode()

	span.SetTag(ext.HTTPCode, status)

	if status == gearbox.StatusInternalServerError {

		b := string(ctx.Context().Response.Body())
		span.SetTag(ext.Error, true)
		span.SetTag(ext.ErrorMsg, b)
	}
}

This a example in "go-chi" https://github.com/DataDog/dd-trace-go/blob/v1/contrib/go-chi/chi/chi.go

I have one problem need pass a my handler ctxOfSpan this variable is type context.Context. I have been thinking in a possible solution like this:

ctx.NextWithCtx(ctxOfSpan)

You have a better idea?

Thank you for you important time. Regards

Is your feature request related to a problem? Please describe.
I have a middleware that checks and parse the JWT. I would like to pass from that middleware to the handler the parsed JWT so I don't need to parse it again. I saw that the fiber has this feature but I didn't saw anything related here.

Describe the solution you'd like
Be able to set in context some data that will be available in the next middleware or handler

Add all missing tests to achieve coverage 100%

Go embedding was released in Go 1.16. This PR is to add an example for how to use it with the filesystem middleware

Hello,

My apologies in advance, if this feature already exists.

I can't find references on the internet or the documentation itself.

I appreciate your guidance, to be able to configure my gearbox instance with CORS support.

Thank you.

I think we need to implement something like Active Job on Rails for job scheduling

• We need to write some test for the worker's implementation
• We need to implement an adaptor for the FastHttp worker-pool

Please add an inbuilt way to render html files to a route. I suggest you add it to the context interface of the gearbox struct. The method itself should accept the address of the html file as the argument and render the page. It can also be used to edit the html file from within the go sever. I suggest this because I had a hard time rendering my html file on my web server I wrote with gearbox framework.

Describe the bug
Just above this part of your homepage the link to your documentation point to an unsafe location:

To Reproduce
Just click the provided link

Expected behavior
Your doc

Actual behavior
Some redirections to something not related to gearbox

Is your feature request related to a problem? Please describe.
It would be great to customize the error handlers that are used for the various potential errors. For example a simple HTML 500-error or some additional logic.

Describe the solution you'd like
Something akin to NotFound() like NotAllowed(), InternalServerError(), IAmATeapot()

I'd love to use the project because of it's awesome performance but we need Prometheus metrics for our production env so is it possible for you to write Prometheus middlewares for it?

In #16, we introduced to Fallback handler, but it's not exposed in gearbox interface to be used. We need to allow user to use it. The name of that method it could be (Fallback, or Default, etc.)

global middlewares are applied on all incoming requests that already have a matching handler

These metrics should be considered

• Introduce Use method to add user's middleware function
• Middleware functions are executed in the same order they were added (chained)
• Middleware can decide to stop flow or continue

Hi,

This project has not had any activity for more than two years and links redirects to other pages.

please participate in the Tech Empower benchmark

I initialize routes like this:

func (h *HttpSrv) createRoutes() {
	h.g.Get("/dsp/cpc/:sspId", h.handleGetBids)
	h.g.Post("/dsp/cpc/:sspId", h.handlePostBids)

	h.g.Get("/dsp-native/cpc/:sspId", h.handleNativeGetBids)
	h.g.Post("/dsp-native/cpc/:sspId", h.handleNativePostBids)

	h.g.Get("/dsp-popunder/cpc/:sspId", h.handlePopunderGetBids)
	h.g.Post("/dsp-popunder/cpc/:sspId", h.handlePopunderPostBids)
}

func (h *HttpSrv) handleGetBids(ctx gearbox.Context) {
	mw, ok := sspinfo.GetSspProcessorsPool().GetPushProcessor(ctx.Param("sspId"))
	if ok {
		mw.ExecQueryParamsHandler(ctx)
	} else {
		ctx.Status(gearbox.StatusNotFound)
	}
}

And catching the next panic:

panic: runtime error: slice bounds out of range [1:0]
goroutine 2027 [running]:
github.com/gogearbox/gearbox.(*node).matchRoute(0xc000557c00, 0xc031aa2900, 0x0, 0xc030b9b320, 0xc02e463f68, 0xc032ece400, 0x3)
	/go/pkg/mod/github.com/gogearbox/[email protected]/tree.go:134 +0x289
github.com/gogearbox/gearbox.(*router).Handler(0xc0005d0c80, 0xc0323d7400)
	/go/pkg/mod/github.com/gogearbox/[email protected]/router.go:163 +0x745
github.com/valyala/fasthttp.(*Server).serveConn(0xc02e54c000, 0xa69b38, 0xc031106420, 0x0, 0x0)
	/go/pkg/mod/github.com/valyala/[email protected]/server.go:2193 +0x128e
github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc02e468640, 0xc032c9b0a0)
	/go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:223 +0xba
github.com/valyala/fasthttp.(*workerPool).getCh.func1(0xc02e468640, 0xc032c9b0a0, 0x905520, 0xc032c9b0a0)
	/go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:195 +0x35
created by github.com/valyala/fasthttp.(*workerPool).getCh
	/go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:194 +0x11f

What the reason could be?

First of all I am not sure maybe there is an work around for that but i cant find it.

Here is the use case:
I like to use opencensus.io for request tracking and prometheus for metrics export.
If you are using standard net/http you can do that like this

import "contrib.go.opencensus.io/exporter/prometheus"

// Then create the actual exporter
pe, err := prometheus.NewExporter(prometheus.Options{
    Namespace: "demo",
})

pe is http.handler

Probably if you are using standalone fasthttp you can use fasthttpadaptor to wrap the net/http handler.
Are there an option to attach fasthttp.RequestHandler directly to the gearbox ?

We need to be able to parse the path parameters if the request is Get.
we need to be able to parse the path parameters AND the body if POST and be able to decode it to a struct.
if a struct is not provided then parse to a map.
It should be able to detect from the header the body type and if it's a JSON then it should parse the json
If XML then parse it as XML.

We need to be able to support HTTPS websites and SSL certificates.

Allow user to specify parameters in paths while adding routes (e.g. gearbox.Get("/user/:name, ...)).

Discussions / suggestions is open here

• The proper delimiter (:name, or {name})
• How these parameters will be passed to handler

Currently, we allow user to register handlers to predefined methods (e.g GET, POST, etc.).

We need to allow user to register handlers with custom methods. This can be done through a new method called Method that takes custom method as a parameter alongside handlers like (Get, Post, etc.) methods

Current code base of our router needs improvements to enhance it performance

• Re-struct current code base
• Drop support of regexes since it's not widely used and it affects our performance

Hi,

I'm leading a benchmarking project => https://github.com/the-benchmarker/web-frameworks

The purpose of this project is to compare performances (other indicators will come), based on a fairest, as we can, comparison for go frameworks (comparing on the same version, production-grade environments, same compilation options ...).

Do you mind that I (or any member) add gearbox ? ❤️

Regards,

Add a startup message to be logged in console when service is started that contains:

• the name of gearbox (it would be great if a banner is added too)
• current version of gearbox
• listening on port (http server started on :....)

Support prefork with flag in settings (default is disabled). Preforks will enhance performance for services that don't use global states as will be be multiple forks of service.

Useful references
What is a pre-fork worker model for a server?
fasthttp/prefork

After supporting middlewares in #9

Let's start writing common middlewares that will be useful for users

Logger Middleware should have these metrics:

• User can specify the writer (io, or file, etc.)
• User can select output is colored or not
• Log message should contain (at least)
  • method
  • path
  • remote address
  • timestamp
  • protocol
  • body
• all message elements are optional and selectable for user

When an exception happens then we need to be able to recover from crashes and be able to process the next request without any problems.

Allow to specify a customized handler if request's path and method do not match any with any handler

Allow user to be able to register routes under prefix
e.g

routes are slice of paths, methods, handlers that user want to register a prefix
gb.Group("/users", routes)

I'm curious about the choice for a Ternary Search Tree instead of just a normal map. I wrote a quick benchmark an no matter how I configure it the map always uses 90% less memory and is 10% to 500% faster depending on the hit rate. Do you have some benchmark to show why it is better for your case?

package main

import (
	"container/list"
	"math/rand"
	"runtime"
	"testing"
)

// Basic Implementation for Ternary Search Tree (TST)

// tst returns Ternary Search Tree
type tst interface {
	Set(word []byte, value interface{})
	Get(word []byte) interface{}
	GetString(word string) interface{}
}

// Ternary Search Tree node that holds a single character and value if there is
type tstNode struct {
	lower  *tstNode
	higher *tstNode
	equal  *tstNode
	char   byte
	value  interface{}
}

// newTST returns Ternary Search Tree
func newTST() tst {
	return &tstNode{}
}

// Set adds a value to provided key
func (t *tstNode) Set(key []byte, value interface{}) {
	if len(key) < 1 {
		return
	}

	t.insert(t, key, 0, value)
}

// Get gets the value of provided key if it's existing, otherwise returns nil
func (t *tstNode) Get(key []byte) interface{} {
	length := len(key)
	if length < 1 || t == nil {
		return nil
	}
	lastElm := length - 1

	n := t
	idx := 0
	char := key[idx]
	for n != nil {
		if char < n.char {
			n = n.lower
		} else if char > n.char {
			n = n.higher
		} else {
			if idx == lastElm {
				return n.value
			}

			idx++
			n = n.equal
			char = key[idx]
		}
	}
	return nil
}

// Get gets the value of provided key (string) if it's existing, otherwise returns nil
func (t *tstNode) GetString(key string) interface{} {
	return t.Get([]byte(key))
}

// insert is an internal method for inserting a []byte with value in TST
func (t *tstNode) insert(n *tstNode, key []byte, index int, value interface{}) *tstNode {
	char := key[index]
	lastElm := len(key) - 1

	if n == nil {
		n = &tstNode{char: char}
	}

	if char < n.char {
		n.lower = t.insert(n.lower, key, index, value)
	} else if char > n.char {
		n.higher = t.insert(n.higher, key, index, value)
	} else {
		if index == lastElm {
			n.value = value
		} else {
			n.equal = t.insert(n.equal, key, index+1, value)
		}
	}

	return n
}

// RandBytes generates random string from English letters
func RandBytes() []byte {
	const letterBytes = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
	b := make([]byte, rand.Intn(100))
	for i := range b {
		b[i] = letterBytes[rand.Intn(len(letterBytes))]
	}
	return b
}

var (
	keys = make([][]byte, 1000)
	T    tst
	M    = make(map[string]interface{})

	Sink interface{}
)

func init() {
	T = newTST()
	rand.Seed(0)

	for i := 0; i < len(keys); i++ {
		keys[i] = RandBytes()
	}

	rand.Seed(0)
	var b, a runtime.MemStats
	runtime.ReadMemStats(&b)
	for i := 0; i < 500; i++ { // Only fill half of the keys so we get a 50% hit rate.
		T.Set(keys[i], &list.Element{})
	}
	runtime.ReadMemStats(&a)
	println(a.Alloc - b.Alloc)

	rand.Seed(0)
	runtime.ReadMemStats(&b)
	for i := 0; i < 500; i++ { // Only fill half of the keys so we get a 50% hit rate.
		M[string(keys[i])] = &list.Element{}
	}
	runtime.ReadMemStats(&a)
	println(a.Alloc - b.Alloc)
}

func BenchmarkTST(b *testing.B) {
	for n := 0; n < b.N; n++ {
		Sink = T.Get(keys[(n*31)%len(keys)]) // Lookup a semi random key (using rand.Intn is too heavy here compared to the operation we are benchmarking).
	}
}

func BenchmarkMap(b *testing.B) {
	for n := 0; n < b.N; n++ {
		Sink = M[string(keys[(n*31)%len(keys)])] // Lookup a semi random key (using rand.Intn is too heavy here compared to the operation we are benchmarking).
	}
}

Describe the bug

package main

import (
	"log"

	"github.com/gogearbox/gearbox"
)

func main() {
	gb := gearbox.New()
	logMiddleware := func(ctx gearbox.Context) {
		log.Printf("log message!")
		ctx.Next()
	}
	gb.Use(logMiddleware)
	routes := []*gearbox.Route{
		gb.Get("/id", func(ctx gearbox.Context) {
			ctx.SendString("User X")
		}),
		gb.Delete("/del", func(ctx gearbox.Context) {
			ctx.SendString("Deleted")
		}),
	}

	accountRoutes := gb.Group("/account", routes)
	gb.Group("/api", accountRoutes)
	gb.Start(":3000")
}

go run main.go
curl http://localhost:3000/api/account/id
output: Deleted
To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Actual behavior
curl http://localhost:3000/api/account/id

My x86 system result "Deleted"
but my amd64 system result "User X" , it's so strange

Version/Commit
gear1.1.1 , go1.15.7

After landing #32, Code will not be covered. so we need to add tests

Add tests for Start method

gogearbox.com is a redirect to malware filled ads

Fasthttp recommends to avoid conversation between []byte and string.

We can avoid that in our routing process entirely. Through

• update Ternary Search Tree to get and set keys in []byte
• parsing and constructing routing tree in []byte not string