softwarekang / knetty Goto Github PK
View Code? Open in Web Editor NEWKnetty is a network communication framework written in Go based on the event-driven architecture.
License: Apache License 2.0
Knetty is a network communication framework written in Go based on the event-driven architecture.
License: Apache License 2.0
Now:
copying the connection byteBuffer to the session layer creates a new []byte, which can cause frequent GC when sending a large amount of data.
Suggestion:
Implement pooling for the connection buffer, so that each time the data read from the opposite end and the bytebuffer object returned to the upper session layer use the same address.
Currently, knetty monitors the IO read time and can only execute On Message serially. In order to improve performance, it needs to support multi-threaded execution of tasks.
Need disruptor , goroutinePool
1.Currently, Knetty only supports running on BSD systems.
Background:
Suppose that we could need to monitor whole net connection and tracing for upper application if meet packet loss, high latency, etc..
How to do:
To support monitor connection and do package tracing if can use some package tracing\monitoring technology like epbf\pt if needed.
add issues template for various types such as Feature, Bug, Docs, and Question.
Now:
Knetty logging output using package fmt, need a logging framework instead.
Want:
Supports logging configuration files and logging framework
现状
期望
Graceful shutdown of client and server failed
package main
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"github.com/Softwarekang/knetty"
"github.com/Softwarekang/knetty/session"
)
func main() {
// setting optional options for the server
options := []knetty.ServerOption{
knetty.WithServiceNewSessionCallBackFunc(newSessionCallBackFn),
}
// creating a new server with network settings such as tcp/upd, address such as 127.0.0.1:8000, and optional options
server := knetty.NewServer("tcp", "127.0.0.1:8000", options...)
// Initializing the server in a goroutine so that
// it won't block the graceful shutdown handling below
go func() {
if err := server.Server(); err != nil && errors.Is(err, http.ErrServerClosed) {
log.Printf("run server: %s\n", err)
}
}()
// Wait for interrupt signal to gracefully shutdown the server with
quit := make(chan os.Signal)
// kill (no param) default send syscall.SIGTERM
// kill -2 is syscall.SIGINT
// kill -9 is syscall.SIGKILL but can't be caught, so don't need to add it
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
log.Println("shutting down server...")
// The context is used to inform the server it has 5 seconds to finish
// the request it is currently handling
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := server.Shutdown(ctx); err != nil {
log.Fatal("server starting shutdown:", err)
}
log.Println("server exiting")
}
// set the necessary parameters for the session to run.
func newSessionCallBackFn(s session.Session) error {
s.SetCodec(&codec{})
s.SetEventListener(&helloWorldListener{})
s.SetReadTimeout(1 * time.Second)
s.SetWriteTimeout(1 * time.Second)
return nil
}
type helloWorldListener struct {
}
func (e *helloWorldListener) OnMessage(s session.Session, pkg interface{}) {
data := pkg.(string)
fmt.Printf("server got data:%s\n", data)
}
func (e *helloWorldListener) OnConnect(s session.Session) {
fmt.Printf("local:%s get a remote:%s connection\n", s.LocalAddr(), s.RemoteAddr())
}
func (e *helloWorldListener) OnClose(s session.Session) {
fmt.Printf("server session: %s closed\n", s.Info())
}
func (e *helloWorldListener) OnError(s session.Session, err error) {
fmt.Printf("session: %s got err :%v\n", s.Info(), err)
}
type codec struct {
}
func (c codec) Encode(pkg interface{}) ([]byte, error) {
if pkg == nil {
return nil, errors.New("pkg is illegal")
}
data, ok := pkg.(string)
if !ok {
return nil, errors.New("pkg type must be string")
}
if len(data) != 5 || data != "hello" {
return nil, errors.New("pkg string must be \"hello\"")
}
return []byte(data), nil
}
func (c codec) Decode(bytes []byte) (interface{}, int, error) {
if bytes == nil {
return nil, 0, errors.New("bytes is nil")
}
if len(bytes) < 5 {
return nil, 0, nil
}
data := string(bytes)
if len(bytes) > 5 {
data = data[0:5]
}
if data != "hello" {
return nil, 0, errors.New("data is not 'hello'")
}
return data, len(data), nil
}
.
.
.
Background:
Many of networking library support private net proto, private proto can be secure and can support delicate flow control.
How to do:
Extened basic TCP with some flow control and secure encryt, first version just for demo and add some example application to use it.
knetty file suffix with *.go need apache license
see Apache License,Version 2.0
HOW TO APPLY THE APACHE LICENSE TO YOUR WORK
A good project starts with a clear code specification.
I will add two ci lint as follows:
Server、Client provides SetLogger func for users.
User logger instead of package fmt methods in kneety
session 的设计为性能提升带来了较大的负担
now,knetty just support tcp protocol.support udp is necessary
Knet needs a session mechanism based on connections to provide apis for users to use. Similar to netty mechanism
Currently, the timeout read function for a TCP connection uses a for loop to implement a check for data being readable.
for {
if !t.isActive() {
return 0, merr.ConnClosedErr
}
// todo: Using a backoff mechanism to optimize the read function
if t.inputBuffer.Len() == 0 {
time.Sleep(1 * time.Second)
continue
}
return t.inputBuffer.Read(p)
}
1.In the case where there is no data readable for a long time, the for loop generates a large amount of unnecessary CPU consumption.
2.Reducing CPU consumption through mechanisms such as back off may result in higher data processing latency.
By using chan to block the goroutine and waking up the goroutine when data is readable, the CPU consumption can be reduced.
Now:The thread-safe byteBuffer used by the connection is a FIFO model, but memory is not reused, causing GC pressure.
Want:Supporting a lock-free and thread-safe RingBuffer to address the issue of memory reuse.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.