andersfylling / disgord Goto Github PK

the docs says "For example, /channels/:channel_id and /channels/:channel_id/messages/:message_id both take channel_id into account when generating rate limits since it's the major parameter."

Meaning the rate limit of "/channels/:channel_id" and "/channels/:channel_id/messages/:message_id" is not the same, however the parameter of channel_id is a part of the rate limiter such that the rate limits are:

1. "/channels/:channel_id"
2. "/channels/:channel_id/messages"

Currently this library use the same rate limit for both: "/channels/:channel_id"

when the function client.Disconnect() is called, the socket client should disconnect properly. Sending a disconnect packet, and waiting for the discord response before shutting down.

event_dispatcher.go is large, and might get larger in the future. It needs to be refactored and can follow the same file pattern as rest_* and struct_*. However, the methods triggerChan and triggerCallbacks should have a switch that groups together event types based on their file category (channel, guild, user, etc.). These events would then be handled in a non-exported function within the given files.

This splits up the demultiplexer; the client holds the demultiplexer to identify group of events which is then passed down to the relevant function which deals with all the specific triggering and building logic (consider using the component pattern).

After creating a custom unmarshaller for the socket layer, major performance improvements was noticed:

goos: linux
goarch: amd64
pkg: github.com/andersfylling/disgord/websocket
BenchmarkEvent_CustomUnmarshal_smallJSON-8       20000000           114 ns/op
BenchmarkEvent_Unmarshal_smallJSON-8              3000000           528 ns/op
BenchmarkEvent_CustomUnmarshal_largeJSON-8       10000000           195 ns/op
BenchmarkEvent_Unmarshal_largeJSON-8                 5000        222485 ns/op
PASS

While the marshaller ignores the byte array at d, it is still significantly faster with small JSON's (114ns/op vs. 528ns/op, on my laptop).

With this in mind, I suggest creating custom json unmarshallers for every discord data structure in the repository to reduce any wait time.

Note! The json.Marshaler and json.Unmarshaler interfaces should be implemented on the data structures. Doing so allows us to still use json.Unmarshal(...) intuitively.

checkout ffjson

Edit2: Due to adding voice, json.RawMessage is tmp used until further testing.

Get inspired by the atom contributing guidelines: https://github.com/atom/atom/blob/master/CONTRIBUTING.md#git-commit-messages

Months ago, Disgord implemented the pro-actor pattern; this would run handlers in parallel, making locking of shared resources required. However, that pattern has been removed.

Disgord now uses the reactor pattern and allows the use of channels. Neither of which, in themselves, causes the need for locking. Handlers in the reactor pattern are run in sequence, and the way channels are implemented doesn't cause any parallel use of the data structures.

The cache layer is run before the data structures, from the events are dispatched. And caching is also done on responses from REST requests before the user/developer is given access to said structs.

Once a Operation code 10 is received; The client should begin sending Opcode 1 Heartbeat payloads every heartbeat_interval milliseconds, until the connection is eventually closed or terminated. This OP code is also bidirectional. The gateway may request a heartbeat from you in some situations, and you should send a heartbeat back to the gateway as you normally would.

The following list suggests methods to simplify Discord interaction.

1. AddRoleTo(*Role, []*Member) + remove
2. AddRolesTo([]*Role, []*Member) + remove
3. GetAllMembers(session)

output:

------
TODO
Implement event handler for ``, data: 

------

DEBU[2018-09-11 01:10:58] <-:                                          
ERRO[2018-09-11 01:10:58] readObjectStart: expect { or n, but found , error found in #0 byte of ...||..., bigger context ...||... 
DEBU[2018-09-11 01:10:58] <-:                                          
ERRO[2018-09-11 01:10:58] readObjectStart: expect { or n, but found , error found in #0 byte of ...||..., bigger context ...||... 
DEBU[2018-09-11 01:10:58] <-:                                          
ERRO[2018-09-11 01:10:58] readObjectStart: expect { or n, but found , error found in #0 byte of ...||..., bigger context ...||... 
panic: repeated read on failed websocket connection

goroutine 69 [running]:
github.com/gorilla/websocket.(*Conn).NextReader(0xc0000fe9a0, 0xc000204ea0, 0x4067a5, 0xc000034180, 0xc000204f70, 0xc000204f01)
        /home/anders/go/src/github.com/gorilla/websocket/conn.go:967 +0x356
github.com/gorilla/websocket.(*Conn).ReadMessage(0xc0000fe9a0, 0xc000204f70, 0x1, 0x7de0c0, 0xc0004af000, 0x8c2d40, 0xc00021abb0)
        /home/anders/go/src/github.com/gorilla/websocket/conn.go:1028 +0x2f
github.com/andersfylling/disgord/websocket.(*Client).readPump(0xc000134000)
        /home/anders/go/src/github.com/andersfylling/disgord/websocket/client.go:325 +0xcd
created by github.com/andersfylling/disgord/websocket.(*Client).Connect
        /home/anders/go/src/github.com/andersfylling/disgord/websocket/client.go:427 +0x2a5
exit status 2

There was shown interest for having a command register for bots.

Because of this I suggest creating a Bot interface. But not to mix any of this into the existing disgord.Session interface. Instead the Bot interface will hold a disgord.Session instance and enforce some bot behavior rules as written later in this article.

disgord.NewBotInstance:
An exported function for constructing the Bot called NewBotInstance and will handle the following:

• default disgord.Config setup if not provided in disgord.BotConfig.SessionConfig.
• Returns a disgord.Bot interface and an error

disgord.Bot

• RegisterCmd
  Register a command. It takes the argument of a *BotCommand struct which holds the handler, command name, description, etc.
• SetState
  Makes it easy to quickly change bot behavior. An example would be the state "silence"; the bot do not send out any messages, and is currently viewed as in-active. Users should be able to register command handlers for specific states (to keep code cleaner), and create their own custom states. Another state could be "normal" which implicits the default behavior, or debug to start debugging (without recompiling/restarting). This does also require the state to be stored persistently as the bot might have to restart, be it suddenly or planned.
• RegisterService
  Long-running task that should not be handled in a command. It will have access to the disgord.Session interface.
• On (wrapper for disgord.Session.On)
  For listening to events, when a command does just not qualify.
• Start
  Starts the bot instance. Always connects to the socket API (which is required to make the entire REST api functional)
• Stop
  Stops the bot instance (graceful)
• Session
  Which simply returns the disgord.Session interface (the actual instance used by the bot)

disgord.BotConfig:

• SessionConfig *disgord.Config
• Name string
  Name of the bot. If not set, it will default to the Discord name.
• SourceURL string
  A URL for contributors. GitHub, Bitbucket, whatever you feel like.
• Description
  explanation what the bot does
• CommandPrefix string
  A prefix applied to all commands
• State
  default: normal. The "bot mood" or core behaviour as explained above in the Bot interface.
• Token string (required)
  Discord Token for interacting with Discord
• Commands []*disgord.BotCommand
  Commands the bot should support. Alternatively, they can be registered live.
• Services []*disgord.BotService
  For long-running tasks that should run all the time or until is terminated by the bot application or another process. These must take a context.Context as their first parameter to support cancelation. Alternatively, they can be registered live.
• EventHandlers map[string]interface{}
  When a BotCommand is restricting the behavior of the bot, a custom event listener can be used instead to give more freedom to the developer. Alternatively, they can be registered live.

disgord.BotCommand:

• Trigger string (required)
  Also the name of the command. eg. !ping. Here "ping" is the Trigger for the command.
• ShortDesc string (required)
  A short description for the command.
• LongDesc string
  A in depth description of what the command does. Usually with examples. If not set, it defaults to ShortDesc.
• Handler *BotCommandHandler
  Default handler for the command. This is run whenever a handler can not be found for a given state. If no handler is registered and a handler is not found in the StateHandlers, nothing is executed. Exception is logged.
• StateHandlers map[string]*BotCommandHandler
  Allows for binding different handlers to specific bot states.
• Flags interface{}
  Allow terminal type args. go-args package will be used for parsing these. used to generate a *arg.Parser. eg. !chessgame --start --name="something" --player1=@user1 --player2=@user2
• Permissions
  Set the discord permissions required for this command. It's just a step to improve performance.
• BotFriendly bool
  Set to true if this command can be trigger by other bots. Default behavior is false.

disgord.BotService:

• Name (required)
• ShortDesc (required)
  Short description of the behavior/goal.
• LongDesc
  In-depth description. defaults to ShortDesc on nil/empty.
• Func *disgord.BotServiceFunc
  the function to be run
• Data map[interface{}]interface{}
  Allows for accessible storage for the services. This data can also be accessed by commands.

Rules:

• BotCommand will not be triggered by itself.
• BotCommand can be triggered by another bot, but this must be specified in *disgord.BotCommand. There must be a variable to easily check if a command was triggered by a bot as well.

Please discuss the matter. All comments are welcome. If you only want to express thoughts such as "I like this" or "I dislike this" without any reason. Use emoji's / reactions such as thumbs up and thumbs down.

see the following URL for tips and tricks.

https://godoc.org/github.com/fluhus/godoc-tricks

As of the way this project is written now, this issue is complete once golint does not show an issue.
https://goreportcard.com/report/github.com/andersfylling/disgord#golint

https://github.com/json-iterator/go

When reconnect takes place, the sequence code is overwritten and lost; forcing a invalid session and a identity packet to be sent.

Change the way heartbeats are stored in the disgord instance

Every socket package is categorized based on their operation value (0 - 11) and discord events can be extracted when the operation code is 0.

There are still events that directly affect the socket connection, and needs to be handled on the socket layer.

The events are READY and RESUMED.

There has not been any effort into designing a library that can execute REST requests without authentication. As there are REST endpoints in the Discord API that does not require authentication, this must be fixed.

might increase the time, but avoids any desyncs if the local system is offset/incorrect.
Update the discord and local time whenever a response from discord is processed.

Keep a private struct with the values:

• local time
• discord time

with the method:

• Now() time.Time // adds the offset between local and discord time and calculates current timestamp

Assumptions:
the local clock counts seconds correctly.

Thanks to meew0 on Discord API server for pointing this out:

when ratelimiting, you should get the "current" time from the HTTP Date header, instead of 
the local computer's time, to avoid problems because of time desynchronisation. This isn't 
written down anywhere, I know, but most libs do it and there's really no reason not to

One of the earlier requirements was that when unmarshalling JSON set A into struct a, then marshalling of struct a should give the exact same json data as A. That was a requirement as I wanted Disgord to have the option to behave as a Discord server proxy. That's not happening anymore as this is causing issues with data integrity.

Now the only goal is to make sure we don't overwrite correct data with simple default values when the incoming JSON data is of a partial object.

https://github.com/valyala/fasthttp

https://discordapp.com/developers/docs/resources/channel#create-message

Currently the client structs rewrite the caching and concurrency logic for every api call, which is identical except the types used. rewrite this using interfaces to reduce code size and potential lazy bugs.

Disgord depends on the disconnect channel, and when this is closed (during reconnect), the user cannot stop Disgord using interrupts.

If the gateway ever issues a disconnect to your client, it will provide a close event code that you can use to properly handle the disconnection. A full list of these close codes can be found in the Response Codes documentation.

https://discordapp.com/developers/docs/topics/opcodes-and-status-codes#gateway-close-event-codes

run tests with the race flag to identify the issue

Some events and the operation 7 requires reconnecting the client and the discord gateway.

todo:

1. check if the expected stored rate limit info is incorrect when receiving a response

issues:

1. This is concurrent programming, remember to lock the bucket.

when the current values, requests left before rate limited, subtracted by one does not match the rate limits in the header response; output to log, the url + bucket key.

LRFU (Least Recently / Frequently Used), a combination of LRU (Least Recently Used) and LFU (Least Frequently Used).

When running emoji unit tests, I get rate limited (try again in <1s). However, since the default timeout of the http client is 10 second, the expected behavior is to wait until Disgord can send the request.

A brief install guide would be handy for newcomers to the project. :)

Instead of checking the error type by casting in a switch or if sentence, cast it once and get access to a error ID.

if disgordErr, ok := err.(*DisgordErr); ok && disgordErr.ID == disgord.DiscordGatewayDown {
    // ...
}

Current way of handling custom errors (which adds a bunch of extra code):

switch err.(type) {
    case disgord.UnsupportedEventType:  
        // ...
}

Most of the functions are similar on a stupid level.

The json unmarshalling should be a function of it's own such that we can easily swap json packages in case we want to try a third party.

There's a ton of hardcoded URL's.

On operation code 0, the event data should be pushed to disgord for transforming the data byte array into correct discord objects. Disgord can then dispatch those events correctly to users.

Event handlers should not be allowed to run in parallel, and developers should utilise channels for handling an event in parallel. The only reason is for convenience and readability.

By enforcing handlers to run in sequence, the behavior won't drastically change, causing software to break just because of a boolean in the config file. The developers can go through the code with the expectation of handlers to run in sequence, while seeing the use of channels they will know that any other functions/use cases of the same channel will cause a parallel environment, without having to check the config file for Disgord.