dustin / gomemcached

A memcached binary protocol toolkit for go.

gomemcached

This is a memcached binary protocol toolkit in go.

It provides client and server functionality as well as a little sample server showing how I might make a server if I valued purity over performance.

Server Design

The basic design can be seen in gocache. A storage server is run as a goroutine that receives a MCRequest on a channel, and then issues an MCResponse to a channel contained within the request.

Each connection is a separate goroutine, of course, and is responsible for all IO for that connection until the connection drops or the dataServer decides it's stupid and sends a fatal response back over the channel.

There is currently no work at all in making the thing perform (there are specific areas I know need work). This is just my attempt to learn the language somewhat.

gomemcached's Issues

License

Hi! What about license for the code? Please, add one. I think BSD-style license is a good choise. Go (http://golang.org/LICENSE) and a lot of go libraries use it.

Are you using consistent hashing in your gomemcached package ?

Are you using consistent hashing in your gomemcached package ?

test failure

Looking at packaging this in Fedora, a test is failing:

Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.gisnVz
+ umask 022
+ cd /home/orion/fedora/zabbix/golang-github-dustin-gomemcached
+ cd gomemcached-a2284a01c143e355985d192edf3b62a053747c70
+ LDFLAGS=' -X github.com/dustin/gomemcached/version=0 -X github.com/dustin/gomemcached/version.commit=a2284a01c143e355985d192edf3b62a053747c70'
+ GO_TEST_FLAGS='-buildmode pie -compiler gc'
+ GO_TEST_EXT_LD_FLAGS='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  '
+ go-rpm-integration check -i github.com/dustin/gomemcached -b /home/orion/fedora/zabbix/golang-github-dustin-gomemcached/gomemcached-a2284a01c143e355985d192edf3b62a053747c70/_build/bin -s /home/orion/fedora/zabbix/golang-github-dustin-gomemcached/gomemcached-a2284a01c143e355985d192edf3b62a053747c70/_build -V 0-0.1.20230224gita2284a0.fc39 -C a2284a01c143e355985d192edf3b62a053747c70 -p /home/orion/BUILDROOT/golang-github-dustin-gomemcached-0-0.1.20230224gita2284a0.fc39.x86_64 -g /usr/share/gocode -r '.*example.*'
Testing    in: /home/orion/fedora/zabbix/golang-github-dustin-gomemcached/gomemcached-a2284a01c143e355985d192edf3b62a053747c70/_build/src
         PATH: /home/orion/fedora/zabbix/golang-github-dustin-gomemcached/gomemcached-a2284a01c143e355985d192edf3b62a053747c70/_build/bin:/home/orion/.local/bin:/home/orion/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin:/opt/local/rsi/idl/bin
       GOPATH: /home/orion/fedora/zabbix/golang-github-dustin-gomemcached/gomemcached-a2284a01c143e355985d192edf3b62a053747c70/_build:/usr/share/gocode
  GO111MODULE: off
      command: go test -buildmode pie -compiler gc -ldflags " -X github.com/dustin/gomemcached/version=0 -X github.com/dustin/gomemcached/version.commit=a2284a01c143e355985d192edf3b62a053747c70 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  '"
      testing: github.com/dustin/gomemcached
github.com/dustin/gomemcached
PASS
ok      github.com/dustin/gomemcached   0.004s
github.com/dustin/gomemcached
PASS
ok      github.com/dustin/gomemcached   0.004s
github.com/dustin/gomemcached/client
PASS
ok      github.com/dustin/gomemcached/client    0.003s
github.com/dustin/gomemcached/client
PASS
ok      github.com/dustin/gomemcached/client    0.003s
github.com/dustin/gomemcached/debug
?       github.com/dustin/gomemcached/debug     [no test files]
github.com/dustin/gomemcached/server
# github.com/dustin/gomemcached/server [github.com/dustin/gomemcached/server.test]
./server_test.go:149:2: errored declared and not used
FAIL    github.com/dustin/gomemcached/server [build failed]

replace call to Errorf with Fatalf in gocache.go

the log package has been updated: http://golang.org/pkg/log/ . Old calls to Error* must be replaced with Fatal*

Avoid allocating new slice for data on every Receive()

In MCRequest.Receive, a new buffer is allocated for every receive for the body of the request:

buf := make([]byte, klen+elen+bodyLen)

It would be great if this function could instead accept a buffer as an argument so that buffers could be re-used across requests. The code could use cap(buf) to ensure that the buffer is long enough, and buf = buf[:klen+elen+bodyLen] to limit its length. Only if it buf is too small should it allocate its own (and potentially warn the caller that the buffer was not used).

Missing null check on transmit

Saw this crash:

github.com/dustin/gomemcached/client.(*Client).Transmit(0x0, 0xf8401c52d0, 0xf8401bf740, 0x1e0000001e, 0xf8402bb610, ...)
        /Users/dustin/go/src/github.com/dustin/gomemcached/client/mc.go:68 +0x29
github.com/dustin/gomemcached/client.(*Client).GetBulk(0x0, 0x3d9, 0xf8402c9000, 0x100000001, 0x0, ...)
        /Users/dustin/go/src/github.com/dustin/gomemcached/client/mc.go:231 +0x349
github.com/couchbaselabs/go-couchbase.(*Bucket).doBulkGet(0xf8400fd500, 0xf8401003d9, 0xf8402c9000, 0x100000001, 0xf84014e5a0, ...)
        /Users/dustin/go/src/github.com/couchbaselabs/go-couchbase/client.go:179 +0x171
created by github.com/couchbaselabs/go-couchbase.(*Bucket).GetBulk
        /Users/dustin/go/src/github.com/couchbaselabs/go-couchbase/client.go:210 +0x2fb

Better cas interface

Need to support aborting and deleting during cas

Installation of GoMemcached client fails

When installing go-couchbase, it installs this package as well and it fails saying this.

# github.com/dustin/gomemcached/client
/usr/lib/go/src/pkg/github.com/dustin/gomemcached/client/mc.go:339: function ends without a return statement
/usr/lib/go/src/pkg/github.com/dustin/gomemcached/client/tap_feed.go:110: function ends without a return statement

Client howto?

Hi,

First thanks for this piece of code (and for memcache).
You seems to be the only go memcache binary client out there!

I'm just wondering if i should use only one connection (memcached.Connect) and use it in every goroutine (are connections goroutine safe?) or should i use one connection per goroutine or ...

Also is the readme still accurate (There is currently no work at all in making the thing perform (there are specific areas I know need work). This is just my attempt to learn the language somewhat.)?

Are you or other people succesfully using it "in production"?

Thanks in advance

[client] Unknown command

This is the error message:

MCResponse status=UNKNOWN_COMMAND, opcode=SASL_LIST_MECHS,
 opaque=0, msg: Unknown command

This is the code:

func Memcache() {
    client, err := memcached.Connect("tcp", "127.0.0.1:11211")
    if err != nil {
        log.Fatalf("Error connecting: %v", err)
    }

    resp, err := client.Auth("memcache", "password")
    if err != nil {
        log.Fatalf("auth error: %v", err)
    }
    log.Printf("Auth response = %v", resp)
}

Implement basic opcode dispatch

Right now, all commands return "unknown command". It should be trivial to implement a new module that dispatches based on opcode lookup in a map or even array.

Clean up main

I'd like to ask for the port on the commandline as well as use a less messy set of binding functions.

Updating a key resets it's expiry

https://github.com/dustin/gomemcached/blob/master/client/mc.go#L389

Why is this done? It appears to ignore the expiry value passed in and reset it to 0.

In our system, this leads to updated keys never expiring. The fix is easy, but wondering why this was done in the first place?