iwanbk / rimcu

Redis server-assisted client side caching Go library

rimcu - Redis server-assisted client side caching Go library

Rimcu is Go library for Redis server-assisted client side caching. In other words, it is a combination of Redis cient library and in memory cache library.

System Requirements

Redis 6 or newer, with it's client side caching feature

How it works

It caches the Redis data in your server's RAM and sync it to Redis server when the data changed. So you don't need to always ask the Redis server to get your cache data.

It supports three modes:

• RESP2: single node Redis with RESP2 protocol, it is the default one
• RESP2ClusterProxy: Redis cluster with RESP2 protocol and front proxy
• RESP3: single node Redis with RESP3 protocol, not fully tested yet

Examples

Features

Features	Status	Description
Metrics Client	❌ 🔧	Configurable metrics client
Password Support	❌ 🔧
Strings	❌ 🔧	redis strings data type
list	❌ 🔧	redis list data type
hash	❌ 🔧	redis hash data type

Connection Pool

Features	Status	Description
Single Pool	❌ 🔧	Single conn pool for all cache types
Vanilla Redigo Pool	❌ 🔧	Reuse vanilla redigo pool
Max Number of Connections	✅
Waiting for connection with timeout	✅
Idle connection checking	❌ 🔧
Healthcheck	❌ 🔧

Caches

We categorize the cache based on the Redis data types mention in https://redis.io/docs/manual/data-types/. We need to do this because each type of cache will be stored differently.

StringsCache

StringsCache is cache for redis strings data type.

Implemented Commands

• Setex
• Get
• Del
• MSet (waiting support at RESP2)
• MGet (waiting support at RESP2)
• Append

ListCache (RESP2)

IT IS UNDER REWORK

Old implementation can be found at https://github.com/iwanbk/rimcu/blob/v0.01/resp2/listcache.go#L33

ListCache is cache for Redis list data type which uses RESP2 protocol. It is still in very early development phase. See the godoc page for more explanation.
 
Implemented commands:
- [x]LPOP
- [x]RPUSH
- [x]GET (it is Rimcu specific command)
- [ ]...

Development

Local Test

export TEST_REDIS_ADDRESS=127.0.0.1:6379 # replace with your redis 6 server
go test ./...

TODO

Features	Status	Description
Unify inmem cache	❌ 🔧	resp2 & resp3 currently using two different memcache lib

CREDITS

• redigo redis package is copied and modified to this repo. It is used to provide RESP2 support.

rimcu's Issues

[cluster-proxy] support prefix based subscription

[cluster-proxy] [notif]selective disconnect handling cleanup data

currently, if any of the notification connection closed, it will clear ALL memcache data
ideas:

• instead of using client ID like the single Redis case, we might be able to use the node addr (next question: how if the slot moved because of the rebalancing?)

[notif/cluster] node discovery

List of nodes that need to be subscribed is currently need to be hardcoded

[config] complete the pool config

Pool Enhancement

Proposal

Add new fields in pool struct that contains

• min idle connection .
• max active connection . if 0 unlimited. represent total connection
• idle time out . for closing connection if idle for certain period.
• active to identify active connections

In conn struct

• add createdAt and lastUsedAt
• create goroutine in pool to check obsolete connections by using above params

in get

• check if connections is obsolete or not by using parameter idle timeout and send Do("PING") to redis.
• if there is available connections in iddle connecitons, use it. otherwise create connection

in put

• if active > maxActive, remove it
• otherwise add it into iddle connections

[notif/cluster] choose master/slave for notification subscription

• need to be configurable.
• fallback to master if slave is down

depends on #32

[data pool] use user created pool instead of create our own

[cluster] handle master/slave on-off