kristoff-it / zig-okredis

Zero-allocation Client for Redis 6+

OkRedis

OkRedis is a zero-allocation client for Redis 6+

Handy and Efficient

OkRedis aims to offer an interface with great ergonomics without compromising on performance or flexibility.

Project status

OkRedis is currently in alpha. The main features are mostly complete, but a lot of polishing is still required.

Everything mentioned in the docs is already implemented and you can just zig run example.zig to quickly see what it can do. Remember OkRedis only supports Redis 6 and above.

Zero dynamic allocations, unless explicitly wanted

The client has two main interfaces to send commands: send and sendAlloc. Following Zig's mantra of making dynamic allocations explicit, only sendAlloc can allocate dynamic memory, and only does so by using a user-provided allocator.

The way this is achieved is by making good use of RESP3's typed responses and Zig's metaprogramming facilities. The library uses compile-time reflection to specialize down to the parser level, allowing OkRedis to decode whenever possible a reply directly into a function frame, without any intermediate dynamic allocation. If you want more information about Zig's comptime:

• Official documentation
• What is Zig's Comptime? (blog post written by me)

By using sendAlloc you can decode replies with arbrirary shape at the cost of occasionally performing dynamic allocations. The interface takes an allocator as input, so the user can setup custom allocation schemes such as arenas.

Quickstart

const std = @import("std");
const okredis = @import("./src/okredis.zig");
const SET = okredis.commands.strings.SET;
const OrErr = okredis.types.OrErr;
const Client = okredis.Client;

pub fn main() !void {
    const addr = try std.net.Address.parseIp4("127.0.0.1", 6379);
    var connection = try std.net.tcpConnectToAddress(addr);
    
    var client: Client = undefined;
    try client.init(connection);
    defer client.close();

    // Basic interface
    try client.send(void, .{ "SET", "key", "42" });
    const reply = try client.send(i64, .{ "GET", "key" });
    if (reply != 42) @panic("out of towels");


    // Command builder interface
    const cmd = SET.init("key", "43", .NoExpire, .IfAlreadyExisting);
    const otherReply = try client.send(OrErr(void), cmd);
    switch (otherReply) {
        .Nil => @panic("command should not have returned nil"),
        .Err => @panic("command should not have returned an error"),
        .Ok => std.debug.print("success!", .{}),
    }
}

Available Documentation

The reference documentation is available here.

• Using the OkRedis client

  • Connecting
  • Buffering
  • Evented vs blocking I/O
  • Pipelining
  • Transactions
  • Pub/Sub

• Sending commands

  • Base interface
  • Command builder interface
  • Validating command syntax
  • Optimized command builders
  • Creating new command builders
  • An afterword on command builders vs methods

• Decoding Redis Replies

  • Introduction
  • The first and second rule of decoding replies
  • Decoding Zig types
    • Void
    • Numbers
    • Optionals
    • Strings
    • Structs
  • Decoding Redis errors and nil replies as values
    • Redis OK replies
  • Allocating memory dynamically
    • Allocating strings
    • Freeing complex replies
    • Allocating Redis Error messages
    • Allocating structured types
  • Parsing dynamic replies
  • Bundled types
  • Decoding types in the standard library
  • Implementing decodable types
    • Adding types for custom commands (Lua scripts or Redis modules)
    • Adding types used by a higher-level language

Extending OkRedis

If you are a Lua script or Redis module author, you might be interested in reading the final sections of COMMANDS.md and REPLIES.md.

Embedding OkRedis in a higher level language

Take a look at the final section of REPLIES.md.

TODOS

• Implement remaining command builders
• Better connection management (ipv6, unixsockets, ...)
• Streamline design of Zig errors
• Refine support for async/await and think about connection pooling
• Refine the Redis traits
• Pub/Sub

zig-okredis's Issues

consider making client.sendAlloc() respects sentinel-terminated Element type

Hi, I'm trying to export a fn that returns ZRANGE reply into a shared object, that's what I done so far:

fn redis_zrevrange()

/// caller needs to free the result
export fn redis_zrevrange(ckey: [*:0]const u8, start: i64, stop: i64) [*:null]const ?[*:0]const u8 {
    const zero = allocator.allocSentinel(?[*:0]u8, 0, null) catch unreachable;

    const client: *Client = if (global_client) |cl| cl else return zero;
    const key = mem.span(ckey);

    const reply = client.sendAlloc([]const []const u8, allocator, .{ "ZRANGE", key, start, stop, "REV" }) catch |err| {
        log.err("ZRANGE failed: {}", .{err});
        return zero;
    };
    defer okredis.freeReply(reply, allocator);

    var result = allocator.allocSentinel(?[*:0]u8, reply.len, null) catch |err| {
        log.err("allocate memory failed: {}", .{err});
        return zero;
    };
    errdefer allocator.free(result);

    for (reply) |member, i| {
        const cmember = allocator.dupeZ(u8, member) catch |err| {
            log.err("allocate memory failed: {}", .{err});
            // todo: protential leak of other members
            return zero;
        };
        result[i] = cmember.ptr;
    }
    defer allocator.free(zero);

    return result;
}

It has to allocate twice (one for client.sendAlloc, one for allocator.allocSentinel&dupZ) to get the final result, I'd like to have client.sendAlloc([:null]const ?[:0]const u8) so that the reply can be coerced to the final result implicitly. Do you think it's doable?

HINCRBY

gimme gimme

incorrect reply.len for PING command

Hi, I failed to find the document for PING command, that's how I managed to send it (on redis 7.0.9, okredis master, zig 0.10.1):

const std = @import("std");
const print = std.debug.print;

const okredis = @import("okredis");

pub fn main() !void {
    const stream = try std.net.connectUnixSocket("/run/user/1000/redis.sock");
    var client: okredis.Client = undefined;
    try client.init(stream);
    defer client.close();

    {
        const expect = "PONG";
        const reply = try client.send(okredis.types.FixBuf(expect.len), .{"ping"});
        print("len=actually:{d},buf:{d} {s}\n", .{ reply.len, reply.buf.len, reply.toSlice() });
    }
}

it outputs:

thread 463599 panic: index out of bounds
/srv/playground/olds.nvim/vendor/okredis/src/types/fixbuf.zig:15:28: 0x239dac in .okredis.types.fixbuf.FixBuf(4).toSlice (main)
            return self.buf[0..self.len];
                           ^
/srv/playground/olds.nvim/src/main.zig:15:119: 0x237086 in main (main)
        print("len=actually:{d},buf:{d} {s}\n", .{ reply.len, reply.buf.len, std.fmt.fmtSliceEscapeLower(reply.toSlice()) });
                                                                                                                      ^
/opt/zig-linux-x86_64-0.10.1/lib/std/start.zig:614:37: 0x209ca0 in std.start.posixCallMainAndExit (main)
            const result = root.main() catch |err| {
                                    ^
/opt/zig-linux-x86_64-0.10.1/lib/std/start.zig:376:5: 0x209a51 in std.start._start (main)
    @call(.{ .modifier = .never_inline }, posixCallMainAndExit, .{});
    ^
The following command terminated unexpectedly:
cd /srv/playground/olds.nvim && /srv/playground/olds.nvim/zig-cache/o/fa922d6a27f914caf978a7c27dde5eb3/main
error: the following build command failed with exit code 1:
/srv/playground/olds.nvim/zig-cache/o/b45aa06f1ff9d8bf716e1faebfff32fb/build /opt/zig-linux-x86_64-0.10.1/zig /srv/playground/olds.nvim /srv/playground/olds.nvim/zig-cache /home/haoliang/.cache/zig run

with this change:

print("len=actually:{d},buf:{d} {s}\n", .{ reply.len, reply.buf.len,std.fmt.fmtSliceEscapeLower(&reply.buf) });

I think it's reply.len being incorrect:

len=actually:12297829382473034410,buf:4 PONG

the solutions I found so far:

• a bigger FixBuf(4+1)
• a given payload for PING: .{"ping", "PONG"}

So, do you think that's a problem belongs to okredis? or it's a inconsistent behavior of redis?

Which version of zig should OKRedis use ?

version ?

Tests currently fail

After the Allocgate changes, most of the code was updated in 55e2c8f. However, the tests still fail as at some point the parser is trying to parse an *anyopaque field (probably originating from std.mem.Allocator since the Allocgate changes).

how should I get the reply of ZRANGE command?

Hi, according to the doc of ZRANGE command, it will return array replies, I managed to know I can .send(i64, .{"zadd", "key", 1, "a"} to build the ZADD command, but I failed to find a way to get the reply of ZRANGE, which is supposed to be form like []const []const u8. Cound you give me some hints on this?