nrk / predis-async

Asynchronous PHP client library for Redis built on top of ReactPHP

Predis\Async

Asynchronous (non-blocking) version of Predis, the full-featured PHP client library for Redis, built on top of React to handle evented I/O. By default Predis\Async does not require any additional C extension to work, but it can be optionally paired with phpiredis to sensibly lower the overhead of serializing and parsing the Redis protocol.

Predis\Async is currently under development but already works pretty well. The client foundation is being built on top of the event loop abstraction offered by React, an event-oriented framework for PHP that aims to provide everything needed to create reusable libraries and long-running applications using an evented approach powered by non-blocking I/O. This library is partially tested on HHVM, but support for this runtime should be considered experimental.

Contributions are highly welcome and appreciated, feel free to open pull-requests with fixes or just report issues if you encounter weird behaviors and blatant bugs.

Main features

• Wide range of Redis versions supported (from 2.0 to 3.0 and unstable) using profiles.
• Transparent key prefixing for all known Redis commands using a customizable prefixing strategy.
• Abstraction for MULTI / EXEC transactions (Redis >= 2.0).
• Abstraction for PUBLISH / SUBSCRIBE contexts (Redis >= 2.0).
• Abstraction for MONITOR contexts (Redis >= 1.2).
• Abstraction for Lua scripting (Redis >= 2.6).
• Ability to connect to Redis using TCP/IP or UNIX domain sockets.
• Redis connections can be established lazily, commands are queued while the client is connecting.
• Flexible system for defining and registering custom sets of supported commands or profiles.

Installing

Predis\Async is available on Packagist. It is not required to have the phpiredis extension loaded as suggested since the client will work anyway using a pure-PHP protocol parser, but if the extension is detected at runtime then it will be automatically preferred over the slower default. It is possible to force the client to use the pure-PHP protocol parser even when the extension is detected simply by passing ['phpiredis' => false] in the array of client options.

Example

<?php
require __DIR__.'/../autoload.php';

$loop = new React\EventLoop\StreamSelectLoop();
$client = new Predis\Async\Client('tcp://127.0.0.1:6379', $loop);

$client->connect(function ($client) use ($loop) {
    echo "Connected to Redis, now listening for incoming messages...\n";

    $logger = new Predis\Async\Client('tcp://127.0.0.1:6379', $loop);

    $client->pubSubLoop('nrk:channel', function ($event) use ($logger) {
        $logger->rpush("store:{$event->channel}", $event->payload, function () use ($event) {
            echo "Stored message `{$event->payload}` from {$event->channel}.\n";
        });
    });
});

$loop->run();

Differences with Predis

Being an asynchronous client implementation, the underlying design of Predis\Async is different from the one of Predis which is a blocking implementation. Certain features have not been implemented yet (or cannot be implemented at all), just to name a few you will not find the usual abstractions for pipelining commands and creating cluster of nodes using client-side sharding. That said, they share a common style and a few basic classes so if you used Predis in the past you should feel at home.

Contributing

If you want to work on Predis\Async, it is highly recommended that you first run the test suite in order to check that everything is OK, and report strange behaviours or bugs. When modifying the code please make sure that no warnings or notices are emitted by PHP by running the interpreter in your development environment with the error_reporting variable set to E_ALL | E_STRICT.

The recommended way to contribute to Predis\Async is to fork the project on GitHub, create new topic branches on your newly created repository to fix or add features (possibly with tests covering your modifications) and then open a new pull request with a description of the applied changes. Obviously you can use any other Git hosting provider of your preference.

Please follow a few basic commit guidelines before opening pull requests.

Project

• Source code
• Issue tracker

Author

• Daniele Alessandri (twitter)

License

The code for Predis\Async is distributed under the terms of the MIT license (see LICENSE).

predis-async's Issues

Uncaught Error: Call to undefined method React\EventLoop\Timer\Timer::setData()

Hi Team,

I am using this plugin into my project.

I am getting the following error in my project logs...Can someone plz help on this

My s/w specification:

Php 7.0
"predis/predis": "1.1.1",
"predis/predis-async": "dev-master"

PHP Fatal error: Uncaught Error: Call to undefined method React\EventLoop\Timer\Timer::setData() in /var/www/html/vendor/predis/predis-async/src/Connection/AbstractConnection.php:179
Stack trace:
#0 /var/www/html/vendor/predis/predis-async/src/Connection/AbstractConnection.php(158): Predis\Async\Connection\AbstractConnection->armTimeoutMonitor(5, Object(Closure))
#1 /var/www/html/vendor/predis/predis-async/src/Connection/AbstractConnection.php(209): Predis\Async\Connection\AbstractConnection->createResource(Object(Closure))
#2 /var/www/html/vendor/predis/predis-async/src/Client.php(178): Predis\Async\Connection\AbstractConnection->connect(Object(Closure))
#3 /var/www/html/commands/Queue/QueueConnector.php(86): Predis\Async\Client->connect(Object(Closure))
#4 /var/www/html/commands/Queue/QueueConnector.php(43): CLI\Queue\QueueConnector->createRedisClient(Object(CLI\Configuration\Configuration), Object(CLI\Handler\s3UploadHandler))
#5 /var/www/html/commands/Jobs/S3Upload.php(13): CLI\Queue\QueueConnector->__construct(Object(CLI in /var/www/html/vendor/predis/predis-async/src/Connection/AbstractConnection.php on line 179

Thanks in advance

How can I run a blPop in a continuous loop?

I've got the blocking list pop example working from the repo, however, I'm unable to get the blpop working in a continuous loop.

$connection = "tcp://localhost:6379";
$consumer = new \Predis\Async\Client( $connection )
$consumer->connect(
    function ( $consumer ) {
        echo "Connected to Redis, blpop'ing on queue:*\n";
        $consumer->blpop(
            'queue:normal',
            0,
            function ( $response ) use ( $consumer ) {
                list( $priority, $job ) = $response;

                if ( $priority ):
                    $message = "`%s` from channel `%s`.\n";
                    $feedback = sprintf( $message, $job, $priority );
                    echo $feedback;
                endif;

                return TRUE;
            });
    });
$consumer->getEventLoop()->run();

When I run this code with items in the queue, it will process the first one but then halt and hang in the terminal. Is there a way to have this blpop continuously block and then pop an item when it finds one?

Question about use of async_connect with Predis-Async

I wasn't sure where to ask this so I picked here.

I'm testing out using Redis to collect my user analytics events for realtime viewing. While it is being developed I want the transaction to have minimal impact on page load (ideally none) and to give up very quickly if the Redis server isn't responding.

At first I was (incorrectly) doing $client->connect proceeded immediately by

try{
    $client->getEventLoop()->run();
} catch

It is now my understanding that the idea of the eventLoop is to set those commands up and then run the event loop later, like via AJAX after the page loads (right?).

If you need more background information let me know.

Here's my issue. I can't figure out what, if anything, setting async_connect to true does. I'm just generally confused. Is user-facing performance/responsiveness not the goal of asynchronous PHP like it is with asynchronous javascript? What approach should I use to unobtrusively write to Redis?

predis-async with lists

Is it possible to use predis-async as a listener to a list in Redis?
When ever there is an item in a list, pop it and run a callback and if there is not an item wait for an item.
I want to use a list as buffer.

HHVM compatibility?

First published message in channel after reconnect is lost

Below exampel script. Run it and than publish messages to "queue:0:cmd":
publish queue:0:cmd test
publish queue:0:cmd quit
publish queue:0:cmd test
publish queue:0:cmd quit
You will see response from monitor:
$php ./test
Connected to Redis, now listening for incoming messages cmd...
Received message test from channel queue:0:cmd [type: message].
Received message quit from channel queue:0:cmd [type: message].
Connected to Redis, now listening for incoming messages cmd...
Received message quit from channel queue:0:cmd [type: message].

As you can see second "test" message is lost. In same time "redis-cli" points that all messages were delivered:

127.0.0.1:6379> publish queue:0:cmd test
(integer) 1
127.0.0.1:6379> publish queue:0:cmd quit
(integer) 1
127.0.0.1:6379> publish queue:0:cmd test
(integer) 1
127.0.0.1:6379> publish queue:0:cmd quit
(integer) 1

"libvent" lib is used for event loop.

require __DIR__ . '/vendor/autoload.php';


$loop = \React\EventLoop\Factory::create();

$client = new Predis\Async\Client('tcp://127.0.0.1:6379', $loop);

$client->connect(function ($client) {
    echo "Connected to Redis, now listening for incoming messages cmd...\n";

    $client->pubSubLoop('queue:0:cmd', function ($event, $pubsub) {
        $message = "Received message `%s` from channel `%s` [type: %s].\n";

        $feedback = sprintf($message,
            $event->payload,
            $event->channel,
            $event->kind
        );

        echo $feedback;

        if ($event->payload === 'quit') {
            $pubsub->quit();
        }
    });
});


$loop->run();

$client->connect(function ($client) {
    echo "Connected to Redis, now listening for incoming messages cmd...\n";

    $client->pubSubLoop('queue:0:cmd', function ($event, $pubsub) {
        $message = "Received message `%s` from channel `%s` [type: %s].\n";

        $feedback = sprintf($message,
            $event->payload,
            $event->channel,
            $event->kind
        );

        echo $feedback;

        if ($event->payload === 'quit') {
            $pubsub->quit();
        }
    });
});


$loop->run();

PubSubLoop, stops after redis is restared

• if I restart redis via CLI, then Predis\Async\Client::pubSubLoop() stops with listening

I'm wondering is there a way of 're-invoke pubSubLoop()', without re-runing the script

EDIT: got it
if !redis-async->isConnected()
redis-async->connect(function() { redis-async->pubSubLoop(['psubscribe'=>'*'], function() {....}) })

Port predis aysnc to phastlight

Hi,
Phastlight(https://github.com/phastlight/phastlight) is a new node.js inspired tool that's built on top of libuv.
It has timer functionality(https://github.com/phastlight/phastlight#server-side-timer) that I believe will be faster than the timer that is simulated in php, it would be nice if there is a predis-aysnc port to phastlight.

Best Regards,
Jim

Display what worker ran a scheduled job when we get a failure

When I go to this URL resource /resque/failed/list I can see some failed jobs. However my Redis has more than 1 worker and I am not sure what is the server that has managed that scheduled job, so I cannot investigate (easily) why the scheduled job has failed because I cannot see the worker that managed the job. Can you add a new column on that list please?

Using predis-async from within React Http Server

CPU spikes to 100% regardless of the event loop backend I use and feed to Predis\Async\Connection\StreamConnection.
Can't seem to pinpoint the issue, but it seems to have creeped up when I moved to my new server (php 5.4, new linux kernel - lots of variables in the equation).

CPU only spikes after a command is issued to redis.
I'm going to keep trying to track down exactly what is causing this in my free time, I just wanted to let you know.

To recreate:

• Create a React\Http\Server using the example in the React repo.
• Instantiate a redis connection using predis-async manually and feed it the same event loop instance you used to create your React\Http\Server.
• in http->on('request') - issue a simple command to $redis via your manual connection using predis-async. The http response doesn't need to contain any redis related results, but be sure to respond/end the http request.
• start the react HTTP server and issue a GET request via your browser to the running server. Your cpu should now be at 100% on one core.

Install via composer

I'm trying to install it via composer but i requires ext-phpiredis. Is there a way to force it without phpiredis?

Connect / disconnect without inside commands never stops

$client = new Predis\Async\Client('tcp://127.0.0.1:6379');

$client->connect(function ($client) {
    $client->disconnect();
});

$client->getEventLoop()->run();

Call to undefined function Predis\Async\Connection\phpiredis_reader_create()

Hello, i got this error

PHP Fatal error:  Uncaught Error: Call to undefined function Predis\Async\Connection\phpiredis_reader_create() in /home/mark/test-task/room/src/backend/websocket/vendor/predis/predis-async/lib/Predis/Async/Connection/StreamConnection.php:75 

this is composer.json

  "require": {
        "cboden/ratchet": "^0.4.1",
        "react/socket": "^1.1.0",
        "predis/predis": "^0.8",
        "predis/predis-async": "^0.2.3",
        "clue/redis-react": "^2.2"
    }

Release predis-async on react 0.4.* as stable

Could you tag current master as next stable version? I'm working on a project using react 0.4 and predis-async and I wouldn't like to maintain dependencies on dev versions.

btw Thank you for Predis, it's a great library!

AUTH with predis-async

Is it possible to use the AUTH command with Predis-async ?

Predis\Configuration\Options invokes any callable in its input, meaning on_error has to be wrapped

Predis\Configuration\Options#L104 invokes any callable to get its value. As a result on_error needs to be wrapped otherwise the following error occurs:

Missing argument 3 for ...{closure}(), called in .../vendor/predis/predis/src/Configuration/Options.php on line 104

Update event-loop constraint to 0.3.*

React event-loop is at 0.3.* now.

Memory leak $commands on AbstractConnection | PhpiredisStreamConnection

Hello,

i got a deamon in php who do many thousand sub/unsub redis channel.
After few hours i use 1Go ram.
I found [commands:protected] => SplQueue Object become more and more larger, maybe all sub/unsub is save inside without clean it ?

"$commands" from class Predis\Async\Connection\PhpiredisStreamConnection

Consumer unsubscribe triggers Catchable fatal error

<?php
require __DIR__ . '/vendor/autoload.php';
ini_set('display_errors', 'on');
error_reporting(E_ALL);
function pubsub($message, $client)
{
}
$loop = \React\EventLoop\Factory::create();
$client = new Predis\Async\Client('tcp://127.0.0.1:6379', $loop);
$consumer = $this->redis->pubSubLoop('test', 'pubsub');
$consumer->unsubscribe('test');

This code triggers catchable fatal error on PHP 5.6:

PHP Catchable fatal error:  Argument 2 passed to Predis\Async\Client::pubSubLoop() must be callable, null given, called in redis_test.php on line 10 and defined in /media/sf_share/nevesta.info/vendor/predis/predis-async/src/Client.php on line 307

Is this abandoned?

Is this abandoned?

Update to allow predis 1.*