Async Simple request about http-client

[rdlowrey]

Actually, no -- while the end goal is ultimately the same the underlying systems are completely different. I have some fundamental disagreements with how that project approaches concurrency. I don't believe the answer to concurrency in PHP is to try to create nonblocking versions of every function. Also, that project has a lot of problems when it comes to cross-OS support. It relies on extensions for all of its underlying functionality and many of them are POSIX-only which leaves windows users out in the cold. Note that you can run Artax as well as the multiprocessing amp code just fine with native PHP. Also, I tire of PHP projects claiming to "support HTTP." If you look closely you'll find that the phpdaemon.io HTTP client and server are essentially useless for all but the most trivial scenarios.

While non-blocking IO is a useful paradigm at times, it's not for everyone. IMO the goal should not be to try to fit everything into callbacks and non-blocking IO but instead to present an interface to the user that seems synchronous.

[rdlowrey]

Well, you're right ... it's not really for that. I personally haven't ever found a justification for purposefully terminating once the send completes. Beyond that, it's kind of difficult to do (though I could make some small changes and allow it).

That said, what's your use case and why do you not care about getting an acknowledgement from the other end?

[CMCDragonkai]

For example, I have a web socket server in PHP, that needs to contact my "normal" CRUD application. Both applications live on the same server. The web socket server needs to send a message to the CRUD application. There's a couple of ways of doing this, via async CLI or async HTTP. The web socket server is event based, so it can't be waiting for the response. This request could be a request for authentication, or a simple message. The returned message is actually separate, but via a ZMQ socket message. This is all because the CRUD application is the one that processes the authentication/authorisation, while the web socket server needs to handle web socket connections but needs to authenticate any connections coming in.

Now you're probably wondering why separate these two. Well I would like to be able to async request and then async retrieve. But I haven't found any methodologies to do this. So I could async request for authentication, and then async retrieve and process those results when it comes back, but not wait for it and block the event loop. Do you have any ideas on how to do that.

[rdlowrey]

Well it would be fairly straight-forward to do with the AsyncClient with one caveat: your websocket server would need to run on the same event reactor that underlies the client. This is unlikely and I assume you're probably using Ratchet as your websocket server since you're looking for a PHP solution to your problem (unless you're using Mongrel or something homegrown, perhaps?).

The real issue is using a websocket server that can't interact directly with the actual HTTP server and needs to proxy communications between the two. This is a problem I've been working to solve for several months. I have a combination HTTP/Websocket server written and implemented in PHP. It serves static files faster than apache, handles requests in the event loop at speeds comparable to node.js (which makes it ~3x faster than anything based on the React libraries [ratchet]) and it has a ton of other features. I'm not going to say it's the greatest thing in the history of PHP, but ... it's pretty much the greatest thing in the history of PHP. I'm using it to host this on my laptop right this moment. Unfortunately, it's in a private repo at this time and I don't plan to open up private testing access for people for one to two more weeks.

If you're serious about doing websockets in PHP you might be interested so I thought I'd throw it out there.

If not, I can probably still work out an async solution using Artax to communicate between your websocket server and your http server. But as these servers are operating on the same machine might it not be better to hack together a solution using a shared in-memory key-value store like APCu? And if you need to support future scaling where the websocket server and HTTP server aren't on the same box you could probably use something like redis. Another (perhaps better?) option may be to use the new-ish pthreads extension. I've worked with it a good bit and it would allow you to do what you want without worrying about blocking semantics at all -- you could simply make the request in a separate thread and actually wait for the response to verify that everything worked as expected.

In any case, I'm interested in this general area of things. Probably best if you want to continue discussion to do it via email instead of this already somewhat off-topic issue-tracker thread :)

If you'd like to do that I'm @gmail.com with the same user name as my github. Feel free to email me there (though I'm going to sleep for the night right now). Have a good one.

[CMCDragonkai]

Interesting. You've presented a couple solutions, one being your new (not yet released) event driven framework, and the other being some message brokers.

However, I'm still confused as to how the HTTP CRUD application gets triggered by the message broker. HTTP crud applications only run when an http request comes in or when a cli executes it. Even if my web socket server did some magic with those brokers, how does my HTTP crud application "know about it" and actually does something, like authenticate the request?

Sure I'd like to continue the discussion.

[rdlowrey]

Last thing, then I'm really going to sleep! A solution that actually matches your original question would be to schedule the request to fire using the async Artax client but not actually run the Amp reactor underlying the HTTP client. Then, in each iteration of your websocket server's event loop you could Amp\Reactor::tick(). This should actually work quite well and would yield the added benefit of actually receiving the responses instead of throwing them away. I'll put together some example code to demonstrate it tomorrow.

[CMCDragonkai]

Alright see you tomorrow, but what's the Amp\Reactor::tick() for? To make the requests actually run? This is like javascript's timer right.

[rdlowrey]

Yes it's similar. The tick call will result in writable request data being written to the socket without blocking and likewise if there's any to be read it will pull that in. When there's a response to be had the callback you originally assigned when you scheduled the request will be notified. It will likely make more sense with an example.

[CMCDragonkai]

So you're saying I can drop the ZMQ system for the retrieval part? That I can do async request and async retrieval just using the async client, but forcing the ticks?

This is the part that I'm talking about:

"When there's a response to be had the callback you originally assigned when you scheduled the request will be notified."

How does that work, does waiting for the response to execute the callback block the event loop (in this case the event loop is the React loop)? In JS it doesn't, but not sure with PHP.

[CMCDragonkai]

BTW, I checked out Aerys. It looks really really cool. Hope you release it soon. I was despairing at the lack of PHP's websocket options. I am using Ratchet and it's so limited. Make a PHP version of Sails.js would be awesome.

[rdlowrey]

Isolated Example

First, here's an example of how you would tell the Amp event reactor to manually retrieve the request:

<?php

require dirname(__DIR__) . '/autoload.php'; // <-- autoloader script

// Create an async client and its underlying Amp reactor,
// but don't start the reactor:
$ampReactor = (new Amp\ReactorFactory)->select();
$client = new Artax\AsyncClient($ampReactor);

$isComplete = FALSE;

$onResponse = function(Artax\Response $r) use (&$isComplete) {
    echo $r->getStatus(), "\n"; $isComplete = TRUE;
};
$onError = function(Exception $e) use (&$isComplete) {
    echo $e, "\n"; $isComplete = TRUE;
};

// Tell the client we want to make this request, but don't run the event reactor.
$client->request('http://www.google.com', $onResponse, $onError);

while (!$isComplete) {
    $ampReactor->tick();
}

// Now the program dies because there's nothing left to do

And here's how it would work (roughly) using ratchet.

You'll have to insert the appropriate bits in the right place in your websocket application, but hopefully this will get you moving in the right direction:

<?php

// Create an async client and its underlying Amp reactor, but don't start the reactor
$ampReactor = (new Amp\ReactorFactory)->select();
$asyncHttpClient = new Artax\AsyncClient($ampReactor);

// This is the event loop underlying your Ratchet websocket server
$loop = React\EventLoop\Factory::create();

// Start your websocket server here
// ...
// ...
// Inside your websocket application you need access to the three objects created above.
// When something happens in your websocket app that triggers the need for an HTTP request
// you'll do the following:

$timer = $loop->addPeriodicTimer(0.001, [$ampReactor, 'tick']);

$onHttpClientResponse = function(Artax\Response $response, Artax\Request $request) use ($timer) {
    // ... Do stuff with the HTTP response here ...

    // Stop the Amp reactor from ticking since we're finished with the request
    // Alternatively you may want more complex logic here since you're probable making multiple
    // HTTP requests at the same time.
    $timer->cancel();
};

$onHttpClientError = function(Exception $e, Artax\Request $request) {
    echo $request->getUri(), " failed :(\n";

    // Similar to the above
    $timer->cancel();
};


$myPutRequest = (new Artax\Request)->setUri('http://127.0.0.1:80/resource')->setMethod('PUT')->setBody('some data');
$asyncHttpClient->request($myPutRequest, $onHttpClientResponse, $onHttpClientError);

// ... continue on inside your websocket application code. You want to have the Amp reactor tick
// as long as you have requests that haven't returned. Once any outstanding requests have come back
// you can cancel the $timer.

[CMCDragonkai]

Thanks for the explanation! Just to confirm, does this mean I don't need ZMQ for the second leg for the crud app to respond back to the websocket app. Instead what you are proposing is an all in one solution?From: Daniel Lowrey Sent: Wed, Jul 10, 2013 01:45To: rdlowrey/Artax [email protected]: Roger Qiu [email protected]: Re: [Artax] Async Simple request (#6)Isolated Example

First, here's an example of how you would tell the Amp event reactor to manually retrieve the request:

select(); $client = new Artax\AsyncClient($ampReactor); $isComplete = FALSE; $onResponse = function(Artax\Response $r) use (&$isComplete) { echo $r->getStatus(), "\n"; $isComplete = TRUE; }; $onError = function(Exception $e) use (&$isComplete) { echo $e, "\n"; $isComplete = TRUE; }; // Tell the client we want to make this request, but don't run the event reactor. $client->request('http://www.google.com', $onResponse, $onError); while (!$isComplete) { $ampReactor->tick(); } // Now the program dies because there's nothing left to do And here's how it would work (roughly) using ratchet. You'll have to insert the appropriate bits in the right place in your websocket application, but hopefully this will get you moving in the right direction: select(); $asyncHttpClient = new Artax\AsyncClient($ampReactor); // This is the event loop underlying your Ratchet websocket server $loop = React\EventLoop\Factory::create(); // Start your websocket server here // ... // ... // Inside your websocket application you need access to the three objects created above. // When something happens in your websocket app that triggers the need for an HTTP request // you'll do the following: $timer = $loop->addPeriodicTimer(0.001, [$ampReactor, 'tick']); $onHttpClientResponse = function(Artax\Response $response, Artax\Request $request) use ($timer) { // ... Do stuff with the HTTP response here ... ``` // Stop the Amp reactor from ticking since we're finished with the request // Alternatively you may want more complex logic here since you're probable making multiple // HTTP requests at the same time. $timer->cancel(); ``` }; $onHttpClientError = function(Exception $e, Artax\Request $request) { echo $request->getUri(), " failed :(\n"; ``` // Similar to the above $timer->cancel(); ``` }; $myPutRequest = (new Artax\Request)->setUri('http://127.0.0.1:80/resource')->setMethod('PUT')->setBody('some data'); $asyncHttpClient->request($myPutRequest, $onHttpClientResponse, $onHttpClientError); // ... continue on inside your websocket application code. You want to have the Amp reactor tick // as long as you have requests that haven't returned. Once any outstanding requests have come back // you can cancel the $timer. —Reply to this email directly or view it on GitHub.

[rdlowrey]

That's correct. There's no need for zmq to communicate with the HTTP server -- Artax does that for you. If you have trouble I'll be better able to help if you can show me a bit of code that you're trying to work with.

[CMCDragonkai]

Wow. Never thought PHP could have something similar to the AJAX based callback system. At any case, I'm currently working on PolyAuth an auth library. I'll keep this bookmarked and come back to it as soon as I can. Perhaps you could even work in a promise system: https://github.com/reactphp/promise

If it works, this seriously makes it much more simpler to do then what I originally imagined.

[rdlowrey]

Yeah I've been planning to write a promise library as well as part of the amp code --my work on the server has just been sapping all my time. I'll get to it, though.

[CMCDragonkai]

Are you getting inspiration from this similar project? http://daemon.io/

[rdlowrey]

Also, good news: I've resolved my main sticking points with the Amp library (though they haven't been pushed yet). With any luck I'll be able to finish the interface that sits on top of ext/pthreads for dispatching asynchronous calls within the next few days so I can officially tag a v0.1.0 and get amp/artax onto composer.

[CMCDragonkai]

Sounds good. So it requires pthreads extension? Know where I can get one for windows?

[rdlowrey]

No extension needed -- I'm just working on a pthreads adapter so you can use ext/pthreads to dispatch asynchronous calls if you like.

[rdlowrey]

Oh, also I meant to share this link with you. I don't know why php.net makes it so difficult to find but ...

http://windows.php.net/downloads/pecl/releases/

You can download many of the more common php extensions for windows there.