Add timeout option to Browser about reactphp-buzz

[asm89]

@clue What is your idea for the API? I guess there could be:

• a default timeout setting
• override per call

[clue]

This will very likely get an API somewhat similar to #15.

$browser = $browser->withOptions(array('timeout' => 10));
$browser->get(…)

Any thoughts?

[clue]

I guess it makes sense to build this on top of cancellation support (#43) and then relying on react/promise-timer for the timeout cancellation.

[shtse8]

any updates? I want to have the timeout option.

[clue]

There are currently no immediate plans to build this from my end (no demand at the moment and more important issues such as #66), but I would be really happy to accept PRs 👍

(If you need this for a commercial project and you want a quote, please check out my profile and send me an email)

[r7l]

I would support a feature like that. I am trying to use make requests with SOCKS and it seems like that the response is never coming if the SOCKS server went down. Maybe i am doing it wrong but when using the example code, the response will not be called.

[clue]

For the reference, as of #75 the README explicitly suggests using promise cancellation like this:

$promise = $browser->get($url);

$loop->addTimer(2.0, function () use ($promise) {
    $promise->cancel();
});

Similarly, one can also use the react/promise-timer primitives like this:

$promise = Promise\timeout($browser->get($url), 2.0, $loop);

Additionally, this component obeys the default connector timeout and allows explicitly passing a custom connection timeout:

$connector = new \React\Socket\Connector($loop, array(
    'timeout' => 10.0
));

$browser = new Browser($loop, $connector);

@r7l A similar configuration applies to the clue/socks-react package and its documentation explicitly mentions SOCKS connection timeouts: https://github.com/clue/php-socks-react#connection-timeout

This means that we have plenty of options to implement "timeout options". I wonder what would be a sane way to integrate this into this library. In particular, timeout values have somewhat different implications if one downloads a huge response body or only processes its response headers. Any thoughts anybody? Also, PRs would be much appreciated 👍

[Rakdar]

To me it looks like there are three levels of different timeouts:

1. Low-Level connection-timeout as provided by \React\Socket\Connector.
2. High-Level header-timeout, in case you want to use process the body in a streaming manner.
3. High-Level response-incomplete-timeout, where the body should have been recived/buffered completly.

All three make sense and have their own use-cases. Case 1 is already covered in the Connector.

But 2 and 3 make the most sense in a this-or-that implementation. The header-timeout if $browser->withOptions(array('streaming' => true), otherwise the response-incomplete-timeout should be triggered.

Of course we could simply set only one timeout, and depending if the response should be streamed or not, trigger the corresponding error as explained before.

[clue]

Sounds good to me! I agree that the timeout should probably apply to the promise only as this is the main thing a user "waits" for. This means that if streaming is enabled, it does not take into account the full response body, but only the response headers. I think this makes sense for now and we can live with this if we add some documentation for this. We can still implement a "read timeout" (or similar) in the future for streaming responses 👍

[ondrejmirtes]

Hi, I kind of expected that Browser would play well with TimeoutConnector (because of the compatible interfaces) but it does not 😊

$client = new Browser($this->loop, new \React\Socket\TimeoutConnector(new \React\Socket\Connector($this->loop), 5, $this->loop));

Is there any easy way to make it work with that? I kind of got lost in the promise maze of the codebase 😊 I can solve this with my own timer promise and cancel() but would like to help you with it 😊

[clue]

@ondrejmirtes They do in fact play well together :-) However, they operate on different layers. The TimeoutConnector (or the preferred timeout option for the Connector) only sets a timeout for the underlying TCP/IP (or TLS) connection.

In other words, this only limits how long it may take to establish the underlying connection. It does however not limit how long the application layer HTTP request may take.

I think we all agree that it makes sense to also support a timeout on this layer in the future (in case anybody feels like sponsoring this development, feel free to reach out!). For now, you can use any of the ways documented in #1 (comment) above, such as:

$promise = Promise\timeout($browser->get($url), 2.0, $loop);

[ondrejmirtes]

Yes, I used that, thank you 😊

[roke22]

Hello,
If i use the timeout with this code:

$loop->addTimer(10, function () use ($promise) {
    $promise->cancel();
});

The problem is now always have to wait 10 seconds, i want to finish the loop when all the http request are finish or when the timeout is reached. How to do it? How can i cancel the addtimer function i scheluded if the request its done before?

Sorry for my english :(
Best Regards.

[ondrejmirtes]

$timer = $loop->addTimer(10, function () use ($promise) {
    $promise->cancel();
});
$timer->cancel();

[roke22]

Seems that is not working. The idea is to cancel the timer if the response fuction is done with this code and finish the loop as soon as the all request respond.

foreach ($arrayData as $data){
    $promise = $client->post($data['url'], $data['header'], $data['post'])->then(
        function (ResponseInterface $response) use (&$result, $i, &$timer) {
            $result[] = (string)$response->getBody();
            $timer[$i]->cancel();
        },
        function (Exception $error) use (&$errors, $i, &$timer) {
            $errors[] =  $error->getMessage();
            $timer[$i]->cancel();
        }
    );
    $timer[$i] = $loop->addTimer(15, function () use ($promise) {
        $promise->cancel();
    });
    $i++;
}

But still waiting the 15 seconds even is the request load in 2 seconds. Seems that cancel flags the function to not be done but need to execute to check if have to execute or not and the loop still waiting the 15 seconds.

There is no way to "put out" that addTimer function i put on the queue to get out so the loop finish as soon as possible?

I hope you understand me, sorry for my english. :)

[clue]

@roke22 Have you seen the above example code?

By default, this project does not currently implement a timeout on its own. The following code sends a simple GET request:

$promise = $browser->get($url);

For your use case I would recommend also installing https://github.com/reactphp/promise-timer. You can then simply wrap this in a timeout() call which takes care of starting a timer and also of stopping this timer again if the request is completed. It can simply be wrapped around this library like this:

$promise = \React\Promise\Timer\timeout(
    $browser->get($url),
    2.0,
    $loop
);

For the reference: I agree that this is an often requested feature, therefore I've assigned this to the next release milestone. This means that this will be implemented in the not too far future so you don't have to take care of this manually anymore.

[ondrejmirtes]

Sorry I was wrong, there's no Timer::cancel() method, so your code should crash. The thing you should call is $loop->cancelTimer($timer);

If your code is not crashing, then the fail promise callback is not called. Also, try using done instead of then - that way, all unhandled exceptions will get thrown to the main loop.

Don't apologize, your English is fine :)

[roke22]

Appreciate your help. I used the response that Clue give me and works like a charm if the website have no problems. But if i did with a website that have some problems of connectivity or something similar the timeout is infinite.

I do not know if that is a bug or a error about how i am doing it. I give you a full working example:

<?php
use Clue\React\Buzz\Browser;
use Psr\Http\Message\ResponseInterface;
require 'vendor/autoload.php';

try {

    $loop = React\EventLoop\Factory::create();
    $client = new Browser($loop);
   
    $starttimeTotal = microtime(true);
    $result = array();
    $errors = array();

    $promise = \React\Promise\Timer\timeout($client->get("http://live.xtravelsystem.com")->then(
        function (ResponseInterface $response) use (&$result, $starttimeTotal) {
            $tmpResult = utf8_encode((string)$response->getBody());
            $endtime = microtime(true);
            $time = number_format(($endtime - $starttimeTotal), 2, ',', '.');
            $result[] = array ("http://live.xtravelsystem.com" => $tmpResult, "timeRequest" => $time );
        },
        function (Exception $error) use (&$errors, $starttimeTotal) {
            $endtime = microtime(true);
            $time = number_format(($endtime - $starttimeTotal), 2, ',', '.');
            $errors[] = array ("http://live.xtravelsystem.com" => $error->getMessage(), "timeRequest" => $time );
        }
    ),10.0,$loop);

    $loop->run();

    $endtimeTotal = microtime(true);
    $totalTime = number_format(($endtimeTotal - $starttimeTotal), 2, ',', '.');
   
    echo json_encode(array("error" =>$errors, "data" => $result, "timeSeconds" => $totalTime));
} catch (Exception $e) {
    echo 'Exception: ',  $e->getMessage(), "\n";
}
?>

Thank you very much.

PD: Now website is working, i am going to try to reproduce on my own server and give you a url to test it.

[clue]

@roke It looks like you might be affected by #99 which was fixed via #110 just yesterday. I've just released v2.4.0 of this project, so this should work after you update to the latest version

Just released minor feature / bug fix release v2.4.0 of clue/reactphp-buzz with improved support for promise cancellation and HTTP authentication. Efficiently process multiple concurrent (RESTful) HTTP requests with @ReactPHP 🐘🎉 #php https://t.co/42vdd0NVol

— Christian Lück (@another_clue) 2. Oktober 2018

[roke22]

Thank you very much. Now is working and the request is canceled after the time.

Appreciate your help.