GithubHelp home page GithubHelp logo

clue / reactphp-buzz Goto Github PK

View Code? Open in Web Editor NEW
352.0 17.0 38.0 474 KB

[Deprecated] Simple, async PSR-7 HTTP client for concurrently processing any number of HTTP requests, built on top of ReactPHP.

Home Page: https://clue.engineering/2018/introducing-reactphp-buzz

License: MIT License

PHP 100.00%

reactphp-buzz's Introduction

Deprecation notice

This package has now been migrated over to react/http and only exists for BC reasons.

$ composer require react/http

If you've previously used this package, upgrading should take no longer than a few minutes. All classes have been merged as-is from the latest v2.9.0 release with no other significant changes, so you can simply update your code to use the updated namespace like this:

// old
$browser = new Clue\React\Buzz\Browser($loop);
$browser->get($url);

// new
$browser = new React\Http\Browser($loop);
$browser->get($url);

See react/http for more details.

The below documentation applies to the last release of this package. Further development will take place in the updated react/http, so you're highly recommended to upgrade as soon as possible.

Legacy clue/reactphp-buzz Build Status

Simple, async PSR-7 HTTP client for concurrently processing any number of HTTP requests, built on top of ReactPHP.

This library is heavily inspired by the great kriswallsmith/Buzz project. However, instead of blocking on each request, it relies on ReactPHP's EventLoop to process multiple requests in parallel. This allows you to interact with multiple HTTP servers (fetch URLs, talk to RESTful APIs, follow redirects etc.) at the same time. Unlike the underlying react/http-client, this project aims at providing a higher-level API that is easy to use in order to process multiple HTTP requests concurrently without having to mess with most of the low-level details.

  • Async execution of HTTP requests - Send any number of HTTP requests to any number of HTTP servers in parallel and process their responses as soon as results come in. The Promise-based design provides a sane interface to working with out of bound responses.
  • Standard interfaces - Allows easy integration with existing higher-level components by implementing PSR-7 (http-message) interfaces, ReactPHP's standard promises and streaming interfaces.
  • Lightweight, SOLID design - Provides a thin abstraction that is just good enough and does not get in your way. Builds on top of well-tested components and well-established concepts instead of reinventing the wheel.
  • Good test coverage - Comes with an automated tests suite and is regularly tested in the real world.

Table of contents

Support us

We invest a lot of time developing, maintaining and updating our awesome open-source projects. You can help us sustain this high-quality of our work by becoming a sponsor on GitHub. Sponsors get numerous benefits in return, see our sponsoring page for details.

Let's take these projects to the next level together! ๐Ÿš€

Quickstart example

Once installed, you can use the following code to access a HTTP webserver and send some simple HTTP GET requests:

$loop = React\EventLoop\Factory::create();
$client = new Clue\React\Buzz\Browser($loop);

$client->get('http://www.google.com/')->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump($response->getHeaders(), (string)$response->getBody());
});

$loop->run();

See also the examples.

Usage

Request methods

Most importantly, this project provides a Browser object that offers several methods that resemble the HTTP protocol methods:

$browser->get($url, array $headers = array());
$browser->head($url, array $headers = array());
$browser->post($url, array $headers = array(), string|ReadableStreamInterface $contents = '');
$browser->delete($url, array $headers = array(), string|ReadableStreamInterface $contents = '');
$browser->put($url, array $headers = array(), string|ReadableStreamInterface $contents = '');
$browser->patch($url, array $headers = array(), string|ReadableStreamInterface $contents = '');

Each of these methods requires a $url and some optional parameters to send an HTTP request. Each of these method names matches the respective HTTP request method, for example the get() method sends an HTTP GET request.

You can optionally pass an associative array of additional $headers that will be sent with this HTTP request. Additionally, each method will automatically add a matching Content-Length request header if an outgoing request body is given and its size is known and non-empty. For an empty request body, if will only include a Content-Length: 0 request header if the request method usually expects a request body (only applies to POST, PUT and PATCH HTTP request methods).

If you're using a streaming request body, it will default to using Transfer-Encoding: chunked unless you explicitly pass in a matching Content-Length request header. See also streaming request for more details.

By default, all of the above methods default to sending requests using the HTTP/1.1 protocol version. If you want to explicitly use the legacy HTTP/1.0 protocol version, you can use the withProtocolVersion() method. If you want to use any other or even custom HTTP request method, you can use the request() method.

Each of the above methods supports async operation and either fulfills with a ResponseInterface or rejects with an Exception. Please see the following chapter about promises for more details.

Promises

Sending requests is async (non-blocking), so you can actually send multiple requests in parallel. The Browser will respond to each request with a ResponseInterface message, the order is not guaranteed. Sending requests uses a Promise-based interface that makes it easy to react to when an HTTP request is completed (i.e. either successfully fulfilled or rejected with an error):

$browser->get($url)->then(
    function (Psr\Http\Message\ResponseInterface $response) {
        var_dump('Response received', $response);
    },
    function (Exception $error) {
        var_dump('There was an error', $error->getMessage());
    }
);

If this looks strange to you, you can also use the more traditional blocking API.

Keep in mind that resolving the Promise with the full response message means the whole response body has to be kept in memory. This is easy to get started and works reasonably well for smaller responses (such as common HTML pages or RESTful or JSON API requests).

You may also want to look into the streaming API:

  • If you're dealing with lots of concurrent requests (100+) or
  • If you want to process individual data chunks as they happen (without having to wait for the full response body) or
  • If you're expecting a big response body size (1 MiB or more, for example when downloading binary files) or
  • If you're unsure about the response body size (better be safe than sorry when accessing arbitrary remote HTTP endpoints and the response body size is unknown in advance).

Cancellation

The returned Promise is implemented in such a way that it can be cancelled when it is still pending. Cancelling a pending promise will reject its value with an Exception and clean up any underlying resources.

$promise = $browser->get($url);

$loop->addTimer(2.0, function () use ($promise) {
    $promise->cancel();
});

Timeouts

This library uses a very efficient HTTP implementation, so most HTTP requests should usually be completed in mere milliseconds. However, when sending HTTP requests over an unreliable network (the internet), there are a number of things that can go wrong and may cause the request to fail after a time. As such, this library respects PHP's default_socket_timeout setting (default 60s) as a timeout for sending the outgoing HTTP request and waiting for a successful response and will otherwise cancel the pending request and reject its value with an Exception.

Note that this timeout value covers creating the underlying transport connection, sending the HTTP request, receiving the HTTP response headers and its full response body and following any eventual redirects. See also redirects below to configure the number of redirects to follow (or disable following redirects altogether) and also streaming below to not take receiving large response bodies into account for this timeout.

You can use the withTimeout() method to pass a custom timeout value in seconds like this:

$browser = $browser->withTimeout(10.0);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // response received within 10 seconds maximum
    var_dump($response->getHeaders());
});

Similarly, you can use a bool false to not apply a timeout at all or use a bool true value to restore the default handling. See withTimeout() for more details.

If you're using a streaming response body, the time it takes to receive the response body stream will not be included in the timeout. This allows you to keep this incoming stream open for a longer time, such as when downloading a very large stream or when streaming data over a long-lived connection.

If you're using a streaming request body, the time it takes to send the request body stream will not be included in the timeout. This allows you to keep this outgoing stream open for a longer time, such as when uploading a very large stream.

Note that this timeout handling applies to the higher-level HTTP layer. Lower layers such as socket and DNS may also apply (different) timeout values. In particular, the underlying socket connection uses the same default_socket_timeout setting to establish the underlying transport connection. To control this connection timeout behavior, you can inject a custom Connector like this:

$browser = new Clue\React\Buzz\Browser(
    $loop,
    new React\Socket\Connector(
        $loop,
        array(
            'timeout' => 5
        )
    )
);

Authentication

This library supports HTTP Basic Authentication using the Authorization: Basic โ€ฆ request header or allows you to set an explicit Authorization request header.

By default, this library does not include an outgoing Authorization request header. If the server requires authentication, if may return a 401 (Unauthorized) status code which will reject the request by default (see also the withRejectErrorResponse() method below).

In order to pass authentication details, you can simple pass the username and password as part of the request URL like this:

$promise = $browser->get('https://user:[email protected]/api');

Note that special characters in the authentication details have to be percent-encoded, see also rawurlencode(). This example will automatically pass the base64-encoded authentication details using the outgoing Authorization: Basic โ€ฆ request header. If the HTTP endpoint you're talking to requires any other authentication scheme, you can also pass this header explicitly. This is common when using (RESTful) HTTP APIs that use OAuth access tokens or JSON Web Tokens (JWT):

$token = 'abc123';

$promise = $browser->get(
    'https://example.com/api',
    array(
        'Authorization' => 'Bearer ' . $token
    )
);

When following redirects, the Authorization request header will never be sent to any remote hosts by default. When following a redirect where the Location response header contains authentication details, these details will be sent for following requests. See also redirects below.

Redirects

By default, this library follows any redirects and obeys 3xx (Redirection) status codes using the Location response header from the remote server. The promise will be fulfilled with the last response from the chain of redirects.

$browser->get($url, $headers)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // the final response will end up here
    var_dump($response->getHeaders());
});

Any redirected requests will follow the semantics of the original request and will include the same request headers as the original request except for those listed below. If the original request contained a request body, this request body will never be passed to the redirected request. Accordingly, each redirected request will remove any Content-Length and Content-Type request headers.

If the original request used HTTP authentication with an Authorization request header, this request header will only be passed as part of the redirected request if the redirected URL is using the same host. In other words, the Authorizaton request header will not be forwarded to other foreign hosts due to possible privacy/security concerns. When following a redirect where the Location response header contains authentication details, these details will be sent for following requests.

You can use the withFollowRedirects() method to control the maximum number of redirects to follow or to return any redirect responses as-is and apply custom redirection logic like this:

$browser = $browser->withFollowRedirects(false);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // any redirects will now end up here
    var_dump($response->getHeaders());
});

See also withFollowRedirects() for more details.

Blocking

As stated above, this library provides you a powerful, async API by default.

If, however, you want to integrate this into your traditional, blocking environment, you should look into also using clue/reactphp-block.

The resulting blocking code could look something like this:

use Clue\React\Block;

$loop = React\EventLoop\Factory::create();
$browser = new Clue\React\Buzz\Browser($loop);

$promise = $browser->get('http://example.com/');

try {
    $response = Block\await($promise, $loop);
    // response successfully received
} catch (Exception $e) {
    // an error occured while performing the request
}

Similarly, you can also process multiple requests concurrently and await an array of Response objects:

$promises = array(
    $browser->get('http://example.com/'),
    $browser->get('http://www.example.org/'),
);

$responses = Block\awaitAll($promises, $loop);

Please refer to clue/reactphp-block for more details.

Keep in mind the above remark about buffering the whole response message in memory. As an alternative, you may also see one of the following chapters for the streaming API.

Concurrency

As stated above, this library provides you a powerful, async API. Being able to send a large number of requests at once is one of the core features of this project. For instance, you can easily send 100 requests concurrently while processing SQL queries at the same time.

Remember, with great power comes great responsibility. Sending an excessive number of requests may either take up all resources on your side or it may even get you banned by the remote side if it sees an unreasonable number of requests from your side.

// watch out if array contains many elements
foreach ($urls as $url) {
    $browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
        var_dump($response->getHeaders());
    });
}

As a consequence, it's usually recommended to limit concurrency on the sending side to a reasonable value. It's common to use a rather small limit, as doing more than a dozen of things at once may easily overwhelm the receiving side. You can use clue/reactphp-mq as a lightweight in-memory queue to concurrently do many (but not too many) things at once:

// wraps Browser in a Queue object that executes no more than 10 operations at once
$q = new Clue\React\Mq\Queue(10, null, function ($url) use ($browser) {
    return $browser->get($url);
});

foreach ($urls as $url) {
    $q($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
        var_dump($response->getHeaders());
    });
}

Additional requests that exceed the concurrency limit will automatically be enqueued until one of the pending requests completes. This integrates nicely with the existing Promise-based API. Please refer to clue/reactphp-mq for more details.

This in-memory approach works reasonably well for some thousand outstanding requests. If you're processing a very large input list (think millions of rows in a CSV or NDJSON file), you may want to look into using a streaming approach instead. See clue/reactphp-flux for more details.

Streaming response

All of the above examples assume you want to store the whole response body in memory. This is easy to get started and works reasonably well for smaller responses.

However, there are several situations where it's usually a better idea to use a streaming approach, where only small chunks have to be kept in memory:

  • If you're dealing with lots of concurrent requests (100+) or
  • If you want to process individual data chunks as they happen (without having to wait for the full response body) or
  • If you're expecting a big response body size (1 MiB or more, for example when downloading binary files) or
  • If you're unsure about the response body size (better be safe than sorry when accessing arbitrary remote HTTP endpoints and the response body size is unknown in advance).

You can use the requestStreaming() method to send an arbitrary HTTP request and receive a streaming response. It uses the same HTTP message API, but does not buffer the response body in memory. It only processes the response body in small chunks as data is received and forwards this data through ReactPHP's Stream API. This works for (any number of) responses of arbitrary sizes.

This means it resolves with a normal ResponseInterface, which can be used to access the response message parameters as usual. You can access the message body as usual, however it now also implements ReactPHP's ReadableStreamInterface as well as parts of the PSR-7's StreamInterface.

$browser->requestStreaming('GET', $url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    $body = $response->getBody();
    assert($body instanceof Psr\Http\Message\StreamInterface);
    assert($body instanceof React\Stream\ReadableStreamInterface);

    $body->on('data', function ($chunk) {
        echo $chunk;
    });

    $body->on('error', function (Exception $error) {
        echo 'Error: ' . $error->getMessage() . PHP_EOL;
    });

    $body->on('close', function () {
        echo '[DONE]' . PHP_EOL;
    });
});

See also the stream download example and the stream forwarding example.

You can invoke the following methods on the message body:

$body->on($event, $callback);
$body->eof();
$body->isReadable();
$body->pipe(React\Stream\WritableStreamInterface $dest, array $options = array());
$body->close();
$body->pause();
$body->resume();

Because the message body is in a streaming state, invoking the following methods doesn't make much sense:

$body->__toString(); // ''
$body->detach(); // throws BadMethodCallException
$body->getSize(); // null
$body->tell(); // throws BadMethodCallException
$body->isSeekable(); // false
$body->seek(); // throws BadMethodCallException
$body->rewind(); // throws BadMethodCallException
$body->isWritable(); // false
$body->write(); // throws BadMethodCallException
$body->read(); // throws BadMethodCallException
$body->getContents(); // throws BadMethodCallException

Note how timeouts apply slightly differently when using streaming. In streaming mode, the timeout value covers creating the underlying transport connection, sending the HTTP request, receiving the HTTP response headers and following any eventual redirects. In particular, the timeout value does not take receiving (possibly large) response bodies into account.

If you want to integrate the streaming response into a higher level API, then working with Promise objects that resolve with Stream objects is often inconvenient. Consider looking into also using react/promise-stream. The resulting streaming code could look something like this:

use React\Promise\Stream;

function download(Browser $browser, string $url): React\Stream\ReadableStreamInterface {
    return Stream\unwrapReadable(
        $browser->requestStreaming('GET', $url)->then(function (Psr\Http\Message\ResponseInterface $response) {
            return $response->getBody();
        })
    );
}

$stream = download($browser, $url);
$stream->on('data', function ($data) {
    echo $data;
});

See also the requestStreaming() method for more details.

Legacy info: Legacy versions prior to v2.9.0 used the legacy streaming option. This option is now deprecated but otherwise continues to show the exact same behavior.

Streaming request

Besides streaming the response body, you can also stream the request body. This can be useful if you want to send big POST requests (uploading files etc.) or process many outgoing streams at once. Instead of passing the body as a string, you can simply pass an instance implementing ReactPHP's ReadableStreamInterface to the request methods like this:

$browser->post($url, array(), $stream)->then(function (Psr\Http\Message\ResponseInterface $response) {
    echo 'Successfully sent.';
});

If you're using a streaming request body (React\Stream\ReadableStreamInterface), it will default to using Transfer-Encoding: chunked or you have to explicitly pass in a matching Content-Length request header like so:

$body = new React\Stream\ThroughStream();
$loop->addTimer(1.0, function () use ($body) {
    $body->end("hello world");
});

$browser->post($url, array('Content-Length' => '11'), $body);

If the streaming request body emits an error event or is explicitly closed without emitting a successful end event first, the request will automatically be closed and rejected.

HTTP proxy

You can also establish your outgoing connections through an HTTP CONNECT proxy server by adding a dependency to clue/reactphp-http-proxy.

HTTP CONNECT proxy servers (also commonly known as "HTTPS proxy" or "SSL proxy") are commonly used to tunnel HTTPS traffic through an intermediary ("proxy"), to conceal the origin address (anonymity) or to circumvent address blocking (geoblocking). While many (public) HTTP CONNECT proxy servers often limit this to HTTPS port443 only, this can technically be used to tunnel any TCP/IP-based protocol, such as plain HTTP and TLS-encrypted HTTPS.

$proxy = new Clue\React\HttpProxy\ProxyConnector(
    'http://127.0.0.1:8080',
    new React\Socket\Connector($loop)
);

$connector = new React\Socket\Connector($loop, array(
    'tcp' => $proxy,
    'dns' => false
));

$browser = new Clue\React\Buzz\Browser($loop, $connector);

See also the HTTP CONNECT proxy example.

SOCKS proxy

You can also establish your outgoing connections through a SOCKS proxy server by adding a dependency to clue/reactphp-socks.

The SOCKS proxy protocol family (SOCKS5, SOCKS4 and SOCKS4a) is commonly used to tunnel HTTP(S) traffic through an intermediary ("proxy"), to conceal the origin address (anonymity) or to circumvent address blocking (geoblocking). While many (public) SOCKS proxy servers often limit this to HTTP(S) port 80 and 443 only, this can technically be used to tunnel any TCP/IP-based protocol.

$proxy = new Clue\React\Socks\Client(
    'socks://127.0.0.1:1080',
    new React\Socket\Connector($loop)
);

$connector = new React\Socket\Connector($loop, array(
    'tcp' => $proxy,
    'dns' => false
));

$browser = new Clue\React\Buzz\Browser($loop, $connector);

See also the SOCKS proxy example.

SSH proxy

You can also establish your outgoing connections through an SSH server by adding a dependency to clue/reactphp-ssh-proxy.

Secure Shell (SSH) is a secure network protocol that is most commonly used to access a login shell on a remote server. Its architecture allows it to use multiple secure channels over a single connection. Among others, this can also be used to create an "SSH tunnel", which is commonly used to tunnel HTTP(S) traffic through an intermediary ("proxy"), to conceal the origin address (anonymity) or to circumvent address blocking (geoblocking). This can be used to tunnel any TCP/IP-based protocol (HTTP, SMTP, IMAP etc.), allows you to access local services that are otherwise not accessible from the outside (database behind firewall) and as such can also be used for plain HTTP and TLS-encrypted HTTPS.

$proxy = new Clue\React\SshProxy\SshSocksConnector('me@localhost:22', $loop);

$connector = new React\Socket\Connector($loop, array(
    'tcp' => $proxy,
    'dns' => false
));

$browser = new Clue\React\Buzz\Browser($loop, $connector);

See also the SSH proxy example.

Unix domain sockets

By default, this library supports transport over plaintext TCP/IP and secure TLS connections for the http:// and https:// URL schemes respectively. This library also supports Unix domain sockets (UDS) when explicitly configured.

In order to use a UDS path, you have to explicitly configure the connector to override the destination URL so that the hostname given in the request URL will no longer be used to establish the connection:

$connector = new React\Socket\FixedUriConnector(
    'unix:///var/run/docker.sock',
    new React\Socket\UnixConnector($loop)
);

$browser = new Browser($loop, $connector);

$client->get('http://localhost/info')->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump($response->getHeaders(), (string)$response->getBody());
});

See also the Unix Domain Sockets (UDS) example.

API

Browser

The Clue\React\Buzz\Browser is responsible for sending HTTP requests to your HTTP server and keeps track of pending incoming HTTP responses. It also registers everything with the main EventLoop.

$loop = React\EventLoop\Factory::create();

$browser = new Clue\React\Buzz\Browser($loop);

If you need custom connector settings (DNS resolution, TLS parameters, timeouts, proxy servers etc.), you can explicitly pass a custom instance of the ConnectorInterface:

$connector = new React\Socket\Connector($loop, array(
    'dns' => '127.0.0.1',
    'tcp' => array(
        'bindto' => '192.168.10.1:0'
    ),
    'tls' => array(
        'verify_peer' => false,
        'verify_peer_name' => false
    )
));

$browser = new Clue\React\Buzz\Browser($loop, $connector);

get()

The get(string|UriInterface $url, array $headers = array()): PromiseInterface<ResponseInterface> method can be used to send an HTTP GET request.

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump((string)$response->getBody());
});

See also example 01.

For BC reasons, this method accepts the $url as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

post()

The post(string|UriInterface $url, array $headers = array(), string|ReadableStreamInterface $contents = ''): PromiseInterface<ResponseInterface> method can be used to send an HTTP POST request.

$browser->post(
    $url,
    [
        'Content-Type' => 'application/json'
    ],
    json_encode($data)
)->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump(json_decode((string)$response->getBody()));
});

See also example 04.

This method is also commonly used to submit HTML form data:

$data = [
    'user' => 'Alice',
    'password' => 'secret'
];

$browser->post(
    $url,
    [
        'Content-Type' => 'application/x-www-form-urlencoded'
    ],
    http_build_query($data)
);

This method will automatically add a matching Content-Length request header if the outgoing request body is a string. If you're using a streaming request body (ReadableStreamInterface), it will default to using Transfer-Encoding: chunked or you have to explicitly pass in a matching Content-Length request header like so:

$body = new React\Stream\ThroughStream();
$loop->addTimer(1.0, function () use ($body) {
    $body->end("hello world");
});

$browser->post($url, array('Content-Length' => '11'), $body);

For BC reasons, this method accepts the $url as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

head()

The head(string|UriInterface $url, array $headers = array()): PromiseInterface<ResponseInterface> method can be used to send an HTTP HEAD request.

$browser->head($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump($response->getHeaders());
});

For BC reasons, this method accepts the $url as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

patch()

The patch(string|UriInterface $url, array $headers = array(), string|ReadableStreamInterface $contents = ''): PromiseInterface<ResponseInterface> method can be used to send an HTTP PATCH request.

$browser->patch(
    $url,
    [
        'Content-Type' => 'application/json'
    ],
    json_encode($data)
)->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump(json_decode((string)$response->getBody()));
});

This method will automatically add a matching Content-Length request header if the outgoing request body is a string. If you're using a streaming request body (ReadableStreamInterface), it will default to using Transfer-Encoding: chunked or you have to explicitly pass in a matching Content-Length request header like so:

$body = new React\Stream\ThroughStream();
$loop->addTimer(1.0, function () use ($body) {
    $body->end("hello world");
});

$browser->patch($url, array('Content-Length' => '11'), $body);

For BC reasons, this method accepts the $url as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

put()

The put(string|UriInterface $url, array $headers = array()): PromiseInterface<ResponseInterface> method can be used to send an HTTP PUT request.

$browser->put(
    $url,
    [
        'Content-Type' => 'text/xml'
    ],
    $xml->asXML()
)->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump((string)$response->getBody());
});

See also example 05.

This method will automatically add a matching Content-Length request header if the outgoing request body is a string. If you're using a streaming request body (ReadableStreamInterface), it will default to using Transfer-Encoding: chunked or you have to explicitly pass in a matching Content-Length request header like so:

$body = new React\Stream\ThroughStream();
$loop->addTimer(1.0, function () use ($body) {
    $body->end("hello world");
});

$browser->put($url, array('Content-Length' => '11'), $body);

For BC reasons, this method accepts the $url as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

delete()

The delete(string|UriInterface $url, array $headers = array()): PromiseInterface<ResponseInterface> method can be used to send an HTTP DELETE request.

$browser->delete($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump((string)$response->getBody());
});

For BC reasons, this method accepts the $url as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

request()

The request(string $method, string $url, array $headers = array(), string|ReadableStreamInterface $body = ''): PromiseInterface<ResponseInterface> method can be used to send an arbitrary HTTP request.

The preferred way to send an HTTP request is by using the above request methods, for example the get() method to send an HTTP GET request.

As an alternative, if you want to use a custom HTTP request method, you can use this method:

$browser->request('OPTIONS', $url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump((string)$response->getBody());
});

This method will automatically add a matching Content-Length request header if the size of the outgoing request body is known and non-empty. For an empty request body, if will only include a Content-Length: 0 request header if the request method usually expects a request body (only applies to POST, PUT and PATCH).

If you're using a streaming request body (ReadableStreamInterface), it will default to using Transfer-Encoding: chunked or you have to explicitly pass in a matching Content-Length request header like so:

$body = new React\Stream\ThroughStream();
$loop->addTimer(1.0, function () use ($body) {
    $body->end("hello world");
});

$browser->request('POST', $url, array('Content-Length' => '11'), $body);

Note that this method is available as of v2.9.0 and always buffers the response body before resolving. It does not respect the deprecated streaming option. If you want to stream the response body, you can use the requestStreaming() method instead.

requestStreaming()

The requestStreaming(string $method, string $url, array $headers = array(), string|ReadableStreamInterface $body = ''): PromiseInterface<ResponseInterface> method can be used to send an arbitrary HTTP request and receive a streaming response without buffering the response body.

The preferred way to send an HTTP request is by using the above request methods, for example the get() method to send an HTTP GET request. Each of these methods will buffer the whole response body in memory by default. This is easy to get started and works reasonably well for smaller responses.

In some situations, it's a better idea to use a streaming approach, where only small chunks have to be kept in memory. You can use this method to send an arbitrary HTTP request and receive a streaming response. It uses the same HTTP message API, but does not buffer the response body in memory. It only processes the response body in small chunks as data is received and forwards this data through ReactPHP's Stream API. This works for (any number of) responses of arbitrary sizes.

$browser->requestStreaming('GET', $url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    $body = $response->getBody();
    assert($body instanceof Psr\Http\Message\StreamInterface);
    assert($body instanceof React\Stream\ReadableStreamInterface);

    $body->on('data', function ($chunk) {
        echo $chunk;
    });

    $body->on('error', function (Exception $error) {
        echo 'Error: ' . $error->getMessage() . PHP_EOL;
    });

    $body->on('close', function () {
        echo '[DONE]' . PHP_EOL;
    });
});

See also ReadableStreamInterface and the streaming response for more details, examples and possible use-cases.

This method will automatically add a matching Content-Length request header if the size of the outgoing request body is known and non-empty. For an empty request body, if will only include a Content-Length: 0 request header if the request method usually expects a request body (only applies to POST, PUT and PATCH).

If you're using a streaming request body (ReadableStreamInterface), it will default to using Transfer-Encoding: chunked or you have to explicitly pass in a matching Content-Length request header like so:

$body = new React\Stream\ThroughStream();
$loop->addTimer(1.0, function () use ($body) {
    $body->end("hello world");
});

$browser->requestStreaming('POST', $url, array('Content-Length' => '11'), $body);

Note that this method is available as of v2.9.0 and always resolves the response without buffering the response body. It does not respect the deprecated streaming option. If you want to buffer the response body, use can use the request() method instead.

submit()

Deprecated since v2.9.0, see post() instead.

The deprecated submit(string|UriInterface $url, array $fields, array $headers = array(), string $method = 'POST'): PromiseInterface<ResponseInterface> method can be used to submit an array of field values similar to submitting a form (application/x-www-form-urlencoded).

// deprecated: see post() instead
$browser->submit($url, array('user' => 'test', 'password' => 'secret'));

For BC reasons, this method accepts the $url as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

send()

Deprecated since v2.9.0, see request() instead.

The deprecated send(RequestInterface $request): PromiseInterface<ResponseInterface> method can be used to send an arbitrary instance implementing the RequestInterface (PSR-7).

The preferred way to send an HTTP request is by using the above request methods, for example the get() method to send an HTTP GET request.

As an alternative, if you want to use a custom HTTP request method, you can use this method:

$request = new Request('OPTIONS', $url);

// deprecated: see request() instead
$browser->send($request)->then(โ€ฆ);

This method will automatically add a matching Content-Length request header if the size of the outgoing request body is known and non-empty. For an empty request body, if will only include a Content-Length: 0 request header if the request method usually expects a request body (only applies to POST, PUT and PATCH).

withTimeout()

The withTimeout(bool|number $timeout): Browser method can be used to change the maximum timeout used for waiting for pending requests.

You can pass in the number of seconds to use as a new timeout value:

$browser = $browser->withTimeout(10.0);

You can pass in a bool false to disable any timeouts. In this case, requests can stay pending forever:

$browser = $browser->withTimeout(false);

You can pass in a bool true to re-enable default timeout handling. This will respects PHP's default_socket_timeout setting (default 60s):

$browser = $browser->withTimeout(true);

See also timeouts for more details about timeout handling.

Notice that the Browser is an immutable object, i.e. this method actually returns a new Browser instance with the given timeout value applied.

withFollowRedirects()

The withTimeout(bool|int $$followRedirects): Browser method can be used to change how HTTP redirects will be followed.

You can pass in the maximum number of redirects to follow:

$new = $browser->withFollowRedirects(5);

The request will automatically be rejected when the number of redirects is exceeded. You can pass in a 0 to reject the request for any redirects encountered:

$browser = $browser->withFollowRedirects(0);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // only non-redirected responses will now end up here
    var_dump($response->getHeaders());
});

You can pass in a bool false to disable following any redirects. In this case, requests will resolve with the redirection response instead of following the Location response header:

$browser = $browser->withFollowRedirects(false);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // any redirects will now end up here
    var_dump($response->getHeaderLine('Location'));
});

You can pass in a bool true to re-enable default redirect handling. This defaults to following a maximum of 10 redirects:

$browser = $browser->withFollowRedirects(true);

See also redirects for more details about redirect handling.

Notice that the Browser is an immutable object, i.e. this method actually returns a new Browser instance with the given redirect setting applied.

withRejectErrorResponse()

The withRejectErrorResponse(bool $obeySuccessCode): Browser method can be used to change whether non-successful HTTP response status codes (4xx and 5xx) will be rejected.

You can pass in a bool false to disable rejecting incoming responses that use a 4xx or 5xx response status code. In this case, requests will resolve with the response message indicating an error condition:

$browser = $browser->withRejectErrorResponse(false);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // any HTTP response will now end up here
    var_dump($response->getStatusCode(), $response->getReasonPhrase());
});

You can pass in a bool true to re-enable default status code handling. This defaults to rejecting any response status codes in the 4xx or 5xx range with a ResponseException:

$browser = $browser->withRejectErrorResponse(true);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // any successful HTTP response will now end up here
    var_dump($response->getStatusCode(), $response->getReasonPhrase());
}, function (Exception $e) {
    if ($e instanceof Clue\React\Buzz\Message\ResponseException) {
        // any HTTP response error message will now end up here
        $response = $e->getResponse();
        var_dump($response->getStatusCode(), $response->getReasonPhrase());
    } else {
        var_dump($e->getMessage());
    }
});

Notice that the Browser is an immutable object, i.e. this method actually returns a new Browser instance with the given setting applied.

withBase()

The withBase(string|null|UriInterface $baseUrl): Browser method can be used to change the base URL used to resolve relative URLs to.

If you configure a base URL, any requests to relative URLs will be processed by first prepending this absolute base URL. Note that this merely prepends the base URL and does not resolve any relative path references (like ../ etc.). This is mostly useful for (RESTful) API calls where all endpoints (URLs) are located under a common base URL.

$browser = $browser->withBase('http://api.example.com/v3');

// will request http://api.example.com/v3/example
$browser->get('/example')->then(โ€ฆ);

You can pass in a null base URL to return a new instance that does not use a base URL:

$browser = $browser->withBase(null);

Accordingly, any requests using relative URLs to a browser that does not use a base URL can not be completed and will be rejected without sending a request.

This method will throw an InvalidArgumentException if the given $baseUrl argument is not a valid URL.

Notice that the Browser is an immutable object, i.e. the withBase() method actually returns a new Browser instance with the given base URL applied.

For BC reasons, this method accepts the $baseUrl as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

Changelog: As of v2.9.0 this method accepts a null value to reset the base URL. Earlier versions had to use the deprecated withoutBase() method to reset the base URL.

withProtocolVersion()

The withProtocolVersion(string $protocolVersion): Browser method can be used to change the HTTP protocol version that will be used for all subsequent requests.

All the above request methods default to sending requests as HTTP/1.1. This is the preferred HTTP protocol version which also provides decent backwards-compatibility with legacy HTTP/1.0 servers. As such, there should rarely be a need to explicitly change this protocol version.

If you want to explicitly use the legacy HTTP/1.0 protocol version, you can use this method:

$newBrowser = $browser->withProtocolVersion('1.0');

$newBrowser->get($url)->then(โ€ฆ);

Notice that the Browser is an immutable object, i.e. this method actually returns a new Browser instance with the new protocol version applied.

withResponseBuffer()

The withRespomseBuffer(int $maximumSize): Browser method can be used to change the maximum size for buffering a response body.

The preferred way to send an HTTP request is by using the above request methods, for example the get() method to send an HTTP GET request. Each of these methods will buffer the whole response body in memory by default. This is easy to get started and works reasonably well for smaller responses.

By default, the response body buffer will be limited to 16 MiB. If the response body exceeds this maximum size, the request will be rejected.

You can pass in the maximum number of bytes to buffer:

$browser = $browser->withResponseBuffer(1024 * 1024);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // response body will not exceed 1 MiB
    var_dump($response->getHeaders(), (string) $response->getBody());
});

Note that the response body buffer has to be kept in memory for each pending request until its transfer is completed and it will only be freed after a pending request is fulfilled. As such, increasing this maximum buffer size to allow larger response bodies is usually not recommended. Instead, you can use the requestStreaming() method to receive responses with arbitrary sizes without buffering. Accordingly, this maximum buffer size setting has no effect on streaming responses.

Notice that the Browser is an immutable object, i.e. this method actually returns a new Browser instance with the given setting applied.

withOptions()

Deprecated since v2.9.0, see withTimeout(), withFollowRedirects() and withRejectErrorResponse() instead.

The deprecated withOptions(array $options): Browser method can be used to change the options to use:

The Browser class exposes several options for the handling of HTTP transactions. These options resemble some of PHP's HTTP context options and can be controlled via the following API (and their defaults):

// deprecated
$newBrowser = $browser->withOptions(array(
    'timeout' => null, // see withTimeout() instead
    'followRedirects' => true, // see withFollowRedirects() instead
    'maxRedirects' => 10, // see withFollowRedirects() instead
    'obeySuccessCode' => true, // see withRejectErrorResponse() instead
    'streaming' => false, // deprecated, see requestStreaming() instead
));

See also timeouts, redirects and streaming for more details.

Notice that the Browser is an immutable object, i.e. this method actually returns a new Browser instance with the options applied.

withoutBase()

Deprecated since v2.9.0, see withBase() instead.

The deprecated withoutBase(): Browser method can be used to remove the base URL.

// deprecated: see withBase() instead
$newBrowser = $browser->withoutBase();

Notice that the Browser is an immutable object, i.e. the withoutBase() method actually returns a new Browser instance without any base URL applied.

See also withBase().

ResponseInterface

The Psr\Http\Message\ResponseInterface represents the incoming response received from the Browser.

This is a standard interface defined in PSR-7: HTTP message interfaces, see its ResponseInterface definition which in turn extends the MessageInterface definition.

RequestInterface

The Psr\Http\Message\RequestInterface represents the outgoing request to be sent via the Browser.

This is a standard interface defined in PSR-7: HTTP message interfaces, see its RequestInterface definition which in turn extends the MessageInterface definition.

UriInterface

The Psr\Http\Message\UriInterface represents an absolute or relative URI (aka URL).

This is a standard interface defined in PSR-7: HTTP message interfaces, see its UriInterface definition.

For BC reasons, the request methods accept the URL as either a string value or as an UriInterface. It's recommended to explicitly cast any objects implementing UriInterface to string.

ResponseException

The ResponseException is an Exception sub-class that will be used to reject a request promise if the remote server returns a non-success status code (anything but 2xx or 3xx). You can control this behavior via the withRejectErrorResponse() method.

The getCode(): int method can be used to return the HTTP response status code.

The getResponse(): ResponseInterface method can be used to access its underlying ResponseInterface object.

Install

The recommended way to install this library is through Composer. New to Composer?

This project follows SemVer. This will install the latest supported version:

$ composer require clue/buzz-react:^2.9

See also the CHANGELOG for details about version upgrades.

This project aims to run on any platform and thus does not require any PHP extensions and supports running on legacy PHP 5.3 through current PHP 7+ and HHVM. It's highly recommended to use PHP 7+ for this project.

Tests

To run the test suite, you first need to clone this repo and then install all dependencies through Composer:

$ composer install

To run the test suite, go to the project root and run:

$ php vendor/bin/phpunit

The test suite also contains a number of functional integration tests that send test HTTP requests against the online service http://httpbin.org and thus rely on a stable internet connection. If you do not want to run these, they can simply be skipped like this:

$ php vendor/bin/phpunit --exclude-group online

License

This project is released under the permissive MIT license.

Did you know that I offer custom development services and issuing invoices for sponsorships of releases and for contributions? Contact me (@clue) for details.

reactphp-buzz's People

Contributors

asm89 avatar carusogabriel avatar clue avatar eislambey avatar holtkamp avatar masakielastic avatar mmoreram avatar rakdar avatar samnela avatar seregazhuk avatar simonfrings avatar wyrihaximus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

reactphp-buzz's Issues

Look into HTTPS support

HTTPS is supported right nowโ€ฆkind ofโ€ฆ

This should work reasonably well in recent PHP versions. Several changes to the underlying react/socket-client component and PHP itself contribute to quite a different behavior.

For example, accessing https://svn.apache.org does not work in PHP 5.3, because it does not use the correct SNI headers.

At the very least, this needs some documentation to raise awareness.

getCode is wrong

In documentation says to get the status code is getCode() but actually is getStatusCode(). Also if use foreach to get every URL from array and do $client->get() inside foreach loop. Would this slowdown asynchronous loop from ReactPHP?

Edit:
Just tried and if i go with foreach through $urls and then call $client->get(); is not going asynchronous so how to pass links to asnyc loop?

Add simple streaming API

An experimental streaming API has been implemented via #31. However, it's quite cumbersome to work with.

In particular, one still has to wait on a promise progress event until one can access a stream object.

Instead, we should add a simple API that returns a stream right away and then emits events on it once the HTTP response comes in.

Setting Headers

Hopefully someone can explain, firstly, if we can set headers such as a user agent, secondly, how to do so? I have been trying to manipulate so many ways to add a custom user agent and referrer I have gotten lost in what I have done! I am finding myself trying to implement the same methods over and over.. As an example of I'm looking for, with a Guzzle HTTP client we can do - $client->setUserAgent('Test/123'); - but I am using Clue\React\Socks\Client as SocksClient. Looking at the source code I cannot find a function for this, and also looking at the guzzle source and can see it implemented in there. I am not yet advanced enough to apply this myself, although I will try.. :/ can any one help?

[question] How to access this array?

Hello,
Just some noob-ish question from me,
How to get this (green boxed) data:
data image
My code
App/MyScraper/Bukalapak.php

<?php
namespace App\MyScraper;

use Clue\React\Buzz\Browser;
use function React\Promise\all;
use React\Promise\PromiseInterface;
use Psr\Http\Message\ResponseInterface;
use Symfony\Component\DomCrawler\Crawler;
use App\MyScraper\Objects\BukalapakObject;


class Bukalapak
{
    private $browser;

    public function __construct(Browser $browser)
    {
        $this->browser = $browser;
    }
    
    public function scrape(string ...$urls): PromiseInterface
    {
        $promises = array_map(function ($url) {
            return $this->extractFromUrl($url);
        }, $urls);

        return all($promises);
    }
    
    private function extract(string $responseBody) : BukalapakObject
    {
        $crawler = new Crawler($responseBody);

        $name        = $crawler->filter('.c-product-detail__name')->text();
        $price       = $crawler->filter('.c-product-detail-price')->attr('data-reduced-price');
        $image       = $crawler->filter('.c-product-image-gallery__main img')->attr('src');
        $stock       = preg_replace('/[^0-9]+/', '', $crawler->filter('.qa-pd-stock')->text());
        $weight      = preg_replace('/[^0-9]+/', '', $crawler->filter('.qa-pd-weight-value')->text());
        $condition   = $crawler->filter('.qa-pd-condition-value > span.c-label')->text();
        $description = trim($crawler->filter('.qa-pd-description')->html(), "\n");
        $assurance   = 'Tidak';
        $courier     = 'jner|j&tr';

        return new BukalapakObject($name, $price, $image, $stock, $weight, $condition, $assurance, $courier);
    }
    
    private function extractFromUrl(string $url) : PromiseInterface
    {
        return $this->browser->get($url)->then(function (ResponseInterface $response) {
            return $this->extract((string) $response->getBody());
        });
    }
}

App/MyScraper/Helpers/BukalapakHelper.php

<?php
namespace App\MyScraper\Helpers;

use Goutte;

class BukalapakHelper
{
	public static function fetchAllProductsLink(string $shopUrl)
	{
	    $goutte = Goutte::request('GET', $shopUrl);
	    $data = $goutte->filter('.product-display')->each(function ($node) {
	        return [
	            'https://www.bukalapak.com'
	                .explode('?', $node->filter('.js-tracker-product-link')->attr('href'))[0]
	        ];
	    });
	    unset($goutte);
	    return array_column($data, 0);
	}
	
	public static function writeToCSV(string $fileName, array $data = [])
	{
	    $handle = fopen($fileName, 'w');
	    foreach ($data as $value) {
	        fputcsv($handle, [key($data), $value]);
	    }
	    fclose($handle);
	}
}

App/MyScraper/Objects/BukalapakObject.php

<?php
namespace App\MyScraper\Objects;

final class BukalapakObject
{
    public $name;
    public $price;
    public $image;
    public $stock;
    public $weight;
    public $condition;
    public $assurance;
    public $courier;


    public function __construct(
        string $name,
        string $price,
        string $image,
        string $stock,
        string $weight,
        string $condition,
        string $assurance,
        string $courier
    )
    {
        $this->name = $name;
        $this->price = $price;
        $this->image = $image;
        $this->stock = $stock;
        $this->weight = $weight;
        $this->condition = $condition;
        $this->assurance = $assurance;
        $this->courier   = $courier;
    }
}

Controller (App/Http/Controllers/TestController.php)

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;

use Clue\React\Buzz\Browser;
use App\MyScraper\Bukalapak;
use App\MyScraper\Helpers\BukalapakHelper;

class TestController extends Controller
{

	public function index()
	{
		$shop = 'https://www.bukalapak.com/u/attiqahijab';
		$urls = BukalapakHelper::fetchAllProductsLink($shop);

		$loop = \React\EventLoop\Factory::create();
		$browser = new Browser($loop);
		$scraper = new Bukalapak($browser);
		
		$data = $scraper->scrape(...$urls)->then(function($result) {
			return $result;
		});
		$loop->run();

		print_r($data);

		return view('backend.test');
	}

}

I also use this laravel-goutte library.

Documentation for Browser API (docblocks)

The Browser API is currently documented in the README.md, but the class does not have any docblocks otherwise. We should add some docblocks to ease static analysis and/or IDE auto-completion.

Initial request data is lost when streaming response

Hi,

In react HttpClient when response is parsed "data" event is immediately triggered after response is created. As a result, if promise that is waiting for response is issued in the next tick, data is lost (as no one handled that part of body after headers).

Simpliest solution would be wrapping response body with paused buffering stream.

documentation should include some reference/comparison to guzzle

This looks like a nice library but I'm currently using Guzzle and I'm trying to understand why I should use this.

It would be helpful if the documentation included some reference guzzle and explain what the differences are - why someone would choose one over the other.

Eg. Perhaps, Buzz is much faster but offers fewer customization options. So if you need raw speed Buzz is better but if you need X, Y or Z, then Guzzle is a better choice.

If the two are basically the same but Buzz is easier to use - that's also a selling point. But something that can help me understand why I should switch.

Add basic stream control to sync streams

Hey,

There is always a chance that input and output streams have different bandwidth. Like if we are reading a file and sending it somewhere it is read at much higher speed. Result is simple - more and more data is buffered into memory.

Possible solution could be usage of buffered stream, that pauses itself when buffer overflows + "drain" listener on output stream, that resumes it.

Setting header X-MBX-APIKEY is not working

Hi there,

I'm trying to set the X-MBX-APIKEY header, however somehow it's not getting through. Is there anything wrong with this code? (I'm 100% certain the endpoint and key are correct, tested in Postman)

At worst I should get a 400 (Bad Request), but I'm getting 401 (Unauthorized). It's not sending the key correctly it seems. Here's my code:

$this->client->get($endpoint, [
    'X-MBX-APIKEY: ' . $secret['key']
]);

Failing connector will not be detected

This is kind of an esoteric situation, but using a Connector that immediately rejects will not be detected, because the error handler will be attached after the actual error has been raised.

Expose transaction settings to outside

The Transaction has several settings such as followRedirects, maxRedirects and obeySuccessCode. However, there's currently no way to control these from the calling script.

Custom headers not re-used when following redirect

By default the Browser object follows HTTP redirect codes.

However, it seems that when doing so, any custom headers that were used on the initial request are not re-used when following the redirect. Not sure whether this should be default HTTP behaviour (but I think so?)

I suppose re-using the headers should be done here https://github.com/clue/php-buzz-react/blob/ba4eb355e9ffe4bb2a9071ccc69698d48da548dc/src/Io/Transaction.php#L136

Not sure, but maybe something like:

$request = $this->messageFactory->request($method, $location, $request->getHeaders());

The reason how I found out: we use custom headers to simulate the "Google Bot" by using a UserAgent header like Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

After a redirect, the default UserAgent seems to be used: React/alpha defined at https://github.com/reactphp/http-client/blob/d779a3b3c91ee19742839df7395b366da4471d12/src/RequestData.php#L29

Timeouts no longer throw TimeoutException

return \React\Promise\Timer\timeout($deferred->promise(), $timeout, $this->loop)->then(null, function ($e) {
if ($e instanceof TimeoutException) {
throw new \RuntimeException(
'Request timed out after ' . $e->getTimeout() . ' seconds'
);
}
throw $e;
});

Related to 32de457

Timeouts no longer throw a TimeoutException, instead throwing a generic RuntimeException. This makes it more difficult to programmatically handle timeouts.

Could this change be reverted or is there a specific reason for it? Could the code throw a more specific exception instead please?

Implement PSR-18

I've tried to implement PSR-18 to this library. But as you (@clue) told me a year ago, it might be impossible to do sync requests with this library. In guzzle we do something like:

public function sendRequest(RequestInterface $request): ResponseInterface
{
    return $this->sendAsyncRequest($request)->wait();
}

Nevertheless, I'm about to open the vote on the specification and would like to hear your comments before that.

See the specification: https://github.com/php-fig/fig-standards/blob/master/proposed/http-client/http-client.md
Read the blog post: https://medium.com/php-fig/the-http-client-psr-9c2535132980

Using done() on request returns a null value

Hello, I recently started using your library within the context of an application powered by ReactPHP. I encountered a problem with the promises returned from the calls made via the client.

...
//function body
$client = new Browser($loop);

$request = $client->get(url, headers)->done(function ($response) {
  return (string) $response->getBody();
});

return $request;
...

The snippet of code above is a small part of a larger application. The problem I am encountering is that the request returns a null value and yet using the echo directive on the response prints a valid API response. I would like to know how to fix this.

Browser in php-fpm Docker times out and never fetches data (default LoopInterface implementation)

Prerequisites:

  • The script is run in a php-fpm Docker container
  • The command is wrapped in an instance of Symfony\Component\Console\Command\Command
  • Host machine MacOS latest

Test:

$loop = React\EventLoop\Factory::create();

$loop->addTimer(11, function()use($loop) {
            echo "Timed out\n";
            $loop->stop();
        });

$browser = new Browser($loop);

            /** @var Promise $promise */
            $promise = $browser->get('http://ukr.net/');
            $promise->done(
                function(Response $data){
                    echo 'Fulfilment code =: ' . $data->getCode();
                },
                function($reason) {
                    echo 'Rejection reason: ' . (string)$reason;
                }
            );

$loop->run();

Please note that requesting the same URL through php curl OR curl in console works just fine.

The result is always a Timed out output.

Am I doing something wrong? If not, where do I start to dig to find out what the issue is?

Connection refused from destination server..

Hi @clue,
I have been using this scripts for months for my asynchronous process http submit and request, but suddenly it has been a week that my destination server was refused to receive connection for it.
My question is what kind of specific trouble for connection refused? it was close off tcp port?
The Exception file show that error from react tcpconnector.php?
Regards,
Roy

Post request loosing content and not getting correct response

Please excuse if this is not the right place to ask but i haven't found any forum or chat.

I am trying to do a POST request with Buzz to another server that is running some Slim PHP application. Somehow the content is lost on the receiving end and the same in reverse. The request itself comes through perfectly and the other code works as well as it was tested by using some JS.

The other thing would be the response. It is always giving me null no matter what i am sending back.

Is there anything i am missing? The request itself is correctly identified as POST on the recieving end and it works fine with all my GET requests. The documentation or examples are slim on that topic.

Here is the testing code that is not working for me (i need those TLS options because of self signed SSL):

$content['var'] = 'bla';

$content = http_build_query($content);

$connection = new Connector($loop, [
    'tls' => [
        'verify_peer' => false,
        'verify_peer_name' => false,
        'allow_self_signed' => true,
   ]
]);

$browser = new Browser($loop, $connection);

$browser = $browser->withOptions(array(
    'followRedirects' => true,
    'maxRedirects' => 10,
    'obeySuccessCode' => true,
    'streaming' => false,
));

$browser->post($url, array(), $content)->then(function(ResponseInterface $response) {
    $output = (string)$response->getBody();
});

Explicitly close connection after response body to support servers ignoring Connection: close request header

Some HTTP/1.1 servers do not honor the Connection: Close header. bufferResponse will never resolve when making requests to these.

Here is an example fix:
jmoo@a9c27ec

The only problem is that \React\Promise\Stream\buffer doesn't resolve when maxLength is reached and changing it to resolve would be a BC break. We'd have to roll our own or start a discussion over on https://github.com/reactphp/promise-stream about adding a $resolveOnMaxLengthReached flag or something.

Related to #89

Fix cancellation of redirected requests

While working on an implementation for clue/reactphp-mq#6, I noticed that some requests do no seem to be cancelled properly, i.e. they keep running despite their promises being cancelled. A similar issues came up in clue/reactphp-mq#7 as it relies on this project for its example. It turns out that this affects cancellation of requests that are issued after the initial request is successfully completed (i.e. redirected requests). Cancellation of operations that have been started instantly worked as expected.

This will eventually also be addressed upstream via reactphp/promise#99, but it's currently not decided when this version will be tagged.

Upgrade to new ReactPHP components

This component uses deprecated SocketClient which has been replaced by Socket. Also, the HttpClient will be merged into the Http component. Both changes will involve a BC break to our (advanced) API, so I'd like to wait for the ReactPHP component to become more stable for updating this library.

Getting acknowledgement response, not the result response

Hi there,

I'm calling the Binance API (https://github.com/binance-exchange/binance-official-api-docs/blob/master/rest-api.md - POST /api/v3/order) using this code:

$response = yield $this->client->post($endpoint, ['X-MBX-APIKEY' => $exchange['key']]);
$response = json_decode($response->getBody());

This is done using this package and Recoil (hence the yield). However the result I get back is the ACK response, not the RESULT response.

How can I make it so that I get the second response?

Thanks so much,
Maarten

Empty body on POST request not compatible with some legacy servers (411 Length Required)

When sending a HTTP request with a method which has a request body (e.g. POST or PUT) but with an empty body (size=0), then Buzz does not apply a Content-Length header, despite there being no body and as such it violates the HTTP RFC which requires a Content-Length header (or an actual body with chunked transfer).

Looking at the code the body seems to be handled as if it were a real stream, even if the size is known and 0. According to PSR-7 getSize() returns null if the size is unknown, otherwise an integer if the size is known.
grafik

As such this implementation is non-compliant with PSR-7. Is there any reason for this check to be there?
I'm not familiar whether all streams, or only a subset of streams, return 0 if the size is unknown. But PSR-7 compliant streams[1] always return null for unknown sizes, so wouldn't it make sense to only respect 0 as unknown size if the stream is not a PSR-7 stream?

[1] e.g. https://github.com/ringcentral/psr7/blob/360faaec4b563958b673fb52bbe94e37f14bc686/src/Stream.php#L135-L157

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.