amphp / http-client Goto Github PK

View Code? Open in Web Editor NEW

701.0 701.0 67.0 4.86 MB

An advanced async HTTP client library for PHP, enabling efficient, non-blocking, and concurrent requests and responses.

Home Page: https://amphp.org/http-client

License: MIT License

PHP 99.92% Shell 0.08%

amphp async http http-client https php revolt

http-client's People

Contributors

Stargazers

Watchers

http-client's Issues

AsyncClient stops after few socket errors

Easy to reproduce with current stable version (0.7.0) and this code (to fix it - just add socket clearing in doError() function):

<?php

require(__DIR__ . '/vendor/autoload.php');

$reactor = (new Alert\ReactorFactory)->select();
$client = new Artax\AsyncClient($reactor);

$client->setOption('maxconnections', 5);
$client->setOption('connecttimeout', 1);

// Generate a request URI for each letter a-z
$requests = array_map(function($alpha) { return 'http://8.8.8.8/search?q=' . $alpha; }, range('a', 'z'));

// We need to track how many requests remain so we can stop the program when they're all finished
$unfinishedRequests = count($requests);

// What to do when an individual request completes
$onResponse = function(Artax\Response $response, Artax\Request $request) use (&$unfinishedRequests, $reactor) {
    echo $request->getUri(), ' -- ';
    echo 'HTTP/', $response->getProtocol(), ' ', $response->getStatus(), ' ', $response->getReason(), "\n";
    echo $unfinishedRequests, PHP_EOL;
    if (!--$unfinishedRequests) {
        $reactor->stop();
    }
};

// What to do if a request encounters an exceptional error
$onError = function(Exception $e, Artax\Request $request) use (&$unfinishedRequests, $reactor) {
    echo $request->getUri(), " failed (", get_class($e), ") :(\n";
    echo $unfinishedRequests, PHP_EOL;
    if (!--$unfinishedRequests) {
        $reactor->stop();
    }
};

// The reactor IS our task scheduler and the program runs inside it. Nothing will happen until the
// event reactor is started, so release the hounds!
$reactor->run(function() use ($client, $requests, $onResponse, $onError) {
    echo 'Requesting ', count($requests), ' URIs ...', "\n";
    foreach ($requests as $uri) {
        $client->request($uri, $onResponse, $onError);
    }
});

And another issue on current master branch - script just terminates after 5th failure.

ByPass SSL certificate validation with Artax

Please see issue here:

http://stackoverflow.com/questions/23732447/bypass-ssl-certificate-validation-with-artax

High load instability

Hi - so as discussed I'm seeing weird stuff happen - that may be being caused by a bug in PHP. I'm going to dump some stuff in here though, so that it can be referenced.

After recompiling PHP with debugging enabled I'm seeing an error when running my app through strace:

PHP Notice:  Undefined offset: 281 in /home/github/Bastion/Bastion/vendor/amphp/artax/lib/SocketPool.php on line 163
PHP Notice:  Trying to get property of non-object in  /home/github/Bastion/Bastion/vendor/amphp/artax/lib/SocketPool.php on line 164

I have no idea if it's actually a real issue or whether it's an artifact caused by system instability.

Question : Non blocking request

Hi,

I have a question regarding non blocking request. I wanted to send the request, but I don't care to wait for the response. Is there a way to achieve it ?

Say calling a url : http://example.com/post/hello?w=222 . Just a dummy one.

Thank you.

Update Readme

The code manually implements the HTTP over TCP sockets; as such it has no dependency on PHP's disastrous curl_* API and requires no non-standard PHP extensions.

It's not cool to make derogatory comments about other implementations of functionality.

I've never thought curl to be disastrous and never had any problems whatsoever with it.

'disastrous' should be removed.

Fatal errors

Tested on windows 8.1 x64:

Fatal error: Undefined class constant 'OP_VERBOSE' in Artax\examples\005_persistent_cookies.php on line 15

Fatal error: Cannot use object of type Artax\ResourceIterator as array in Artax\lib\FormBody.php on line 167

Catchable fatal error: Argument 1 passed to After\some() must be of the type array, null given, called in Artax\examples\008_multi_request.php on line 21 and defined in Artax\vendor\rdlowrey\after\lib\functions.php on line 56

Redirect to HTTPS throws exception

The request to:
"http://api.github.com/repositories/6269013/tags?owner=aws&repo=aws-sdk-php&page=1"

In the example below should (probably) be redirected to the HTTPS version of the URI. Instead an exception is thrown. I haven't touched #62 so there are probably two reactors being made....not sure if that's relevant.

<?php

require_once(realpath(__DIR__).'/../vendor/autoload.php');

$client = new Amp\Artax\Client;

use Amp\Artax\Progress;

$callback = function($update) {

    $knownStates = [   
        Progress::CONNECTING => "Progress::CONNECTING",
        Progress::SENDING_REQUEST => "Progress::SENDING_REQUEST",
        Progress::AWAITING_RESPONSE => "Progress::AWAITING_RESPONSE",
        Progress::REDIRECTING => "Progress::REDIRECTING",
        Progress::READING_LENGTH => "Progress::READING_LENGTH",
        Progress::READING_UNKNOWN => "Progress::READING_UNKNOWN",
        Progress::COMPLETE => "Progress::COMPLETE",
    ];

    $message = "Unknown state ".$update['request_state'];

    if (isset($knownStates[$update['request_state']])) {
        $message = $knownStates[$update['request_state']];
    }

    echo $message."\n";    
};

for ($x=0 ; $x<1 ; $x++) {

    $randomWord = md5(md5(time()).$x);

    $request = "http://api.github.com/repositories/6269013/tags?owner=aws&repo=aws-sdk-php&page=1";

    $promise = $client->request($request);

    $promise->watch(new \Amp\Artax\Progress($callback));
    $response = $promise->wait();
    /** @var $response \Amp\Artax\Response */
    $previous = $response;

    while ($previous) {
        echo "URI: ".$previous->getRequest()->getUri()."\n";
        $previous = $previous->getPreviousResponse();
    }
}

Fatal error: Uncaught exception 'Amp\Artax\SocketException' with message 'Socket disconnected prior to write completion :(' in /documents/projects/github/Bastion/Bastion/vendor/amphp/artax/lib/BufferWriter.php on line 43

Amp\Artax\SocketException: Socket disconnected prior to write completion :( in /documents/projects/github/Bastion/Bastion/vendor/amphp/artax/lib/BufferWriter.php on line 43

Call Stack:
    0.0002     232872   1. {main}() /documents/projects/github/Bastion/Bastion/src/test.php:0
    0.0245    2217824   2. Amp\Future->wait() /documents/projects/github/Bastion/Bastion/src/test.php:39

PHP Fatal error:  Uncaught exception 'Amp\Artax\SocketException' with message 'Socket disconnected prior to write completion :(' in /documents/projects/github/Bastion/Bastion/vendor/amphp/artax/lib/BufferWriter.php:43
Stack trace:
#0 /documents/projects/github/Bastion/Bastion/vendor/amphp/artax/lib/BufferWriter.php(75): Amp\Artax\BufferWriter->doWrite()
#1 /documents/projects/github/Bastion/Bastion/vendor/amphp/amp/lib/NativeReactor.php(153): Amp\Artax\BufferWriter->Amp\Artax\{closure}(Object(Amp\NativeReactor), 8, Resource id #94)
#2 /documents/projects/github/Bastion/Bastion/vendor/amphp/amp/lib/NativeReactor.php(115): Amp\NativeReactor->selectActionableStreams(119.9999)
#3 /documents/projects/github/Bastion/Bastion/vendor/amphp/amp/lib/Future.php(90): Amp\NativeReactor->tick()
#4 /documents/projects/github/Bastion/Bastion/src/test.php(39): Amp\Future->wait()
#5 {main}
  thrown in /documents/projects/github/Bastion/Bastion/vendor/amphp/artax/lib/BufferWriter.php on line 43

Problem in Uri::parseQueryParameters

If uri has array data, like params[]=1&params[]=2, or params[1][0]=1&params[1][1]=2

I had error, at line 440, because function rawurldecode can't decode arrays:
$values = array_map('rawurldecode', array_values($parameters));

Taking the information outside $onResponse

I'm using the example available in the documentation for requestMulti but I can not manage to move the response information outside the callback function $onResponse to work with it.

<?php

$client = new Artax\Client;

//how to take $myArray outside it
$onResponse = function($requestKey, Artax\Response $response) {
    $myArray[$requestKey] = $response->getBody();
};
$onError = function($requestKey, Exception $error) {
    echo 'Error: (', $requestKey, ') ', $error->getMessage(), "\n";
};
$requests = [
    'google' => 'http://www.google.com',
    'google news' => 'http://news.google.com',
    'bing' => 'http://www.bing.com',
    'yahoo' => 'http://www.yahoo.com',
    'php' => 'http://www.php.net'
];

$client->requestMulti($requests, $onResponse, $onError);

//I want to show the array with all the responses here, for example
print_r($myArray);

I'm using Laravel framework in case it helps in some way with my issue.

Multiple Reactor being made

The singleton is on the function Amp\react() but this is not used in the Amp\Artax\Client::__construct function, and so the singleton is skipped.

Allegedly this is 'crossing the streams' bad.

Fatal error in blocking client - Possibly related to Expect/Continue

I think the new expectContinue functionality is breaking blocking clients under certain circumstances. I'm having difficulty producing a simple reproduction script that works in an external environment though.

I am using a single client, and creating new request objects on demand as necessary, all requests are being sent to the same site using POST and having form data.

The first request returns control to the caller with a 417 (Expectation Failed) and the second request (about 5 seconds later) causes:

Fatal error: Call to a member function cancel() on a non-object in AsyncClient.php on line 564

As bodyDrainObservation is not set on the second request for some reason.

It seems to be a network local squid proxy returning the 417.

$client->setOption('expectContinue', false); fixes the issue, hence my assumption it is related to this.

Problem with a simple GET request

There is a problem with your simple GET example if you try to fetch a page from a non-existing domain such as:

<?php
$client = new Artax\Client;
$response = $client->request('http://www.gccccccccoogle.com');
echo "Response status code: ", $response->getStatus(), "\n";
echo "Response reason:      ", $response->getReason(), "\n";
echo "Response protocol:    ", $response->getProtocol(), "\n";
print_r($response->getAllHeaders());
echo $response->getBody();
?>

The script keeps looping until php timeout is reached or if is set to 0, indefinitely with no error or a response.

I am using Artax-0.3.3 and AMP-0.1.0 on a Windows 7 machine

Am I missing something here?

Is there a way to have just 100 request going in the same time? What are limitations really?

So I want to scrape 1 million URLs. That's a lot of time and memory to be used. How many requests are made by artax in the same time? Is there a way to redefine it? Can I have just X request's active in any given time?

Passing values to FormBody

Some code:

$formBody = new \Artax\FormBody;
$formBody->addField('tags', $this->parameters['tags']);

When the value of $this->parameters['tags'] is null, the error InvalidArgumentException: Invalid field value; scalar or array expected is generated. I realise that passing a null value isn't technically possible - but shouldn't Artax convert it to an empty string?
When the value is an array, there is a fatal error here as is_scalar($somearray) is false, and it tries to call the count function on the array.

cheers
Dan

How to install it?

Please add to README installation instructions for Windows

Performance

Did you do some perf testing of Artax against curl, guzzle or something like that?

I am curious how your php userland solution competes against the curl extension and competitors...

Error on rc-5 release

Deprecated: Amp\ReactorFactory is deprecated and scheduled for removal. Please update code to use the Amp\getReactor() function instead. in D:\wwwroot\gp_dev_patrick\vendor\amphp\amp\lib\ReactorFactory.php on line 16

It works with dev-master. Would be great if you could release another version with the fix that is already in dev-master so that we don't have to use dev-master as version.

Thanks :)

Unknown socket exception

I'm getting the error below 'sometimes' - which is totally vague I know. It doesn't reproduce in simple code, but effectively all I'm doing is:

<?php

require_once "../vendor/autoload.php";

use Amp\Artax\Client as ArtaxClient;

$reactor = \Amp\getReactor();

$client1 = new ArtaxClient($reactor);
$client2 = new ArtaxClient($reactor);

$promise1 = $client1->request('http://www.google.com');
$response1 = \Amp\wait($promise1);

$promise2 = $client2->request('https://github.com/danack/imagick-demos/archive/1864936a93649bfd94cdad3b905b51eb6b416ac0.tar.gz');
$response2 = \Amp\wait($promise2);

As I said, that doesn't actually show the issue.

Unexpected exception of type DomainException running Deployer`: Unknown socket: Resource id #186
#0 .../vendor/amphp/artax/lib/HttpSocketPool.php(114): Amp\Artax\SocketPool->checkin(Resource id #186)
#1 .../vendor/amphp/artax/lib/Client.php(543): Amp\Artax\HttpSocketPool->checkin(Resource id #186)
#2 .../vendor/amphp/artax/lib/Client.php(773): Amp\Artax\Client->assignParsedResponse(Object(Amp\Artax\RequestCycle), Array)
#3 .../vendor/amphp/artax/lib/Client.php(461): Amp\Artax\Client->processDeadSocket(Object(Amp\Artax\RequestCycle))
#4 .../vendor/amphp/artax/lib/Client.php(436): Amp\Artax\Client->onReadableSocket(Object(Amp\Artax\RequestCycle))
#5 .../vendor/amphp/amp/lib/NativeReactor.php(185): Amp\Artax\Client->Amp\Artax\{closure}(Object(Amp\NativeReactor), 25, Resource id #186)
#6 .../vendor/amphp/amp/lib/NativeReactor.php(151): Amp\NativeReactor->selectActionableStreams(98.0418)
#7 .../vendor/amphp/amp/lib/functions.php(534): Amp\NativeReactor->tick()
#8 .../src/.../Deployer/Deployer.php(103): Amp\wait(Object(Amp\Future))
#9 .../src/.../Deployer/Deployer.php(48): ...\Deployer\Deployer->downloadPackage('danack', 'imagick-demos', Object(GithubService\Model\Commit))
#10 [internal function]: ...\Deployer\Deployer->run()
#11 .../vendor/danack/auryn/lib/ReflectionMethodExecutable.php(27): ReflectionMethod->invokeArgs(Object(...\Deployer\Deployer), Array)
#12 [internal function]: Auryn\ReflectionMethodExecutable->__invoke()
#13 .../vendor/danack/auryn/lib/AurynInjector.php(125): call_user_func_array(Array, Array)
#14 .../vendor/danack/auryn/lib/AurynInjector.php(117): Auryn\AurynInjector->executeInternal(Array, Array, false)
#15 .../vendor/danack/auryn/lib/Provider.php(36): Auryn\AurynInjector->execute(Array, Array, false)
#16 .../bin/cli(106): Auryn\Provider->execute(Array, Array)
#17 {main}

Versions being used are:

amp: c21068fc946f5bfd692297e927773a732f205279
artax: 3e4c077

And running on OSX with the NativeReactor.

Watcher seems to fire to early / can't tell if being processed

So I want to duplicate the cool multi-bar progress monitor that was in one of the earlier versions.

What's happening is that the progress callback is being fired for each of the requests before the request is being started by the Artax\Client.

There doesn't seem to be any entry in the array passed as the data to tell if the request has actually been started to be sent yet...which leads to the screen being filled with lots of bars that say [DETERMINING_LENGTH] - a lot more than concurrent connections.

I guess either the watcher shouldn't be fired until the client actually starts the request on it's way, or there should be an entry in the data that indicates whether the request has been started, or if it's still waiting to start.

Different response using PHP 5.5

Fetching the following URL with Artax results in a different response on 5.5 than on 5.6 or 7.0. I still can't figure out why. file_get_contents works fine on all versions.

https://opacplus.bib-bvb.de/TouchPoint_touchpoint/start.do?Query=205%3D%22BV021286998%22&Language=De&SearchProfile=

Stack overflow when making a request from within a scheduled call.

The issue seems to stem from doAlarmCallback() waiting for the callback to return before updating / unsetting the alarm.

The request() invokes tick() which trickles down to doAlarmCallback() where the callback is executed again (since it was not updated/unset yet) and we recurse forever.

Reproduction script.

$reactor = (new \Alert\ReactorFactory)->select();
$client = new \Artax\Client($reactor);

$reactor->immediately(function() use ($client) {
    $client->request('http://www.google.co.uk');
});

$reactor->run();

Result

PHP Fatal error: Maximum function nesting level of '100' reached, aborting!

DNS Resolution can block

Currently host names are resolved using PHP's native blocking DNS resolution functions. This is the only aspect of the client that does not operate in a non-blocking manner. In best-case scenarios where lookups happen quickly this doesn't negatively affect performance, but slow or failed lookups for DNS A records can hamper event loop execution when performing multiple requests in parallel.

I get no Class 'Amp\Artax\Client' found error

I get no Class 'Amp\Artax\Client' found error:

Fatal error: Class 'Amp\Artax\Client' not found in D:\HTML\xamp\htdocs\Websites\legoScraper\public_html\index.php on line 13

With your example:
https://github.com/amphp/artax/blob/master/examples/008_multi_request.php

Just after installation. That's my first attemt to run your lib.

Dumping require returns:
object(Composer\Autoload\ClassLoader)#1 (7) { ["prefixLengthsPsr4":"Composer\Autoload\ClassLoader":private]=> array(1) { ["A"]=> array(1) { ["Addr"]=> int(5) } } ["prefixDirsPsr4":"Composer\Autoload\ClassLoader":private]=> array(1) { ["Addr"]=> array(1) { [0]=> string(72) "DIR\vendor/daverandom/addr/lib/Addr" } } ["fallbackDirsPsr4":"Composer\Autoload\ClassLoader":private]=> array(0) { } ["prefixesPsr0":"Composer\Autoload\ClassLoader":private]=> array(2) { ["L"]=> array(1) { ["LibDNS"]=> array(1) { [0]=> string(69) "DIR\vendor/daverandom/libdns/src" } } ["A"]=> array(1) { ["Artax"]=> array(1) { [0]=> string(63) "DIR\vendor/amphp/artax/src" } } } ["fallbackDirsPsr0":"Composer\Autoload\ClassLoader":private]=> array(0) { } ["useIncludePath":"Composer\Autoload\ClassLoader":private]=> bool(false) ["classMap":"Composer\Autoload\ClassLoader":private]=> array(0) { } }

Connect Timeout starts at the wrong place

You should probably change the connect timeout to start when the connect attempt is actually initiated and not when the request is first dispatched.

see: http://chat.stackoverflow.com/transcript/message/14816764#14816764

Wrong common name

PHP Warning: stream_socket_enable_crypto(): Peer certificate CN='xy.de' did not match expected CN='kelunik.com' in /git/kelunik/acme-client/vendor/amphp/socket/lib/functions.php on line 286
PHP Fatal error: Uncaught exception 'Amp\Socket\CryptoException' with message 'Crypto negotiation failed: Connection reset by peer' in /git/kelunik/acme-client/vendor/amphp/socket/lib/functions.php:292

I think we should suppress that warning inside Artax if we have the exception.

Support proxy authentication

Currently there is no means to support Proxy-Authorization: headers ... needs to be fixed :)

Feature Request - FormBody() methods addFields() and addFiles() for arrays

Assuming this is the correct place for this.

Can we please have methods to support adding fields to FormBody() as an array please.

Example:

FormBody::addFields(array $associativeFieldMap); 
FormBody::addFiles(array $arr);

Reactor doesn't stop

Not entirely impossible that I've done something silly - but I think the code below should exit, instead the reactor never returns from the ->run().

<?php

require_once(realpath(__DIR__).'/../vendor/autoload.php');

$reactor = Amp\reactor();
$client = new \Amp\Artax\Client($reactor);
$client->setOption(\Amp\Artax\Client::OP_MS_KEEP_ALIVE_TIMEOUT, 3);

$callback = function () {
    echo "I r callback!!\n";
};

$promise = $client->request("http://www.bing.com/asdasdasdsd");
$promise->when($callback);

if (false) {
    //if this block is enabled, there really shouldn't be anything left
    //for the reactor to process. It seems to make no difference.
    echo "Waiting:\n";
    $promise->wait();
}

echo "Running reactor: \n";
//$this->readStreams in NativeReactor.php:114 has an entry and so 
//the reactor never exits.
$reactor->run();

echo "Fin.";

SSL Error If Connection Not Closed

I get SSL routines:SSL3_GET_RECORD:wrong version number if Connection: close header is not set for request. Below is log with request that it fails on.

[2014-02-07 16:21:12] backlog_dev.INFO: requesting: https://api.stackexchange.com/2.1/questions/21455504;21454831;21451041;21430477;21439157;21439177;21428068;21430868;21430108;21339492;21426612;21426266;21425038;21424874;21424907;21423449;21423677;21423695;21423699;21415393;21412560;21406317;21405177;21399681;21399720;21399143;21398708;21398983;21399558;21399552;21399299;21394487;21394482;18799799;21369214;21383186;21276752;21380235;21371907;21370342;21370244;21369702;21367508;21368804;21365785;21368224;21368209;20098708;21368006;21367380;21367772;21367748;21367619;21365785;21363943;21363486;5100893;7426685;21361334;21353848;21352530;21349364;7974562;21346587;21345691;21339828;21337290;21336450;21336049;21332891;21331766;21331624;21329637;21322704;21322109;21319946;21319712;21311213;21307210;21307210;8496621;21284137;18903876;21302976;19696418;20088554;5242319;21302114;21292919;21292324;21292532;21292436?filter=%21%29qPt9%2AYH%28FStcRW8jDsel4v&key=pMxerkFG8E257Xblt5BUHA%28%28&pagesize=100&site=stackoverflow.com [] {"url":"/backlog/api","ip":"127.0.0.1","http_method":"GET","server":"localhost","referrer":"NULL","uid":"fb3c12f","memory_usage":"1.75 MB","memory_peak_usage":"1.75 MB","file":"CVBacklogUI/src/CvRing/Backlog/StackExchange/StackApi.php","line":98,"class":"CvRing\\Backlog\\StackExchange\\StackApi","function":"questionsExist"}
[2014-02-07 16:21:12] backlog_dev.CRITICAL: request failed: exception 'Artax\SocketException' with message 'stream_socket_enable_crypto(): SSL operation failed with code 1. OpenSSL Error messages:
error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number' in CVBacklogUI/vendor/rdlowrey/artax/src/Artax/Socket.php:145
Stack trace:
#0 CVBacklogUI/vendor/rdlowrey/artax/src/Artax/Socket.php(131): Artax\Socket->enableSockEncryption()
#1 CVBacklogUI/vendor/rdlowrey/alert/src/Alert/NativeReactor.php(100): Artax\Socket->Artax\{closure}(19, Resource id #57)
#2 CVBacklogUI/vendor/rdlowrey/alert/src/Alert/NativeReactor.php(73): Alert\NativeReactor->selectActionableStreams(4, 954500)
#3 CVBacklogUI/vendor/rdlowrey/artax/src/Artax/Client.php(53): Alert\NativeReactor->tick()
#4 CVBacklogUI/src/CvRing/Backlog/StackExchange/StackApi.php(99): Artax\Client->request('https://api.sta...')
#5 CVBacklogUI/src/CvRing/Backlog/StackExchange/ChatCrawler.php(73): CvRing\Backlog\StackExchange\StackApi->questionsExist(Array)
#6 CVBacklogUI/src/CvRing/Backlog/StackExchange/ChatCrawler.php(54): CvRing\Backlog\StackExchange\ChatCrawler->crawlTranscript()
#7 CVBacklogUI/src/CvRing/Backlog/BacklogCore.php(55): CvRing\Backlog\StackExchange\ChatCrawler->getQuestionIds()
#8 CVBacklogUI/src/CvRing/Backlog/Presenter/BacklogPresenter.php(54): CvRing\Backlog\BacklogCore->getSourceData('api')
#9 [internal function]: CvRing\Backlog\Presenter\BacklogPresenter->indexAction('api')
#10 CVBacklogUI/vendor/rdlowrey/auryn/lib/Auryn/Executable.php(46): ReflectionMethod->invokeArgs(Object(CvRing\Backlog\Presenter\BacklogPresenter), Array)
#11 [internal function]: Auryn\Executable->__invoke('api')
#12 CVBacklogUI/vendor/rdlowrey/auryn/lib/Auryn/Provider.php(404): call_user_func_array(Array, Array)
#13 CVBacklogUI/vendor/rdlowrey/arya/lib/Arya/Application.php(326): Auryn\Provider->execute('CvRing\Backlog\...', Array)
#14 CVBacklogUI/vendor/rdlowrey/arya/lib/Arya/Application.php(160): Arya\Application->routeRequest()
#15 CVBacklogUI/src/app.php(92): Arya\Application->run()
#16 CVBacklogUI/web/index.php(33): require_once('C...')
#17 {main} [] {"url":"/backlog/api","ip":"127.0.0.1","http_method":"GET","server":"localhost","referrer":"NULL","uid":"fb3c12f","memory_usage":"1.75 MB","memory_peak_usage":"1.75 MB","file":"CVBacklogUI/src/CvRing/Backlog/StackExchange/StackApi.php","line":114,"class":"CvRing\\Backlog\\StackExchange\\StackApi","function":"questionsExist"}

hardcoded `http://` prevents viewing the page

Simply opening the page https://resume.github.io/ will trigger the error.

Blocked loading mixed active content "http://yui.yahooapis.com/2.7.0/build/reset-fonts-grids/reset-fonts-grids.css"
Blocked loading mixed active content "http://code.jquery.com/jquery-1.8.1.min.js"

Invalidate Artax cache

I noticed that even if the body of a request is updated, Artax keep it in cache for a lot of minutes (maybe hours)... how can I invalidate the cache?

Thanks

Request hangs untill SSL terminates on OSX

On OSX, the request below seems to sit at 'AWAITING_RESPONSE' until the SSL is about to be terminated. Looking in Wireshark the data is being transferred.

<?php

require __DIR__.'/../vendor/autoload.php';

use Amp\Artax\Client as ArtaxClient;
use Amp\Artax\Request;

$states = [
    \Amp\Artax\Progress::CONNECTING => 'CONNECTING',
    \Amp\Artax\Progress::SENDING_REQUEST => 'SENDING_REQUEST',
    \Amp\Artax\Progress::AWAITING_RESPONSE => 'AWAITING_RESPONSE',
    \Amp\Artax\Progress::REDIRECTING => 'REDIRECTING',
    \Amp\Artax\Progress::READING_LENGTH => 'READING_LENGTH',
    \Amp\Artax\Progress::READING_UNKNOWN => 'READING_UNKNOWN',
    \Amp\Artax\Progress::COMPLETE => 'COMPLETE',
    \Amp\Artax\Progress::ERROR => 'ERROR',
];

$watchCallback = function($update) {
    global $states;

    if (isset($update[0]) && array_key_exists($update[0], $states)) {
        echo "State: ".$states[$update[0]]."\n"; 
    }
};


$uri = "https://rapidgator.net/article/premium";
$reactor = Amp\reactor();
$client = new ArtaxClient();
$request = new Request();
$request->setUri($uri);
$request->setHeader('User-Agent', "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36");

$client->setOption(\Amp\Artax\Client::OP_MS_CONNECT_TIMEOUT, 5000);
$client->setOption(ArtaxClient::OP_HOST_CONNECTION_LIMIT, 3);
$client->setOption(ArtaxClient::OP_MS_KEEP_ALIVE_TIMEOUT, 5000);

//Enabling this makes the transfer timeout correctly.
//$client->setOption(ArtaxClient::OP_MS_TRANSFER_TIMEOUT, 5000);

/*
Interestingly, enabling this block and adding `var_dump($options);      exit(0); ` to https://github.com/rdlowrey/nbsock/blob/master/lib/Encryptor.php#L112 causes a zend_mm_heap corrupted

$cryptoOptions =[
    'verify_peer' => false,
    'verify_peer_name' => false, 
    'peer_name' => 'rapidgator.net',
    'CN_match' =>  'rapidgator.net',
    'SNI_server_name' => 'rapidgator.net',
    // yeah these values are probably not correct ones to use.
]; */

$options = [\Amp\Artax\Client::OP_CRYPTO, ];

$promise = $client->request($uri, $options);
$promise->watch($watchCallback);
$reactor->run();

echo "Reactor finished.\n";

$result = $promise->wait();

echo "Promised awaited.\n";
var_dump($result);

Missing a ;

On Main Page of artax 2nd example on how to use Artax.

Composer installation

Can this be placed on composer to be installed? Including the AMP library?

Unable to install current master branch via composer

Command output:

$ ./composer.phar require rdlowrey/artax:dev-master
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for rdlowrey/artax dev-master -> satisfiable by rdlowrey/artax[dev-master].
    - rdlowrey/artax dev-master requires bagder/ca-bundle dev-master -> no matching package found.

Out of memory

I built a crawler based on artax using the ParallelCrawler example from v 0.7. Everything works smooth except for one thing: the memory footprint. It always increases by each new url that has been consumed.
Here is an example that demonstrates that memory increases as we add more urls into the mix:

<?php
Amp\run(function() {
    $client = new Amp\Artax\Client;

    $tmpRequests = [];
    $requests = ['http://bing.com','http://google.com','http://microsoft.com','http://yandex.com','http://twitter.com','http://live.com','http://wordpress.com','http://imgur.com','http://pinterest.com','http://ask.com','http://msn.com','http://alexa.com','http://dropbox.com'];
    for($i=0;$i<50;$i++) $tmpRequests[] = $requests[array_rand($requests)];

    // Dispatch two requests at the same time
    $promiseArray = $client->requestMulti($tmpRequests);

    try {
        // Yield control until all requests finish (magic sauce)
        echo "Memory::start::".number_format(memory_get_usage(true));

        $responses = (yield Amp\all($promiseArray));

        echo "Memory::end::".number_format(memory_get_usage(true));

        foreach($responses as $oneResponse)
        {
            var_dump($oneResponse->getStatus(), $oneResponse->getStatus()); 
        }

    } catch (Exception $e) {
        echo $e;
    }
});

Referer header when following a location from an encrypted resource to an un-encrypted resource

Currently Artax only checks the autoReferer option before sending the origin location as part of the Referer header. According to RFC 2616 15.1.3 ...

Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol.

This additional check should be added to the next bugfix release. Also, as per the following spec excerpt an option to allow such behavior if desired will be added:

it is strongly recommended that the user be able to select whether or not the Referer field is sent.

It looks like redirects are not (correctly) followed

I try do a POST to the twitter login form which results in a redirect (302) on successful login, but it looks like the redirect is not being followed (even though it is enabled in the client).

Demo: https://gist.github.com/PeeHaa/b9f91d4545b3db424ab9

The response contains the 302 status code and the Location header:

private 'status' => int 302
private 'reason' => string 'Found' (length=5)
private 'protocol' (Artax\Message) => string '1.1' (length=3)
private 'headers' (Artax\Message) => 
  'location' => 
    array (size=1)
      0 => string 'https://twitter.com/' (length=20)

Change the credentials {twitter_un}, {twitter_pass} at the bottom to your twitter credentials.

Client set max connection

Is OP_HOST_CONNECTION_LIMIT the correct option for setting the max number of connections?

Basically I'm setting $client->setOption(ArtaxClient::OP_HOST_CONNECTION_LIMIT, 4);. What I'm seeing is that the maximum concurrent connections appears to increase during the program execution.

I'm setting a watcher for each request when it's made, with each watcher callback set an ID. In the watcher callback I'm printing the Watcher ID out along with the 'progress bar'.

After the programs been running a while, snipping away the bars, and ordering the watcher IDs from a snapshot of the output I see 262, 264, 266,268, 272,278, 305,551, 555,557, 561,569, 569,571, 600,606, 610,620, 604,608, 620,724, i.e. there seem to be far more than 4 connections active, and also more than the default max connections.

It may be related, it seems that the progress slows down dramatically during the progress of the execution....despite the bandwidth usage not decreasing. Almost as if the bandwidth was being distributed amongst increasing numbers of transfers.

stream_socket_enable_crypto returns FALSE on large number of concurrent connections

Hi, I am having an issue where stream_socket_enable_crypto on the method enableSockEncryption() is constantly returning FALSE when I open a large number of concurrent connections to a same host. It appears to hit a limit at > 3000 concurrent connections. According to the phpdocs, this means that the negotiation failed.

The server is running openSSL v1.0.1e.

Has anyone encountered a similar situation before? I am not sure if this is a server, openssl extension, PHP, Nginx or client limitation.

Would be grateful if anyone can shed some light on the matter.

Warm regards,

Alvin

Connection not timing out

When hitting a certain domain several times consecutively it eventually stalls at ->request() and doesn't seem to timeout. (Will send you the URL in gtalk)

$url = '';

$client = new Amp\Artax\Client;
$client->setOption(Amp\Artax\Client::OP_MS_CONNECT_TIMEOUT, 5000);
$client->setOption(Amp\Artax\Client::OP_MS_KEEP_ALIVE_TIMEOUT, 5000);

function getResponse($client, $uri)
{
    echo $uri . "\n";
    try
    {
        $request = (new \Amp\Artax\Request)
            ->setAllHeaders([
                'User-Agent' => 'Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36',
                'Content-Type' => 'text/plain; charset=utf-8'
            ])
            ->setUri($uri);

        return \Amp\wait($client->request($request));
    }
    catch (\Exception $error)
    {
        echo $error->getMessage();
    }

    return false;
}

while(1) {
    getResponse($client, $url);
}

CA Bundle shenanigans.

I know that you are already planning on changing how the bundling works but:

i) The bundle is being picked up from the wrong directory at the moment...it's being taken from
`/home/github/Bastion/Bastion/vendor/rdlowrey/acesync/lib/../vendor/bagder/ca-bundle/ca-bundle.crt'

That worked for me for a bit due to having an old version of Acesync there previously, but when I completely cleaned the directory out, it's not being installed.

ii) Unless I'm missing something, there doesn't seem to be a way to set the CA file as an option in Artax\Client . It looks like it the options array gets merged in the Encryptor class through the 'cafile' option, but that isn't an allowed option in the Client.

Setting the UA string

Add option to set the UA string per instance of a client instead of per request.

Question on cache

Hi,

Does artax can handle http cache, so it can understand a 304 response?

Unknown socket DomainException

This is from the v1.0.0-beta release of rdlowrey/artax - opening as I can't see any commits that would have changed the behaviour.

Seems somewhat reproducible.


Unexpected exception of type DomainException running Bastion: Unknown socket: Resource id #345
#0 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/artax/lib/HttpSocketPool.php(114): Artax\SocketPool->checkin(Resource id #345)
#1 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/artax/lib/Client.php(502): Artax\HttpSocketPool->checkin(Resource id #345)
#2 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/artax/lib/Client.php(472): Artax\Client->assignParsedResponse(Object(Artax\RequestCycle), Array)
#3 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/artax/lib/Client.php(459): Artax\Client->parseSocketData(Object(Artax\RequestCycle))
#4 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/artax/lib/Client.php(442): Artax\Client->consumeSocketData(Object(Artax\RequestCycle), 'HTTP/1.1 304 No...')
#5 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/artax/lib/Client.php(422): Artax\Client->onReadableSocket(Object(Artax\RequestCycle))
#6 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/alert/lib/NativeReactor.php(99): Artax\Client->Artax\{closure}(2351, Resource id #345, Object(Alert\NativeReactor))
#7 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/alert/lib/NativeReactor.php(70): Alert\NativeReactor->selectActionableStreams(5.4281)
#8 /documents/projects/github/Bastion/Bastion/vendor/rdlowrey/alert/lib/NativeReactor.php(36): Alert\NativeReactor->tick()
#9 /documents/projects/github/Bastion/Bastion/src/bootstrap.php(102): Alert\NativeReactor->run()
#10 [internal function]: getArtifacts(Object(Bastion\Config\Config), Object(Bastion\ArtifactFetcher), Object(Alert\NativeReactor), '/documents/proj...')
#11 /documents/projects/github/Bastion/Bastion/vendor/danack/auryn/lib/Executable.php(23): ReflectionFunction->invokeArgs(Array)
#12 [internal function]: Auryn\Executable->__invoke(Object(Bastion\Config\Config), Object(Bastion\ArtifactFetcher), Object(Alert\NativeReactor), '/documents/proj...')
#13 /documents/projects/github/Bastion/Bastion/vendor/danack/auryn/lib/AurynInjector.php(125): call_user_func_array(Array, Array)
#14 /documents/projects/github/Bastion/Bastion/vendor/danack/auryn/lib/AurynInjector.php(117): Auryn\AurynInjector->executeInternal('getArtifacts', Array, false)
#15 /documents/projects/github/Bastion/Bastion/vendor/danack/auryn/lib/Provider.php(36): Auryn\AurynInjector->execute('getArtifacts', Array, false)
#16 /documents/projects/github/Bastion/Bastion/bin/bastion(83): Auryn\Provider->execute('getArtifacts', Array)

I think my code had a bug and was making requests to the same URLs repeatedly.

Partial downloads and then determining mime type

Is there a way to partially download a resource and then check the mime type of that resource?

Curl apparently do it with range header, but that doesn't work for all servers. I suppose Artax using sockets might have more control.

But also subsequently, how would one determine the mime type?

The reason for this is that I need some way of determining the filesize and filetype random urls and then deciding whether to download them or not. So I can't exactly download the entire resource and then check what it is.

Http caching: cache-control, etags, if-modifed-since, if-none-match

I found your project today and browsed the sources... Didn't found something related to http caching and all those points mentioned in the title.

Whats your opinion on those topics?

Are they already implemented and I was just unable to find them?

Are those http features out of the projects scope?
Would you accept PRs regarding this topics?
Any advice how to implement them with artax?

rdlowrey/alert is private

We're using v0.7.0 and would like to update to v0.7.1. However, amphp/artax depends on rdlowrey/alert and that repo is apparently private (git asks for username/password).

We're updating to v1.0.0 as soon as it's stable but for now it would be great if "rdlowrey/alert" was public or "amphp/artax" depended on something else.

- Tom

Async Simple request

I'm not sure, but I don't think the async client is for this. But how do you do a simple asynchronous GET, POST, PUT or DELETE. That is, simply sending the request, but not caring about the retrieval or waiting for the retrieval?

Cannot update resolved promise

I have this nasty error and pisses me off.. This is the code i use:

$body = (new FormBody)
    // ->addField('name', 'torrents')
    ->addFile('torrents', './torrents/Suits.S04E01.1080p.WEB-DL.DD5.1.H.264-NTb.torrent')
;

$request = (new Request)
    ->setUri('http://localhost:8080/command/upload')
    ->setMethod('POST')
    ->setBody($body)
;
try {
    $response = \Amp\wait((new Client)->request($request));
} catch (Amp\Artax\ClientException $e) {
    echo $e;
}

This is the error im getting:

PHP Fatal error:  Uncaught exception 'LogicException' with message 'Cannot update resolved promise' in /media/ady/Storage_Drive/Dropbox/www/tripper/vendor/amphp/amp/lib/Future.php:94
Stack trace:
#0 /media/ady/Storage_Drive/Dropbox/www/tripper/vendor/amphp/artax/lib/Client.php(481): Amp\Future->update(Array)
#1 /media/ady/Storage_Drive/Dropbox/www/tripper/vendor/amphp/artax/lib/Client.php(473): Amp\Artax\Client->parseSocketData(Object(Amp\Artax\RequestCycle))
#2 /media/ady/Storage_Drive/Dropbox/www/tripper/vendor/amphp/artax/lib/Client.php(456): Amp\Artax\Client->consumeSocketData(Object(Amp\Artax\RequestCycle), 'HTTP/1.1 200 OK...')
#3 /media/ady/Storage_Drive/Dropbox/www/tripper/vendor/amphp/artax/lib/Client.php(436): Amp\Artax\Client->onReadableSocket(Object(Amp\Artax\RequestCycle))
#4 /media/ady/Storage_Drive/Dropbox/www/tripper/vendor/amphp/amp/lib/NativeReactor.php(178): Amp\Artax\Client->Amp\Artax\{closure}(Object(Amp\NativeReactor), 3, Resource id #80)
#5 /media/ady/Storage_Drive/Dropbox/www/tripper/vendor/amphp/amp/li in /media/ady/Storage_Drive/Dropbox/www/tripper/vendor/amphp/amp/lib/Future.php on line 94

amphp / http-client Goto Github PK

http-client's People

Contributors

Stargazers

Watchers

Forkers

http-client's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs