Both https://massivescale.net/celery-php/li_celery-php.html and https://massivescale.net/celery-php are blank pages.

While trying to use celery-php with celery 3.1.0, I get an error. My testcase is phpunit CeleryTest unittest.php:

AMQPQueueException: Server channel error: 406, message: PRECONDITION_FAILED - parameters for queue 'php_52cc14ecebb576.01794390' in vhost '/' not equivalent

Additionally in celery worker:

PreconditionFailed: {'exc_message': "Queue.declare: (406) PRECONDITION_FAILED - parameters for queue 'php_52cc18c1edf3a2.30349364' in vhost '/' not equivalent", 'exc_type': 'PreconditionFailed'}

The celery4 branch contains a working Celery Protocol v2 implementation for PhpAmqpLib. The changes are not yet ported to the other backends.

It should be very straightforward for PECL AMQP - basically add support for application headers, and don't bind to an exchange when checking results.

For Redis we'll need to reverse engineer the serialization used for headers, which shouldn't be a major task too.

Pull requests welcome if you want this code to be released soon!

Hello. First, thanks very much for this project.

I had some problems connecting to the AMQP Server. I've tried to run the test.php, and I've got the same issue every time:

    PHP Fatal error:  Uncaught exception 'AMQPConnectionException' with message 'Library error: Connection reset by peer - Potential login failure.' in /home/vbox/dev_ressources/celery-php/celery.php:101

I've started the Celery Service by using the files in testscenario and this is the log of celery:
-------------- celery@ocb03 v3.0.19 (Chiastic Slide)
---- **** -----
--- * *** * -- Linux-3.2.0-29-generic-x86_64-with-Ubuntu-12.04-precise
-- * - **** ---

• ** ---------- [config]
• ** ---------- .> broker: amqp://guest@localhost:5672//
• ** ---------- .> app: tasks:0x332ebd0
• ** ---------- .> concurrency: 1 (processes)
• *** --- * --- .> events: OFF (enable -E to monitor this worker)
  -- ******* ----
  --- ***** ----- [queues]
  -------------- .> celery: exchange:celery(direct) binding:celery

I've also configured in test.php like this:
$c = new Celery('127.0.0.1', 'guest', '', '');

Any help will be fully appreciated !
Zhenyi

how does this library address the autoscaling of workers with celery?

It seems that when I use custom states the function isReady returns true on the first state change. Because of caching it is never retried.

Adding a function to clear the cache does work around it and allows me to eventually get the right result. I tried coming up with a patch, but did find some odd issues with timing so first just a bug report.

AFAICS, there isn't support for Celery Canvas primitives like signatures, group, chain, chord, etc. Is that correct?

Has Canvas support been considered or possibly attempted? It's a critical need we have and I'd be up to attempt developing the feature if it makes sense in this client.

Thoughts?

Hi gjedeer,

Many thanks for this library. I'm using your library in a php project through composer. But I'm getting an error when it tries to run this code line:
File: amqplibconnectorssl.php
Line: 4
Sentence: require_once('vendor/autoload.php');

Once the lib is installed through composer the structure is:

Obviously, when I deploy my application composer installs or updates my dependencies and it generates the autoload.php, that I use along the whole application. I guess that this require tries to find its own autoload. But, when it is a package for Composer this does not work in this way.
I set the path to my generated vendor/autoload.php file for your require_once('vendor/autoload.php') and all works fine. But it is a workaround and I need a final solution in order to leave the application running in production.

I'm look forward to hear from you.
Many thanks.
Kind regards.

TypeError

list indices must be integers, not str

kombu/transport/redis.py in _do_restore_message at line 468

payload['headers']['redelivered'] = True

I apologize if this is a stupid question - I'm only just realizing that I don't need a result_backend in Celery if I monitor the worker events.

So with celery-php - is a results_backend required? I do not need to persistently store the result of a task, nor do I need to query it more than once. Can I get the results from the task event?

I'd like to replace a webservice call with a Celery worker, but am having a terrible time wrapping my head around how I can communicate the results back without a results_backend.

We have RabbitMQ all set up, and are using it for a small number of workers.

Many thanks for any help,

Geoff

Currently, celery-php lets me specify only one set of credentials, but real-life scenarios of celery usage often involve separate queues for tasks and replies.

I've done some testing and confirmed ETA doesn't work (and presumably other user provided params). ETA was moved to the headers in task protocol 2, so an array_merge of the user-provided params to PostTask() over the defaults will probably resolve it.

Attempting to send a task through RabbitMQ with bad credentials takes 3 seconds to timeout instead of immediately failing.

Hi,

I'm trying to run the test.php file on Windows 7 OS/PHP 5.5.19 and I installed all dependencies using composer.json file. But when I run php test.php command I got:

Fatal error: Class 'AMQPLibConnector' not found in \path\to\celery-php\amqp.php on line 55

And I realized that this error comes up with:

if(class_exists('PhpAmqpLib\Connection\AMQPConnection'))
{
    require_once('amqplibconnector.php');
    require_once('amqplibconnectorssl.php');

}

Where it couldn't find PhpAmqpLib\Connection\AMQPConnection class. I think something with namespacing is wrong. Do you have any idea how should I fix it?

p.s. I tried to fix it by removing the "if" condition but I got the similar error in another line of code.

A test needs written to verify ETA support works. It's not currently working in the celery4 branch.

When using composer to import this project into an existing project a fatal require_once error is thrown because the celery-php/amqplibconnector.php file is trying to require the composer autoload file again.

Thanks for your work!

I'm new to laravel, and have never used this package, so I might be wrong in this assumption.

Laravel packages I have installed in the past have a 'provider' php file. Would one need to be created to use with the laravel framework?

If incorrect credentials are used, we receive:

The website encountered an unexpected error. Please try again later.

PhpAmqpLib\Exception\AMQPProtocolConnectionException: ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile. in PhpAmqpLib\Connection\AbstractConnection->connection_close() (line 661 of vendor/php-amqplib/php-amqplib/PhpAmqpLib/Connection/AbstractConnection.php).

call_user_func(Array, Object) (Line: 199)
PhpAmqpLib\Channel\AbstractChannel->dispatch('10,50', '', NULL) (Line: 341)
PhpAmqpLib\Channel\AbstractChannel->wait(Array) (Line: 212)
PhpAmqpLib\Connection\AbstractConnection->connect() (Line: 180)
PhpAmqpLib\Connection\AbstractConnection->__construct('incorrect', 'guest', '/', , 'AMQPLAIN', NULL, 'en_US', Object, 0) (Line: 50)
PhpAmqpLib\Connection\AMQPStreamConnection->__construct('localhost', 5672, 'incorrect', 'guest', '/') (Line: 78)
AMQPLibConnector->GetConnectionObject(Array) (Line: 188)
CeleryAbstract::InitializeAMQPConnection(Array) (Line: 170)
CeleryAbstract->BuildConnection(Array) (Line: 125)
CeleryAdvanced->__construct(Array, Array) (Line: 195)
[...]

To catch this exception, we need to do something like:

    use CeleryAdvanced;
    use PhpAmqpLib\Exception\AMQPProtocolConnectionException;

    [...]

    try {
            $queue = new CeleryAdvanced($broker, $backend);
            return $queue;
    }
    catch(AMQPProtocolConnectionException $e) {
            [...]
    }

It'd be more convenient to add a new exception (CeleryConnectionException), as we then wouldn't need to concerns ourselves with the underlying library.

Celery itself supports several types of transport where the client only one.

Would you be interested in adding an abstraction layer on top of https://github.com/php-enqueue/psr-queue . It is Java message service like queue abstraction layer. It already supports amqp, redis and amazon sqs (those are supported by celery) and others.

We could work on this if there is a green light from your side on such an addition.

We will need an ability not only schedule tasks from php but also to execute php tasks from celery. It is something will do after this one.

Hi,

After much head scratching, I finally figured out why my 'get' was not returning results.

It turns out that ff the Celery worker is configured with

CELERY_TRACK_STARTED = True
CELERY_RESULT_BACKEND = 'amqp'

the task status changes to 'STARTED', but never moves to ''SUCCESS'. (at least for me!). So using .get() never returns a result

Changing to

CELERY_TRACK_STARTED = False

and switching to PHP AMQP implementation (so that I can expire the results), solved all my headaches.

Thanks for a very useful script!

This could just be a bug to the celery4 branch I'm working on, but when calling PostTask() and overriding the default 'celery' exchange it ends up creating a 'celery' queue. Is there a valid reason for this?

I'm trying to avoid using the default 'celery' queue/exchange.

With either the php or pecl/c driver, when I produce messages very fast, I get stuck in an infinite loop.

I've been at this all day, so my brain is a little fuzzy.

It gets caught in the channel wait() on $ch->close() when you use the php driver. I can't recall the issue with the C driver because I just gave up.

It happens when I use a low delay between calls, but it happens with long delays (although much less commonly.)

Serious issue IMO.

Is there any way to do publisher confirms with celery-php? I need to make sure that my messages are safe.

Seems like there is a way to use php-amqplib, which has publisher confirms:
712444f

but it hasn't been merged in yet. How are people handling message durability and making sure things aren't dropped??

so in the testscenerio folder, your'e defining a task, in the traditional way the celeryproject.org docs define it.

The question is this, how is PHP interacting with this new defined task written in Python?

function PostTask($task, $args) {
...
$params = array(
'content_type' => 'application/json',
'content_encoding' => 'UTF-8',
'immediate' => false,
'delivery_mode' => 2, // < ------------------
);
}

Hi,
when I looked into your code I found that you have not used the 'password' patrameter while initailizing the Predis\Client object in GetConnectionObject function of redisconnector.php file.

Is there any particular reason for ommitting this.
Because when I pass the password I could connect to redis.

Or is there any other way of connecting to password protected redis from celery-php.

Please let me know if I am missing anything.

If my task is running for like 100 seconds without any status change but I want to retrieve every 20 seconds the status the following behaviour applies:

The first status message will contain the status like RUNNING but the second message will be empty/not existing. Only if the status changes again (to something like SUCCESS) I will receive another status message, which is not empty/existing.

How can tell the celery-php client to not delete the result (status message)?

I have tried to get the status information with the default python celery client. I can read the status message as often as I want between status changes without the python client deletes it.

If RabbitMQ throttles the producers for any reason (low disk / memory) an infinite loop is encountered by celery-php or an underlying library. I would expect it to fail after a timeout.

Composer is warning that Package videlalvaro/php-amqplib is abandoned when installing celery-php

$ tools/cli/composer.phar require massivescale/celery-php
Using version ^2.1 for massivescale/celery-php
./composer.json has been updated
  - Installing videlalvaro/php-amqplib (v2.6.3)
    Downloading: 100%

  - Installing massivescale/celery-php (2.1.1)
    Downloading: 100%

Package videlalvaro/php-amqplib is abandoned, you should avoid using it. Use php-amqplib/php-amqplib instead.

Is it feasible to replace this dependency?

composer show massivescale/celery-php shows this:
name : massivescale/celery-php
descrip. : PHP client for Celery task queue
keywords : AMQP, celery, cron, python, queue, task
versions : dev-master, dev-peclonly, dev-php-amqplib, dev-celery31, dev-redis
type : library
license : BSD-2-Clause
source : [git] https://github.com/gjedeer/celery-php.git 0132462
dist : [zip] https://api.github.com/repos/gjedeer/celery-php/zipball/0132462faeb22b7900b742da3f8af34ccffd6235 0132462
names : massivescale/celery-php

autoload
classmap
celery.php

requires
ext-amqp >=1.0.0

1. There are no versions here only dev branches.
2. Would you be open to fixing composer.json and branches as a start to at least try and expose some version information. From the looks of it master is "2.0-dev" and peclonly is "1.0-dev".?
3. composer.json should NOT hard-code version information.
4. You should be using branch aliases for dev branches to map version information. I could create Pull Request(s) to fix this if you are willing to merge them.
5. Would be nice to cut some sort of tag(s)/release(s) to let users use stable version of the library and to be able to track changes between versions somehow without relying on "dev-master" or master branch. (which has potential to break things, also harder to pin things to specific version(s)).

Using Redis as a queue is more and more popular. Now that we have a queue abstraction, adding it should be much more feasible.

c2a0181
http://docs.celeryproject.org/en/master/getting-started/brokers/redis.html#broker-redis

This issue refers to the master branch (3.0.0-dev)

Desired: Run travis tests with celery 3.1.x (e.g. 3.1.25), alongside other supported celery versions, clarify documentation. This can be done via the travis env: directive (E.g. CELERY_INSTALLATION=celery==3.1.25 could be used to cause travis to run pip install celery==3.1.25 redis.

Observed: Celery 4.1.0 is installed in travis tests. From the documentation, I'm not clear if celery 4.1 is officially supported yet, and limitations aren't summarized.

• Also, latest builds are failing according to https://travis-ci.org/gjedeer/celery-php/branches (EDIT: never mind, PR 108 may have a fix)

• Also, I'm not able to get the unit tests to pass with celery 3.1.x (3.1.18 or 3.1.25). I created an equivalent dockerized test runner to run tests locally, for which tests pass in Celery 4.1.0, but many fail in 3.1.x (nothing else changed except dependencies of celery). (https://github.com/TysonAndre/celery-php/blob/master/run_docker_test.sh)

  Is celery 3.1 supported in the latest master commits?

Celery 4.1 installation can be seen in https://travis-ci.org/gjedeer/celery-php/jobs/318734521#L894

$ venv/bin/pip install celery redis
Collecting celery
/home/travis/build/gjedeer/celery-php/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/travis/build/gjedeer/celery-php/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading celery-4.1.0-py2.py3-none-any.whl (400kB)
    100% |████████████████████████████████| 409kB 2.2MB/s 
Collecting redis
  Downloading redis-2.10.6-py2.py3-none-any.whl (64kB)
    100% |████████████████████████████████| 71kB 4.6MB/s 
Collecting billiard<3.6.0,>=3.5.0.2 (from celery)
  Downloading billiard-3.5.0.3.tar.gz (149kB)
    100% |████████████████████████████████| 153kB 4.1MB/s 
Collecting pytz>dev (from celery)
  Downloading pytz-2017.3-py2.py3-none-any.whl (511kB)
    100% |████████████████████████████████| 512kB 2.2MB/s 
Collecting kombu<5.0,>=4.0.2 (from celery)
  Downloading kombu-4.1.0-py2.py3-none-any.whl (181kB)
    100% |████████████████████████████████| 184kB 4.1MB/s 
Collecting amqp<3.0,>=2.1.4 (from kombu<5.0,>=4.0.2->celery)
  Downloading amqp-2.2.2-py2.py3-none-any.whl (48kB)
    100% |████████████████████████████████| 51kB 9.2MB/s 
Collecting vine>=1.1.3 (from amqp<3.0,>=2.1.4->kombu<5.0,>=4.0.2->celery)
  Downloading vine-1.1.4-py2.py3-none-any.whl
Building wheels for collected packages: billiard
  Running setup.py bdist_wheel for billiard ... done
  Stored in directory: /home/travis/.cache/pip/wheels/85/15/e4/11683b23ab74c2a835845811976e664ab33df7d23c3cb23500
Successfully built billiard
Installing collected packages: billiard, pytz, vine, amqp, kombu, celery, redis
Successfully installed amqp-2.2.2 billiard-3.5.0.3 celery-4.1.0 kombu-4.1.0 pytz-2017.3 redis-2.10.6 vine-1.1.4

php-amqp api has totally changed since 0.x versions

I set rabbitmq to be the Celery Broker, and Redis to be the Celery BACKEND.
and i will get error about timeout when get result from celery.

Is possible to get the result from the Celery backend ( redis ) ?

Tedious compilation of a PECL extension is needed. A pure-PHP implementation would be much easier for beginners to use.

https://github.com/videlalvaro/php-amqplib

I recently upgraded celery-php to version 2.1, posting a large number of tasks no longer works very well.

When posting multiple tasks in a loop like:

for ($i = 0; $i < 1000; $i++) {
    $c = new Celery('localhost', 'myuser', 'mypass', 'myvhost');
    $c->PostTask('tasks.add', array(2,2), false);
}

Connections are left open and eventually PECL AMQPConnection deadlocks waiting for a connection to RabbitMQ.

Before 2.1 celery-php would close the connection after PostTask.

I was never able to use the same Celery instance to post multiple tasks.

Starting with commit: 029c312

celery-php leaves 1 connection open with the following test case:

<?php

require 'celery-php/celery.php';

$c = new Celery('localhost', 'myuser', 'mypass', 'myvhost');
$c->PostTask('tasks.add', array(2,2), false);
while (true) {
}

This is because a connection is created inside Celery::__construct() that never gets closed. The connection used to be closed inside PECLAMQPConnector::PostToExchange(), however now a new connection is built for this function.

Further, after upgrading to master (v2.1) 2 connections are opened that are never closed. One for the broker and one for the backend.

I would like to fix this, but need some clarification. What is the intention?

Is the intention that connections are created eagerly on instantiation? If so, what is the purpose of CeleryAbstract::$isConnected?

Is the intention that connections are created lazily in PostTask()? If so, why are multiple connections created in Celery::__construct()?

Is the intention that a single instantiation of the Celery object can call PostTask multiple times? If so, perhaps it shouldn't disconnect inside PECLAMQPConnector::PostToExchange().

I'd be happy to submit a pull request to fix these open connections if the above can be explained.

Hey,

do you mind to add the newest release 2.0.2 to packagist.org to smooth the way the library can be used in other products?

In release 2.0.2 the version of "videlalvaro/php-amqplib" is still ">=2.4.0". I used 2.0.2 with 2.4.0, but after updating all third parties via composor the 2.5.0 version was installed which breaks one of our usecases because there is a problem with the backward compatibility. It is possible to fix the dependecy version for the next release? E.g. 2.5.x for 2.0.3 and so on? So it is possible to rely that the current release works with the dependency libaries.

This class appears to be causing issues in Celery:

`[2012-02-12 10:35:35,030: WARNING/PoolWorker-2] Couldn't send result for u'php_4f3787e1be39f4.65715100': AMQPChannelException(406, u"PRECONDITION_FAILED - inequivalent arg 'x-expires'for queue 'php_4f3787e1be39f4.65715100' in vhost 'celery': received the value '86400000' of type 'signedint' but current is none", (50, 10), 'Channel.queue_declare'). Retry in 0s.``

Which eventually errors out:

AMQPChannelException: {'exc_message': '(406, u"PRECONDITION_FAILED - inequivalent arg \'x-expires\'for queue \'php_4f3787e1be39f4.65715100\' in vhost \'celery\': received the value \'86400000\' of type \'signedint\' but current is none", (50, 10), \'Channel.queue_declare\')', 'exc_type': 'AMQPChannelException'}

No matter which version of the library I use, it always give me this:

=PROGRESS REPORT==== 1-Apr-2016::15:54:18 ===
supervisor: {<0.30364.13>,rabbit_channel_sup}
started: [{pid,<0.30367.13>},
{name,channel},
{mfargs,
{rabbit_channel,start_link,
[1,<0.30356.13>,<0.30365.13>,<0.30356.13>,
<<"127.0.0.1:63584 -> 127.0.0.1:5672">>,
rabbit_framing_amqp_0_9_1,
{user,<<"harvester">>,
[administrator],
[{rabbit_auth_backend_internal,none}]},
<<"harvester">>,
[{<<"authentication_failure_close">>,bool,
true}],
<0.30357.13>,<0.30366.13>]}},
{restart_type,intrinsic},
{shutdown,70000},
{child_type,worker}]

And I can log into the management plugin by the browser, and also, I can do it using the rabbitmq library directly.

I use the basic example:

$c = new Celery('localhost', 'harvester', 'test', 'harvester', 'celery.harvester', 'celery.harvester');
$result = $c->PostTask('tasks.add', array(2,2));

see https://github.com/gjedeer/celery-php/blob/master/celery.php#L142
I suggest a fix here : #69
the tests are OK

Hi!!

I have a problem, with rabbitq as broker, I see in RabbitMQ Management (in Connections tab) my php script leave always one connection open in every server call.

I will try to explain myself, sorry for my awful english.

My requirements are very simple, my tasks dosen´t need confirmations and I don´t need results tasks, in fact, Celery is working without backend track, I mean: CELERY_RESULT_BACKEND = 'disabled://'

So my typical scenario is like:

.
.
.
$celery = new Celery('localhost','guest','guest','/');
$celery->PostTask("myuntrackedtask", $m_param , false);
.
.
(more my php stuff regardless of celery's world...)
.
.

I see in the RabbitMQ Management tab as connections opens, one for broker and one for backend (although I do not need backend), when the php script is finished one connection is closed but another one is still open... and accumulate along the time many open connections.

Please.. can anybody help me?

Thanks very much!!

Zeniox

ErrorException [ Deprecated ]: Function AMQPQueue::declare() is deprecated:
celery-php/amqppeclconnector.php [ 71 ]

Hello,

How can i dump result from worker ?

My worker in python

demo.py

from celery import Celery celery = Celery('tasks', broker='amqp://guest:guest@localhost:5672//') celery.conf.update( CELERY_RESULT_SERIALIZER='json', CELERY_TASK_SERIALIZER='json', CELERY_TASK_RESULT_EXPIRES=None, CELERY_TRACK_STARTED = False, ) @celery.task(queue='demo', name='add') def add(x, y): return x + y

My php script
require 'celery-php/celery.php'; $c = new Celery('localhost', 'guest', 'guest', '/'); $result = $c->PostTask('tasks.add', array(2,2));

but when i use var_dump($result); i can't dump result from woker.
In terminal show
Received task: tasks.add[php_52b1759141a8b3.43107845] Task tasks.add[php_52b1759141a8b3.43107845] succeeded in 0.000701383920386s: 4

Show what i wrong some thing ?

Thanks so much !

Hi,

I am sure I am missing something here.
I just want to post a task which should be executed after some countdown.
Is there any way I can post such a task from celery-php?

I think since the recently, the param vhost for Rabbitmq needs to be "/" if you're using the guest user. I just had this problem where I couldn't connect to rabbitmq, and I was getting this error:

{amqp_error,access_refused,
"access to vhost '' refused for user 'guest'",
'connection.open'}}

because I wasn't passing a vhost that was authorize to the guest user. Maybe I'm wrong, but I just wanted to share this since I lost an hour on this. Maybe it could be added to this readme.

I started integrating celery-php into my application. I missed the step to install php-pecl-amqp (on Fedora). When connecting to Celery, this results in a segfault in PHP. It would be nice if this error could be detected and fail with a message that I can easily understand and act on. Other PHP extensions and libraries tend to do this.

It looks like the original PHPDocumentor docstring format is not compatible with the latest versions. This results in me having to maintain an outdated installation.

Related: #73