GithubHelp home page GithubHelp logo

yii2-beanstalk's Introduction

yii2-beanstalk

Yii2 beanstalkd web and console component which is an interface on the top of pda/pheanstalk. Thanks Paul Annesley for such a complete work.

How to use?

Installation with Composer

Just add the line under require object in your composer.json file.

{
  "require": {
  ...
    "udokmeci/yii2-beanstalk" : "^1.0.0"
  }
}

Then run

$> composer update

Configuration

Now add following lines in to your main and console configuration file under components

'beanstalk'=>[
    'class' => 'udokmeci\yii2beanstalk\Beanstalk',
    'host' => '127.0.0.1', // default host
    'port' => 11300, //default port
    'connectTimeout' => 1,
    'sleep' => false, // or int for usleep after every job 
],

Now add following in to your console configuration only.

...
'params' => $params,
// add you controller with name and class name next to params.
'controllerMap' => [
    'worker' => [
        'class' => 'app\commands\WorkerController',
    ]
],

Producing

Now if everything is fine, you can run beandstalkd and access to controller as

\Yii::$app->beanstalk
    ->putInTube('tube', $mixedData, $priority, $delay);

$mixedData is added on v1.0 for complex usage. Anything else then string will be send as json format. So you can sent anything within it suppoted by json.

Worker

For worker it also has a built in controller which runs an infinite loop and wait for new jobs. Most of the work is done in BeanstalkController. All you have to do is to create a controller and action like below.

Controller

Create a controller under your commands folder. Give the name anything you want to it and extend your controller from udokmeci\yii2beanstalk\BeanstalkController

Example Controller
<?php
namespace app\commands;

use udokmeci\yii2beanstalk\BeanstalkController;
use yii\helpers\Console;
use Yii;

class WorkerController extends BeanstalkController
{
  // Those are the default values you can override

  const DELAY_PRIORITY = "1000"; //Default priority
  const DELAY_TIME = 5; //Default delay time

  // Used for Decaying. When DELAY_MAX reached job is deleted or delayed with 
  const DELAY_MAX = 3; 

  public function listenTubes(){
    return ["tube"];
  }

  /**
   *
   * @param Pheanstalk\Job $job
   * @return string  self::BURY
   *                 self::RELEASE
   *                 self::DELAY
   *                 self::DELETE
   *                 self::NO_ACTION
   *                 self::DECAY
   *  
   */
  public function actionTube($job){
	    $sentData = $job->getData();
	    try {
    	   // something useful here



           if($everythingIsAllRight == true){
                fwrite(STDOUT, Console::ansiFormat("- Everything is allright"."\n", [Console::FG_GREEN]));
                //Delete the job from beanstalkd
                return self::DELETE; 
           }

           if($everythingWillBeAllRight == true){
                fwrite(STDOUT, Console::ansiFormat("- Everything will be allright"."\n", [Console::FG_GREEN]));
                //Delay the for later try
                //You may prefer decay to avoid endless loop
                return self::DELAY; 
           }

           if($IWantSomethingCustom==true){
                Yii::$app->beanstalk->release($job);
                return self::NO_ACTION;
           }

           fwrite(STDOUT, Console::ansiFormat("- Not everything is allright!!!"."\n", [Console::FG_GREEN]));
           //Decay the job to try DELAY_MAX times.
           return self::DECAY; 

           // if you return anything else job is burried.
	    } catch (\Exception $e) {
            //If there is anything to do.
            fwrite(STDERR, Console::ansiFormat($e."\n", [Console::FG_RED]));
            // you can also bury jobs to examine later
            return self::BURY;
	    }
	}
}
Running Worker

Running console is the easiest part. Run ./yii Your controller

$> php ./yii worker

Controller will tell you, whether there are actions for correspanding tubes or beanstalk server is accessible and which tubes are listening currently. The controller handles with signals. So exit whenever you want, reserved job will not be hanged.

Any forks are welcome.

yii2-beanstalk's People

Contributors

anatolym avatar drsdre avatar elvenpath avatar g1k avatar githubjeka avatar larryli avatar mitrm avatar molotoksoftware avatar rzumstein avatar udokmeci avatar vitaliykoziy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yii2-beanstalk's Issues

Readme File semi-colon missing

There is a error in the following line under controller section. One semi colon is missing. Is there is any way i can edit this?

 if($IWantSomethingCustom==true) {
                Yii::$app->beanstalk->release($job);
                return self::NO_ACTION
}

Releases

Could we get some versioned releases (Semver preferably), Since you are actively working on the code, I'd like to ensure that my development and deployed environments are running the same version.

DEADLINE_SOON or TIMED_OUT isn't handled as a reply on reserve()

In BeanstalkController.php at line 156, you reserve a job, and when you got a response, you pass it to the statsJob function. However, when a DEADLINE_SOON or a TIMED_OUT is received (because the script that reserved the job crashed for example), reserve will return false. This will cause the statsJob function to receive an empty $job, and that causes beanstalk to reply with a NOT_FOUND, causing an Exception.

$job = $bean->reserve();
$jobStats = $bean->statsJob($job);

Name for actions

Please add the conversion action games titles.

public function listenTubes()
    {
        return ['signup_verify'];
    }
public function actionSignupVerify($job)

It is an error:

/s/http — php ./yii worker
Process Control Extension is loaded. Signal Handling Registered!
Not Listening default tube since there is no action defined. actionDefault
Not Listening signup_verify tube since there is no action defined. actionSignup_verify
No tube found to listen!

Can't use tube named index

We had a tube named index (to index records into elasticsearch), but since PHP7, that isn't possible anymore.

It seems that there is a actionIndex() defined in the BeanstalkController, and for defining a tube, you have to create a function called actionTubename($job). In PHP 5.6 or lower this gave no issue, but since PHP7, it notices that the function signature of actionIndex($job) is different in it's parent class (actionIndex($job)).

Maybe this behaviour should be documented to not use "index" as tube-name, since I don't think this is a easy fix without rewriting the beforeAction-function.

Integration test

Hi,

Running the unit test, the environment is obviusly test, when a job is put and after that is execute, ¿Exists any way to define YII_ENV to test in the console command?

*I thought a craft way, sending YII_ENV and in the top o action redefine the db component... but it does not convince me

Exception: 'Job 5 NOT_FOUND: does not exist or is not reserved by client'

Hi.
I have issue:
exception 'Pheanstalk\Exception\ServerException' with message 'Job 5 NOT_FOUND: does not exist or is not reserved by client' in /var/www/site/vendor/pda/pheanstalk/src/Command/ReleaseCommand.php:63

My code executes about 77 s. I think, that is why I have this issue.
What can I do for fix this?
Help please.
Thks.

Latest version causing insane worker CPU usage

I just updated to your latest version and all of workers (running with supervisor) started going crazy (400%) total server cpu usage.

I'm running php 7.1. I just downgraded back to 1.0.1 and everything seems fine again.

PHP7, declaration of action is not compatible

Hi!

class ImporterController extends BeanstalkController
{
    public function listenTubes()
    {
        return [TUBE_FEED_IMPORT];
    }

    /**
     * @param Job $job
     * @return string
     */
    public function actionIndex(Job $job)
    {
        // ...
    }
}

Running this worker in strict PHP7 generates warning:

PHP Strict Warning 'yii\base\ErrorException' with message 'Declaration of ImporterController::actionIndex() should be compatible with udokmeci\yii2beanstalk\BeanstalkController::actionIndex()'

Database is require while it shouldn't

The beforeAction method is reconnecting to the database (to fix for dropped connections probably), but we are not using a database on our server for this project.

All database-related commands should be optional since this forces us to use a database just to interface with beanstalkd.

How to use logger?

Messages from Yii::error() are not working.

<?php
namespace console\controllers;
use udokmeci\yii2beanstalk\BeanstalkController;
use Yii;
use yii\helpers\Console;

class WorkerClickController extends BeanstalkController
{
    public function listenTubes()
    {
        return ['click'];
    }

    /**
     *
     * @param Pheanstalk\Job $job
     * @return string  self::BURY | self::RELEASE | self::DELAY | self::DELETE | self::NO_ACTION | self::DECAY
     */
    public function actionClick($job)
    {
        Yii::error('test error');  // ignored
        Console::stderr('test error' . "\n", Console::FG_RED);
    }
}

Limit number of tubes to be watched

Hi:

Is there any way to listen only in a predefined list of tubes??

Is server has:

  • tube1
  • tube2
  • tube3

somethind like:

public function listenOnlyTubes(){
return ["tube1","tube2"];
}

so, this run not try to do works from tube3 tube.

Multiple workers

What if I want to run several workers, what would be the best implementation?

Possible race condition

Exception 'Pheanstalk\Exception\ServerException' with message 'Job 318940 NOT_FOUND: does not exist or is not reserved by client' in /var/www/social.zone/ybs/vendor/pda/pheanstalk/src/Command/ReleaseCommand.php:64
Stack trace:
#0 /var/www/social.zone/ybs/vendor/pda/pheanstalk/src/Connection.php(145): Pheanstalk\Command\ReleaseCommand->parseResponse('NOT_FOUND', NULL)
#1 /var/www/social.zone/ybs/vendor/pda/pheanstalk/src/Pheanstalk.php(395): Pheanstalk\Connection->dispatchCommand(Object(Pheanstalk\Command\ReleaseCommand))
#2 /var/www/social.zone/ybs/vendor/pda/pheanstalk/src/Pheanstalk.php(269): Pheanstalk\Pheanstalk->_dispatch(Object(Pheanstalk\Command\ReleaseCommand))
#3 [internal function]: Pheanstalk\Pheanstalk->release(Object(Pheanstalk\Job), '1000', 300)
#4 /var/www/social.zone/ybs/vendor/udokmeci/yii2-beanstalk/src/udokmeci/yii2beanstalk/Beanstalk.php(94): call_user_func_array(Array, Array)
#5 /var/www/social.zone/ybs/vendor/udokmeci/yii2-beanstalk/src/udokmeci/yii2beanstalk/BeanstalkController.php(376): udokmeci\yii2beanstalk\Beanstalk->__call('release', Array)
#6 /var/www/social.zone/ybs/vendor/udokmeci/yii2-beanstalk/src/udokmeci/yii2beanstalk/BeanstalkController.php(376): udokmeci\yii2beanstalk\Beanstalk->release(Object(Pheanstalk\Job), '1000', 300)
#7 /var/www/social.zone/ybs/vendor/udokmeci/yii2-beanstalk/src/udokmeci/yii2beanstalk/BeanstalkController.php(318): udokmeci\yii2beanstalk\BeanstalkController->executeJob('actionPre', Object(Pheanstalk\Job))
#8 /var/www/social.zone/ybs/vendor/yiisoft/yii2/base/Controller.php(152): udokmeci\yii2beanstalk\BeanstalkController->beforeAction(Object(yii\base\InlineAction))
#9 /var/www/social.zone/ybs/vendor/yiisoft/yii2/console/Controller.php(119): yii\base\Controller->runAction('', Array)
#10 /var/www/social.zone/ybs/vendor/yiisoft/yii2/base/Module.php(454): yii\console\Controller->runAction('', Array)
#11 /var/www/social.zone/ybs/vendor/yiisoft/yii2/console/Application.php(167): yii\base\Module->runAction('worker', Array)
#12 /var/www/social.zone/ybs/vendor/yiisoft/yii2/console/Application.php(143): yii\console\Application->runAction('worker', Array)
#13 /var/www/social.zone/ybs/vendor/yiisoft/yii2/base/Application.php(375): yii\console\Application->handleRequest(Object(yii\console\Request))
#14 /var/www/social.zone/ybs/yii(30): yii\base\Application->run()
#15 {main}

PHP7 signal handler

Hi,

Stopping workers with Ctrl+C stopped working with PHP7. It looks like the behavior of declare (ticks = 1); changed a bit. I'm trying to find out what needs to be changed and will PR as soon as I find something, but maybe you already know how to solve the issue.

SQLSTATE[HY000] [1040] Too many connections

Hi,
I didn't understand the purpose of setDBSessionTimeout() but that function never close MySQL connection so in case of many jobs, they start to fail because MySQL answer:

SQLSTATE[HY000] [1040] Too many connections

I mean, this code here, is really doing something useful?

if ($this->_lasttimereconnect == null) {
$this->_lasttimereconnect = time();
$this->setDBSessionTimeout();
}
if (time() - $this->_lasttimereconnect > 60 * 60) {
$this->getDb()->close();
$this->getDb()->open();
Yii::info(Yii::t('udokmeci.beanstalkd', "Reconnecting to the DB"));
$this->setDBSessionTimeout();
$this->_lasttimereconnect = time();
}

For what I can see, that code only handle a timer, but that timer will be never used outside of that piece of code... but maybe I'm missing something.

If you think that code was not needed, I'll be happy to submit a PR.

please use version tags

There are many commits since 1.0.0. Risky to use it in production without version tags. Thank you!

`db` component usage hardcode

Hi there.
In file BeanstalkController.php from line 305 to 316 there is a lot of logic with a lot of assumptions.

  1. First of all, why do you assume that any yii2 application has 'db' component? In each getDb() you are trying to get it without any checks. It results in Exception 'yii\base\InvalidConfigException' with message 'Unknown component ID: db'. In my app I don't have db component, but have mongodb.
  2. Second, why do you assume that every yii2 that uses yii2-beanstalk needs to reconnect to db periodically? What if my app doesn't even use db in beanstalk workers? Moreover, I think it's developer's responsibility to make sure that db connection is up and available. If you want to put this logic in your extension, at least give it option to disable. For example, in Laravel this check happens by catching exceptions after any db query and checking if it's caused by has gone away error. If it is, it's just reconnect and executes query again.
  3. Why do you assume that reconnection every hardcoded 60 * 60 seconds will help? This value should correlate with wait_timeout of the db.
  4. Making some hardcoded text query in BeanstalkController.php:140 is a very big assumtion. I don't think you should every do that. You don't know what database the app uses and don't know what it supports. And there is one more 31536000 value without any ability to change or disable it.

How to avoid default tube listening?

Worker controller:

    public function listenTubes()
    {
        return ['request'];
    }

Console:

Process Control Extension is loaded. Signal Handling Registered!
Not Listening default tube since there is no action defined. actionDefault
Listening request tube.
No method found for job's tube!

After that controller exits.
It happens every run if default tube is not empty.

Even if I add this:

    public function getTubes()
    {
        return ['request'];
    }

I got the same error:

Process Control Extension is loaded. Signal Handling Registered!
Listening request tube.
No method found for job's tube!

I18n error

2016-09-26 09:27:49
[-][-][-][error][yii\i18n\PhpMessageSource::loadMessages] The message
file for category 'udokmeci.beanstalkd' does not exist:
/var/www/html/site/yii2-app-advanced/console/messages/ru/udokmeci.beanstalkd.php

Tube Name can be one word only

Tube name has no problem with having word but action is not triggered if it needs conversion. Only one word has convertion.

Needed to fix

Reconnection

After lost connection to beanstalkd this "compoent" does not reinit watching session.
so after "reconnection" your worker is listening for no one tubes in infinite loop

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.