GithubHelp home page GithubHelp logo

swarrot / swarrotbundle Goto Github PK

View Code? Open in Web Editor NEW
90.0 4.0 59.0 367 KB

A symfony bundle for swarrot integration

License: MIT License

PHP 90.04% Makefile 0.28% Twig 9.68%
symfony bundle php symfony-bundle swarrot rabbitmq rabbit consumer

swarrotbundle's Introduction

Swarrot

Build Status Scrutinizer Quality Score Latest Stable Version Latest Unstable Version

Swarrot is a PHP library to consume messages from any broker.

Installation

The recommended way to install Swarrot is through Composer. Require the swarrot/swarrot package:

$ composer require swarrot/swarrot

Usage

Basic usage

First, you need to create a message provider to retrieve messages from your broker. For example, with a PeclPackageMessageProvider (retrieves messages from an AMQP broker with the pecl amqp package:

use Swarrot\Broker\MessageProvider\PeclPackageMessageProvider;

// Create connection
$connection = new \AMQPConnection();
$connection->connect();
$channel = new \AMQPChannel($connection);
// Get the queue to consume
$queue = new \AMQPQueue($channel);
$queue->setName('global');

$messageProvider = new PeclPackageMessageProvider($queue);

Once it's done you need to create a Processor to process messages retrieved from the broker. This processor must implement Swarrot\Processor\ProcessorInterface. For example:

use Swarrot\Processor\ProcessorInterface;
use Swarrot\Broker\Message;

class Processor implements ProcessorInterface
{
    public function process(Message $message, array $options): bool
    {
        echo sprintf("Consume message #%d\n", $message->getId());

        return true; // Continue processing other messages
    }
}

You now have a Swarrot\Broker\MessageProviderInterface to retrieve messages and a Processor to process them. So, ask the Swarrot\Consumer to do its job :

use Swarrot\Consumer;

$consumer = new Consumer($messageProvider, $processor);
$consumer->consume();

Using a stack

Heavily inspired by stackphp/builder you can use Swarrot\Processor\Stack\Builder to stack your processors. Using the built in processors or by creating your own, you can extend the behavior of your base processor. In this example, your processor is decorated by 2 other processors. The ExceptionCatcherProcessor which decorates your own with a try/catch block and the MaxMessagesProcessor which stops your worker when some messages have been consumed.

use Swarrot\Processor\ProcessorInterface;
use Swarrot\Broker\Message;

class Processor implements ProcessorInterface
{
    public function process(Message $message, array $options): bool
    {
        echo sprintf("Consume message #%d\n", $message->getId());
        
        return true; // Continue processing other messages
    }
}

$stack = (new \Swarrot\Processor\Stack\Builder())
    ->push('Swarrot\Processor\MaxMessages\MaxMessagesProcessor', new Logger())
    ->push('Swarrot\Processor\ExceptionCatcher\ExceptionCatcherProcessor')
    ->push('Swarrot\Processor\Ack\AckProcessor', $messageProvider)
;

$processor = $stack->resolve(new Processor());

Here is an illustration to show you what happens when this order is used:

this

Processors

Official processors

Create your own processor

To create your own processor and be able to use it with the StackProcessor, you just need to implement ProcessorInterface and to take another ProcessorInterface as first argument in constructor.

Deprecated processors & message providers / publishers

In order to reduce swarrot/swarrot dependencies & ease the maintenance, some processors & message providers / publishers have been deprecated in 3.x version. They will be deleted in 4.0.

If you use those deprecated classes you could create your own repository to keep them or we could create a dedicated repository under the swarrot organisation if you're willing to help to maintain them.

Message providers / publishers

  • SQS Message provider (in 3.5.0)
  • Stomp message providers (in 3.6.0)
  • Stomp message publishers (in 3.7.0)
  • Interop message publishers & providers (in 3.7.0)

Processors

  • SentryProcessor (in 3.5.0)
  • RPC related processors (in 3.5.0)
  • NewRelicProcessor (in 3.7.0)

Inspiration

License

Swarrot is released under the MIT License. See the bundled LICENSE file for details.

swarrotbundle's People

Contributors

adrienbrault avatar altahrim avatar antoox avatar atailouloute avatar blaugueux avatar eguilly avatar glutamatt avatar greg0ire avatar iamluc avatar j0k3r avatar jdecool avatar jderusse avatar jlepeltier avatar juliendufresne avatar k-phoen avatar lepiaf avatar lyrixx avatar metfan avatar mroca avatar notfloran avatar odolbeau avatar olaurendeau avatar pierrelemee avatar pvgnd avatar samnela avatar serhiiosm avatar sroze avatar stof avatar thisisareku avatar tucksaun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

swarrotbundle's Issues

Install Fail

I tried this morning to install swarrotBundle (composer require swarrot/swarrot-bundle), and :
(i use PHP7.3)

Using version ^1.7 for swarrot/swarrot-bundle
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Restricting packages listed in "symfony/symfony" to "5.0.*"
Nothing to install or update
Generating autoload files
Executing script cache:clear [KO]
 [KO]
Script cache:clear returned with error code 255
!!  
!!  Fatal error: Declaration of Swarrot\SwarrotBundle\DataCollector\SwarrotDataCollector::collect(Symfony\Component\HttpFoundation\Request $request, Symfony\Component\HttpFoundation\Response $response, ?Exception $exception = NULL) must be compatible with Symfony\Component\HttpKernel\DataCollector\DataCollectorInterface::collect(Symfony\Component\HttpFoundation\Request $request, Symfony\Component\HttpFoundation\Response $response, ?Throwable $exception = NULL) in /srv/vendor/swarrot/swarrot-bundle/DataCollector/SwarrotDataCollector.php on line 65
!!  Symfony\Component\ErrorHandler\Error\FatalError {#5745
!!    -error: array:4 [
!!      "type" => 64
!!      "message" => "Declaration of Swarrot\SwarrotBundle\DataCollector\SwarrotDataCollector::collect(Symfony\Component\HttpFoundation\Request $request, Symfony\Component\HttpFoundation\Response $response, ?Exception $exception = NULL) must be compatible with Symfony\Component\HttpKernel\DataCollector\DataCollectorInterface::collect(Symfony\Component\HttpFoundation\Request $request, Symfony\Component\HttpFoundation\Response $response, ?Throwable $exception = NULL)"
!!      "file" => "/srv/vendor/swarrot/swarrot-bundle/DataCollector/SwarrotDataCollector.php"
!!      "line" => 65
!!    ]
!!    #message: "Compile Error: Declaration of Swarrot\SwarrotBundle\DataCollector\SwarrotDataCollector::collect(Symfony\Component\HttpFoundation\Request $request, Symfony\Component\HttpFoundation\Response $response, ?Exception $exception = NULL) must be compatible with Symfony\Component\HttpKernel\DataCollector\DataCollectorInterface::collect(Symfony\Component\HttpFoundation\Request $request, Symfony\Component\HttpFoundation\Response $response, ?Throwable $exception = NULL)"
!!    #code: 0
!!    #file: "./vendor/swarrot/swarrot-bundle/DataCollector/SwarrotDataCollector.php"
!!    #line: 65
!!  }
!!  
Script @auto-scripts was called via post-update-cmd

Installation failed, reverting ./composer.json to its original content.

In Symfony\Component\HttpKernel\DataCollector\DataCollectorInterface:

public function collect(Request $request, Response $response, \Throwable $exception = null);

In Swarrot\SwarrotBundle\DataCollector\SwarrotDataCollector:

public function collect(Request $request, Response $response, \Exception $exception = null)

Yes, class Exception implements Throwable, but usage of interface inheritance in the context of method signature is not allowed (no?).

It's appear when i install sf recipe.

It's seems to work without..

Unable to manage heartbeats with PECL provider

Hi,

Here is our stack :

  • Symfony 4.1.6
  • swarrot/swarrot-bundle : v1.5.1
  • RabbitMQ Server : 3.7
  • HaProxy 1.7
  • Using the default "pecl" provider/factory

Our problem

  1. Our HaProxy has a timeout that close the connection between our application and RabbitMQ.
  2. According to RabbitMQ specs, the use of heartbeats is highly recommended (https://www.rabbitmq.com/heartbeats.html)
  3. Do you know any implementation to manage heartbeats using the PECL provider with the SwarrotBundle ?

Thanks

Problem with symfony/dotenv

My .env

SWARROT_RABBIT_MQ_HOST=rabbitmq
SWARROT_RABBIT_MQ_PORT=15672
SWARROT_RABBIT_MQ_LOGIN=admin
SWARROT_RABBIT_MQ_PASSWORD=admin

And has configuration:

swarrot:
    connections:
        rabbitmq:
            host: %env(SWARROT_RABBIT_MQ_HOST)%
            port: %env(SWARROT_RABBIT_MQ_PORT)%
            login: %env(SWARROT_RABBIT_MQ_LOGIN)%
            password: %env(SWARROT_RABBIT_MQ_PASSWORD)%

On try clear cache i got:
Invalid type for path "swarrot.connections.rabbitmq.port". Expected int, but got string.

As i know dotenv uses strings only (like bash etc..)

I think ->integerNode('port')->defaultValue(5672)->end() should be changed to scalar with normalization to int.

PS.: Used symfony/skeleton 3.4

Upgrade to 1.8 on Symfony 3.4

Hi

On one of my project I tried to pass to https://packagist.org/packages/swarrot/swarrot-bundle#v1.8.0 (being on v1.6)

But I have a failing test (the test does nothing more that trying to get all services from the container)

1) Tests\ContainerTest::testGetService
TypeError: Argument 2 passed to Swarrot\SwarrotBundle\Processor\ServicesResetter\ServicesResetterProcessorConfigurator::__construct() must implement interface Symfony\Contracts\Service\ResetInterface, instance of Symfony\Component\HttpKernel\DependencyInjection\ServicesResetter given

Symfony\Contracts\Service\ResetInterface seems to be for sf 4.*

FYI I am on Symfony 3.4LTS with https://packagist.org/packages/swarrot/swarrot#v3.7.0

Thank you,

Should BlackholePublisher publish events as well?

I currently want to inspect messages that have been sent during my tests, without messages reaching my real broker. It's not possible out of the box as the BlackholePublisher does not collect data and is not plugged to the event dispatcher.

Before working on a patch, I want to be sure that it doesn't break the event dispatching meaning if events are sent from the BlackholePublisher.

Any thoughts on that matter?

Durable Message config

Hi,

We begin to use your bundle and the lib behind, it works well

I have 2 questions about our first implementation:

  • to have a durable message, we need to add some properties in the message, do you confirm this behavior?
            $message = new Message(json_encode([
                'channel' => SlackWebHookApi::CHANNEL_TECH_LOG,
                'text' => 'RabbitMq'.$i
            ]), ['delivery_mode' => 2]); // AMQP_DURABLE
            $messagePublisher->publish('slack_publisher', $message);

If i did not set the ['delivery_mode' => 2], if we publish message and restart the rabbitmq server, message is lost (queue empty)

  • do you confirm that the ack processor is kind of mandatory, because if not set, the message is never removed from the queue? (so this #100 could be a very good idea)

Many thanks :)

Processor stack by consumer

@odolbeau How is it possible to define different processor stacks by consumer ?

For example i want a consumer with the retry and an other one without the retry processor.

Symfony Messenger and Swarrot compatibility

Hello,
I would like to know if Swarrot is "compatible" with Messenger or whether it makes no point of using both ? I use Messenger for internal bus and I dispatch messages on a rabbitmq but there is no retry mechanism and seems like there are many options missing that I can find here. Can we configure Messenger as a custom broker for instance ?

Thank you very much

Port cannot be configured by env var

As the ENV var is resolved at runtime, port is casted into int by the bundle configuration class: https://github.com/swarrot/SwarrotBundle/blob/master/DependencyInjection/Configuration.php#L90

Configuration used:

swarrot:
    connections:
        event:
          host: '%env(APP_MESSAGE_BROKER_HOST)%'
          port: '%env(APP_MESSAGE_BROKER_PORT)%'
          login: '%env(APP_MESSAGE_BROKER_LOGIN)%'
          password: '%env(APP_MESSAGE_BROKER_PASSWORD)%'
bash-5.0$ php bin/console debug:config swarrot

Current configuration for extension with alias "swarrot"
========================================================

swarrot:
    connections:
        event:
            host: '%env(APP_MESSAGE_BROKER_HOST)%'
            port: 0
            login: '%env(APP_MESSAGE_BROKER_LOGIN)%'
            password: '%env(APP_MESSAGE_BROKER_PASSWORD)%'

PHP 7.1?

php-version: 7.0.10
symfony-version: 3.4
swarrot-bundle: 1.5.0 (current version)

| Implementing "Symfony\Component\HttpKernel\DataCollector\DataCollectorInterface" without the "reset()" method is deprecated since Symfony 3.4 and will be unsupported in 4.0 for class "Swarrot\SwarrotBundle\DataCollector\SwarrotDataCollector".
-- | --

Hi,

Above deprecation triggered on symfony version 3.4 and problem already seems to be fixed by the pull request but unfortunately it is not possible to upgrade to latest version of swarrot-bundle from php 7.0.26.

swarrot/swarrot-bundle v1.5.1 requires php >=7.1 -> your PHP version (7.0.10) does not satisfy that requirement.

I am also confused with a breaking change from 1.5.0 to 1.5.1, any clean work around without modifying the file manually?

Thank you

Custom processor stack

Actually the processor stack definition is highly coupled to commands and other stuff, so adding our own processor into the mix imply to rewrite the whole command stuff.

My proposal would introduce a new interface : ProcessorFactoryInterface and a new way of configuring the processor stack :

processors_stack:
    retry: retry.processor.factory
    my_processor: my_processor_factory_service
    another_processor: another_processor_service

Each value in the processors_stack list would be a service id representing a ProcessorFactoryInterface service or a ProcessorInterface service

If it's a ProcessorInterface then the processor is just add to the stack, if it's a factory it will add input option / argument to the command and generate the service depending on the command options value.

So the ProcessorFactoryInterface would look like this :

interface ProcessorFactoryInterface
{
    /**
     * Create service given a specific input
     */
    public function create(InputInterface $input);

    /**
     * Add input argument / option to the command
     */
    public function configure(CommandInterface $command);
}

WDYT ?

Symfony 4.3 deprecations

As of Symfony 4.3, the SwarrotBundle generates deprecation notices :

The "Swarrot\SwarrotBundle\Event\MessagePublishedEvent" class extends "Symfony\Component\EventDispatcher\Event" that is deprecated since Symfony 4.3, use "Symfony\Contracts\EventDispatcher\Event" instead.

Calling the "Symfony\Component\EventDispatcher\EventDispatcherInterface::dispatch()" method with the event name as the first argument is deprecated since Symfony 4.3, pass it as the second argument and provide the event object as the first argument instead.

SVG version of the logo

The profiler integration should be updated to the new design
The way to do it has not yet been documented, but I can work on it (I know how it works as I worked on the system allowing bundles to support both 2.7 and 2.8 properly).
However, the new profiler expects icons to be defined in SVG (2.7 already uses SVG rather than PNG btw, making the profiler retina-ready, but it still allowed to use both).

Do you have an SVG version of the logo ?

Allow users to attach aliases to Swarrot commands

Allow users to attach aliases to Swarrot commands

As a user of the Swarrot consumption command, I'd like to be allowed to use my command name.

In practice, it's probably more intuitive to refer to a command name my:custom:action than swarrot:consume:my_custom_action.

Considering that since SF3.2, command aliases are supported, it would be great to allow users to define their aliases on top of current Swarrot configuration

Set 'retry_exchange' option

Hi,

I want to set 'retry_exchange' extra options for retry processor, but i have an error when i want to execute generated consumer. My context is in SwarrotBundle v1.5.1

ex:

In OptionsResolver.php line 667:

  The option "retry_exchange" does not exist. Defined options are: "connection", "max_execution_time", "poll_interval", "queue", "requeue_on_error", "retry_attempts", "retry_fail_log_levels_map", "retr
  y_key_pattern", "retry_log_levels_map".

This is my configuration:

swarrot:
    consumers:
        order_create_consumer:
            processor: onepoint.order.create.processor
            middleware_stack:
                 - configurator: swarrot.processor.exception_catcher
                 - configurator: swarrot.processor.ack
                 - configurator: swarrot.processor.max_execution_time
                 - configurator: swarrot.processor.retry
            extras:
              max_execution_time: 120
              retry_exchange: 'toto'
              retry_attempts: 2

Do you have any idea ?

Retrieve channel to perform manual action on a queue

I would like to get the number of messages from a queue without re-connecting manually to the broker.

The getChannel method from both AmqpLibFactory & PeclFactory looks great to achieve that. But there are both protected so I can't access them.

I would like to achieve sth like that (from inside a project):

public function isQueueEmpty()
{
    $message = $this
        ->get('swarrot.factory.default')
        ->getChannel('rabbitmq')
        ->basic_get('queue_name');

    if (null === $message) {
        return true;
    }

    return $message->delivery_info['message_count'] < 0;
}

And I know that swarrot.factory.default might not have the getChannel method since it's not in the FactoryInterface ๐Ÿ™‚.

Should I open a PR to put getChannel public and add it to the FactoryInterface?
Or do you have a better option?

Setup exchanges/queues/bindings

Hi,

Do you have any plans to support creating rabbitmq infra (exchanges/queues/bindings...) automatically ?

It would be great if we could describe it in the service configuration, then run a command to setup everything.

Thank you for this bundle !

Upgrade Doctrine dependencies

Hello,

I've recently updated doctrine/common to 3.1.0 and composer didn't rise the fact that SwarrotBundle was using ^2.9.
As I'm using the configuration

swarrot:
    consumers:
        my_consumer:
            middleware_stack: 
                 - configurator: swarrot.processor.doctrine_object_manager

this rise me the error of using the wrong ManagerRegistry (as path changed in doctrine).

Is there a way to fix this ? Am I doing something wrong ?

Thanks

make queue declare before consume queue

Hi.
I have only consumer in my application.
And when I run it, the error "NOT_FOUND - no queue 'test' in vhost '/'" is occured.
Could you make "queue_declare" before consume queue?

Default middleware stack

I think it would be handy to be able to define a default middleware stack that would server as a basis for every consumer. I discovered the swarrot.processor.ack and I can't imagine why one would ever not use it. I can provide a PR for that.

1.6.0 Port mapping with variables

Hello :).

Something looks weird on the version 1.6.0 with the port mapping specifically when we are using environnement variable mapping.

In the yaml :

rabbitmq_port: '%env(int:RABBITMQ_PORT)%'

The value of 'port' index in this case is 0;

It looks like the parameter is processed and converted to string before being replaced by the environnement value.

Something in there is maybe wrong with "beforeNormalization" c95a65e

Best regards.

Why the message type is not used as the queue name by default

I know this is not necessary true, but it could be so useful.

Given this configuration:

swarrot:
    messages_types:
        crawler.update_db:
            exchange: 'redirectionio'
            routing_key: 'crawler.update_db'

Instead of

bin/console bin/console  swarrot:consume:crawler_update_db crawler.update_db

I would like to run

bin/console bin/console  swarrot:consume:crawler_update_db

Support configuring the connection with a URL

On Paas platforms like Heroku, it is common for addons to expose their credentials as an URL (amqp://login:password@host:post/vhost in the case of RabbitMQ). In my project, I currently rely on prependExtensionConfiguration in my own DI extension to parse the URL into the format of the bundle. What do you think about supporting the url-based configuration in the bundle directly ?

Question about stacking processors

Hi

According to https://github.com/swarrot/swarrot#using-a-stack, the order of the stacked processors is important.

Given this config: (v1.5.1)

notification_consumer:
            processor: core.component.rabbitmq.notification_processor
            middleware_stack:
                - configurator: swarrot.processor.signal_handler             
                - configurator: swarrot.processor.exception_catcher      
                - configurator: swarrot.processor.ack                               
                - configurator: swarrot.processor.max_messages           
            extras:
                max_messages: 100

What is the best order? from top (last processor executed) to bottom (first executed after our processor)
I ask this question because checking my logs today, I've seen this:

[2018-04-15 01:32:55] rabbitmq.INFO: [Ack] Message #19 has been correctly ack'ed {"swarrot_processor":"ack"} {"uid":"078067949a"}
[2018-04-15 01:55:43] rabbitmq.INFO: [MaxMessages] Max messages have been reached (20) {"swarrot_processor":"max_messages"} {"uid":"078067949a"}
[2018-04-15 01:55:43] rabbitmq.INFO: [Ack] Message #20 has been correctly ack'ed {"swarrot_processor":"ack"} {"uid":"078067949a"}

and the logic would be:

[2018-04-15 01:32:55] rabbitmq.INFO: [Ack] Message #19 has been correctly ack'ed {"swarrot_processor":"ack"} {"uid":"078067949a"}
[2018-04-15 01:55:43] rabbitmq.INFO: [Ack] Message #20 has been correctly ack'ed {"swarrot_processor":"ack"} {"uid":"078067949a"}
[2018-04-15 01:55:43] rabbitmq.INFO: [MaxMessages] Max messages have been reached (20) {"swarrot_processor":"max_messages"} {"uid":"078067949a"}

Is there a good practice for this kind of stack? (exception, handling signal, max execution and ack of course)

Thank you for your time

Use CompilerPass rather than Extension

i don't know if it is only a personnal POV, but I think that manipulating the container in the Extension configuration is not a really good practice ; I think it should be manipulated within a compiler pass.

I'll try to do a PR as soon as I can, if the idea is acknowledged. :)

Add extension point to provider

By defining services directly in the extension, it's very hard to add a our own provider.
It could be easier with:

  • moving the logic in a compiler pass
  • using a provider_service instead of a provider key

Problem when type hinting method process in PHP7

Hi,
we are using a PHP 7.2 stack with SwarrotBundle and we 're font of a issue that can't passed PHP CS-Fixer rules:

We need to type-hinting the processor return with ?bool when using the middleware configurator swarrot.processor.ack.

So the resposability of ack/no'ack depends on this configurator.

How can we do that without "re-create" the ack processor ?

Thanks in advance

configuration problem with latest commit

Hi,
since latest commit , we have a problem on composer update

here is the stacktrace

Exception trace:
() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/symfony/symfony/src/Symfony/Component/Config/Definition/Builder/ExprBuilder.php:208
Symfony\Component\Config\Definition\Builder\ExprBuilder->end() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/swarrot/swarrot-bundle/DependencyInjection/Configuration.php:37
Swarrot\SwarrotBundle\DependencyInjection\Configuration->getConfigTreeBuilder() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/symfony/symfony/src/Symfony/Component/Config/Definition/Processor.php:50
Symfony\Component\Config\Definition\Processor->processConfiguration() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/swarrot/swarrot-bundle/DependencyInjection/SwarrotExtension.php:27
Swarrot\SwarrotBundle\DependencyInjection\SwarrotExtension->load() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Compiler/MergeExtensionConfigurationPass.php:50
Symfony\Component\DependencyInjection\Compiler\MergeExtensionConfigurationPass->process() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/DependencyInjection/MergeExtensionConfigurationPass.php:39
Symfony\Component\HttpKernel\DependencyInjection\MergeExtensionConfigurationPass->process() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Compiler/Compiler.php:117
Symfony\Component\DependencyInjection\Compiler\Compiler->compile() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/ContainerBuilder.php:613
Symfony\Component\DependencyInjection\ContainerBuilder->compile() at /Users/xav/Workspace/Sites/vhosts/redpill/app/bootstrap.php.cache:636
Symfony\Component\HttpKernel\Kernel->initializeContainer() at /Users/xav/Workspace/Sites/vhosts/redpill/app/bootstrap.php.cache:415
Symfony\Component\HttpKernel\Kernel->boot() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:70
Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /Users/xav/Workspace/Sites/vhosts/redpill/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:124
Symfony\Component\Console\Application->run() at /Users/xav/Workspace/Sites/vhosts/redpill/app/console:22

Thanks

Reset application after each consummed message

As of Symfony 4.4, we can reset application ($this->getApplication()->reset();) in a Symfony Command.

This is usefull to reset Monolog (when finger crossed is used for example), and free resources while waiting for messages.

Consumer on multiple queues

Hi,
I'm trying to make one of my consumer consume multiple queues but I can't find the command nor the configuration to do so.
Is it possible ?

Thanks

Command options are ignored when extra config is set

I have defined a comsumer with the following extras:

            extras:
                max_messages: 100
                max_execution_time: 300

And I want to consume only one message so I execute the following command :

sf swarrot:consume:voip_valorize_call citadel_voip_valorize_calls --max-messages 1

But the option max-messagesis ignored :

[MaxMessages] Max messages have been reached (100)

I think the problem came from this line : https://github.com/swarrot/SwarrotBundle/blob/master/Command/SwarrotCommand.php#L120

The + ignore already defined config, we can use a array_mergeto fix that.

dot in messages-types are lost in the command help

For exemple:

swarrot:
    messages_types:
        crawler.update_db:
            exchange: 'redirectionio'
            routing_key: 'crawler.update_db'

but the --help give that:

Description:
  Consume message of type "crawler_update_db" from a given queue

The dot has been replaced by and underscore

The options "retry_attempts", "retry_key_pattern" do not exist. Defined options are: "connection", "max_execution_time", "max_messages", "poll_interval", "queue", "requeue_on_error", "signal_handler_signals".

This shows up when trying to run consumer with -R or --no-retry option:

[Symfony\Component\OptionsResolver\Exception\UndefinedOptionsException]
The options "retry_attempts", "retry_key_pattern" do not exist. Defined options are: "connection", "max_execution_time", "max_messages", "poll_interval", "queue", "requeue_on_error", "signal_handler_signals".

Exception trace:
() at /var/www/symfony/vendor/symfony/symfony/src/Symfony/Component/OptionsResolver/OptionsResolver.php:685
Symfony\Component\OptionsResolver\OptionsResolver->resolve() at /var/www/symfony/vendor/swarrot/swarrot/src/Swarrot/Consumer.php:72
Swarrot\Consumer->consume() at /var/www/symfony/vendor/swarrot/swarrot-bundle/Command/SwarrotCommand.php:97

Make Swarrot commands lazy

Hello.

I have issues about commands created by SwarrotBundle that are booted even if I don't call them.
Dependencies of these commands are then loaded like the processors and their dependencies.

I don't know if this is done in purpose but I think it is better if we make them lazy.

However This problem can be solved by adding to all commands the tag
{ name: console.command, command: swarrot.command.generated.foo_consumer }
in Swarrot\SwarrotBundle\DependencyInjection\SwarrotExtension and removing commands registrations in Swarrot\SwarrotBundle\SwarrotBundle because Sf > 3 registrate them well with the tags.

I haven't dig further so can you tell me if it can work.

Thank you!

The ObjectManagerProcessor is added at the wrong place in the stack

Currently, the object manager is not reset or cleared when an exception happens in the processing, because the logic is not reached. This makes it partially useless given that the entityManager generally happens because of an exception (it is always the case, unless you close it yourself for a weird reason. Doctrine only closes itself on exceptions).
The ObjectManagerProcessor should be added in the stack outside the ExceptionCatcherProcessor, not inside it.

Make the swarrot logger configurable

Swarrot logs a lot. It can be useful to be able to use a custom logger (with custom logging rules) to be able to configure the verbosity.

TODO:

  • Deprecate the processor_logger configuration key
  • Create a new logger configuration to replace the previous one
  • Use it everywhere (even in the command which use the default logger for now)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.