Queue Service about phalcon

[ruudboon]

+1 for an common interface with at least a redis adapter. The queue interop looks like a good example. They are trying to pitch it to fig, if that get’s approved it would be even better.

[erisrayanesh]

However the Beanstalkd project is abandoned but if its latest stable version works good without any critical or security issues for most of currently using developers, it would be unnecessary to remove its adapter until Phalcon 5.x . So it is a good idea to encourage developers for using other developing and progressing queue service projects with new featured adapters in Phalcon 4.x .

[stamster]

@Zaszczyk
We're talking about interoperability here - having a layer of abstraction which will allow devs to switch the concrete queue service on one place (IoC service), similar like we have with DB wrapper/adapters.

"new and poor Phalcon adapter"?
So why we have a framework there at first place? Let's do everything with composer and bundle up a custom framework per each project of ours then.

[stamster]

The good abstraction layer is a necessity here - so the underlying concrete adapter implementation does not matter that much like it was the case with Beanstalk.

[niden]

The way I was (personally) looking at it, it would be similar to what happens in PSR. You create an Queue class that consumes the interop-queue interface and offers functionality to Phalcon applications.

Then you can inject an adapter that implements that interface, whatever the adapter is for (redis, nats etc.).

[scrnjakovic]

@jellisii did you just miss @niden mentioning beanstalkd is being dropped in v4? Which, btw, was announced way back. Beanstalkd was ditched for a reason - it's uncertain future and stability as every now and then, maintainer disappears like a fart in the wind.

I'm all in for creating an interface with at least redis implementation, unless someone else has already worked out the abstraction, in which case we can just port it to zephir.

[CameronHall]

I agree. Considering it is an abandoned project, it won't be getting any updates. This means there's minimal if any maintenance debt within the adapter. I believe that we should leave it in 4.0.x at least on the basis that there will be minimal if any maintenance overhead and the argument to remove it is to "fix something that isn't broken".

Alternatively, we could move it to the incubator and not supply any adapters in the framework.

[niden]

@scrnjakovic Yes, this should get in in 4.1, provided we have bandwidth (as always).

We already have discussed with the queue-interop team and have worked with them to create an extension for the queue service (similar to PSR).

Using that interface we will work on creating several adapters that will come with the framework and also offer the ability to create new ones for any queue.

https://github.com/queue-interop/queue-interop <- the spec
https://github.com/queue-interop/ext <- the extension

[scrnjakovic]

Sooo, are we actually going somewhere with this?

I can say that I loved how simple and easy to use Beanstalkd adapter was. For an example, all of the redis PHP implementations I've seen are overly complicated and opinionated e.g. introduce worker scripts that spawn other workers, don't get along with supervisord, require jobs as classes and then call run() or similar methods to do the job and what not, forcing their own logic, naming conventions and architecture onto a developer.

[CameronHall]

Queue Interop looks good. I'd be happy with creating an adapter so we can supply a queue-interop compatible client.

[JABirchall]

I think a QueueAdapaterInterface would be good but I also think phalcon should implement an out the box queue adapter for some of the popular queue services.

ref: #153
Some good comments on that NFR for possible implimentation.

[benwills]

I second the idea for default integration with Redis.

[jellisii]

A implementation of the BeanstalkD adapter would be nice. I have a few projects that rely on BeanstalkD to get work done, so this isn't exactly a unbiased request.

I still advocate for an adapter-style interface that multiple implementations can be written to. That does appear what queue interop seems to be driving for. I do understand that because there's not a good base like ANSI SQL to start from that there's difficulty around it.

[Jurigag]

Definitely there should be option to implement our own adapter and just switch to any messaging system we want like beanstalkd, redis, rabbitmq, apache kafka etc. Interop looks promising.

[david-duncan]

What would this implement that packages like interop don't already implement?

Are we proposing a wrapper for a wrapper?

[Jurigag]

Yea that's a good question if we need actually any adapter and built-in queues for phalcon.

[Zaszczyk]

I think Phalcon team should not focus on queue adapter. There are a lot of queues, which have own mature and stable libraries/plugins/adapters. Developers will prefer to use them, than new and poor Phalcon adapter.

[aat2703]

I actually like the way Laravel handles queues with a class per queue action...

SendEmail::dispatch(); // Job is now dispatched to queue for example AWS SQS

https://laravel.com/docs/5.7/queues

[jellisii]

@scrnjakovic I did not miss that, but if there's a queue adapter, as has been suggested using queue-interop, having the ability to import BeanstalkD functionality from something like incubator would be nice.

I use BeanstalkD in several projects because Phalcon made it easy to do so. Deprecation of the existing feature for 4.x and dropping for 5.x would make the transition easier for existing users of that particular Phalcon feature, but it sounds like that option is no longer available.

[stamster]

@jellisii BeanstalkD is an abandoned project, w/o any bright future. Sadly, even the new maintainer gave up on the project soon after getting access to it. So that shall be removed from Phalcon for sure.

Once upon a time, I was also a happy user of BeanstalkD via Phalcon.

[scrnjakovic]

The latest version was released in 2014 and numerous bugs were reported since then, none of which were fixed. If you want to use it, no one is stoping you, you still can. It's just not going to be the part of the core.

[erisrayanesh]

@scrnjakovic I'm not a fan of the Beanstalkd project and current Phalcon queue service which is now supporting only the Beanstalkd. I'm just saying that like the idea of @niden and @jellisii about creating an adapter interface as what Phalcon does for caching, database or logger, it still could implement an adapter for the Beanstalkd. Because as I reviewed the beanstalkd's commits, probably it's going to be active again since some minor changes committed couple of month ago and discussions made about making release here.

[Rajeshr34]

BeanstalkD ++

[slothentic]

Beanstalkd had a release recently, and is still maintained. Please reconsider pulling it back into Phalcon. I'm using it in several projects and it works great. Other MQs have proven to be unnecessarily complex, beanstalkd strikes the right balance.

[JABirchallHR]

It looks like beanstalkd ownership has been transfered over to @ysmolsky

[Rid]

Is there any way to run beanstalk as-is on Phalcon 4.x? I've just finished migrating 3 projects over to v4 however if the queue adapter is no longer available or no suitable alternatives are provided, I'll need to revert.

Does anyone have any advice on a good queue system which can be used with minimal changes?

[slothentic]

This looks like the most commonly used library for PHP/Beanstalkd: https://github.com/pheanstalk/pheanstalk

Beanstalkd repo getting more updates, last one is 4 days ago. I think its safe to say the project is back to life, anyone on team Phalcon want to revisit whether to include Beanstalk queue into the core project?

[Rid]

@slothstronaut thanks, I think I will make a service using Pheanstalk to replace the queue adapter which should be able to provide the same methods so that integration is seamless with the old version.

https://github.com/beanstalkd/beanstalkd had it's first release in a year and seems to be in good hands with @ysmolsky, so I'm happy to continue using it.

I think it's simplicity makes it a great choice for integration with Phalcon, however more choice is always a good thing, so if it's possible to use beantalkd through a general adapter such as https://github.com/queue-interop/queue-interop then I'm all for it, but I think it deserves more priority than is currently being given.

[scrnjakovic]

@Rid you can still use beanstalkd the old way in v4: take a look at the source from v3 and port it to PHP from Zephir. There may even be PHP version in the incubator. Either way, I think phalcon’s beanstalkd client relies on pheanstalk, so you can utilize that directly. You have plenty of choices.

[Rid]

@crnjakovic Thanks for the tip, I ended up porting the source of v3 from Zephir to PHP, there is no version in the incubator. The code is pretty compact and isn't relying on pheanstalk so it was a pretty simple job.