Comments (11)
In principle, yes. Even more so if it can be kept as simple as possible. Could you shortly describe how you envision it behaving, in rough terms?
from beanstalkc.
I was looking at doing a rough port of how the Ruby client works - some operations read from all hosts in the pool & collate the results, other operations work against a specific host.
I also prefer simple - and am thinking of a separate class BeanstalkPool or similar, but there might need to be some changes to the core code.
Thanks
from beanstalkc.
I assume you are talking of beaneater? I'll have a look again at how they do pooling.
from beanstalkc.
Yes, beaneater.
from beanstalkc.
Hi there.
As part of my job I forked and wrote a simple Pool-class. I haven't looked at beaneater, but it sounds like I'm doing exactly what you talk about. I haven't made a pull-request yet because I thought we would come across various issues while coding up against the pool, and I have managed to catch a bug or two this way. I will make a pull-request when I feel it is deserves to be a part of this great client!
from beanstalkc.
I think having pool is essential for such a client. I have noticed that Doerge has made additions with Pool class [1]. Earl, what do you think about merging into your original library?
from beanstalkc.
Doerge, do you mind if I work on your fork with Pooled connections to cut some rough edges? :-)
from beanstalkc.
If someone prepares a pull-request, I'll be happy to review it. Doerge mentioned that he came across various issues he wanted to shake out first; I don't know what the status of that is.
from beanstalkc.
Hiya.
We are using my branch in production and it has been a while since I came across any issues. I don't know if that justifies a pull-request? (I'm new to github'ish..)
@ysmolsky Fork away! I would love to see some polish! For the sake of my coding-bettering what rough edges have you noticed?
edit:
I logged 3 issues with the pool btw:
https://github.com/Doerge/beanstalkc/issues?state=open
from beanstalkc.
Well, the polish I meant is about code style mostly. But more importantly is that reconnection does not work as expected for your fork, b/c of issues you mentioned by that link.
For example the most obvious is the we need to reuse and rewatch all those tubes we were using and watching prior the disconnect. I am going to implement them in fork with BlockingConnection class...
Also I don't really like the idea of having multiple beastalkd server and pulling reserve() from random instances. Thinking about having something more reliable..
from beanstalkc.
Cool! I'm looking forward to learn from your polishing!
@reconnecting
Cool! I'll keep an eye on your fork :)
@reserving from random instances:
Yeah I did consider peeking at the different bstalkds but for our scenario I went with the simple solution at first. Setting a time out on the reserve and then trying the next in line should be a better solution.
from beanstalkc.
Related Issues (20)
- All 4 workers got the same reserved job ???? HOT 1
- peek_ready() gets job from 'default' tube even if it's not being watched. HOT 2
- Change failure mode for missing YAML parser (PyYAML) dependency HOT 2
- Add put-unique HOT 2
- Cannot use "with" syntax with beanstalkc.Connection HOT 2
- stats_tube('non-existent-tube') Raises SocketErr HOT 1
- Please make a new release HOT 1
- IPv6 support
- 8-bit transparency under Python 3 HOT 3
- documentation/protocol issue with peek_ready() HOT 2
- Refactor to support delayed connections HOT 2
- Add Support For Connection Pool
- CommandFailed: ('delete', 'NOT_FOUND', []) HOT 4
- How to actively delete a delayed job before it is been consumed?
- when I run 'sudo pip install beanstalkc',sth wrong. HOT 2
- recv() block process
- Introducing a constant for the "default" tube name HOT 2
- yaml.load() without Loader=... is deprecated HOT 1
- Does not work with Python 3.9 HOT 2
- [Errno 104] Connection reset by peer
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from beanstalkc.