Comments (11)
Have you been able to reproduce this? I'm not quite sure where this is failing
from billiard.
This looks like what happens when you raise an exception that doesnt pickle correctly.
http://packages.python.org/celery/userguide/tasks.html#creating-pickleable-exceptions
from billiard.
Yes, that could be the case. I will look into that.
from billiard.
This seems to have been the case, sorry about that. Quite surprising information about the exceptions.
from billiard.
Experiencing the same error since upgrading to 3.0 It seems that anything that raises an unpickleable exception can take the Celery workers.
I see the issue was closed -- was this fixed?
from billiard.
We are being bitten by this. We have fixed all our own exceptions, but it is quite hard to manage all possible exceptions from libraries. In this case it is the "requests" package that has exceptions that cannot be pickled, however I think celery should handle this better so it doesn't crash the worker.
from billiard.
Same here -- there's no way we can go through every library we use to check that all its exceptions are pickleable, so it'd be great if Celery itself could gracefully handle them.
from billiard.
It used to jam stuff into an UnpickleableExceptionWrapper iirc so this probably is a bug :)
from billiard.
Good to know!
from billiard.
Fixed by the commit above, please upgrade to celery 3.0.6
from billiard.
Thanks a lot Ask
from billiard.
Related Issues (20)
- ModuleNotFoundError: No module named 'billiard.py3' HOT 1
- Run integration test of GA CI
- 4.0.0: sphinx warnings (mainly `reference target not found`) HOT 2
- dynamic pool size with gnumake jobclient / jobserver
- ExceptionWithTraceback object has no attribute __cause__ HOT 4
- NameError on Pool.shrink() HOT 5
- Hard Timeout Triggers AttributeError on Windows HOT 1
- billiard==4.0.2 and celery==5.2.7 HOT 1
- Python 3.11: '_Code' object has no attribute 'co_positions' HOT 20
- Worker exited prematurely: signal 9 (SIGKILL) Job: 10.
- TypeError when using `soft_timeout` or `timeout` on `Pool`
- Add support for ExceptionGroups - PEP-0654 HOT 2
- Trouble with calling process
- Task randomly get stuck (between parent and child)
- Replacing `mktemp` usage in multiple files HOT 2
- Long hangs when `os.sysconf('SC_OPEN_MAX')` is large
- Process doesn't run target function after call to celery's app.control().inspect() HOT 1
- Billiard hangs when exception is thrown in a finally block
- Python 3.13: billiard.util.get_logger() is broken HOT 2
- Use `dill.dumps` and `dill.loads` serialization instead of `pickle`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from billiard.