Unrecoverable error in celery worker about billiard

[ask]

Have you been able to reproduce this? I'm not quite sure where this is failing

[public]

This looks like what happens when you raise an exception that doesnt pickle correctly.

http://packages.python.org/celery/userguide/tasks.html#creating-pickleable-exceptions

[holm]

Yes, that could be the case. I will look into that.

[holm]

This seems to have been the case, sorry about that. Quite surprising information about the exceptions.

[lsemel]

Experiencing the same error since upgrading to 3.0 It seems that anything that raises an unpickleable exception can take the Celery workers.

I see the issue was closed -- was this fixed?

[holm]

We are being bitten by this. We have fixed all our own exceptions, but it is quite hard to manage all possible exceptions from libraries. In this case it is the "requests" package that has exceptions that cannot be pickled, however I think celery should handle this better so it doesn't crash the worker.

[lsemel]

Same here -- there's no way we can go through every library we use to check that all its exceptions are pickleable, so it'd be great if Celery itself could gracefully handle them.

[public]

It used to jam stuff into an UnpickleableExceptionWrapper iirc so this probably is a bug :)

[lsemel]

Good to know!

[ask]

Fixed by the commit above, please upgrade to celery 3.0.6

[holm]

Thanks a lot Ask