Ignoring errors during complex object encoding about pyjson_tricks

[mverleg]

There's not a way to totally skip it, maybe that'd be a good addition.

But it's already easy to replace the unencodable objects by null or some other value:

from json_tricks.nonp import DEFAULT_ENCODERS
dumps(your_data, obj_encoders=DEFAULT_ENCODERS + [lambda obj: "(unknown)"])

or

dumps(your_data, extra_obj_encoders=[lambda obj: None])

Is that useful?

(So what it does is add an extra encoder after existing ones, which will replace anything not handled yet by a specific object).

(Don't use obj_encoders with this trick because that comes before the standard encoders, losing specials like numpy array encoding).

[tdorsey]

Thanks for the help!

That's an improvement in that it no longer throws, but still not the behavior I'd expect. The whole object is returned as null, since the lambda appears to be handling the entire object. A minimal example:

samplecred.py

import threading
class SampleCredentials:
        def __init__(self, user, password):
                self.user = user
                self.password = password
                self.lock = threading.RLock()

test.py

import json_tricks as  jt
from json_tricks.nonp import DEFAULT_ENCODERS
from samplecred import SampleCredentials

c = SampleCredentials("johndoe", "abc123")

json = jt.dumps(c, extra_obj_encoders=[lambda RLock: None])
print(json)

json2 = jt.dumps(c, obj_encoders=DEFAULT_ENCODERS + [lambda RLock: None])
print(json2)

json/json2 are both None, but I'd expect to see something along the lines of

 { "user: "johndoe", "password": "abc123", "lock": None }

Is this possible?

[mverleg]

I've added a less hacky way that seems to work better:

from json_tricks import dumps, fallback_ignore_unknown
from json_tricks.np import DEFAULT_ENCODERS
from functools import partial
import threading

class SampleCredentials:
	def __init__(self, user, password):
		self.user = user
		self.password = password
		self.lock = threading.RLock()

c = SampleCredentials("johndoe", "abc123")
json2 = dumps(c, fallback_encoders=[
	partial(fallback_ignore_unknown, fallback_value='(unknown type)')])
print(json2)

I'm also deprecating silence_typeerrors - it was a bad idea to ignore errors, it could lead to more parts being ignored than with the current way.

[mverleg]

Version 3.12.0 is now on pip!

(I've also made json_tricks refuse to encode anything from threading, because python2 and pypy accepted that)

[tdorsey]

Excellent. I can work around the other issues I'm having, so you're fine to close if you prefer.

dumps(vars(server))

got me the info I needed, but just to be thorough:

I'm using the python-plexapi

pip install plexapi

from plexapi.myplex import MyPlexAccount
from json_tricks import dumps

account = MyPlexAccount(user,password)
server = account.resource(serverName).connect()

json = dumps(server, fallback_encoders=[
	partial(fallback_ignore_unknown, fallback_value='unknown')])
print(json)
``` python
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/json_tricks/encoders.py", line 175, in class_instance_encode
    obj.__new__(obj.__class__)
TypeError: Required argument 'code' (pos 1) not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 24, in <module>
    json = jt.dumps(p.__dict__, fallback_encoders = fe)
  File "/usr/local/lib/python3.6/site-packages/json_tricks/nonp.py", line 88, in dumps
    primitives=primitives, fallback_encoders=fallback_encoders, **jsonkwargs).encode(obj)
  File "/usr/local/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.6/site-packages/json_tricks/encoders.py", line 83, in default
    obj = call_with_optional_kwargs(encoder, obj, primitives=self.primitives, is_changed=id(obj) != prev_id)
  File "/usr/local/lib/python3.6/site-packages/json_tricks/utils.py", line 51, in call_with_optional_kwargs
    return callable(*args, **use_kwargs)
  File "/usr/local/lib/python3.6/site-packages/json_tricks/encoders.py", line 178, in class_instance_encode
    'cannot be called, perhaps it requires extra parameters').format(obj, obj.__class__))
TypeError: instance "<function PoolManager.__init__.<locals>.<lambda> at 0x7fa648ec2620>" of class "<class 'function'>" cannot be encoded because it's __new__ method cannot be called, perhaps it requires extra parameters

[mverleg]

Thanks for reporting the problem!

The second problem seems like it is an object with a custom __new__, which unfortunately isn't supported (#16).

The first one looks like it might be a bug, I'll try to look into it!

[mverleg]

My suspicion about why it fails is that it needs to encode a lambda, apparently in PoolManager (there is one in the __init__). Unfortunately that's not possible. I've improved the error message though.

[mverleg]

Based on my comments from 2 years ago, this is solved to the extend it can be solved, right?

I'm going to close it, feel free to re-open if a problem persist that can be solved.