Comments (3)
In case it's helpful, here's my current workaround.
@classmethod
def enc_hook(cls, obj: Any) -> Any:
if isinstance(obj, MappingProxyType):
return obj.copy()
else:
raise NotImplementedError(f"Unknown type: {type(obj)}")
@classmethod
def dec_hook(cls, type: Type, obj: Any) -> Any:
if type is MappingProxyType or get_origin(type) is MappingProxyType:
args = get_args(type)
if len(args) == 2:
key_type: Any = args[0]
value_type: Any = args[1]
return MappingProxyType(msgspec.convert(obj, Dict[key_type, value_type]), dec_hook=cls.dec_hook)
else:
return MappingProxyType(msgspec.convert(obj, dict, dec_hook=cls.dec_hook))
raise NotImplementedError(f"Unknown type: {type(obj)}")
from msgspec.
Thanks for opening this. Your workaround as posted is how I'd handle this today.
For encoding I'd expect only the slightest of speedups if we supported this natively. MappingProxyType
doesn't expose a native API, so the only difference is the call to .copy()
would be made a bit quicker. Supporting this for encoding builtin is very easy to do though.
For decoding, native support could be made quicker since we could avoid the copies and 2nd traversal done by calling msgspec.convert
. Supporting this for decoding is less easy (there's more plumbing needed here) but still doable.
That said, types.MappingProxyType
is a fairly uncommon type to use. Adding additional builtin types increases the maintenance burden on msgspec, generally we only add types that are common or can be handled significantly faster when supported as builtins.
Can you say more about why you're trying to use a MappingProxyType
? Due to how they're implemented, MappingProxyType
objects will always be slower to create, access, encode, and decode. The only thing they give you is pseudo-immutability (and then only if the proxied dict isn't accessible elsewhere).
from msgspec.
Hi @jcrist, thanks for taking a look.
Adding additional builtin types increases the maintenance burden on msgspec, generally we only add types that are common or can be handled significantly faster when supported as builtins.
Given this, I think it's completely reasonable to not implement this suggestion. I also can't make any arguments that MappingProxyType
is widely used, I only saw it in some relatively obscure posts about immutability.
Can you say more about why you're trying to use a MappingProxyType?
Once-deserialized, I'm using Immutability to detect changes to a nested data structure. E.g. I don't need to use ==
, and have that recurse down the whole tree, but instead I can use object identity at the root level (or any sub-leaf I'm interested in). It's also trivial for me to implement an onchange
detection, because I only need to override the setter property at the root level, and that will catch any change anywhere in the entire tree.
That requires callers to remember to create new copies rather than updating existing ones when making changes, hence the requirement for immutability.
There probably are other options, such as subclassing UserDict
and overriding __setitem__
but then the static type checker is blissfully unaware. MappingProxyType seemed to be the simplest solution. The only challenge I ran into was that serializing the structure to disk with msgspec
(and then re-parsing it) doesn't work.
from msgspec.
Related Issues (20)
- json schema generation - differences between pydantic and msgspec HOT 3
- Allow conversion to collection from generator HOT 2
- Porting guide for users coming from `orjson`
- Converting dicts into list with key-reuse HOT 3
- Collecting multiple validation/constraint errors at once HOT 1
- Allow `omit_defaults` to exclude fields when encoded value is `{}` (empty dict)
- Duplicate key detection
- Allow unknown tags, defaulting to tagged base
- Implementing optional bytes type for json. HOT 1
- Update annotation parsing to work with PEP 649 in Python 3.13
- `omit_defaults` does not omit tuples and frozensets HOT 2
- Field Alias Overrides in Subclasses Not Reflected in __struct_encode_fields__
- Convert builtin types to numpy HOT 1
- Is it possible to have the decoding of union of all subclasses of a struct
- Subclasses of frozen Structs causing mypy error: `Cannot inherit non-frozen dataclass from a frozen one` HOT 2
- Datetime without timezone are decoded as str with msgspec.msgpack. in 0.18.6 HOT 1
- Optional[Raw] results in an unexpected validation error at json.decode time HOT 5
- Struct import fails in pytest - Missing requirement argument 'attribute x' or unhashable type: 'StructClass'
- Generate FastAPI docs from msgspec.Struct json schema HOT 2
- Output in Docs not readable in darkmode
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from msgspec.