Comments (6)
Could you provide a minimal example in code?
from cattrs.
Sorry for the delay...
I don't know if it is a minimal one, but here is my example:
from datetime import datetime
from typing import Any, Final, Type, TypeVar
import attrs
from cattrs.preconf.json import make_converter
from cattrs.strategies import configure_tagged_union
T = TypeVar('T')
@attrs.frozen
class Sub:
foo: str = 'bar'
# and some more fields (incl. other attrs types)
@attrs.frozen
class A:
some: str = 'a'
sub: Sub = Sub()
@attrs.frozen
class B:
some: str = 'b'
sub: Sub = Sub()
FrameData = dict | A | B
@attrs.frozen
class Frame:
data: FrameData
_CUSTOMIZED_STRUCTURE_TYPES: Final[set] = {
datetime,
dict,
Frame,
# and some more...
}
_TIMESTAMP_FORMAT: Final[str] = '%Y%m%d_%H%M%S'
def _structure(data: dict[str, Any] | str, to_type: Type[T]) -> T:
match to_type:
case t if t is datetime:
return datetime.strptime(data, _TIMESTAMP_FORMAT)
case t if t is dict:
return _structure_dict(data)
case t if t is Frame:
data.pop('to_add', None)
return conv.structure_attrs_fromdict(data, Frame)
case _:
raise NotImplementedError(f'Unsupported type: {str(to_type)}.')
def _structure_dict(data: dict[str, Any]) -> dict[str, Any]:
structured: dict[str, Any] = data.copy()
for k, v in structured.items():
if isinstance(v, str):
try:
structured[k] = datetime.strptime(v, _TIMESTAMP_FORMAT)
except ValueError:
continue
# something is needed here to call the converter for structuring the other values of the dict
return structured
conv = make_converter()
for data_type in _CUSTOMIZED_STRUCTURE_TYPES:
conv.register_structure_hook(data_type, lambda data, to_type: _structure(data, to_type))
configure_tagged_union(union=FrameData,
converter=conv,
tag_name='_type',
tag_generator=lambda t: t.__name__.casefold(),
default=dict)
As a result of this:
f='{"data": {"a": {"some": "a", "sub": {"foo": "bar"}}, "ts": "20240320_010203", "_type": "dict"}}'
print(conv.loads(f, Frame))
I get this output:
Frame(data={'a': {'some': 'a', 'sub': {'foo': 'bar'}}, 'ts': datetime.datetime(2024, 3, 20, 1, 2, 3)})
But what I need is this output:
Frame(data={'a': A(some='a', sub=Sub(foo='bar')), 'ts': datetime.datetime(2024, 3, 20, 1, 2, 3)})
from cattrs.
Hi Tin,
Is there anything else I should add or is it just a busy schedule?
from cattrs.
Hey,
yeah sorry I got sidetracked by other things.
But since the simple dict objects can also contain attrs instances, it must be possible to call the structuring recursively again.
This is going to be complicated without modeling this more precisely. How do you know a nested dict is supposed to be converter into a class instance and not left as a dict? If a nested dict always means A | B
, then it gets easier.
Frame(data={'a': {'some': 'a', 'sub': {'foo': 'bar'}}, 'ts': datetime.datetime(2024, 3, 20, 1, 2, 3)})
That looks correct given the input. Even if we assume data['a']
is logically typed as FrameData
, it has no _type
field and so will default to a dict. In other words, how can we tell data['a']
is supposed to be A
?
from cattrs.
But since the simple dict objects can also contain attrs instances, it must be possible to call the structuring recursively again.
This is going to be complicated without modeling this more precisely. How do you know a nested dict is supposed to be converter into a class instance and not left as a dict? If a nested dict always means
A | B
, then it gets easier.
I now see that my example is misleading.
My sentence (which you quoted) referred to the following line in the _structure_dict
function:
# something is needed here to call the converter for structuring the other values of the dict
return structured
The goal is not to convert an arbitrary dict into an attrs instance. But dict-values, which in turn can be attrs instances (with datetime), should also be converted accordingly. However, they are not recognized as such before the datetime conversion (due to the special format).
To achieve this, the converter would have to be called again within the _structure_dict
function. Something like this:
# something is needed here to call the converter for structuring the other values of the dict
return conv.structure(structured, dict)
Of course, this doesn't work because it creates an endless loop.
With attrs classes I use conv.structure_attrs_fromdict
to achieve this (as in the case of Frame
).
If necessary, I can rework the example. (But that will certainly not be until the week after next.)
from cattrs.
Yeah, a simplified example would be good.
To achieve this, the converter would have to be called again within the _structure_dict function. Something like this: ... Of course, this doesn't work because it creates an endless loop.
You can just call _structure_dict
on each value yourself, right? You don't even need to jump back into cattrs. It won't create an endless loop since it will stop when there are no values.
from cattrs.
Related Issues (20)
- Constructor selection for structuring HOT 7
- Tagged Union: How to do structure/unstructure hook(s) for particular member type? HOT 4
- A dictionary gets structured into a list without errors/warnings HOT 2
- Question: Correct use of converters HOT 3
- Documentation does not explain unstruct_collection_overrides' input HOT 4
- Detailed validation exception groups with hook errors HOT 2
- cattrs effectively disables attrs validation HOT 3
- Covert to defaultdict instead of plain dict HOT 4
- How to recursively unstructure with hooks? HOT 5
- Dataclass structuring fails with misleading error message.
- Default field conversion conflicts with user-defined field converter function HOT 3
- [macOS] Error when trying to run tests: `ImportError: cannot import name 'CodecOptions' from 'bson'` HOT 2
- Incorrect dispatch in Python 3.12 HOT 4
- Why does structuring of datetime in attrs class need a structure hook? HOT 1
- `prefer_attrib_converters` doesn't work when the field is aliased HOT 2
- Register structure hook only for optional types HOT 2
- TYPE_CHECKING and init=False HOT 5
- Problem with tagged union example HOT 3
- Subclass disambiguation for nested structures. HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cattrs.