Comments (11)
Hey @MisterVladimir! Thanks for contributing.
I'm happy to support your usecase and your suggestions look sound. Would you be open to trying these changes in a pull request and see if the tests pass? You can remove superfluous test cases, if any.
from addict.
Thanks @mewwts. I'm working on it now, namely, on getting test_addict.Tests to run the same tests on Dict, or a child class. Expect a pull request this evening.
from addict.
I'm a little new to Travis, tests, etc., so apologies for polluting the commit history with incremental bug fixes, but it looks like tests pass on my machine with "python setup.py test" and with Travis. The second commit I made, which changed the test script, built with Python 3.6+ but not Python 2, probably because I used "super()" instead of the old "super(cls, obj)". That's been remedied in subsequent commits. Looks like the package builds with Python 2 and 3.
from addict.
No worries @MisterVladimir, I've wrestled countless times with that!
I've gone ahead and merged #102.
Thanks!
from addict.
I've released 2.2.0 as well https://github.com/mewwts/addict/releases/tag/v2.2.0.
Cheers!
from addict.
I've just ran into several problems with this change, which caused breaking changes with my project, though it is understandable why they occurred after reading this issue. I just wanted to say that existing projects could break due to various assumptions we used to have before this change, so perhaps this might be a heads-up for others who also found their projects breaking from today onwards.
Firstly, I have a base class which extends Dict
and some subclasses, like as follows:
from addict import Dict
class BaseStrategy(Dict):
name = None
def __init__(self):
super(BaseStrategy, self).__init__()
# This line breaks because the class variable
# name is the same as the instance variable name.
self.name = self.__class__.name
class ExampleStrategy(BaseStrategy):
name = 'EXAMPLE'
example = ExampleStrategy()
The error we get:
Traceback (most recent call last):
File "poc/setattr.py", line 19, in <module>
example = ExampleStrategy()
File "poc/setattr.py", line 12, in __init__
self.name = self.__class__.name
File "/usr/local/lib/python2.7/dist-packages/addict/addict.py", line 27, in __setattr__
"'{0}' is read-only".format(name))
AttributeError: 'Dict' object attribute 'name' is read-only
As I found out, this is bad practice so I was okay to rectify my code accordingly and rename the class variable to be a different name from my member variable.
Secondly, perhaps slightly more concerning, is that the new __missing__
method creates a new instance without passing the correct arguments to __init__
. This could break projects which assumes that addict returns {}
for missing attributes for subclasses of Dict, or if we assume that the parent class is addict.Dict
and therefore we can just initialise member variables through chaining:
from addict import Dict
class CustomDict(Dict):
def __init__(self, value):
super(CustomDict, self).__init__()
# The problem is that I do not initialize self.my as a Dict(),
# since I previously assumed that this would work.
self.my.custom.value = value
class CustomDict2(Dict):
def __init__(self, value, **kwargs):
super(CustomDict2, self).__init__()
self.my.custom.value = value
class CustomDict3(Dict):
def __init__(self, value=None, **kwargs):
super(CustomDict3, self).__init__()
self.my.custom.value = value
# Try running any of the following lines on its own
# custom_dict = CustomDict('hello world')
# custom_dict2 = CustomDict2('hello world')
# custom_dict3 = CustomDict3('hello world')
# The correct fix would be to initialize self.my as a Dict() first.
# This also breaks when I try to access a non-existent attribute:
class CustomDict4(Dict):
def __init__(self, value, **kwargs):
super(CustomDict4, self).__init__()
self.value = value
custom_dict4 = CustomDict4(1)
assert not custom_dict4.missing_attr
# The correct fix would be to do:
# assert 'missing_attr' not in custom_dict4
Some of the errors:
Traceback (most recent call last):
File "poc/getitem.py", line 26, in <module>
custom_dict = CustomDict('hello world')
File "poc/getitem.py", line 10, in __init__
self.my.custom.value = value
File "/usr/local/lib/python2.7/dist-packages/addict/addict.py", line 62, in __getattr__
return self.__getitem__(item)
File "/usr/local/lib/python2.7/dist-packages/addict/addict.py", line 65, in __missing__
return self.__class__(__parent=self, __key=name)
TypeError: __init__() got an unexpected keyword argument '__key'
Traceback (most recent call last):
File "poc/getitem.py", line 39, in <module>
assert not custom_dict4.missing_attr
File "/usr/local/lib/python2.7/dist-packages/addict/addict.py", line 62, in __getattr__
return self.__getitem__(item)
File "/usr/local/lib/python2.7/dist-packages/addict/addict.py", line 65, in __missing__
return self.__class__(__parent=self, __key=name)
TypeError: __init__() takes exactly 2 arguments (1 given)
Although this was a minor version bump, it may have caused quite drastic inconvenience for others as well. I'm not really sure what recommendation to give, but just wanted to mention this.
from addict.
Thank you so much @irvinlim for pointing this out!
First of all, sorry for breaking this for you. I was unaware of the implications this would have for clients inheriting from addict.Dict
and the test cases did not cover the issue. Please pin addict
to a version that works for you while we figure out what's sensible here.
If I understand correctly, @MisterVladimir's test-case is so that for a subclass of addict.Dict
, getting a missing variable on the subclass should return an instance of that subclass.
Before this change setting a missing attribute on the subclass would return a addict.Dict
instance.This was the functionality you were depending on @irvinlim.
The problem arises when the subclass defines its own __init__
function with custom parameters. The following line will try to call __init__
on the class without knowing its parameters.
Lines 64 to 65 in cf29d47
As far as I can see it's not possible to support both these usecases at the same time. On the one hand, I feel like what we just merged is the more expected functionality, however inheriting from addict
and accepting custom parameters in __init__
will break if you don't implement __missing__
on your own. So it's kind of a halfway there solution.
On the other hand, this probably breaks the appeal of inheriting from addict
in config classes and the like.
I'm torn, but I'm open to reverting. I need to think some more about it.
Do you guys have any other thoughts on this @irvinlim and @MisterVladimir?
from addict.
I can see where @MisterVladimir is coming from with the fix, as he expects the descendant nodes to be the same class as the object itself, meanwhile I expected my Dict subclasses to inherit the dot-notation elegance from addict.
Actually, I don't think there's a right answer to the behaviour when subclassing addict.Dict
, because I suppose you wrote this library to allow elegant quick dot-notation access, and not meant as a base class of sorts. Because you did not design for this, I suppose any breaking changes might be reasonable.
My only gripe would be that error messages are now much more cryptic, due to the fact that users unaware of the internal implementation would see messages like this:
TypeError: __init__() got an unexpected keyword argument '__key'
when in fact the error is due to a missing attribute or method, perhaps due to a typo. In fact, a KeyError
thrown might be more meaningful, but it doesn't help that it's still another backwards-incompatible change for those expected it to return a new, empty Dict()
.
I am perfectly okay with this change actually, because the breaking changes didn't take me very long to fix, but I just wonder how many others may be depending on the same functionality as I did.
from addict.
Thanks for the lively discussion, everyone. I agree that the most concerning issue is
the new '''missing''' method creates a new instance without passing the correct arguments to '''init'''.
Basically v2.2.0 throws an error if the child class' __init__
has "regular" arguments and sets keyword arguments to the default values. Would this be too ugly a fix for your problem, @irvinlim's?
def __missing__(self, key):
try:
return self.__class__(__parent=self, __key=name)
except: #maybe limit to TypeError and RecursionError
return Dict(__parent=self, __key=name)
I never realized one would want to implement an __init__
in child classes. I see a Dict subclass still as a data structure whose instances don't reach back and modify parent nodes. Specifically, my use case is to store info extracted from a zip archive, whose files often have silly naming schemes. It helps to access keys via an integer index. Perhaps down the line I'd add a method for recursively writing the dict to an hdf5 file. Although I too enjoy the convenience of the dot notation -- which I preserve in my subclasses -- I'm open to better data types for storing this info. Which leads me to:
On the other hand, this probably breaks the appeal of inheriting from addict in config classes and the like.
@mewwts, would you be so kind as to link to an example? I actually need to make a config file for my own program, so a pointer would be a big help to this n00b. : )
from addict.
Hey @MisterVladimir.
I thought of a config class just as something like
class Config:
s=13
n=12
which you can then pass around.
I'm closing this now as it doesn't seem to be a problem for more users. @irvinlim thanks for using addict and reporting this.
from addict.
I'm reopening this as several users has hinted towards issues with the new implementation.
from addict.
Related Issues (20)
- Is it possible to forbid accessing missing keys? HOT 4
- Dict adds quote when retrieving str values HOT 1
- Dict() inconsistent with dict() initialization HOT 4
- Adding support for merge operator `|` and `|=` (as python 3.9 dict merge) HOT 1
- Maintainership & Altrenatives HOT 4
- 'function' object has no attribute when using freeze as a field
- Dict.__missing__ implementation leads to surprising behaviour HOT 1
- google-colab 1.0.0 has requirement ipykernel~=4.10 HOT 1
- copy broken by 2.4.0 HOT 1
- Arrays in dictionaries, how to denote HOT 1
- Shouldn't dictionary keys be encapsulated with u''? Isn't everything in python 3 Unicode?
- Suggestion on attributes containing dashes
- Dict Attributes not available to Autocomplete and not present in __dict__ HOT 4
- have stubs for minimal type annotation on addict
- Somehow inherit from TypedDict
- Keys starting with __ can't be referenced with dotted notation inside an object
- Creating a Dict from instances of classes that inherit from NamedTuple
- Support inheritance in merge operators `|` and `|=`
- Consider changing the "About" line for this repo?
- What would it be like to add support for Pandas?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from addict.