Design of lazy evaluation for some properties about koalas

[rxin]

Does Python automatically materialize all the properties when the class is created? If yes, I definitely agree we shouldn't create "values" eagerly ...

[garawalid]

Yes, it does.
Here is an example. I implemented values as property in _Frame class.

    @property
    def values(self):
        return self.toPandas().values

Running this code will show us that the property values contains already the data as numpy array.

df = pd.DataFrame({'a': [4, 2, 3, 4, 8, 6],
                   'b': [1, 2, 9, 4, 2, 4],
                   'c': ["one", "three", "six", "seven", "one", "5"]},
                   index=[10, 20, 30, 40, 50, 60])

ddf = self.spark.from_pandas(df)

A screen capture from the debugger stopped after creating ddf

[AbdealiLoKo]

A debugging tool would typically get all attributes in dir() and request for every attribute. Checking it in a debugger may not be an accurate test

[garawalid]

@AbdealiJK
Thanks! You are right, it's not an accurate test.
I tried it with this simple test, and the message doesn't appear when we create the dataframe.

@property
def values(self):
    print("executed !")
    return self.toPandas().values

[garawalid]

So, we can say that properties are lazy by default!

[rxin]

Thanks. Looks like we are safe here. I'm going to close this ticket. Feel free to reopen if there is an issue in the future.