What should do the DataGrid.view method Python side? about ipydatagrid

[kaiayoung]

On the TS side, the default behavior is to update in place. For example, if @phosphor/datagrid/DataGrid.model is a ViewBasedJSONModel, model.updateView() will replace its current view with the new one. Looking at my code now, the (intended) runtime behavior isn't quite clear, but calling model.updateView() with no arguments should revert the model back to its initial view. Making a .resetView() or something would likely be more clear in this case, and could easily be exposed in the widget.

On the widget side, I can't think of any use cases offhand in which users would want to create a new widget, but don't see any harm in including an option to do so, though perhaps as a .copy(), .createNewWidget(), inplace=False or something that more explicitly communicates that a new widget is being created. Random thoughts:

• I think that updating in place should be the default widget behavior unless the user explicitly indicates that they want a copy. We've seen many users creating and returning new bqplot figures to "update" data rather than just assigning new data to the marks. Since the widget comms stays open even if the widget is no longer referenced in the notebook, the unreferenced bqplot figures (and marks that contain a full dataset) are never garbage collected and can eventually use up all available memory on the heap. For many notebooks, this isn't an issue, but can become one when users are making "dashboard" apps that are meant to stay open for longer periods of time than a single notebook
• We'll want to verify that calling .close() on the widget actually frees up all of the resources on the JS side.

[martinRenou]

So, do you think it would make sense if everything is a view?

from json import load
from ipydatagrid import JSONGrid

with open('./cars.json') as fobj:  
    data = load(fobj)

datagrid = JSONGrid(data=data)
# Here, datagrid is actually a view, so that you can call update:
datagrid.update([Filter('Origin', '=', 'Europe'), Sort('Displacement')]) 

And we include the option for copying later, possibly never. I really don't have an opinion on it. I suppose having everything in-place is fine.

[kaiayoung]

The ViewBasedJSONModel item manages its own views, so the user doesn't even have to know about its concept of views, as its functionally interchangeable with any other data model. This does bring up an interesting question though, I hadn't really considered the level of support we want to have for other data models. It could possibly make sense to have multiple widgets, as the interface of the widget could need to make assumptions about what the data model can do.

My initial thought is that the main "DataGrid" widget should have the ViewBasedJSONModel as it's model, so users can quickly get something with all of the sorting/filter/menus and other features without having to write lots of code in the notebook (our main userbase are not developers), but we also expose some more generic grids that could be customized more?

Some ideas:

from ipydatagrid import DataGrid, JSONGrid, BaseGrid
import pandas as pd

with open('./cars.json') as fobj:  
    json_data = load(fobj)

# Creates a grid with a user-provided model (which would need a widget wrapper),
# which would let people use/create their own models
grid_1 = BaseGrid(model=JSONModel(data))

# Creates a grid that subclasses BaseGrid and uses the phosphor JSONModel data
# model
grid_2 = JSONGrid(data)

# Creates a grid that subclasses BaseGrid, with the ViewBasedJSONModel as its
# model, and exposes additional functionality available in the model. This grid
# would have context menus that would expose the .updateView() options.
df = pd.read_csv('./cars.json')
grid_3 = DataGrid(df)

# Calls .updateView() on the model, which updates the view in place
grid_3.transform([sort, filter, some_other_transform])

# Resets the grid back to its initial state (ie, the view created in its constructor)
grid_3.reset()

# Replaces the data in the model with a new dataset, then rerenders the grid
grid_3.update_data(new_df)

I also don't really have a strong opinion about copying. I've only seen people copying widgets to dynamically create UI's, which probably would have been better handled with factory functions.

[martinRenou]

The JSONGrid might not be the more optimized data model for communication between the back-end and the front-end, as we won't be able to use binary buffers for the data communication.

Anyway, I am totally fine with exposing the ViewBasedJSONModel for the base DataGrid class Python side for now. I'll do that quickly. And we can add more data models later on if need be.

[martinRenou]

Done in #7

[martinRenou]

The JSONGrid might not be the more optimized data model for communication between the back-end and the front-end, as we won't be able to use binary buffers for the data communication

I am wrong. I tested with a million cells. It takes ~1 second to send the data to the front-end. And then the rendering of the grid is instantaneous.

[kaiayoung]

There is definitely some room for optimization in the future! @mbektasbbg actually created a phosphor-based widget in the past that kept all of the data in the Python kernel and only sent what was in the viewport to the front end, which I would love to have a look at!

An in-memory data model will get us fairly far for now, but developing a StreamingViewBasedModel and transforms or something in the future would be good. My initial uneducated guess was to have a "data" widget that was solely for serving data to the front end. Then, we could have StreamingTransform classes that tell the "data" widget to transform itself in the kernel and then send the updated data for the current viewport.

[martinRenou]

I guess we can close this PR now that #7 is merged