Make this into a package? about jupyter-widgets-for-data-science-guis

[janfreyberg]

I agree that that would be great. I'm currently developing this under janfreyberg/superintendent, but I have no real knowledge of mechanical turk, so @AlJohri if you have experience with it maybe you could help?

[pbugnion]

my initial suggestion would have been to allow passing a function that displays the content

I think that makes sense. To make it somewhat easier for the user, you could have factory methods that either generate the lambda, or wrap the entire constructor -- e.g.

InteractiveLabeller.from_dataframe(model, X, columns_to_display=None) # and more kwargs related to dataframes
# equivalent to InteractiveLabeller(model, lambda X, rowindex: getattr(X, 'iloc')[rowindex])

InteractiveLabeller.from_images(model, img_array)
#  equivalent to InteractiveLabeller(model, plt.imshow(X[rowindex, :]))

Internally, you could implement this with an output widget.

I'd definitely like this to be open source and I can manage it, no problem!

Great -- I think this could be a really nice use case for widgets!

[pbugnion]

Superintendent is great, so I'm closing this.

[pbugnion]

I think building a widget that allows a human to correct misclassified labels is a great idea!

One challenge is that the information that allows a human to guess the correct label will be very dependent on the training data. In this repo, it's the memo attached to a transaction, but if your training data was the MNIST dataset, you would probably want to see an image. Getting the API right would be challenging.

In terms of moving forward, I don't think I could commit the time to maintain another open source package. If this is something you're interested in, I would be very happy to assist and to contribute to development. If you're interested in managing this, we could have an initial chat next week to think about interfaces etc.?

Finally, I would encourage open sourcing from the beginning, either under the ASI organisation or under your name (which you choose depends on how you want to share both the ownership and the maintenance burden). It'll have more impact than keeping it in an internal repository.

[janfreyberg]

Yes, that's what I was thinking. In terms of the interface, my initial suggestion would have been to allow passing a function that displays the content: if it's text in a pandas dataframe this might be lambda X, rowindex: getattr(X, 'iloc')[rowindex], if it's an np array of MNIST it might be lambda X, rowindex: plt.imshow(X[rowindex, :].reshape(28, 28)).

Talking about it next week would be very helpful, yes - let's fix a time for that tomorrow. I'd definitely like this to be open source and I can manage it, no problem!

[AlJohri]

Just wanted to add that this would be really cool if somehow one could directly integrate mechanical turk into here. Or at least making it flexible enough that it could be done down the line.

[pbugnion]

Thanks for the great suggestion. I agree that that a Mechanical Turk integration makes a lot of sense.

[AlJohri]

good stuff @janfreyberg! I'll check it out.