dtherrick / ds-notebooks Goto Github PK

Looks good @dtherrick! Nothing stands out as needing to be changed in the example, but we'll need to be thoughtful about the way we turn this walkthrough into a challenge.

We've got the main ask at the top, which is to use Spark to determine if we can predict whether a review is positive or negative based on the language in the review.

• How much information do you think we'll need to include to make that a reasonable ask?
• What specific information should they include in their work?
• How are they going about this work? Is it going to be done in a Jupyter notebook? Will they need to use Docker somehow? (And if so, I think they'll need more guidance on setting that up.)

Then at the bottom, there are some extension tasks:

Here's where you can go from here:

Think about resampling the overall dataset to better balance positive and negative reviews.
Use a different method to tokenize and convert the text to numeric (TF/IDF, etc).
Adjust the parameters of your classifier.

Do you want to incorporate those tasks into the challenge itself, or will they just live in the solution?