Strange behaviour of REPL when running first few Ch3 examples about spark-the-definitive-guide

[bllchmbrs]

Not sure what the issue is, since it still works (when they're separate) I'm just going to close this out.

[ljalbinson]

I appreciate it is not an issue for your book.
However, isn't it a little worrying that the combined case fails? I don't think it should. But I'm happy to be corrected.
And, of course, I'm happy to report it elsewhere if that is more appropriate (assuming it is a bug and not my lack of understanding).
Kind regards, Lawrence

[bllchmbrs]

It's probably something to do with the implicits and the REPL compiler, could you try running the first line alone (import ...) and then running the rest as one block?

[ljalbinson]

Putting the import first doesn't solve the problem; however, it leads to a useful error message, namely:

<console>:18: error: Unable to find encoder for type stored in a Dataset. Primitive types (Int, ....

In this post the case class is moved out of the current scope and the problem is solved, so it seems that it is an implicit resolution issue.

I have tried this with the Ch3 example and it works.

So, in summary, declare the case class as a first snippet and then declare the import, the two vals and the filter/map/take as a second snippet and it is fine.

[bllchmbrs]

There we are. Glad we figured it out!