Establish a spark session by providing masternode & application name.
If you want to work on your own Classes like House, Car then convert DF to dataset and then convert back dataset to dataframe.
Reselient Distributed Dataset
Directed Acyclic Graph Recipe containing instructions on what to do in a stepwise fashion. Procedural steps that are single directional. When actions are performed, then only the recipe is being prepared i.e. then only all the instructions are executed.
Optimizes the graph by reducing the number of steps needed to do the same thing more efficiently. It is smart enough to decide whether filtering needs to be ececuted 1st or order by even if the code is poorly written.