alinauman / pyspark-and-sparksql-basics

The objective is to demonstrate how to run Spark with PySpark and execute common functions.

PySpark-and-SparkSQL-Basics

How to implement Spark with Python Programming

Apache Spark is a cluster computing system that offers comprehensive libraries and APIs for developers and supports languages including Java, Python, R, and Scala. SparkSQL can be represented as the module in Apache Spark for processing unstructured data with the help of DataFrame API. Python is revealed the Spark programming model to work with structured data by the Spark Python API which is called as PySpark.

Tutorial URL: Link

Data-Set Description

The data-set for this task has been downloaded from the below mentioned Kaggle Data-set, New York Times Best Sellers.

Link: Data-set

Gathered from the New York Times API for Hardcover Fiction best sellers from June 7, 2008 to July 22, 2018.

The API can be found here: API

Collected data includes the book title, author, the date of the best seller list, the published date of the list, the book description, the rank (this week and last week), the publisher, number of weeks on the list, and the price.