caroljunq / sptrans-data-analysis

Characterization of a bus mobility dataset from the city of São Paulo

• Master Degree project in Computer Networks and Distributed System

• University: UFSCar - Universidade Federal de São Carlos - https://www.ppgcc.ufscar.br/en?set_language=en

• Professor advisor: Hermes Senger

• About this repository: This repository contains all codes and Spark snippets used during the project to pre-process, and extract metrics from the Sao Paulo buses dataset. The data engineering methods applied here could be applied in similar datasets.

• Topic: VANETs - Vehicular Network

• Description: Simulation is a widely used technique for validation and testing of routing protocols in vehicular networks (VANETs). These simulations are based on mobility models that must follow the real movement patterns of vehicles to produce realistic and accurate assessments of protocols and other vehicular network solutions. Mobility models and simulations can be based and adjusted using real vehicular mobility traces. These traces are records of vehicles positioning over a period of time, and they incorporate the behavior and dynamics that occur in the day-to-day of a city (traffic jams, the way drivers drive, routines of population or types of vehicles). Understanding the mobility characteristics of these traces helps in feasibility study of vehicular networks, understanding of factors that affect traffic conditions, configuring and conducting realistic simulations so that the design of routing protocols reacts to real vehicle movement behaviors. Therefore, a tool to produce more accurate and realistic results in these simulations is the characterization of vehicle mobility datasets. There are works that already makes this type of characterization, but few of them explore data from large cities like São Paulo. This work characterizes a dataset of real mobility traces of buses in São Paulo, extracting and demonstrating mobility and connectivity metrics found in the literature. Prior to characterization, the dataset was pre-processed using common treatments for this type of data. The metrics extraction and pre-processing were done using the Python language and the big data tool Apache Spark. The objective of this work is to demonstrate the behavior of a large urban centers, and to provide a dataset for the validation, adjustment and conduction of simulations of vehicular networks. The extracted characteristics indicate behavior patterns in the movement of buses that are repeated according to the days of the week (weekdays, Saturdays, and Sundays), period of the day, and regions of the city.

• Link to the monography: https://repositorio.ufscar.br/handle/ufscar/14672

• The resultant pre processed dataset can be found on Kaggle: https://www.kaggle.com/datasets/caroljunq/sao-paulo-bus-mobility-traces-oct-2015