javadch / pyquis

A port of the QUIS data query system to Python

pyquis

A port of the QUIS data query system to Python

pyquis's Issues

LazyPandas

A set of extensions to the Pandas library to utilize QUIS to fetch, join, filter, project, aggregate, and sort data from different data sources, before loading the data into Pandas and data frames.

import pandas as pd
data_iterator = pd.query()
 .source(name="weather", type=csv, path="large_data.csv", separator="tab")  #multiple sources are allowed
 .from(weather) # joins are allowed, assuming multi sources are available
 .select("timestamp, avg(temperature) as avg_temp")
 .where("temperature >= 22")
 .groupby("timestamp")
 .orderby("avg_temp", order="desc")
 .fetch()

• source can be csv, excel, rdb
• csv can be file or folder: folder contains multi-file, each file is a data container
• excel is multi-sheet, each sheet is a container
• db has multi tables, each table is a container