pretzelai / pretzelai

Open-source, browser-local data exploration using DuckDB-Wasm and PRQL

🥨 Pretzel

Live deployed build: https://pretzelai.github.io

Pretzel is an open-source, offline browser-based tool for fast and intuitive data exploration and visualization. It can handle large data files, runs locally in your browser, and requires no backend setup. Pretzel makes it easy to manipulate data via visual chained data transform blocks. It's also reactive - changing a transform block in the chain automatically updates all transform blocks, and charts that follow.

Features

• 🚀 Blazing-fast performance with WebAssembly-based DuckDB and PRQL
• 🔍 Intuitive data exploration with a visual, top-down pipeline of data transformations and visualizations
• 🧠 AI-powered transformation block to help with fast data manipulation
• 🔒 Privacy-first design: run Pretzel AI locally or host it yourself for full control over your data
• 📊 Upcoming features: Local LLM support, API calls, in-browser Python support with Pyodide, save and share workflows securely and canvas-based table rendering

Table of Contents

• Demo video
• Getting started
  • Website (Easiest)
  • Offline standalone app
  • Developers
    • Run locally
    • Host Pretzel
• Optional Configuration
• Implemented Transformation Blocks
• Known Bugs
• Contact

Demo video

Pretzel.mp4

Getting Started

Website (Easiest)

The easiest way to use Pretzel is to visit https://pretzelai.github.io

Offline standalone app

Since Pretzel doesn't have a backend, you can easily install it as a Chrome app and it will work even without internet (for those long flights!)

1. Visit https://pretzelai.github.io in Chrome

2. Click the install app icon

3. Now you can launch Pretzel as a standalone app. It will also work offline, though it may error if you try to use some internet feature (like the AI Block). Just close it and open it again to fix it.

Developers

Run locally

To run Pretzel locally, follow these steps:

1. Clone the repository:

   git clone https://github.com/pretzelai/pretzelai.git
   

2. Install dependencies:

   cd pretzelai
   npm install
   

3. Start the development server:

   npm run start
   

4. Open your browser and navigate to http://localhost:3000

Host Pretzel

To host Pretzel, follow these steps (it's just a static website!):

1. Build the app

npm run build

2. Upload the contents of the dist folder to your hosting. This is what you can find live at https://pretzelai.github.io

Optional configuration

• Bug report box: Update /src/lib/config.ts with your PostHog config to let users report bugs directly on the website
• AI Endpoint: Deploy a cloud function to provide an AI endpoint for users without an OpenAI API key. Check the cloud folder for instructions.

Implemented transformation blocks

• Upload: accepts CSV / Excel (XLSX) files
• Filter: string/number/date filtering including nested filters
• Ask AI: connects to OpenAI to transform user command to SQL
• Pivot: to create a pivot table (you can also go group-by using this - only use the Rows and Values fields)
• Sort: sorts ascending or descending on multiple columns
• Chart: supports line (including multi-line) charts, bar charts (grouped and stacked) and scatter plot
• Create column: make a new column with basic math or use PRQL functions
• Remove columns: easily add/remove columns with visual toggles
• Table: add a table in the middle of your workflow to visualize data in a intermediate step
• Download: export your transformed data in CSV

Known Bugs

• Dates are sometimes parsed incorrectly - existing GH issue here
• Table panel is slow for large datasets. We're planning on moving to a canvas-based table.
• [Rare] Charts axes can sometimes not be ordered correctly

Please report any bugs you find in GitHub issues.

Contact

You can email us at founders [at] withpretzel [dot] com.

We also read all the feedback and bugs you report at the top left of https://pretzelai.github.io

pretzelai's Issues

Some CSV files are not parsing correctly

Example 1: CSV generated from US Census:
https://censusreporter.org/data/table/?table=B06009&geo_ids=860%7C04000US06&primary_geo_id=04000US06

Some other files as well but they do they contain sensitive data

Change Request: add streamlit/gradio/etc. integration instead of standalone app

It would be great if not only as separate standalone app, pretzelai could be integrated in other apps via these streamlit, gradio and similar solutions in python ...

Are multiple sheets of an xlsx file supported?

• I would like to be able to import https://zenodo.org/records/10701706/files/plastchem_db_v1.0.xlsx?download=1 from either remotely or locally
• This file has numerous tabs, so I want to be able to work with each tab being a separate table. If I am not mistaken, currently only the first tab in it looks to be made available as a table.

refactor: codebase and organize interface and models into seperate directories

Hi team!
We can refactor the codebase to a large extent as size of files right now are huge and personally for me it was difficult to filter through the files
We can do the following

1. Seperate interface and models into different functions
2. Create directories for utilies functions and maybe declare a common directory which contain functions used by loads of repos
3. Also utils folder is kind of less self explanatory.

There can be a lot of more changes that can be done. I would like to help on that! Thanks

Switch to Vite

CRA is not great for dev experience, switch to Vite to use native ES module imports.

Import table from postgres connection

Future updates:

• many tables
• import from sql dump

Support for external API calling in new column creation

Let users create new columns from external API calls. Per user feedback:

I have a huge list of leads and want to call the Neverbounce API for each row for email validation. And then afterwards, the prospeo email enrichment API, … This would be a huge use case for me and a lot of other lead gen folks I know

Add python block

Add Pivot support for Columns + values

This is a feedback form user request.
Currently only rows + values supported.
Familiar excel/gsheets support rows+value as well as columns+values

Create a release process

• tagged release branches
• downloadable distributable website

Add PRQL generating option to AI

This is a feedback form user request

Add button/block to remove duplicate rows

Block UI should show how many duplicated where removed

Timestamp processing is flaky

nps_reviews.csv

On this CSV, the parse detects the timestamp string correctly and we pass this to DuckDB but duckdb seems to ignore the timestamp type. When we query the loaded table, duckdb returns floats and the type in the returned schema is also float (which parses correctly to the correct timestamp value).

Find if this is a DuckDB bug and if so, file with bug report with them.

Null literals in CSVs aren't handled properly

If in a CSV, there are values that are literally the string "null", currently, we make the assumption that the CSV has generated incorrectly and that these values should have been empty (i.e. no text between the commas)

We read these values currently as null or missing values. But there can be cases where the null literal should be parsed and stored as the string "null". Fix this.

Filter contains/doesn't contain should be case insensitive

Modify src/components/FilterBlock.tsx to add this functionality

Local build failed

I foundnpm run build failed , and do you need me a pr to fix it ?

Add server code for SQL based AI block

See #3 - we need to replicate this in the Lambda code

Refactor with global state

Motivations:

• Contributors have been struggling to understand the codebase
• Better DX to develop new features or fix bugs

SQL / PRQL syntax highlighting

This is a feedback form user request.
Let's try to use codemirror npm i @uiw/react-codemirror

something like what we are doing for python but for SQL/PRQL
<CodeMirror minHeight="10px" theme={"light"} height="100%" className="w-full border h-[200px]" editable basicSetup={false} extensions={[ python(), minimalSetup({ syntaxHighlighting: true, }), ]} value={code} onChange={setCode} onKeyDown={handleKeyDown} />

Feature: python charts with Plotly and AI

[Bug] clicking "Table" button breaks the pipeline

Pivot Aggregation "Last Value" returns sample variance instead.

When selecting a Pivot Table with "Last Value" as Values aggregation, instead of the last value, the sample variance is returned.

This appears to be due to a typo in [src/components/Pivot.tsx]:(https://github.com/pretzelai/pretzelai/blob/main/src/components/Pivot.tsx#L230)

<SelectItem value="var_samp">Last Value</SelectItem>

Upload and Filterblock shoul include total row information

Upload: how many rows imported. implemented by #46
Filterblock: previous rows -> rows after filtering

Speed up python implementation

Add/remove blocks in the middle of the pipeline

Currently it is only possible to add/remove them at the end

Add support for SQL / PRQL block with toggle

Similar to the AI Block, add support where users could write their own code. There should be a toggle to let people switch between PRQL and SQL

Add archive with static assets to releases

Hi, can we have a zip archive with the static assets already build, as Cyberchef does for every release?

Fix CORS issues with PRQL to make it possible to simply run the `index.html` locally with no server needed

Add support for Parquet and Arrow files