alloc / tusken

100% type-safe query builder compatible with any Postgres client 🐘 Generated table/function types, tree-shakable, implicit type casts, and more

⚠️ This library is currently in alpha. Contributors wanted!

---

tusken

Postgres client from a galaxy far, far away.

• your database is the source-of-truth for TypeScript generated types
• type safety for all queries (even subqueries)
  • all built-in Postgres functions are available and type-safe
  • implicit type casts are accounted for
• minimal, intuitive SQL building
  • shortcuts for common tasks (eg: get, put, and more)
  • identifiers are case-sensitive
• lightweight, largely tree-shakeable
• works with @tusken/cli to easily import CSV files, wipe data, generate a type-safe client, dump the schema for migrations, and more
• you control the pg version as a peer dependency
• query streaming with the .stream method (just install pg-query-stream and run tusken generate)

Migrations?

Use graphile-migrate.

Install

pnpm i tusken@alpha pg postgres-range postgres-interval
pnpm i @tusken/cli@alpha -D

Usage

First, you need a tusken.config.ts file in your project root, unless you plan on using the default config. By default, the Postgres database is assumed to exist at ./postgres relative to the working directory (customize with dataDir in your config) and the generated types are emitted into the ./src/generated folder (customize with schemaDir in your config).

import { defineConfig } from 'tusken/config'

export default defineConfig({
  dataDir: './postgres',
  schemaDir: './src/generated',
  connection: {
    host: 'localhost',
    port: 5432,
    user: 'postgres',
    password: ' ',
  },
  pool: {
    /* node-postgres pooling options */
  },
})

After running pnpm tusken generate -d <database> in your project root, you can import the database client from ./src/db/<database> as the default export.

import db, { t, pg } from './db/<database>'

The t export contains your user-defined Postgres tables and many native types. The pg export contains your user-defined Postgres functions and many built-in functions.

Creating, updating, deleting one row

Say we have a basic user table like this…

create table "user" (
  "id" serial primary key,
  "name" text,
  "password" text
)

To create a user, use the put method…

// Create a user
await db.put(t.user, { name: 'anakin', password: 'padme4eva' })

// Update a user (merge, not replace)
await db.put(t.user, 1, { name: 'vader', password: 'darkside4eva' })

// Delete a user
await db.put(t.user, 1, null)

Getting a row by primary key

Here we can use the get method…

await db.get(t.user, 1)

Selections are supported…

await db.get(
  t.user(u => [u.name]),
  1
)

Selections can have aliases…

await db.get(
  t.user(u => [{ n: u.name }]),
  1
)

// You can omit the array if you don't mind giving
// everything an alias.
await db.get(
  t.user(u => ({ n: u.name })),
  1
)

Selections can contain function calls…

await db.get(
  t.user(u => ({
    name: pg.upper(u.name),
  })),
  1
)

To select all but a few columns…

await db.get(t.user.omit('id', 'password'), 1)

Inner joins

// Find all books with >= 100 likes and also get the author of each.
await db.select(t.author).innerJoin(
  t.book.where(b => b.likes.gte(100)),
  t => t.author.id.eq(t.book.authorId)
)

 

What's planned?

This is a vague roadmap. Nothing here is guaranteed to be implemented soon, but they will be at some point (contributors welcome).

• math operators
• enum types
• domain types
• composite types
• more geometry types
• array-based primary key
• ANY and SOME operators
• transactions
• explicit locking
• views & materialized views
• table inheritance
• window functions
• plugin packages
  • these plugins can do any of:
    • alter your schema
    • seed your database
    • extend the runtime API
  • auto-loading of packages with tusken-plugin-abc or @xyz/tusken-plugin-abc naming scheme
  • add some new commands
    • tusken install (merge plugin schemas into your database)
    • tusken seed (use plugins to seed your database)
• NOTIFY/LISTEN support (just copy pg-pubsub?)
• define Postgres functions with TypeScript
• more shortcuts for common tasks

What could be improved?

This is a list of existing features that aren't perfect yet. If you find a good candidate for this list, please add it and open a PR.

Contributions are extra welcome in these places:

• comprehensive "playground" example
• subquery support is incomplete
  • bug: selectors cannot treat single-column set queries like an array of scalars
• type safety of comparison operators
  • all operators are allowed, regardless of data type
  • see .where methods and is function
• the jsonb type should be generic
  • with option to infer its subtype at build-time from current row data
• missing SQL commands
  • WITH
  • GROUP BY
  • UPDATE
  • MERGE
  • USING
  • HAVING
  • DISTINCT ON
  • INTERSECT
  • CASE
  • etc

tusken's Issues

Tree-shaking doesn't work with Esbuild

evanw/esbuild#1420

We recommend using Rollup when bundling a package that uses Tusken to interact with your Postgres database.

Hasura auth plugin

https://github.com/nhost/hasura-auth/tree/main/migrations 👀

Add `pg.default` placeholder

This tokenizes into DEFAULT, which is a keyword used for inserting/updating a column with the default value for that column. Adding pg.default is useful for expressions that may or may not resolve to the default column value.

Use lateral join when a SetExpression is returned by a selector

Postgres type functions

// A. Cast expression to built-in type
t.text(expr)

// B. Cast primary key to inlined columns
t.post(post => [
  post.id,
  post.text,
  // Map the author ID to a user row. Select the name only.
  t.user(post.author, user => [user.id, user.name]),
])

Exhibit A would replace #3.

Exhibit B would return the following columns:

"id": int4
"text": text
"author.id": int4
"author.name": text

…and Tusken would convert author.id and author.name into author: { id, name } for you.

Blog plugin

https://github.com/facinick/gnippet/tree/master/api/db/migrations 👀

The `numeric` type should accept JS numbers

Currently, it only accepts expressions of type t.numeric and JS strings.

Support spreading a row into a selection mapping

db.select(
  t.tweet(tweet => ({
    ...tweet,
    author: db.get(t.user, tweet.author),
  }))
)

By spreading tweet into the mapping, all columns should be selected and an additional author column is added (which overrides the tweet.author column).

Implementing this should be straight-forward.
Just need to define the ownKeys trap on the proxy handler that tweet uses.

tusken/src/database/selector.ts

Line 17 in 96b4ddd

const columns: any = new Proxy(type, {

Warn when aggregate function is used alongside non-aggregate selection

If an aggregate function is used within a db.select call, you need GROUP BY to include non-aggregate selections.

Current progress

• Generated defineFunction calls use the Aggregate type instead of Output in their type signatures
• TypeScript emits an error when a selection includes both Aggregate and non-aggregate types

CLI autocomplete support with Fig

https://github.com/withfig/autocomplete

Superuser plugin

Write a plugin package that provides superuser functionality, like managing tables, indexes, triggers, roles, etc.

Combining `omit` with `innerJoin` doesn't use fully-qualified column names

In this test, the author and text should be preceded by tweet.

tusken/spec/sql.spec.ts

Lines 56 to 66 in 12f4608

	test('omit column from a joined table', () => {
	expect(
	db
	.select(t.user)
	.innerJoin(t.tweet.omit('id'), ({ user, tweet }) =>
	user.id.equalTo(tweet.author)
	)
	).toMatchInlineSnapshot(
	'SELECT "user".*, author, text FROM "user" INNER JOIN tweet ON "user".id = tweet.author'
	)
	})

JSON array inputs may need explicit cast/stringify

If you're calling a Postgres function that accepts t.json or t.jsonb and you pass a JS array, it will be serialized as a Postgres array (not a JSON array). This is a limitation of node-postgres. It also affects INSERT queries, but the runtime types I'm adding for #14 will fix that in the next alpha version.

Related to brianc/node-postgres#2012

Note that `db.put` does not throw if an update fails

When db.put is used to update one or multiple rows, its promise won't be rejected if one of the updates does nothing because the targeted record doesn't exist.

Array plugin

Tusken plugin that adds Array.prototype methods to Select queries as shorthand for .then(rows => rows.map(...)) etc. Possibly with subquery support too?

UUID v6 plugin

https://gist.github.com/fabiolimace/515a0440e3e40efeb234e12644a6a346

Composite keys

Support tables with a primary key made up of multiple columns

Nested JSON objects/arrays from subqueries

db.select(
  t.tweet(tweet => ({
    // Many-to-one relation
    author: db.get(t.user, tweet.author),
    // Many-to-many relation
    hashtags: db.select(t.hashtag).where(hashtag => hashtag.id.in(tweet.hashtags)),
    // One-to-many relation
    likes: db.select(t.like).where(like => like.tweet.eq(tweet.id)),
  }))
)

That query would produce an array of objects with the following shape:

const result = {
  author: { ... },
  hashtags: [ { ... }, ... ],
  likes: [ { ... }, ... ],
}

Plugin packages

• Plugins found in node_modules are loaded automatically
  • Their name must match a pattern like Babel uses (https://babeljs.io/docs/en/options#name-normalization) but imagine those patterns with tusken instead of babel
• Plugins can alter the generated client, which means they can:
  • Inject runtime code
  • Modify the db, pg, and t objects
  • Modify any TypeScript interface with declaration merging
• Plugins can add their own config options to tusken.config.js
• Plugins can alter the database schema with explicit permission
  • tusken apply brings up a multi-select prompt of unapplied plugins?
• Plugins can add their own database schema for metadata purposes

Umami plugin

https://noted.lol/umami-a-self-hosted-privacy-focused-analytics-platform/
https://github.com/umami-software/umami/blob/master/sql/schema.postgresql.sql

Add helper for converting json array into a set

Whose SQL would be similar to:

FROM json_to_recordset(yourJsonValue -> 'rows') AS rows(columns JSON),
     json_to_record(columns) AS cols(
       "Nombre" TEXT,
       "rutaEsquema" TEXT,
       "TipoDeComponente" TEXT,
       "detalleDelComponente" TEXT)

Consider using "postgres" package

https://github.com/porsager/postgres

Change how the `Database` object is generated

Currently, Rollup cannot treeshake db.xyz access, because ES6 classes are not treeshaked.

Instead of there being a Database class with the generated client having a singleton instance, do the following:

• Move all commands into plugin dependencies of the tusken package, so *default commands* can be tree-shaked.
• Emit database extensions with export const syntax instead of using a subclass, so plugin methods can be treeshaked if not used in a bundle.
• Refactor database state into a POJO stored in a global variable. Then make this POJO the default database context of each database method.
• Now move the database state & methods from ./index.ts to ./database.ts
• Inside the ./index.ts module, do this instead:

  export * as db from './database'
  export * as pg from './functions'
  export * as t from './types'

Vitest helper

Take inspiration from https://github.com/seamapi/ava-postgres

Should `pg` function names be converted to camel case?

So instead of pg.row_to_json, it would be pg.rowToJson 🤔

Stripe sync plugin

https://github.com/supabase/stripe-sync-engine/tree/main/db/migrations

Add `db.list` for getting multiple rows by primary key

We can't just let db.get take an array of keys, because we want to support array-based keys.

// Order is guaranteed to match the `userIds` array.
const rows = await db.list(t.user, userIds)

This is not simply a shorthand method for db.select(...).where(({ pk }) => pk.in(array)), because the result order is guaranteed to match the input array.

Note: When using this method, the primary key will always be included in the resulting objects.

Add `cast` helper

This creates an Expression object that renders the CAST ... AS operator.

cast(20, t.text)

This will require an export const statement for each type in the t namespace, as there needs to be a way to infer the identifier that trails the AS operator when building the query at runtime.

Schema inference

Add a way to infer schema types from a dataset, using the Tusken CLI

Prepared statements

https://www.postgresql.org/docs/current/sql-prepare.html

Prepared statements potentially have the largest performance advantage when a single session is being used to execute a large number of similar statements. The performance difference will be particularly significant if the statements are complex to plan or rewrite, e.g., if the query involves a join of many tables or requires the application of several rules. If the statement is relatively simple to plan and rewrite but relatively expensive to execute, the performance advantage of prepared statements will be less noticeable.

The API would look like this:

const preparedQuery =
  db.select(...)
    .innerJoin(...)
    .innerJoin(...)
    .prepare('unique_name')

// Release from memory
preparedQuery.dealloc()

Note that two separate queries cannot share the same name.

Ideas 🤔

• We could generate a hash that allows automatic naming of prepared statements based on the query.
• We could inject a LRU cache if enabled within tusken.config.ts

Make a documentation site

To start out, we could rip the organization from https://jawj.github.io/zapatos/ 😄

Stop using extract-pg-schema

It accounts for 76% of the CLI's install size (10 of 13MB), and we don't really need it.

ZWS plugin

https://github.com/zws-im/zws/tree/main/prisma/migrations 👀

Job queue plugin

https://github.com/graphile/worker/tree/main/sql 👀

add `pg.array(...dims)` function

SELECT array_fill(1, ARRAY[3]);
SELECT array_fill(1, ARRAY[2,3]);

Add `pg.json_to_record` support

Allow providing a schema object like so…

pg.json_to_record({ a: 1, b: '' }, { a: t.int, b: t.text }).as('row')

…that creates the following SQL…

json_to_record('{"a":1,"b":""}'::json) as row(a int, b text)

Migration generator

Create a plugin package that adds triggers to system tables. These triggers will be used to track added/updated/renamed/deleted columns and such. This event data is used to generate migration scripts (written in TypeScript with Tusken 😄 ). Those migration scripts are used by tusken migrate to generate SQL for graphile-migrate.

For column type changes that require a handwritten expression, you can edit a migration script directly or add a blank one (should we recommend a VS Code plugin that can create a blank migration script from a pre-defined template?).

Type brands for primary/foreign keys

Goal: Extra type safety. Prevent primary keys for one table from being used in a query of another table.

// The generated type of a primary/foreign key
id: t.int4 & t.key<'user.id'>

To convert an external value into a branded key:

// Note: Falsy values are passed through.
const userId = t.user.id(req.searchParams.get('userId'))

Casting like what's shown above doesn't include runtime validation. This is only for type safety.

Feedback plugin

https://github.com/duyet/feedback/tree/main/dashboard/prisma/migrations 👀

Add `pg.distinct`

// SELECT DISTINCT ON(name) FROM "user"
db.select(t.user(user => pg.distinct(user.name)))

// SELECT COUNT(DISTINCT name) FROM "user"
db.select(t.user(user => pg.count(pg.distinct(user.name))))

Give `json_agg` a type parameter to be used when input type is known

This will require the t.json and t.jsonb types to have a type parameter that can narrow the Json type that is used by default in TypeScript result sets.

<T extends t.anyelement>(arg: ColumnRef<T>): Aggregate<T extends t.json ? t.json : t.json<T>>

Multiple databases with one client

If only one database is configured by tusken.config.ts, the current behavior is kept.

db.select(t.user)

If multiple databases are configured like…

export default defineConfig({
  database: ['foo', 'bar'],
})

…then it works like this:

db.foo.select(t.foo.user)
db.bar.select(t.bar.user)

The t.foo.user could be shortened to t.user if and only if db.bar has no user table.