upstash / query

Secondary indexing and query capabilities for Upstash Redis

@upstash/query

Search for Upstash Redis

upstash.com

Note

This project is in the Experimental Stage.

We declare this project experimental to set clear expectations for your usage. There could be known or unknown bugs, the API could evolve, or the project could be discontinued if it does not find community adoption. While we cannot provide professional support for experimental projects, we’d be happy to hear from you if you see value in this project!

@upstash/query offers secondary indexing and search capabilities for Upstash Redis. It is fully managed by Upstash and scales automatically.

This library is tested well but not yet production ready. We are looking for feedback: Please open an issue if you have any questions or suggestions.

Features

• E2E Typesafe: Fully typed API with TypeScript generics to offer the best developer experience.
• Secondary Indexing: Create indexes on your data and query them with a simple API.
• Range Queries: Query your data with range queries. Either numeric or lexicographic.

Quickstart

import { Redis } from "@upstash/redis";
// import { Redis } from "@upstash/redis/cloudflare"; // for Cloudflare Workers
// import { Redis } from "@upstash/redis/fastly"; // for Fastly Compute@Edge
import { Query } from "@upstash/query";

/**
 * Define a custom type for your documents
 */
type User = {
    id: string;
    name: string;
    organization: string;
    email: string;
  };

  /**
   * Initialize the client
   */
  const q = new Query({
    redis: Redis.fromEnv({ automaticDeserialization: false }), // <- important to turn it off as @upstash/query handles deserialization itself
  });

  /**
   * Create your first collection.
   *
   * Please make sure you're passing in a type to take full advantage of @upstash/query
   */
  const users = q.createCollection<User>("users");

  /**
   * Create a searchable index on the collection and specify which terms we are filtering by
   * terms are fully typed as long as you have defined a custom type when creating the collection
   */
  const usersByOrganization = users.createIndex({
    name: "users_by_organization",
    terms: ["organization"],
  });

  const user: User = {
    id: "chronark",
    name: "Andreas Thomas",
    organization: "Upstash",
    email: "[email protected]",
  };
  // Create and store your first user
  await users.set("documentId", user);



  /**
   * Now we can use the previously created index to query by organization
   */
  const upstashEmployees = await usersByOrganization.match({ organization: "Upstash" });
  /**
   * [
   *     {
   *         id: "documentId",
   *         ts: 000, // the timestamp when created or last updated
   *         data: {
   *             id: "chronark",
   *             name: "Andreas Thomas",
   *             organization: "Upstash",
   *             email: "[email protected]"
   *         }
   *     }
   * ]
   */

Documentation

See the documentation for more information.

query's Issues

Query by nested fields does not work

hi! it seems that the library supports creating indexes on nested fields, but in reality I can't make it work.

for example, I have records like this:

{ productId: 'product-1',  data: { name: 'name of the product', productId: 'product-1', score: 1 } }

If I create index on a nested field:

const nestedIndex = productsCollection.createIndex({
        name: "nested_products_ids_index",
        terms: ["data.productId"],
    });

I get 0 results when I query it:

const queryProducts = await productsCollectionIndex.match({
        "data.productId": ['product-1'],
    })

but if I create index using only productId on the top level, it works:

const productsCollectionIndex = productsCollection.createIndex({
        name: "products_ids_index",
        terms: ["productId"],
    });

I get the results. is it supposed to work this way, and nested fields are not supported or is this is a bug? thanks!

Ability to dynamically type terms in createIndex

I want to be able to create a function that dynamically types indexes, however currently I am unable to type terms:

When I hover over terms I get the following type constraint:

Type 'keyof T extends Data ? T : Data[]' is not assignable to type 'Join<NestedPaths<T extends Data ? T : Data>>[]'.

I noticed you are not exporting the Join / NestedPaths types. Is there a better way of achieving this? Am I missing something?

type NestedPaths<T> = T extends string | number | boolean ? [] : {
    [K in Extract<keyof T, string>]: [K, ...NestedPaths<T[K]>];
}[Extract<keyof T, string>];

type Join<T> = T extends [] ? never : T extends [infer F] ? F : T extends [infer F, ...infer R] ? F extends string ? ${F}.${Join<Extract<R, string[]>>} : never : string;