GithubHelp home page GithubHelp logo

aiji42 / prisma-data-proxy-alt Goto Github PK

View Code? Open in Web Editor NEW
200.0 7.0 28.0 1.52 MB

This is a library to alternate and self-host the Prisma Data Proxy (cloud.prisma.io)

License: MIT License

Shell 17.96% TypeScript 75.02% JavaScript 5.00% Dockerfile 2.02%

prisma-data-proxy-alt's Introduction

npm version codecov CI

Alternative Prisma Data Proxy

This is a library to alternate and self-host the Prisma Data Proxy (cloud.prisma.io).

In order to deploy your project to edge runtimes (such as Cloudflare Workers or Vercel Edge Functions) and use Prisma, you will need to use the Prisma Data Proxy.
However, at present, instances can only be built in limited areas, and there are also delays caused by cold standby. This is a very stressful problem.

Therefore, we have created a server library to replace Prisma Data Proxy. With it, you are free from stressful limitations. You can deploy it on any platform in any region you like and use any data source you like, such as Supabase or Planetscale.

No changes are required to your prisma client code, just set the DATABASE_URL to the URL you self-hosted with this library.
This is not an official library, but it works the same as Prisma Data Proxy.

Overview

overview

Performance

Using the Alternative Prisma Data Proxy, a significant reduction in clause latency can be expected regardless of the region of the instance. See here for details.

Setup

If you are using @prsima/client v3, install prisma-data-proxy-alt@^1.
The latest library (v2) suports only @prima/client v4.

yarn add prisma-data-proxy-alt

Include prisma schema in your project. The same schema as the client.

cp your_client_project_path/prisma/schema.prisma ./prisma/schema.prisma

Install prisma and @prisma/client.

yarn add -D prisma
yarn add @prisma/client

Launch proxy server

Give environment variables by creating .env, etc.

PRISMA_SCHEMA_PATH=/absolute/path/for/your/schema.prisma
DATABASE_URL={database URL scheme e.g. postgresql://postgres:pass@db:5432/postgres?schema=public}
DATA_PROXY_API_KEY={random string for authentication}
PORT={server port e.g. 3000}
yarn pdp

This will bring up the proxy server, but it must be SSL-enabled to connect from @prisma/client.
So here are the steps to establish a local connection with SSL using docker-compose and https-portal with a self certificate.

Create entrypoint.sh.

#!/bin/sh

yarn install
exec "$@"

Create docker-compose.yml.

version: '3'

services:
  data-proxy:
    image: node:18-bullseye-slim
    working_dir: /app
    ports:
      - "3000:3000"
    entrypoint: /app/entrypoint.sh
    command: yarn pdp
    environment:
      PRISMA_SCHEMA_PATH: /app/for/your/schema.prisma
      DATABASE_URL: your DATABASE_URL
      DATA_PROXY_API_KEY: your DATA_PROXY_API_KEY
      PORT: "3000"
    volumes:
      - ./:/app:cached
      - node_modules:/app/node_modules
  https-portal:
    image: steveltn/https-portal:1
    ports:
      - "443:443"
    environment:
      STAGE: local
      DOMAINS: 'localhost -> http://data-proxy:3000'
    volumes:
      - ./ssl-certs:/var/lib/https-portal

volumes:
  node_modules:
docker-compose up

Now you can connect with data proxy with DATABASE_URL=prisma://localhost?api_key={DATA_PROXY_API_KEY}.

Deploy

GCP Cloud Run

Create Dockerfile

FROM node:16.15-bullseye-slim as base

RUN apt-get update && apt-get install -y tini ca-certificates \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

WORKDIR /app

FROM base as builder

COPY package.json .
COPY yarn.lock .
COPY prisma/schema.prisma ./prisma/schema.prisma

RUN yarn install

RUN yarn prisma generate

FROM base

COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./package.json

ENV PRISMA_SCHEMA_PATH=/app/node_modules/.prisma/client/schema.prisma

USER node

ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["yarn", "pdp"]

Create cloudbuild.yml

steps:
  - name: 'gcr.io/kaniko-project/executor:latest'
    args:
      - --destination=gcr.io/$PROJECT_ID/prisma-data-proxy-alt:$SHORT_SHA
      - --destination=gcr.io/$PROJECT_ID/prisma-data-proxy-alt:latest
      - --cache=true
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args:
      - run
      - deploy
      - prisma-data-proxy-alt
      - --image
      - gcr.io/$PROJECT_ID/prisma-data-proxy-alt:latest
      - --region
      - $_REGION
      - --allow-unauthenticated
      - --set-env-vars
      - DATABASE_URL=$_DATABASE_URL
      - --set-env-vars
      - DATA_PROXY_API_KEY=$_DATA_PROXY_API_KEY
substitutions:
  _REGION: asia-northeast1
  _DATABASE_URL: your_database_url
  _DATA_PROXY_API_KEY: your_api_key

Create a new trigger from the GCP Cloud Build web console and link it to your repository.

Set _REGION, _DATABASE_URL, and _DATA_PROXY_API_KEY in the substitution values.

  • _REGION: The region of deploy target for Cloud Run
  • _DATABASE_URL: Connection URL to your data source (mysql, postgres, etc...)
  • _DATA_PROXY_API_KEY: Arbitrary string to be used when connecting data proxy. e.g. prisma://your.deployed.domain?api_key={DATA_PROXY_API_KEY}
    (do not divulge it to outside parties)

For Client (on your application)

On the client side, generate the Prisma client in data proxy mode --data-proxy. official document

yarn prisma generate --data-proxy

Set the DATABSE_URL from the domain of the server you deployed and the api key (DATA_PROXY_API_KEY) you set for it.

DATABSE_URL=prisma://${YOUR_DEPLOYED_PROJECT_DOMAIN}?api_key=${DATA_PROXY_API_KEY}

Now you can connect to the (alternative) Data Proxy from your application. ๐ŸŽ‰

Contribution

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

LICENCE

This project is licensed under the MIT License - see the LICENSE file for details

prisma-data-proxy-alt's People

Contributors

aiji42 avatar denniske avatar juniorusca avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

prisma-data-proxy-alt's Issues

Improve the Readme

  • back ground color of overview image
  • write server script file name
  • no use the dist directory
  • gcp setup flows

Recieved corrupt message when running locally

Hi, thank you for your work on this project, it's been a real help using Prisma with Deno.

However, when trying to use the proxy I get an error:

Cannot fetch data from service:
error sending request for url (https://localhost:3851/4.7.1/b7b4fb7095922bd7b7c1126f4ffb522c4a94a7c7baa9310b74ef3789231d8756/graphql): error trying to connect: received corrupt message
    at wn.handleRequestError (file:///.../backend/generated/client/runtime/edge-esm.js:80:18717)
    at wn.handleAndLogRequestError (file:///.../backend/generated/client/runtime/edge-esm.js:80:18249)
    at wn.request (file:///.../generated/client/runtime/edge-esm.js:80:18087)
    at async t._request (file:///.../backend/generated/client/runtime/edge-esm.js:97:2022)
    at async file:///.../backend/src/prisma.ts:27:15
Watcher Process finished. Restarting on file change...

First thing I noticed was that it was trying to use https://. Is this perhaps the issue?

I put a console.log in the beforeMiddleware.js to log every request. Requests sent manually via curl to the proxy-alt server log to the console, but when using the Prisma client nothing gets logged, possibly indicating it never actually reached the server. I have a feeling it's expecting an https response and getting an http one, or something along those lines.

I'd like to run the proxy on localhost for integration tests :)

Connection Pooling/Management

Does this library provide external connection polling/management functionality?

On prisma: [https://www.prisma.io/docs/data-platform/data-proxy#connection-pooling-and-connection-management](data proxy docs):

Data Proxy provides connection pooling so that your application can reuse active connections and never exhaust the maximum number of connections that your database allows.

Connection pooling with the Data Proxy utilizes orchestrated Prisma query engine technology to open, maintain, and close connection pools. It helps applications load-balance and scale database requests based on demand.

A Data Proxy connection pool maintains active database connections and ensures that an application reuses them when new requests come in.

After a quick look through the code, I don't see anything explicit about connection pooling/management, but perhaps this is handled under the hood?

If I self-host this prisma data proxy, will I have to use it in conjunction with https://www.prisma.io/docs/guides/performance-and-optimization/connection-management/configure-pg-bouncer as an external connection pooler?

Do not know how to serialize a BigInt

First of all, thanks for sharing this project!

It works great, but I found one issue: when running a postgres query that returns a BigInt column, the query fails with the following error: "Do not know how to serialize a BigInt"

Further digging shows that the error originates in apollo-server-core:

TypeError: Do not know how to serialize a BigInt
    at prettyJSONStringify (/app/node_modules/apollo-server-core/dist/runHttpQuery.js:314:17)
    at processHTTPRequest (/app/node_modules/apollo-server-core/dist/runHttpQuery.js:233:20)

Can you provide more details about the benchmark setup?

Hey ๐Ÿ‘‹ ,

I am from Prisma and we learned about your awesome work today! Thanks for doing this for the open source community. ๐Ÿ™
We have also seen your interesting benchmarks. Can you provide more details about the setup? We would like to reproduce those results and need the schema file and the test data.

This would help us a lot in tracking this down and improving on our end.

Support plannetscale database

Facts:

  • if i use plannetscale, i have to set relationMode = 'prisma' in schema file
  • the current verison of prisma in this project does not support relationMode. I have looked the lockfile, it is 4.0.0, to support relationMode, the minimal version of prisma is 4.8.0

Solution:
Maybe update the prisma version? I have tired change the version in package.json, and then run build, but it occured error when building.

Prisma Client Docker Build errors

Hello,

I have tried using the docker-compose.yml, and Dockerfile example, and none have worked even though the prisma generate command is ran.

Docker container output log

Error: @prisma/client did not initialize yet. Please run "prisma generate" and try to import it again.

In my package.json I have the following versions of Prisma.

"@prisma/client": "^4.7.1",
"prisma": "^4.7.1",

Could the aforementioned Prisma version be causing the issue?

Is Prisma version v4.0.0 the only version that is currently compatible?

Error opening a TLS connection (unable to get local issuer certificate)

Hi,

I followed the documentation and created a docker-compose file with env variables set for the database (Planetscale Tokyo region), prisma schema path, data proxy API key and port.
When running docker compose up, I get the following error:

...
prisma-data-proxy-alt-https-portal-1  | [cont-init.d] done.
prisma-data-proxy-alt-https-portal-1  | [services.d] starting services
prisma-data-proxy-alt-https-portal-1  | [services.d] done.
prisma-data-proxy-alt-data-proxy-1    | [3/4] Linking dependencies...
prisma-data-proxy-alt-data-proxy-1    | [4/4] Building fresh packages...
prisma-data-proxy-alt-data-proxy-1    | success Saved lockfile.
prisma-data-proxy-alt-data-proxy-1    | Done in 19.17s.
prisma-data-proxy-alt-data-proxy-1    | yarn run v1.22.19
prisma-data-proxy-alt-data-proxy-1    | $ /app/node_modules/.bin/pdp
prisma-data-proxy-alt-data-proxy-1    | prisma:info Starting a mysql pool with 9 connections.
prisma-data-proxy-alt-data-proxy-1    | ๐Ÿ”ฎ Alternative Prisma Data Proxy listening on port 3000
prisma-data-proxy-alt-data-proxy-1    | prisma:info Encountered error during initialization:
prisma-data-proxy-alt-data-proxy-1    | prisma:error 
prisma-data-proxy-alt-data-proxy-1    | Error opening a TLS connection: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1914: (unable to get local issuer certificate)
prisma-data-proxy-alt-data-proxy-1    | /app/node_modules/@prisma/client/runtime/index.js:24071
prisma-data-proxy-alt-data-proxy-1    |                 const err = new PrismaClientInitializationError(json.message, this.clientVersion, json.error_code);
prisma-data-proxy-alt-data-proxy-1    |                             ^
prisma-data-proxy-alt-data-proxy-1    | 
prisma-data-proxy-alt-data-proxy-1    | PrismaClientInitializationError: Error opening a TLS connection: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1914: (unable to get local issuer certificate)
prisma-data-proxy-alt-data-proxy-1    |     at LineStream.<anonymous> (/app/node_modules/@prisma/client/runtime/index.js:24071:29)
prisma-data-proxy-alt-data-proxy-1    |     at LineStream.emit (node:events:513:28)
prisma-data-proxy-alt-data-proxy-1    |     at addChunk (node:internal/streams/readable:324:12)
prisma-data-proxy-alt-data-proxy-1    |     at readableAddChunk (node:internal/streams/readable:297:9)
prisma-data-proxy-alt-data-proxy-1    |     at Readable.push (node:internal/streams/readable:234:10)
prisma-data-proxy-alt-data-proxy-1    |     at LineStream._pushBuffer (/app/node_modules/@prisma/client/runtime/index.js:20137:17)
prisma-data-proxy-alt-data-proxy-1    |     at LineStream._transform (/app/node_modules/@prisma/client/runtime/index.js:20131:8)
prisma-data-proxy-alt-data-proxy-1    |     at Transform._write (node:internal/streams/transform:175:8)
prisma-data-proxy-alt-data-proxy-1    |     at writeOrBuffer (node:internal/streams/writable:392:12)
prisma-data-proxy-alt-data-proxy-1    |     at _write (node:internal/streams/writable:333:10) {
prisma-data-proxy-alt-data-proxy-1    |   clientVersion: '4.2.1',
prisma-data-proxy-alt-data-proxy-1    |   errorCode: 'P1011'
prisma-data-proxy-alt-data-proxy-1    | }
prisma-data-proxy-alt-data-proxy-1    | 
prisma-data-proxy-alt-data-proxy-1    | Node.js v18.7.0
prisma-data-proxy-alt-data-proxy-1    | error Command failed with exit code 1.
prisma-data-proxy-alt-data-proxy-1    | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
prisma-data-proxy-alt-data-proxy-1 exited with code 1

Do you happen to know the reason for that error?
I don't really understand what is meant with (unable to get local issuer certificate). Should that be on my local machine, in Docker or is that something on the Planetscale side? How do I create that?

I am on Mac OS Monterey 12.5

I would appreciate your help a lot!

Regards,
Oka

Does this data proxy support `include`?

Does this data proxy support include?

I tried this query:

    const leaderboardRows = await prisma.leaderboard_row.findMany({
        include: {
            profile: true,
        },
        where: {
            leaderboard_id: leaderboardId,
        },
        skip: start - 1,
        take: count,
        orderBy: {
            ['rank']: 'asc',
        },
    });

and got:

  Invalid `prisma.leaderboard_row.findMany()` invocation:
  Cannot return null for non-nullable field leaderboard_row.profile.

Data proxy logs:

prisma:query SELECT 1
prisma:query SELECT "public"."leaderboard_row"."leaderboard_id", "public"."leaderboard_row"."profile_id", "public"."leaderboard_row"."name", "public"."leaderboard_row"."rank", "public"."leaderboard_row"."rating", "public"."leaderboard_row"."last_match_time", "public"."leaderboard_row"."drops", "public"."leaderboard_row"."losses", "public"."leaderboard_row"."streak", "public"."leaderboard_row"."wins", "public"."leaderboard_row"."updated_at", "public"."leaderboard_row"."rank_country" FROM "public"."leaderboard_row" WHERE "public"."leaderboard_row"."leaderboard_id" = $1 ORDER BY "public"."leaderboard_row"."rank" ASC LIMIT $2 OFFSET $3

When I removed the include the query succeeded:

    const leaderboardRows = await prisma.leaderboard_row.findMany({
        // include: {
        //     profile: true,
        // },
        where: {
            leaderboard_id: leaderboardId,
        },
        skip: start - 1,
        take: count,
        orderBy: {
            ['rank']: 'asc',
        },
    });

Fails to retrieve data with JSON type Postgres

Making a request from the client to a table that has a JSON type field throws an error SyntaxError: "[object Object]" is not valid JSON.

Note that a console log at afterMiddleware.ts
below JSON.parse shows the correct data being returned

Any idea how to run it in a Cloudflare Worker ?

Hi ! I'm trying to run a minimal Prisma data proxy in a Cloudflare Worker and was considering doing this using this project.
Does anyone has already achieved something similar or have informations that could help me ?
Thanks :)

Not working in Vercel

Hi the build is successful but it's not working tho
I just copied the examples in the repo

can someone help me how to deploy it properly in vercel thank you!

image

image

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.