AWS Data API about kysely

[koskimas]

The Driver interface now has these methods

  async beginTransaction(connection: DatabaseConnection): Promise<void> {
    await connection.executeQuery({ sql: 'begin', bindings: [] })
  }

  async commitTransaction(connection: DatabaseConnection): Promise<void> {
    await connection.executeQuery({ sql: 'commit', bindings: [] })
  }

  async rollbackTransaction(connection: DatabaseConnection): Promise<void> {
    await connection.executeQuery({ sql: 'rollback', bindings: [] })
  }

You can override those to create a transaction using the Data API and save the transaction id to the connection object. The same connection is then used throughout the transaction.

Should I be working on this as an external library or a PR to the main library?

Since this dialect won't be easily testable using Kysely's testbed, I think an external library would make more sense. By an external library, I mean just making a package for the dialect. You'd install kysely using something like

npm install kysely kysely-aws-data-api-dialect

or something similar. The dialect package should propably mention kysely as a peer dependency in the package.json and not as a direct dependency to prevent multiple versions of Kysely from being installed.

Not sure if you imagine all dialects being bundled or primarily as an add-on library

I plan to bundle a handful of the most used dialects

[koskimas]

Awesome! Can't wait to see what you come up with.

A small warning though, you'll be working against a moving target for a while. Kysely is still very young, and I'll probably still end up changing the API a little bit before a full launch. I plan to release a 1.0 and write a blog post about it when I feel Kysely is ready for it. We're not far from that, but there are still some things to figure out.

I plan to release Kysely with postgres, mysql and sqlite support. So far only postgres is supported. When I add other databases there's probably going to be some breaking changes.

[thdxr]

Yeah I copied your setup for now but I think I'd eventually setup roll-up to do this for me. Think that'll take care of the .js stuff too

And yes! Please link it. We're going to be writing some guides for our community soon around this library

[koskimas]

Right now DriverConfig is a fixed type that assumes all drivers need host, port, etc ...

That's a good point. A lower level base class could indeed be a good idea. The config given to Kysely.create method also inherits the DriverConfig again making all the host, port etc. properties required.

I wonder if we could simply provide a dialect for the Kysely.create like this:

const db = new Kysely<Database>({
  dialect: new DataApiDialect({
    // Driver config here.
  })
})

and remove the whole DriverConfig interface. That way each dialect could have completely different config. The downside is slightly more complex initialization. What do you think?

I mean, you already CAN give the dialect like that, but currently you also need to provide the config for the Kysely.create method.

I'm also extending the DefaultQueryCompiler to tweak how bindings work and suggest that #bindings be made protected so I can override the appendValue implementation.

Would it be enough to add a protected addbinding method that calls #bindings.push?

Other than that, everything else seems great. I have to say, this is the best Typescript codebase I've ever seen. Sent it around to some people to learn from 😄

Thank you 😊 It really makes me happy to hear that 🍻

[koskimas]

I made the change described in the above message. It simplified things a lot. The new API allows all kinds of dialects to be added with very little limitations. In the new 0.5.0 version the KyselyConfig interface accepted by Kysely.create only has two properties dialect and plugins. clean.

  const db = new Kysely<Database>({
    dialect: new PostgresDialect({
      host: 'localhost',
      database: 'kysely_test',
    })
  })

[thdxr]

Sweet I just got everything working minus transactions yesterday. Updates to DefaultQueryCompiler look good too, will simplify a lot. Will take advantage of these updates today

[koskimas]

Cool. I also removed the Kysely.create function and went back to using the constructor. This time it's final 😅

[thdxr]

Got everything working: https://github.com/serverless-stack/kysely-data-api

Need to write a whole bunch of tests but looking great so far - was so easy to implement this, the abstractions are great.

[koskimas]

I took a look at the AWS data API and It seems that we'll need to further abstract the Driver interface. At the moment, a new transaction is created by simply running a BEGIN SQL command on a normal connection. With the Data API a transaction is created using a separate method call.

I'll add the transaction creation to the Driver class's API so that drivers can implement it however they want.

[thdxr]

Ah that makes sense - no worries about breaking changes. I got a basic implementation working without transactions.

Some thoughts:
Right now DriverConfig is a fixed type that assumes all drivers need host, port, etc. For data api these variables aren't needed and it needs different variables. I worked around this by using the constructor to pass in additional params like this:

export class DataApiDialect implements Dialect {
  private readonly opts: DataApiDialectOpts

  constructor(opts: DataApiDialectOpts) {
    this.opts = opts
  }

  createDriver(config: DriverConfig): Driver {
    return new DataApiDriver(this.opts.driver, config)
  }

I was thinking maybe Dialect and Driver can take a generic (which can default to the current DriverConfig) to specify the type of the config? Or maybe even create an middle abstract class for connection based drivers ConnectionDriver extends Driver<ConnectionDriverConfig> since getDefaultPort() also isn't relevant.

I'm also extending the DefaultQueryCompiler to tweak how bindings work and suggest that #bindings be made protected so I can override the appendValue implementation.

Other than that, everything else seems great. I have to say, this is the best Typescript codebase I've ever seen. Sent it around to some people to learn from 😄

[thdxr]

Awesome thanks for those updates! Should I be working on this as an external library or a PR to the main library? Not sure if you imagine all dialects being bundled or primarily as an add-on library

Personally prefer working inside this codebase so I don't have to create a similar setup

[koskimas]

DefaultQueryCompiler now has these additional methods

  protected get numBindings(): number {
    return this.#bindings.length
  }

  protected appendValue(value: PrimitiveValue): void {
    this.addBinding(value)
    this.append(this.getCurrentParameterPlaceholder())
  }

  protected getCurrentParameterPlaceholder(): string {
    return '$' + this.numBindings
  }

  protected addBinding(binding: any): void {
    this.#bindings.push(binding)
  }

[koskimas]

Nice! I should add a list of 3rd party dialects to the documentation. Is it ok if I link to your package from it?

It seems you used the same structure as Kysely to create both cjs and esm exports. I can't believe this is the best way to do that at the moment 😓 At least I wasn't able to find a better way. Expecially the module-fixup step is horrible.

You need to use the .js suffix for imports for them to work in the esm version though. That's why kysely has another hacky step of running this script to make sure they do.

You can test your package by creating a project with "type": "module" in package.json and adding your package to it. At the moment, you'll get a bunch of errors saying this and that file is not found (because of the missing .js) suffixes.

By the way, I added a PostgresQueryCompiler class yesterday. You could extend that instead of the DefaultQueryCompiler.

[ClearedFram3]

I plan to release Kysely with postgres, mysql and sqlite support. So far only postgres is supported. When I add other databases there's probably going to be some breaking changes.

Everyone seems to be going the support as many as possible route and I can see why. It seems to me that if you were to only support PG you would reduce overhead, workload, and would be deeply integrated giving this library its niche. This article really made me think of it: https://blog.js.wiki/news/2021/wiki-js-3-going-full-postgresql

[koskimas]

@ClearedFram3 The plan is to support the most common ones and provide a clear documented API for third parties to create and maintain other dialects.

Supporting a handful of dialects out of the box isn't such a big deal since Kysely doesn't provide a "greatest common denominator API" like knex for example. With kysely, the generated queries are obvious from the code. For example when you write

db.insertInto('person').values(person).onConflictDoNothing('id')

on any dialect, even the ones that don't support the ON CONFLICT keyword, you still get the query

INSERT INTO "person" ("first_name", "last_name") VALUES ('Foo', 'Bar') ON CONFLICT ("id") DO NOTHING

On mysql you need to use the

db.insertInto('person').ignore().values(person)

which again ~directly translates in to SQL with a INSERT IGNORE INTO "person".

This approach brings it's own challenges and third party dialects may not be able to implement all the features they want, but the code in Kysely remains simple.

Since I'm a big fan of postgres and basically never use anything else if I have the choice, Kysely will probably heavily favour postgres features 😄

[koskimas]

@thdxr I opened a PR in kysely-data-api. sst/kysely-data-api#2

[koskimas]

I think this can now be closed.