GithubHelp home page GithubHelp logo

tada5hi / typeorm-extension Goto Github PK

View Code? Open in Web Editor NEW
189.0 4.0 32.0 9.96 MB

This library provides utitlites to create & drop the database, seed the database and apply URL query parameter(s).

Home Page: https://typeorm-extension.tada5hi.net

License: MIT License

TypeScript 96.75% JavaScript 3.20% Shell 0.05%
typescript database orm json-api json typeorm-mysql typeorm nodejs seed seeder

typeorm-extension's Introduction

Typeorm Extension πŸš€

npm version codecov Master Workflow Known Vulnerabilities Conventional Commits

This is a library to

  • create, drop & seed the (default-) database πŸ”₯
  • manage one or many data-source instances πŸ‘»
  • parse & apply query parameters (extended JSON:API specification & fully typed) to:
    • filter (related) resources according to one or more criteria,
    • reduce (related) resource fields,
    • include related resources,
    • sort resources according to one or more criteria,
    • limit the number of resources returned in a response by page limit & offset

Warning This readme includes the documentation for the upcoming version 3. This is the link for the v2.

Table of Contents

Installation

npm install typeorm-extension --save

Documentation

To read the docs, visit https://typeorm-extension.tada5hi.net

Usage

CLI

If you use esm, the executable must be changed from typeorm-extension to typeorm-extension-esm. The following commands are available in the terminal:

  • typeorm-extension db:create to create the database
  • typeorm-extension db:drop to drop the database
  • typeorm-extension seed:run seed the database
  • typeorm-extension seed:create to create a new seeder

If the application has not yet been built or is to be tested with ts-node, the commands can be adapted as follows:

"scripts": {
    "db:create": "ts-node ./node_modules/typeorm-extension/bin/cli.cjs db:create",
    "db:drop": "ts-node ./node_modules/typeorm-extension/bin/cli.cjs db:drop",
    "seed:run": "ts-node ./node_modules/typeorm-extension/bin/cli.cjs seed:run",
    "seed:create": "ts-node ./node_modules/typeorm-extension/bin/cli.cjs seed:create"
}

To test the application in the context of an esm project, the following adjustments must be made:

  • executable ts-node to ts-node-esm
  • library path cli.cjs to cli.mjs

Read the Seeding Configuration section to find out how to specify the path, for the seeder- & factory-location.

CLI Options

Option Commands Default Description
--root or -r db:create, db:drop, seed:create & seed:run process.cwd() Root directory of the project.
--dataSource or -d db:create, db:drop & seed:run data-source Name (or relative path incl. name) of the data-source file.
--synchronize or -s db:create & db:drop yes Synchronize the database schema after database creation. Options: yes or no.
--initialDatabase db:create undefined Specify the initial database to connect to. This option is only relevant for the postgres driver, which must always to connect to a database. If no database is provided, the database name will be equal to the connection user name.
--name seed:create & seed:run undefined Name (or relative path incl. name) of the seeder.
--preserveFilePaths db:create, db:drop, seed:create & seed:run false This option indicates if file paths should be preserved and treated as if the just-in-time compilation environment is detected.

CLI Examples

Database Create

ts-node ./node_modules/typeorm-extension/bin/cli.cjs db:create  -d src/data-source.ts

Database Drop

ts-node ./node_modules/typeorm-extension/bin/cli.cjs db:drop  -d src/data-source.ts

Seed Run

ts-node ./node_modules/typeorm-extension/bin/cli.cjs seed:run  -d src/data-source.ts

Seed Run Explicit

ts-node ./node_modules/typeorm-extension/bin/cli.cjs seed:run  -d src/data-source.ts --name src/database/seeds/user.ts

Seed Create

ts-node ./node_modules/typeorm-extension/bin/cli.cjs seed:create  --name src/database/seeds/user.ts

Database

An alternative to the CLI variant, is to create the database in the code base during the runtime of the application. Therefore, provide the DataSourceOptions for the DataSource manually, or let it be created automatically:

Create

Example #1

import { DataSource, DataSourceOptions } from 'typeorm';
import { createDatabase } from 'typeorm-extension';

(async () => {
    const options: DataSourceOptions = {
        type: 'better-sqlite',
        database: 'db.sqlite'
    };

    // Create the database with specification of the DataSource options
    await createDatabase({
        options
    });

    const dataSource = new DataSource(options);
    await dataSource.initialize();
    // do something with the DataSource
})();

Example #2

import {
    buildDataSourceOptions,
    createDatabase
} from 'typeorm-extension';

(async () => {
    const options = await buildDataSourceOptions();

    // modify options

    // Create the database with specification of the DataSource options
    await createDatabase({
        options
    });

    const dataSource = new DataSource(options);
    await dataSource.initialize();
    // do something with the DataSource
})();

Example #3

It is also possible to let the library automatically search for the data-source under the hood. Therefore, it will search by default for a data-source.{ts,js} file in the following directories:

  • {src,dist}/db/
  • {src,dist}/database
  • {src,dist}
import { createDatabase } from 'typeorm-extension';

(async () => {
    // Create the database without specifying it manually
    await createDatabase();
})();

To get a better overview and understanding of the createDatabase function, check out the documentation.

Drop

Example #1

import {
    DataSource,
    DataSourceOptions
} from 'typeorm';
import { dropDatabase } from 'typeorm-extension';

(async () => {
    const options: DataSourceOptions = {
        type: 'better-sqlite',
        database: 'db.sqlite'
    };

    // Drop the database with specification of the DataSource options
    await dropDatabase({
        options
    });
})();

Example #2

import {
    buildDataSourceOptions,
    dropDatabase
} from 'typeorm-extension';

(async () => {
    const options = await buildDataSourceOptions();

    // modify options

    // Drop the database with specification of the DataSource options
    await dropDatabase({
        options
    });
})();

Example #3

It is also possible to let the library automatically search for the data-source under the hood. Therefore, it will search by default for a data-source.{ts,js} file in the following directories:

  • {src,dist}/db/
  • {src,dist}/database
  • {src,dist}
import { dropDatabase } from 'typeorm-extension';

(async () => {
    // Drop the database without specifying it manually
    await dropDatabase();
})();

To get a better overview and understanding of the dropDatabase function, check out the documentation.

Instances

Single

The default DataSource instance can be acquired, by not providing any alias at all or using the key default. If no DataSource instance or DataSourceOptions object is deposited initially the method will attempt to locate and load the DataSource file and initialize itself from there.

import { useDataSource } from 'typeorm-extension';

(async () => {
    const dataSource : DataSource = await useDataSource();
})();

Reference(s):

Multiple

It is also possible to manage multiple DataSource instances. Therefore, each additional DataSource must be registered under a different alias. This can be done by either setting the DataSource instance or the DataSourceOptions object for the given alias.

import { DataSource, DataSourceOptions } from 'typeorm';
import { setDataSource, useDataSource } from 'typeorm-extension';

(async () => {
    const secondDataSourceOptions : DataSourceOptions = {
        // ...
    };

    const dataSource = new DataSource(secondDataSourceOptions);
    setDataSource(dataSource, 'second');

    const instance : DataSource = await useDataSource('second');
})();

Reference(s):

Seeding

Seeding the database is fairly easy and can be achieved by following the steps below:

  • Configuration: Specify the seed and factory location by path or object.
  • Entity: Define one or more entities.
  • Factory (optional): Define a factory for each entity for which data should be automatically generated.
  • Seed: Define one or more seed classes to populate the database with an initial data set or generated data by a factory.
  • Execute: Run the seeder(s) with the CLI or in the code base.

Configuration

Seeder paths are configured as glob patterns, making it easy to match all the factory/seeder files in your project without configuration effort:

  • use * to match anything expect slashes and hidden files
  • use ** to match zero or more directories
  • use comma separate values between {} to match against a list of options

Check out the glob documentation for other supported pattern features. It is important to use the posix/unix path separator (/) because the Windows path separator (\) is used to match paths with literal global pattern characters.

The seeder- & factory-location, can be specified via:

  • environment variable(s)
  • extended data-source.ts file
  • runSeeder(s) method options parameter, in case of a direct code base usage

The following values are assumed by default:

  • factory path: src/database/factories/**/*{.ts,.js}
  • seed path: src/database/seeds/**/*{.ts,.js}

Note: When seeder paths are configured as glob patterns, the paths are resolved and sorted in alphabetical order using filenames. This helps to ensure that the seeders are executed in the correct order.

It is possible to define that a seeder is only executed once. This can either be set globally using the seedTacking option or locally using the track property of a seeder class.

data-source.ts

import { DataSource, DataSourceOptions } from 'typeorm';
import { SeederOptions } from 'typeorm-extension';

const options: DataSourceOptions & SeederOptions = {
    type: 'better-sqlite',
    database: 'db.sqlite',

    seeds: ['src/database/seeds/**/*{.ts,.js}'],
    seedTracking: false,
    factories: ['src/database/factories/**/*{.ts,.js}'],
};

export const dataSource = new DataSource(options);

runSeeder(s)

import { DataSource, DataSourceOptions } from 'typeorm';
import { runSeeders, SeederOptions } from 'typeorm-extension';

(async () => {
    const options: DataSourceOptions = {
        type: 'better-sqlite',
        database: 'db.sqlite',
    };

    const dataSource = new DataSource(options);
    await dataSource.initialize();

    runSeeders(dataSource, {
        seeds: ['src/database/seeds/**/*{.ts,.js}'],
        factories: ['src/database/factories/**/*{.ts,.js}']
    });
})();

Entity

To get started, define one or more entities.

user.ts

import {
    Entity,
    PrimaryGeneratedColumn,
    Column
} from 'typeorm';

@Entity()
export class User {
    @PrimaryGeneratedColumn()
    id: number

    @Column()
    firstName: string

    @Column()
    lastName: string

    @Column()
    email: string
}

Factory

To create entities with random data, create a factory for each desired entity. The definition of a factory is optional.

The factory callback provides an instance of the faker library as function argument, to populate the entity with random data.

user.factory.ts

import { setSeederFactory } from 'typeorm-extension';
import { User } from './user';

export default setSeederFactory(User, (faker) => {
    const user = new User();
    user.firstName = faker.name.firstName('male');
    user.lastName = faker.name.lastName('male');
    user.email = faker.internet.email(user.firstName, user.lastName);

    return user;
})

Seed

And last but not least, create a seeder. The seeder can be called by the cli command seed or in the codebase by using the function runSeeder. A seeder class only requires one method, called run and provides the arguments dataSource & factoryManager.

user.seeder.ts

A seeder class must implement the Seeder interface, and could look like this:

import { Seeder, SeederFactoryManager } from 'typeorm-extension';
import { DataSource } from 'typeorm';
import { User } from './user';

export default class UserSeeder implements Seeder {
    /**
     * Track seeder execution.
     *
     * Default: false
     */
    track = false;

    public async run(
        dataSource: DataSource,
        factoryManager: SeederFactoryManager
    ): Promise<any> {
        const repository =  dataSource.getRepository(User);
        await repository.insert([
            {
                firstName: 'Caleb',
                lastName: 'Barrows',
                email: '[email protected]'
            }
        ]);

        // ---------------------------------------------------

        const userFactory = await factoryManager.get(User);
        // save 1 factory generated entity, to the database
        await userFactory.save();

        // save 5 factory generated entities, to the database
        await userFactory.saveMany(5);
    }
}

Execute

Populate the database from the code base:

import { DataSource, DataSourceOptions } from 'typeorm';
import { runSeeders, SeederOptions } from 'typeorm-extension';
import { User } from 'user';

(async () => {
    const options: DataSourceOptions & SeederOptions = {
        type: 'better-sqlite',
        database: 'db.sqlite',
        entities: [User],

        seeds: ['./*.seeder.ts'],
        factories: ['./*.factory.ts']
    };

    const dataSource = new DataSource(options);
    await dataSource.initialize();

    await runSeeders(dataSource);
})();

Populate the database by explicit definitions from the codebase.

import { DataSource, DataSourceOptions } from 'typeorm';
import { runSeeders, SeederOptions } from 'typeorm-extension';
import { User } from 'user';
import UserSeeder from 'user.seeder';
import UserFactory from 'user.factory';

(async () => {
    const options: DataSourceOptions & SeederOptions = {
        type: 'better-sqlite',
        database: 'db.sqlite',
        entities: [User],

        seeds: [UserSeeder],
        factories: [UserFactory]
    };

    const dataSource = new DataSource(options);
    await dataSource.initialize();

    await runSeeders(dataSource);
})();

Query

The query submodule enables query parameter (fields, filter, ...) values to be build, parsed & validated. Therefore, the rapiq library is used under the hood.

The query parameter options (allowed, default, ...) are fully typed πŸ”₯ and depend on the (nested-) properties of the target entity passed to the typeorm query builder.

For explanation proposes, two simple entities with a relation between them are declared to demonstrate the usage of the query utils:

import {
    Entity,
    PrimaryGeneratedColumn,
    Column,
    OneToOne,
    JoinColumn
} from 'typeorm';

@Entity()
export class User {
    @PrimaryGeneratedColumn({unsigned: true})
    id: number;

    @Column({type: 'varchar', length: 30})
    @Index({unique: true})
    name: string;

    @Column({type: 'varchar', length: 255, default: null, nullable: true})
    email: string;

    @OneToOne(() => Profile)
    profile: Profile;
}

@Entity()
export class Profile {
    @PrimaryGeneratedColumn({unsigned: true})
    id: number;

    @Column({type: 'varchar', length: 255, default: null, nullable: true})
    avatar: string;

    @Column({type: 'varchar', length: 255, default: null, nullable: true})
    cover: string;

    @OneToOne(() => User)
    @JoinColumn()
    user: User;
}

In this example routup and the plugin @routup/query is used to handle HTTP requests, but there is also a guide available for express.

import { createServer } from 'node:http';
import type { Request, Response } from 'routup';
import { createNodeDispatcher, Router } from 'routup';
import { createHandler, useQuery } from '@routup/query';

import {
    applyQuery,
    useDataSource
} from 'typeorm-extension';

const router = new Router();
router.use(createHandler());

/**
 * Get many users.
 *
 * Request example
 * - url: /users?page[limit]=10&page[offset]=0&include=profile&filter[id]=1&fields[user]=id,name
 *
 * Return Example:
 * {
 *     data: [
 *         {id: 1, name: 'tada5hi', profile: {avatar: 'avatar.jpg', cover: 'cover.jpg'}}
 *      ],
 *     meta: {
 *        total: 1,
 *        limit: 20,
 *        offset: 0
 *    }
 * }
 * @param req
 * @param res
 */
router.get('users', async (req: Request, res: Response) => {
    const dataSource = await useDataSource();
    const repository = dataSource.getRepository(User);
    const query = repository.createQueryBuilder('user');

    // -----------------------------------------------------

    const { pagination } = applyQuery(query, useQuery(req), {
        defaultAlias: 'user',
        fields: {
            // porfile fields can only be included,
            // if the relation 'profile' is included.
            allowed: ['id', 'name', 'profile.id', 'profile.avatar'],
        },
        filters: {
            // porfile.id can only be used as a filter,
            // if the relation 'profile' is included.
            allowed: ['id', 'name', 'profile.id'],
        },
        pagination: {
            // only allow to select 20 items at maximum.
            maxLimit: 20
        },
        relations: {
            allowed: ['profile']
        },
        sort: {
            // profile.id can only be used as sorting key,
            // if the relation 'profile' is included.
            allowed: ['id', 'name', 'profile.id']
        },
    });

    // -----------------------------------------------------

    const [entities, total] = await query.getManyAndCount();

    return {
        data: entities,
        meta: {
            total,
            ...pagination
        }
    };
});

const server = createServer(createNodeDispatcher(router));
server.listen(80);

Contributing

Before starting to work on a pull request, it is important to review the guidelines for contributing and the code of conduct. These guidelines will help to ensure that contributions are made effectively and are accepted.

License

Made with πŸ’š

Published under MIT License.

typeorm-extension's People

Contributors

ckaeslin avatar dependabot[bot] avatar faloi avatar fossamagna avatar geovannimp avatar guillaumedeslandes avatar inglkruiz avatar mderriey avatar semantic-release-bot avatar shseah601 avatar tada5hi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

typeorm-extension's Issues

`useSeederFactory(Entity).make()` returns Promise

I'm writing the factory for the entity with relations and find that useSeederFactory(Entity).make() returns Promise. It makes the FactoryCallback become the async function while It requires a normal function.
I think make() shouldn't return the Promise.
Did I miss something?
image

Bug: Can't run cli

Versions

  • Node: 20.9.0
  • OS: Linux Mint 21.2 Cinnamon
  • typeorm-extension: 3.1.1

Reproduction

According to this guide I run the command:
ts-node ./node_modules/typeorm-extension/dist/cli/index.js seed
and obtain the error:
Error: Cannot find module './index.js'

In node_modules by path ./node_modules/typeorm-extension/dist/cli/index.js file index.js doesn't exists. Only file index.d.ts.

Best regards, Aleksandr.

Bug: Seed command does not work with compiled seeders

Versions

  • Node: 16.14.2
  • OS: Ubuntu 20.04
  • typeorm: 0.3.6
  • typeorm-extension: 2.1.2

Reproduction

  • Setup a TypeScript project with typeorm and typeorm-extension.
  • Transpile it using tsc or swc.
  • Make sure the dataSource settings are configured to use the compiled files;
seeds: ['dist/seeds/**/*.js'],
  • Run the seed command of the CLI

What is Expected?

The extension should run the seeders as expected.

What is actually happening?

The extension simply ignores the existence of the seeders. I think it has something to do with the fact that the compiled files do not implement the Seeder interface, since it's not supported in plain JavaScript.

Bug: data-source files cannot found via cli:seed command

cli seed command cannot found data-source file in non-default directories. I run command in the following directories

backend(root directory)
β””db
γ€€β””seed
γ€€γ€€β””user.seeder.ts
γ€€β””data-source.ts

I run command with option like this..

typeorm-extension seed -r db
typeorm-extension seed -r ./db

and cli errors
TypeORMError: No connection options were found in any orm configuration files.

Versions

  • Node: 16.13.0
  • OS: mac os 11.5.2
  • typeorm: 0.3.7
  • typeorm-extension: 2.1.7

Reproduction

Additional Details

Steps to reproduce

What is Expected?

What is actually happening?

Feature: support for "extra" connection options (with MySQL)

Is your feature request related to a problem? Please describe.

TypeORM supports passing extra connection options to the underlying driver. This appears to be unsupported by typeorm-extension's simplified view of connection options.. The result is that connection methods such as Unix sockets (see below config) that work fine with TypeORM are unsupported by TypeORM-Extension. This is significant for me because Unix sockets are the primary way to securely to connect to Google Cloud SQL instances from other Google services like Cloud Run (link).

Describe the solution you'd like

Add support for extra connection options (at least for MySQL) so that the below config will be supported.
See here and here for the current implementation in TypeORM.

Describe alternatives you've considered

The only alternative I can think of is to connect via host/port. This is vastly more complicated since the Google-provided mechanism (Unix sockets) makes security simple & transparent.

Additional context

Here is an example of a TypeORM config that works fine with the core library but fails with TypeORM-Extension:

{
  "type": "mysql",
  "extra": {
    "socketPath": "/var/mysqld/mysqld.sock" # for Google it would be: /cloudsql/<project>:<region>:<database>
  },
  "username": "myuser",
  "password": "mypass",
  "database": "mydb"
}

Feature: Standalone DB check if exists

Is your feature request related to a problem? Please describe.

No issue, just a nice to have

Describe the solution you'd like

Something to check if the DB exists and create if it doesn't but a standalone function. Currently calling the createDatabase function where the ifNotExist option is true but would be nice to skip calling the create DB if it already exists.

Describe alternatives you've considered

None

Additional context

Just something that would be nice.

Feature: option to track already executed seeders

Is your feature request related to a problem?

The seed command is not idempotent.

Describe the solution you'd like

It would be nice to have the option to track which seeders have been already executed, so running the seed command won't execute them again. Mostly the same mechanism that TypeORM has for migrations.

Bug: Synchronize not working after DB creation, just times out

Versions

  • Node: Node 16
  • OS: Linux (Azure App service)

Reproduction

Tried to set both synchronisation options to true but has 0 effect, tried increasing timeouts but still no effect. Not sure what else I can do?
Only workaround is to use this to create the database and use Typeorm's initialize function to sync the tables but it's not the greatest solution.

Also locally it works perfectly, syncs everything etc but not on the azure side.

await createDatabase({
        ifNotExist: true,
        synchronize: true,
        options: {
	        synchronize: true,
	        logging: true,
	        logger: 'file',
	        options: {
		        encrypt: true,
	        },
	        type: 'mssql',
	        host: process.env.DATABASE_HOST ?? 'localhost',
	        username: process.env.DATABASE_USERNAME ?? 'sa',
	        password: process.env.DATABASE_PASSWORD ?? 'Password01!',
	        database: `db-${site}`,
	        extra: (process.env.ENVIRONMENT === 'localhost') ? {
		        trustServerCertificate: true,
	        } : {},
	        entities: [...entities],
        },
})

Steps to reproduce

Not sure if it's only an azure thing but deploy to app service and then do the create database function with entities etc...

What is Expected?

Tables should be synchronised once DB is created

What is actually happening?

Timeout occurs even if request & connections timeouts have been increased. As if it's not using those values.

Feature: Add domain name to mssql connection options

Hello! This package seems perfect for a use case I am having problems with. The problem is that the package doesn't pass along domain name in the connection options to the database driver, which prevents me from using our domain controlled SQL accounts.

I suggest adding the domain name parameter to the passed arguments for at least the MSSQL database driver.

Problem with db:create when I use url in ormconfig.json for postgres

Hi!

If I use a url for postgres in ormconfig.json I get a problem.

ormconfig.json

{
   "type": "postgres",
   "url": "postgres://postgres:password@localhost:9000/test",
   "synchronize": true,
   "logging": true,
   "entities": [
      "src/entity/**/*.ts"
   ],
   "migrations": [
      "src/migration/**/*.ts"
   ],
   "subscribers": [
      "src/subscriber/**/*.ts"
   ],
   "cli": {
      "entitiesDir": "src/entity",
      "migrationsDir": "src/migration",
      "subscribersDir": "src/subscriber"
   }
}

And i do

$ npx typeorm-extension db:create

I get

typeorm-extension db:create

Create database.

ΠžΠΏΡ†ΠΈΠΈ:
  -h, --help             ΠŸΠΎΠΊΠ°Π·Π°Ρ‚ΡŒ ΠΏΠΎΠΌΠΎΡ‰ΡŒ                           [Π±ΡƒΠ»Π΅Π²Ρ‹ΠΉ Ρ‚ΠΈΠΏ]
  -r, --root             Path to your typeorm config file
                            [ΠΏΠΎ ΡƒΠΌΠΎΠ»Ρ‡Π°Π½ΠΈΡŽ: "/home/slavav/Projects/typeorm-test"]
  -c, --connection       Name of the connection on which run a query.
                                                       [ΠΏΠΎ ΡƒΠΌΠΎΠ»Ρ‡Π°Π½ΠΈΡŽ: "default"]
  -f, --config           Name of the file with connection configuration.
                                                     [ΠΏΠΎ ΡƒΠΌΠΎΠ»Ρ‡Π°Π½ΠΈΡŽ: "ormconfig"]
  -s, --synchronize      Create database schema for all entities.
                                [возмоТности: "yes", "no"] [ΠΏΠΎ ΡƒΠΌΠΎΠ»Ρ‡Π°Π½ΠΈΡŽ: "yes"]
      --initialDatabase  Specify the initial database to connect to.
  -v, --version          ΠŸΠΎΠΊΠ°Π·Π°Ρ‚ΡŒ Π½ΠΎΠΌΠ΅Ρ€ вСрсии                     [Π±ΡƒΠ»Π΅Π²Ρ‹ΠΉ Ρ‚ΠΈΠΏ]

error: database "test" does not exist
    at Parser.parseErrorMessage (/home/slavav/Projects/typeorm-test/node_modules/pg-protocol/dist/parser.js:287:98)
    at Parser.handlePacket (/home/slavav/Projects/typeorm-test/node_modules/pg-protocol/dist/parser.js:126:29)
    at Parser.parse (/home/slavav/Projects/typeorm-test/node_modules/pg-protocol/dist/parser.js:39:38)
    at Socket.<anonymous> (/home/slavav/Projects/typeorm-test/node_modules/pg-protocol/dist/index.js:11:42)
    at Socket.emit (node:events:390:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:199:23) {
  length: 89,
  severity: 'FATAL',
  code: '3D000',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'postinit.c',
  line: '877',
  routine: 'InitPostgres'
}

If I use ormconfig.json like

{
   "type": "postgres",
   "host": "localhost",
   "port": 9000,
   "username": "postgres",
   "password": "password",
   "database": "test",
   "synchronize": true,
   "logging": true,
   "entities": [
      "src/entity/**/*.ts"
   ],
   "migrations": [
      "src/migration/**/*.ts"
   ],
   "subscribers": [
      "src/subscriber/**/*.ts"
   ],
   "cli": {
      "entitiesDir": "src/entity",
      "migrationsDir": "src/migration",
      "subscribersDir": "src/subscriber"
   }
}

Everything is ok.

Bug: Duplicate shebang lines in v3.1.0

Versions

  • Node: 18.16.0
  • OS: W11 WSL2 - Ubuntu 22.04.3 LTS

Reproduction

Additional Details When updating to typeorm-extension 3.1.0 and using the CLI commands from `node_modules/typeorm-extension/bin/cli.cjs`, i get the exception:
node_modules/typeorm-extension/bin/cli.cjs:2
#!/usr/bin/env node
^

SyntaxError: Invalid or unexpected token

I can see in the cli.cjs file that the shebang line is duplicated. The beginning of the file looks like this:

#!/usr/bin/env node
#!/usr/bin/env node
'use strict';

require('reflect-metadata');
...

Steps to reproduce

  • install typeorm-extension v3.1.0
  • run a cli command like ts-node --project ./tsconfig.json ../node_modules/typeorm-extension/bin/cli.cjs seed:run -d <PATH TO DATASOURCE>

What is Expected?

The command will execute successfully

What is actually happening?

An exception is thrown because the shebang characters (#!) is only allowed on the first line.

Postgres createDatabase ifNotExists leaks connection

const existResult = await executeSimplePostgresQuery(connection, existQuery, false);

You can see that the query to check if the database exists is told not to close the connection (withendConnection set to false), but if the result of the query indicates the database already exists we early return with Promise.resolve() without closing the connection.

I observed this when using createDatabase in a test suite. The symptom was that jest would finish running tests but hang and never exit. If I removed the createDatabase call it worked fine. I was able to hackily work around this by calling createDatabase with ifNotExist set to false and just using a try-catch for the error thrown by the postgres driver, e.g.:

try {
  await createDatabase({ ifNotExist: false, options: AppDataSource.options });
} catch (e) {
  console.log('Error creating test database:', e);
}

Not the end of the world, but would be nice not to have to do this.

TSC Issues

Hey! Thanks for making this work, it's quite useful. However, I'm not sure if I will continue to be able to, as when I run tsc in my project, your library throws some errors as shown below. Any idea if you could fix it up? Changing the args type to any fixes it, but that's not really great.

Errors:

node_modules/typeorm-extension/dist/cli/commands/database/create.d.ts:20:5 - error TS2416: Property 'handler' in type 'DatabaseCreateCommand' is not assignable to the same property in base type 'CommandModule<{}, {}>'.
  Type '(args: DatabaseCreateArguments, exitProcess?: boolean | undefined) => Promise<void>' is not assignable to type '(args: { [argName: string]: unknown; _: (string | number)[]; $0: string; }) => void'.
    Types of parameters 'args' and 'args' are incompatible.
      Type '{ [argName: string]: unknown; _: (string | number)[]; $0: string; }' is missing the following properties from type 'DatabaseCreateArguments': root, connection, config, synchronize

20     handler(args: DatabaseCreateArguments, exitProcess?: boolean): Promise<void>;
       ~~~~~~~

node_modules/typeorm-extension/dist/cli/commands/database/drop.d.ts:17:5 - error TS2416: Property 'handler' in type 'DatabaseDropCommand' is not assignable to the same property in base type 'CommandModule<{}, {}>'.
  Type '(args: DatabaseDropArguments, exitProcess?: boolean | undefined) => Promise<void>' is not assignable to type '(args: { [argName: string]: unknown; _: (string | number)[]; $0: string; }) => void'.
    Types of parameters 'args' and 'args' are incompatible.
      Type '{ [argName: string]: unknown; _: (string | number)[]; $0: string; }' is missing the following properties from type 'DatabaseDropArguments': root, connection, config

17     handler(args: DatabaseDropArguments, exitProcess?: boolean): Promise<void>;
       ~~~~~~~


Found 2 errors.

Thanks!

Feature: npm/yarn workspace support

Is your feature request related to a problem? Please describe.

When working within a NPM workspace the path to node_modules usually isn't ./node_modules

Describe the solution you'd like

It would be nice if typeorm-extension (or rather NPM) could figure out where node_modules are. So all we have to do is

{
  "scripts": {
    "seed": "typeorm-extension seed:run -d src/database/data-source.ts"
  }
}

or just

{
  "scripts": {
    "typeorm-extension": "typeorm-extension-{commonjs, esm}"
  }
}

This would also simplify usage when using a TypeScript data source.

Describe alternatives you've considered

TypeORM solved this by providing typeorm-ts-node-commonjs and typeorm-ts-node-esm https://typeorm.io/using-cli

"scripts": {
    ...
    "typeorm": "typeorm-ts-node-commonjs"
}
npm run typeorm migration:run -- -d path-to-datasource-config

Feature: run migrations with transaction mode specified in data source options

Is your feature request related to a problem? Please describe.

Using createDatabase with DataSourceOptions that specify a value for migrationsTransactionMode ignores that value and runs with the default mode, which is to create a single transaction for all migrations, which causes issues on our end.

Describe the solution you'd like

When the library runs migrations on the database, it'd be great if it could honour the migrationsTransactionMode property if specified.

Describe alternatives you've considered

The workaround I'm using at the moment is to disable synchronization when using createDatabase (which is enabled by default), then new-up a DataSource manually and run the migrations.

Code I'd like to be able to use:

import { createDatabase, dropDatabase } from 'typeorm-extension'
import type { PostgresConnectionOptions } from 'typeorm/driver/postgres/PostgresConnectionOptions'

const dataSourceOptions: PostgresConnectionOptions = {
  [...],
  migrationsTransactionMode: 'each',
}

await createDatabase({
  options: dataSourceOptions,
  initialDatabase: 'my-test-database',
})

Code I'm currently using:

import { createDatabase, dropDatabase } from 'typeorm-extension'
import type { PostgresConnectionOptions } from 'typeorm/driver/postgres/PostgresConnectionOptions'
import { DataSource } from 'typeorm'

const dataSourceOptions: PostgresConnectionOptions = {
  [...],
  migrationsTransactionMode: 'each',
}

await createDatabase({
  options: dataSourceOptions,
  initialDatabase: 'my-test-database',
  // Explicitly disable synchronization
  synchronize: false,
})

// Run migrations manually
const migrationsDataSource = new DataSource(options)
await migrationsDataSource.initialize()
await migrationsDataSource.runMigrations({ transaction: dataSourceOptions.migrationsTransactionMode })
await migrationsDataSource.destroy()

Additional context

N/A

Bug: can't find files in relative path when use in Windows OS

Versions

  • Node: 18.13.0
  • OS: Windows 11

Reproduction

Additional Details

Steps to reproduce

Use typeorm-extension in windows with relative path

What is Expected?

Import seeds and factories

What is actually happening?

typeorm-extension can not find files for seeds and factories

i managed to fixed it by change the path from windows like to unix like as you guys can see below

// eslint-disable-next-line @typescript-eslint/no-var-requires
const os = require('os')

let seedsPath = resolve(__dirname, '../../database/seeders/*{.ts,.js}')
let factoriesPath = resolve(__dirname, '../../database/factories/*{.ts,.js}')

if (os.platform() === 'win32') {
  seedsPath = seedsPath.replace(/\\/g, '/')
  factoriesPath = factoriesPath.replace(/\\/g, '/')
}

Bug: CLI not recognizing data-source files if they have a different name

Versions

  • Node: 8.12.2
  • OS: Ubuntu 20.04

Reproduction

Steps to reproduce

I'm using NestJs.

Configure your package.json like this:

"seed": "ts-node ./node_modules/typeorm-extension/dist/cli/index.js seed -- -d src/database/database.config.ts"

Add a database.config.ts under src/database in your project with the following code:

import { DataSource, DataSourceOptions } from 'typeorm';
import { ConfigService } from '@nestjs/config';
import { config } from 'dotenv';
import { SeederOptions } from 'typeorm-extension';

config();

const configService = new ConfigService();

const options: DataSourceOptions & SeederOptions = {
  type: 'mysql',
  host: configService.get<string>('MYSQL_HOST'),
  port: configService.get<number>('MYSQL_PORT'),
  username: configService.get<string>('MYSQL_USER'),
  password: configService.get<string>('MYSQL_PASSWORD'),
  database: configService.get<string>('MYSQL_DB_NAME'),
  entities: ['dist/**/*.entity{.ts,.js}'],
  migrations: ['dist/migrations/**/*{.ts,.js}'],
  timezone: 'Z',
  logging: true,
  migrationsRun: true,
  synchronize: false,
  ssl: {
    rejectUnauthorized: false,
  },
  seeds: ['src/database/seeds/**/*{.ts,.js}'],
  factories: ['src/database/factories/**/*{.ts,.js}'],
};

export const AppDataSource = new DataSource(options);

Now, if you run the seed script the following error will occur:

TypeORMError: No connection options were found in any orm configuration files.
    at ConnectionOptionsReader.all (/home/jph/zenbit/HiveIn_be/node_modules/src/connection/ConnectionOptionsReader.ts:46:19)
    at async ConnectionOptionsReader.get (/home/jph/zenbit/HiveIn_be/node_modules/src/connection/ConnectionOptionsReader.ts:58:28)

What is Expected?

It would be expected to connect with the database, if I change the file to data-source.ts it will find the connection file.

What is actually happening?

Can't run the CLI script if I have a configuration file with a name that isn't data-source.ts

Feature: Option to disable SQL Queries in the console/logs

Is your feature request related to a problem? Please describe.

No issue/problem, just something that would be nice to have :)

Describe the solution you'd like

I have looked around and haven't managed to find anything in the docs but an option would be great to disable SQL jibber jabber on the console/logs.

Describe alternatives you've considered

None

Additional context

Just to clear up the console/logs during development

Bug: Listener aren't triggered

Versions

  • Node: 16.15
  • OS: Ubuntu

Reproduction

I have a BeforeInsert listener on a function to do something on my entity before insert, and when I use the seed command the function with the listener isn't triggered.

Steps to reproduce

Add a BeforeInsert listener on a entity function and execute the seed command.

What is Expected?

The function content is execute when you trying to save a new entity with the seed command.

What is actually happening?

The listener isn't triggered.

Feature: sort filenames when defining a pattern.

Is your feature request related to a problem? Please describe.

1st thanks for this useful tool. I would like to ask/request if it's possible to sort the seed files when defining a seed pattern.

Describe the solution you'd like

I'm thinking about naming the file starting with numbers so that locter package or this package after resolving the files, sorts by filename.

Imaging I have my seeders named 0_token-mint.seed.ts and 1_token-price.seed.ts therefore they run sequentially cuz token price depends on token mint. I think this should work along with parallelExecution: false and be documented.

Describe alternatives you've considered

I could order my seeds using an array but I found more Dev friendly to use filenames.

Additional context

N/A

Bug: createDatabase does not throw exception when configuration is wrong

Versions

  • Node: 14.9.3
  • OS: MacOS 12.4

Reproduction

Given the following snippet where the database credentials are wrong

        try {
          const dbConfig: any = {
            type: 'mysql',
            host: 'localhost',
            port: 3306,
            database: 'mydatabasename',
            username: 'root',
            password: 'mypassword',
          };

          await createDatabase({
            options: dbConfig,
          });

        } catch (err) {
          console.log('error is', err);
        }
      }

Will hag forever as no exception is thrown nor a result is returned, it just crashes

Screenshot 2022-06-24 at 14 55 40

If I set the correct credentials, it properly creates the database

Additional Details

Steps to reproduce

What is Expected?

An exception is thrown if the connection details are wrong

What is actually happening?

The process crashes without hitting any catch clause

Bug: Seeder crash

Versions

  • Node: v16.19.0
  • OS: macOS Monterey (Intel)
  • lib version: 2.5.2

Steps to reproduce

Configure a library for seeding.

import 'reflect-metadata'
import { DataSource } from 'typeorm'

import { runSeeders } from 'typeorm-extension'
import options from './dsConfig'

const dataSource = new DataSource(options)
console.log('Running Seeder!')

dataSource.initialize().then(async () => {
  await dataSource.synchronize(true)
  await runSeeders(dataSource)
  process.exit()
})

What is Expected?

Database is seeded

What is actually happening?

Crash:

/Users/stanislas/Documents/Projects/qanoune/backend/node_modules/typeorm-extension/dist/seeder/module.js:97
    const clazz = new seeder();
                  ^
TypeError: seeder is not a constructor

Bug: runSeeders throws with: The "path" argument is undefined

Versions

  • Node: Node 18
  • OS: AWS lambda

Reproduction

When executing runSeeders inside an AWS lambda function the following error is thrown:

{
  "errorType": "TypeError",
  "errorMessage": "The \"path\" argument must be of type string or an instance of URL. Received undefined",
  "trace": [
    "TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined",
    "    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)",
    "    at new NodeError (node:internal/errors:405:5)",
    "    at Object.fileURLToPath (node:internal/url:1401:11)",
    "    at Object.<anonymous> (/node_modules/locter/node_modules/glob/src/index.ts:218:13)",
    "    at Module._compile (node:internal/modules/cjs/loader:1256:14)",
    "    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)",
    "    at Module.load (node:internal/modules/cjs/loader:1119:32)",
    "    at Module._load (node:internal/modules/cjs/loader:960:12)",
    "    at Module.require (node:internal/modules/cjs/loader:1143:19)",
    "    at require (node:internal/modules/cjs/helpers:121:18)",
    "    at _tryRequire (file:///var/runtime/index.mjs:1039:14)",
    "    at async _loadUserApp (file:///var/runtime/index.mjs:1055:16)",
    "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:21)",
    "    at async start (file:///var/runtime/index.mjs:1256:23)",
    "    at async file:///var/runtime/index.mjs:1262:1"
  ]
}

I must admit I'm at a loss here. I have no idea why this is failing in an AWS lambda context.

I can run the seed just fine via typeorm-extension/bin/cli.cjs (not using the lambdaHandler)

Additional Details Lambda function:
import { runSeeders } from 'typeorm-extension';
import { AppDataSource } from './myDataSource';

export const lambdaHandler = async () => {
    if (!AppDataSource.isInitialized) {
        await AppDataSource.initialize();
    }

    await runSeeders(AppDataSource);

    return { statusCode: 200, body: 'Seeding completed successfully.' };
};

// myDataSource
export const AppDataSourceOptions: DataSourceOptions & SeederOptions = {
    type: 'postgres',
    ...
    // the error persists with or without seeds and factories props, so I'm thinking it's unrelated
}

export const AppDataSource = new DataSource(AppDataSourceOptions);

Steps to reproduce

  1. Clone https://github.com/snebjorn/typeorm-ext-seed
  2. Install Node, SAM CLI and Docker as described in the README
  3. in the root of the repo run:
    • sam build
    • docker compose up
    • sam local invoke HelloWorldFunction
  4. Observe the "path argument" error

What is Expected?

runSeeders should work.

What is actually happening?

$ sam local invoke HelloWorldFunction
Invoking app.lambdaHandler (nodejs18.x)
Local image is up-to-date
Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.

Mounting ~\sam-app\.aws-sam\build\HelloWorldFunction as /var/task:ro,delegated, inside runtime container
START RequestId: d36f07db-c785-4381-af32-f6bb168c4730 Version: $LATEST
2023-08-24T08:26:25.873Z        undefined       ERROR   Uncaught Exception      {"errorType":"TypeError","errorMessage":"The \"path\" argument must be of type string or an instance of URL. Received undefined","code":"ERR_INVALID_ARG_TYPE","stack":["TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined","    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)","    at new NodeError (node:internal/errors:405:5)","    at Object.fileURLToPath (node:internal/url:1401:11)","    at Object.<anonymous> (/Temp/tmpw3ob8r4s/node_modules/locter/node_modules/glob/src/index.ts:218:13)","    at Module._compile (node:internal/modules/cjs/loader:1256:14)","    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)","    at Module.load (node:internal/modules/cjs/loader:1119:32)","    at Module._load (node:internal/modules/cjs/loader:960:12)","    at Module.require (node:internal/modules/cjs/loader:1143:19)","    at require (node:internal/modules/cjs/helpers:121:18)","    at _tryRequireFile (file:///var/runtime/index.mjs:976:37)","    at _tryRequire (file:///var/runtime/index.mjs:1026:25)","    at _loadUserApp (file:///var/runtime/index.mjs:1055:22)","    at UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:27)","    at start (file:///var/runtime/index.mjs:1256:42)","    at file:///var/runtime/index.mjs:1262:7","    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)"]}
24 Aug 2023 08:26:25,891 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 129 InvokeID=
2023-08-24T08:26:26.666Z        undefined       ERROR   Uncaught Exception      {"errorType":"TypeError","errorMessage":"The \"path\" argument must be of type string or an instance of URL. Received undefined","code":"ERR_INVALID_ARG_TYPE","stack":["TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined","    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)","    at new NodeError (node:internal/errors:405:5)","    at Object.fileURLToPath (node:internal/url:1401:11)","    at Object.<anonymous> (/Temp/tmpw3ob8r4s/node_modules/locter/node_modules/glob/src/index.ts:218:13)","    at Module._compile (node:internal/modules/cjs/loader:1256:14)","    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)","    at Module.load (node:internal/modules/cjs/loader:1119:32)","    at Module._load (node:internal/modules/cjs/loader:960:12)","    at Module.require (node:internal/modules/cjs/loader:1143:19)","    at require (node:internal/modules/cjs/helpers:121:18)","    at _tryRequireFile (file:///var/runtime/index.mjs:976:37)","    at _tryRequire (file:///var/runtime/index.mjs:1026:25)","    at _loadUserApp (file:///var/runtime/index.mjs:1055:22)","    at UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:27)","    at start (file:///var/runtime/index.mjs:1256:42)","    at file:///var/runtime/index.mjs:1262:7","    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)"]}
END RequestId: 3f1cbd2e-9f2e-4dbc-85e2-ce908b03c63e
REPORT RequestId: 3f1cbd2e-9f2e-4dbc-85e2-ce908b03c63e  Init Duration: 0.12 ms  Duration: 1606.12 ms    Billed Duration: 1607 ms        Memory Size: 512 MB     Max Memory Used: 512 MB
{"errorType": "TypeError", "errorMessage": "The \"path\" argument must be of type string or an instance of URL. Received undefined", "trace": ["TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string or an instance of URL. Received undefined", "    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)", "    at new NodeError (node:internal/errors:405:5)", "    at Object.fileURLToPath (node:internal/url:1401:11)", "    at Object.<anonymous> (/Temp/tmpw3ob8r4s/node_modules/locter/node_modules/glob/src/index.ts:218:13)", "    at Module._compile (node:internal/modules/cjs/loader:1256:14)", "    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)", "    at Module.load (node:internal/modules/cjs/loader:1119:32)", "    at Module._load (node:internal/modules/cjs/loader:960:12)", "    at Module.require (node:internal/modules/cjs/loader:1143:19)", "    at require (node:internal/modules/cjs/helpers:121:18)", "    at _tryRequireFile (file:///var/runtime/index.mjs:976:37)", "    at _tryRequire (file:///var/runtime/index.mjs:1026:25)", "    at _loadUserApp (file:///var/runtime/index.mjs:1055:22)", "    at UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:27)", "    at start (file:///var/runtime/index.mjs:1256:42)", "    at file:///var/runtime/index.mjs:1262:7", "    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)"]}

Feature: Async DataSource options for seeding

Is your feature request related to a problem? Please describe.

You cannot use an async data source to seed the database. The seeder needs a sync data source, not a promise. If you give a file that exports a promise of a data source, the seeder will not work and will throw an error saying that it cannot get or load the database options.

OptionsError: The database options could not be located/loaded.

Describe the solution you'd like

To be able to use a file that exports Promise<DataSource> for seeding, as this would allow DataSourceOptions to be fetched asynchronous. This would be more flexibility and scalability for seeding different environments.

Describe alternatives you've considered

As a workaround we have now hardcoded the DataSourceOptions for seeding locally.

Additional context

Feature: Support ES Module Pattern

Is your feature request related to a problem? Please describe.

Currently, it is not possible to use ES modules with this library, which is an issue as I am using ES modules in a lot of projects.

E.g. I have projects where the database and data source files are in a dependency of the services to allow for a shared code base. They are organized in ES Modules. The usage of these modules is not supported in commonJS projects, giving errors when trying to use the seeding feature.

Describe the solution you'd like

Add support for ES modules by switching from commonJS module import to ESNext module import.

This is just a matter of aligning the module import with the library configuration in the tsconfig.json.

Simply switching the configuration from "module": "commonjs", to "module": "ESNext", will do the trick.

Describe alternatives you've considered

The alternative solution is that everyone using this library would be required to be writing their code as commonjs-compatible, even though the library already uses ESNext lib components.

Bug: TypeError: ctx.constructor is not a constructor

Versions

  • Node: 18.18.2
  • OS: ubuntu

What is actually happening?

this is my datasource for seeds

const options: DataSourceOptions & SeederOptions = {
type: 'postgres',
host: process.env.DB_HOST,
port: +process.env.DB_PORT,
username: process.env.DB_USERNAME,
password: process.env.DB_PASSWORD,
database: process.env.DB,
migrations: ['src/database/migration/tenant-information/.ts'],
migrationsTableName: 'migration',
entities: ['src/tenant/**/
.entity.ts'],
logging: true,
synchronize: false,
logger: new TypeormCustomeLog(),
seeds: ['src/database/seeds/tenant-information/*.ts']
}
export const dataSource = new DataSource(options).initialize();

my package json script
"seed:run": "npm run build && ts-node ./node_modules/typeorm-extension/bin/cli.cjs seed:run -d src/shared/typeorm/TenantInformationDbMigration.ts",

when run the script i get this error, my seeds not complete

npm run seed:run

[email protected] seed:run
npm run build && ts-node ./node_modules/typeorm-extension/bin/cli.cjs seed:run -d src/shared/typeorm/TenantInformationDbMigration.ts

[email protected] build
nest build

β„Ή DataSource Directory: ///src/shared/typeorm 5:39:59 PM
β„Ή DataSource Name: TenantInformationDbMigration.ts 5:39:59 PM
typeorm-extension seed:run

Populate the database with an initial data set or generated data by a factory.

Options:
-h, --help Show help [boolean]
--preserveFilePaths This option indicates if file paths should be
preserved. [boolean] [default: false]
-r, --root Root directory of the project.
[default: "//"]
--tsconfig, --tc Name (or relative path incl. name) of the tsconfig
file. [default: "tsconfig.json"]
-d, --dataSource Name (or relative path incl. name) of the data-source
file. [default: "data-source"]
-n, --name Name (or relative path incl. name) of the seeder.
-v, --version Show version number [boolean]

TypeError: ctx.constructor is not a constructor
at new SeederEntity (///node_modules/typeorm-extension/src/seeder/entity.ts:54:29)
at map (///node_modules/typeorm-extension/src/seeder/executor.ts:195:28)
at Array.map ()
at SeederExecutor.buildEntities (///node_modules/typeorm-extension/src/seeder/executor.ts:178:32)
at SeederExecutor.execute (///node_modules/typeorm-extension/src/seeder/executor.ts:46:32)
at async Object.handler (///node_modules/typeorm-extension/bin/cli.cjs:264:9)

Specify initial database to connect to when using db:create

It seems there's no way to specify the initial database to connect to when running db:create, instead the database is assumed to be the same as the user. This seems insufficient, there should be a way to specify the initial database either via ormconfig or via CLI argument.

Please correct me if I'm wrong.

Bug: Exception when creating postgres db with custom schema

Versions

  • Node: 16.16.0
  • pg: 8.7.3
  • typeorm: 0.3.7
  • typeorm-extension: 2.1.5
  • OS: Windows 11

Steps to reproduce

  • Setup a postgres database.
  • Create ormconfig.ts like below
  • Run the npm script db:setup attached below

ormconfig

For me, this is located at /src/config/ormconfig.ts.

export default {
  type: Config.dbDialect,
  host: Config.postgres.host,
  port: Config.postgres.port,
  username: Config.postgres.user,
  password: Config.postgres.password,
  database: Config.postgres.database,
  schema: Config.postgres.schema,
  entities: ['build/db/entities/**/*.js'],
}

npm scripts

"db:setup": "npm run build && cross-env NODE_ENV=development npm run db:drop && cross-env NODE_ENV=development npm run db:create"
"db:create": "typeorm-extension db:create --config ./build/config/ormconfig.js --initialDatabase postgres --synchronize",
"db:drop": "typeorm-extension db:drop --config ./build/config/ormconfig.js",
"build": "npx tsc",

What is Expected?

When specifying a schema in the ormconfig, the db:create should create that schema.

What is actually happening?

Exception when the first CREATE TABLE statement is being executed since the provided schema doesn't exist.

Bug: async datasource is not supported

Versions

  • Node: v18.18.0
  • OS: Ubuntu 22.04

Reproduction

My datasource uses async expression. TypeORM has supported async datasource(typeorm/typeorm#8914), but typeorm-extension doesn't.

// dataSource.ts

export const getDataSource = async (): Promise<DataSource> => {
  if (dataSource !== undefined) {
    return dataSource;
  }

  dataSourceOptions = await dbConfiguration();
  const dataSourceWithSeederOptions: DataSourceOptions & SeederOptions = {
    ...dataSourceOptions,
    seeds: [TestSeeder],
    factories: [UserFactory]
  };

  return new DataSource(dataSourceWithSeederOptions);
};

export const dataSourceInstance = getDataSource();
> ts-node --project ./tsconfig.json ../node_modules/typeorm-extension/bin/cli.cjs seed:run -r database -d dataSource

typeorm-extension seed:run

Populate the database with an initial data set or generated data by a factory.

Options:
  -h, --help               Show help                                   [boolean]
      --preserveFilePaths  This option indicates if file paths should be
                           preserved.                 [boolean] [default: false]
  -r, --root               Root directory of the project.
                                        [default: "/home/sdtc-user/sdtc/api-v2"]
      --tsconfig, --tc     Name (or relative path incl. name) of the tsconfig
                           file.                      [default: "tsconfig.json"]
  -d, --dataSource         Name (or relative path incl. name) of the data-source
                           file.                        [default: "data-source"]
  -n, --name               Name (or relative path incl. name) of the seeder.
  -v, --version            Show version number                         [boolean]

OptionsError: The database options could not be located/loaded.
    at Function.notFound (/home/bbddmm/node_modules/typeorm-extension/src/errors/options.ts:13:16)
    at Object.buildDataSourceOptions (/home/bbddmm/node_modules/typeorm-extension/src/data-source/options/module.ts:71:24)
    at async Object.handler (/home/bbddmm/node_modules/typeorm-extension/bin/cli.cjs:248:35)

Steps to reproduce

  1. export async dataSource
  2. Run seeder(npx ts-node --project ./tsconfig.json ./node_modules/typeorm-extension/bin/cli.cjs seed:run -r database -d dataSource)
  3. OptionsError: The database options could not be located/loaded. has occurred.

What is Expected?

seed:run should work

What is actually happening?

OptionsError: The database options could not be located/loaded. has occurred.

Bug: having trouble to link my seeder using the direct path with .env

Versions

  • Node: 16.19.1
  • OS: MacOS Ventura

Steps to reproduce

Declare your seeder path using .env TYPEORM_SEEDING_SEEDS=src/database/seeders/**/*{.ts,.js} instead of TYPEORM_SEEDING_SEEDS=src/database/seeds/**/*{.ts,.js} (we replaced /seeds to /seeders)

What is Expected?

The library is supposed to identify the seeder using the path

What is actually happening?

The seeder file doesn't react

Additional details

Importing the seeder class or adding the path directly in the seeds key of SeederOptions works fine

You have a reproduction repository here

Bug: Unable to resolve TypeORM decorators (suspect workspace issue)

Versions

  • Node: v20.2.0
  • NPM: 9.6.6
  • OS: Windows 11
  • ts-node: 10.9.1
  • typescript: 5.1.6
  • typeorm-extension: 3.0.1
  • typeorm: 0.3.17

Reproduction

I'm using a normal typeorm datasource, it haven't been modified to use any typeorm-extension features yet

Running the below NPM script

// package.json scripts
"seed": "ts-node ../node_modules/typeorm-extension/bin/cli.cjs seed:run -d src/db/dataSource.ts",

Note the path is ../node_modules

yields:

β„Ή DataSource Directory: path\to\src\db
β„Ή DataSource Name: dataSource.ts
typeorm-extension seed:run

@Entity()
 ~~~~~~~~
error TS1240: Unable to resolve signature of property decorator when called as an expression.
  Argument of type 'undefined' is not assignable to parameter of type 'Object'.

The decorator is

@Entity()
export class MyEntity { ... }

Steps to reproduce

  1. Setup NPM workspace
  2. Make a data-source.ts
  3. Run one of the typeorm-extension scripts like the seed script: npm run ts-node ../node_modules/typeorm-extension/bin/cli.cjs seed:run -d src/db/dataSource.ts

What is Expected?

It shouldn't fail to compile.

What is actually happening?

It's failing to compile. I suspect it has something to do with it being run in a NPM workspace so the path to ../node_modules is different and that screws up stuff

Feature: Decoupling from the typeorm core options reader

Due to the fact that with v0.4.x typeorm no longer supports reading of conenction options via file and env, a standalone reader should be implemented.

Use the possibility and override env naming (TYPEORM_XXX -> DATABASE_XXX).

Bug: unsupported ts-node-dev package in CLI command approach

Versions

  • Node:
  • OS:

Reproduction

Somehow CLI command approach overwrites "src" -> "dist" in paths set by ENV variable

Steps to reproduce

  • Install the project
  • Set "TYPEORM_SEEDING_SEEDS" to "src/data/seeds/*.ts"
  • Run the CLI with "ts-node-dev"

What is Expected?

It should work

What is actually happening?

Seeder files are not found

Bug: Impossible to execute seeds in order

Versions

  • Node: 18.16.0
  • OS: Ubuntu 20.04.5

Reproduction

I named my seeds with 0_country.seeder.ts, 1_client.seeder.ts, etc. to have an order in which I want to launch them, but the CLI command "seed" still don't do it in order, or if it does they are launched asynchronously even with the "parallelExecution" option to false.

Steps to reproduce

  • Add "parallelExecution" option and put it to false
  • Name seeders with a number at the beginning to have them in order
  • Launch CLI command seed

What is Expected?

I would expect my seeds to be launched in order so that the relationships populate correctly.

What is actually happening?

The relationships doesn't populate correctly since the seed doesn't launch in order.

Bug: Seeder Broke in version 2.5.0

Versions

  • Node: 18
  • OS: MAC OS
  • Version: Latest (was working fine with v2.4.1)

Steps to reproduce

import { DataSource } from 'typeorm';
import {createDatabase, runSeeders, SeederOptions} from 'typeorm-extension';

...
<-- fetch datasource from seeder file and pass in function like below -->

try {
  await runSeeders(datasource);
}

What is Expected?

Should seed the tables with data

What is actually happening?

TypeError: clazz.run is not a function\n at runSeeder
typeorm-extension/dist/seeder/module.js:97:18

Please note that the seeding was working fine with 2.4.1 and suddenly after the update to latest version, it has started throwing the error. I have not changed anything in the seed file.

Update - This is possibly due to this Change of Lazy transpiling of seeder (5d092ad)

Bug: TypeError: typeorm_1.DataSource is not a constructor

Versions

  • Node: 16
  • OS: OSX

Reproduction

My project is using typeorm 0.2.34, which doesn't have DataSource so its failing inside of database/utils/synchronize

Additional Details

Steps to reproduce

run db:create with the following ormconfig.js

  database: config.POSTGRES_DATABASE,
  entities: entities,
  host: config.POSTGRES_HOST,
  migrations: migrations,
  password: config.POSTGRES_PASSWORD,
  port: config.POSTGRES_PORT,
  synchronize: false,
  migrationsRun: false,
  logging: false,
  type: 'postgres',
  username: config.POSTGRES_USER,
  uuidExtension: 'pgcrypto',
  cli: {
    migrationsDir: 'db/migrations',
  },
}

What is Expected?

Completes without error

What is actually happening?

The DB seems to be created fine, but there is a type error before the process completes. I downgraded to [email protected] and it works without error

TypeError: typeorm_1.DataSource is not a constructor
    at /Users/joshfee/dev/x-darwin/node_modules/.pnpm/[email protected][email protected]/node_modules/typeorm-extension/dist/database/utils/synchronize.js:16:28
    at Generator.next (<anonymous>)
    at /Users/joshfee/dev/x-darwin/node_modules/.pnpm/[email protected][email protected]/node_modules/typeorm-extension/dist/database/utils/synchronize.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/joshfee/dev/x-darwin/node_modules/.pnpm/[email protected][email protected]/node_modules/typeorm-extension/dist/database/utils/synchronize.js:4:12)
    at synchronizeDatabase (/Users/joshfee/dev/x-darwin/node_modules/.pnpm/[email protected][email protected]/node_modules/typeorm-extension/dist/database/utils/synchronize.js:15:12)
    at /Users/joshfee/dev/x-darwin/node_modules/.pnpm/[email protected][email protected]/node_modules/typeorm-extension/dist/database/driver/postgres.js:73:51
    at Generator.next (<anonymous>)
    at fulfilled (/Users/joshfee/dev/x-darwin/node_modules/.pnpm/[email protected][email protected]/node_modules/typeorm-extension/dist/database/driver/postgres.js:5:58)

Feature: set locale for faker instance

Is your feature request related to a problem? Please describe.

Hi! Thank you for this wonderful library. It's been working well, and I even have a "Show and tell" feature I'll be posting later.

With the new v8 version of Faker that we just upgraded to, it is no longer possible to set locale using faker.locale = "", and instead it is handled via imports using import {faker} from '@faker-js/faker/locale/nb_NO'; or import { fakerDE as faker } from '@faker-js/faker'. However, that option is not available when I don't control which instance this library imports. Are there any solutions to this, or do I have to use my own Faker instance? What is the benefit of using the faker instance provided in the seeder factory callback?

Describe the solution you'd like

Maybe a third parameter to the seeder factory? Or drop faker from the library altogether and leave it to the users?

Describe alternatives you've considered

Importing my own faker instance as per description above.

Bug: Undefined in DataSource variables

Versions

  • Node: 18.17.1
  • OS: Linux 22.04 LTS

Reproduction

When the seed:run command is used as indicated in the documentation, it does not take the values ​​found in the data source file, leaving all fields as undefined or default

Steps to reproduce

Just creating a basic data source file

import { DataSource, DataSourceOptions } from 'typeorm';
import { config } from 'dotenv';
import { SeederOptions } from 'typeorm-extension';
import InitSeeder from './src/database/seeds/test/init.seeder';

config();

const options: DataSourceOptions & SeederOptions = {
  type: 'postgres',
  host: 'process.env.POSTGRES_HOST',
  port: 5432,
  database: 'trippster',
  username: 'postgres',
  password: 'password',
  seeds: [InitSeeder],
  ssl:
    process.env.NODE_ENV === 'production'
      ? { rejectUnauthorized: false }
      : false,
  synchronize: true,
  logging: true,
  entities: ['src/**/*.entity.ts'],
  migrations: ['src/database/migrations/*.ts'],
};

console.log(options);

const dataSource = new DataSource(options);

console.log(dataSource);

export { dataSource };

and using the command

"seed:run": "ts-node ./node_modules/typeorm-extension/bin/cli.cjs seed:run"

What is Expected?

It is expected to connect to the db and load the seeds, but it gives me the following error

Error: SASL: SCRAM-SERVER-FIRST-MESSAGE: client password must be a string

as if I had not provided the data

Debugging I found the error is in the part of the merge where the options supplied by the data source and the ones read by the env

image
image

attached images to show that the data arrives up to that point and then returns the datasource in the cli as if it were undefined

Bug: cannot drop the currently open database

Versions

  • Node: 18.17.0
  • NPM: 9.6.7
  • Yarn: 1.22.19
  • OS: Linux 5.15 Zorin OS 16.2 (based on Ubuntu 20.04)
  • typeorm-extension: 2.8.1 (also same issue on 3.0.0-alpha.9)

Reproduction

Additional Details

This is used withing a NestJS project, where the relevant database definitions, schemas, seeding, etc... are defined in their own library

these are my relevant package.json scripts

{
	// ...
	"typeorm:cli": "ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli",
	"typeorm-extension:cli": "ts-node -r tsconfig-paths/register ./node_modules/typeorm-extension/bin/cli.cjs -d libs/database/src/seeding/datasource.ts --initialDatabase dummydb",
	"db:create": "npm run typeorm-extension:cli db:create",
	"db:drop": "npm run typeorm-extension:cli db:drop",
	"db:seed": "npm run typeorm-extension:cli seed",
	// ...
}

Steps to reproduce

example data-source.ts

export const typeOrmDataSourceoptions: DataSourceOptions = {
    type: "postgres",

    host: "127.0.0.1",
    port: 5432,
    username: "admin",
    password: "dummypass",
    database: "dummydb",

    entities: [
      // entities ...
    ],

    synchronize: false,
    logging: true,
    useUTC: true,

    migrations: [__dirname + "/../migrations/*.{js,ts}"],
};

run the command

npm run db:drop

What is Expected?

The selected database to be dropped with no error

What is actually happening?

The command fails with this error

WARN  Failed to drop database.

ERROR  cannot drop the currently open database

 at Parser.parseErrorMessage (node_modules/.pnpm/[email protected]/node_modules/pg-protocol/src/parser.ts:369:69)
 at Parser.handlePacket (node_modules/.pnpm/[email protected]/node_modules/pg-protocol/src/parser.ts:188:21)
 at Parser.parse (node_modules/.pnpm/[email protected]/node_modules/pg-protocol/src/parser.ts:103:30)
 at Socket.<anonymous> (node_modules/.pnpm/[email protected]/node_modules/pg-protocol/src/index.ts:7:48)
 at Socket.emit (node:events:513:28)
 at Socket.emit (node:domain:489:12)
 at addChunk (node:internal/streams/readable:315:12)
 at readableAddChunk (node:internal/streams/readable:289:9)
 at Socket.Readable.push (node:internal/streams/readable:228:10)
 at TCP.onStreamRead (node:internal/stream_base_commons:190:23)

Proposed solution

The postgres database is always present by default in postgres, before connecting, change the initialDatabase to postgres and from there drop the desired database in context.database in here

const connection = await createSimplePostgresConnection(driver, options, context);

I have tested this and it worked.

-    const connection = await createSimplePostgresConnection(driver, options, context);
+    const connection = await createSimplePostgresConnection(driver, options, { ...context, initialDatabase: "postgres" });

Bug: cannot create or drop database with custom config

i cannot create or drop my database with custom configuration provided in your documentation

this is my code :

import { DataSourceOptions } from 'typeorm'
import { dropDatabase } from 'typeorm-extension'

void (async () => {
  const dataSourceOptions: DataSourceOptions = {
    type: 'postgres',
    host: 'localhost',
    username: 'masb0ymas',
    password: 'mypassword',
    database: 'express-typeorm',
    port: 5432,
  }

  // Create the database with specification of the DataSource options
  await dropDatabase({ options: dataSourceOptions })
})()

run script :

npx ts-node ./src/database/dropDB.ts

the error message says: error: database "masb0ymas" does not exist

Please help me, find solution with the problem.

Versions

  • Node: 14.18.1
  • OS: macOS 11.6

Reproduction

Additional Details

Steps to reproduce

What is Expected?

What is actually happening?

Bug: impossible to run only one seeder

Versions

  • Node: 16.13
  • OS: OSX
  • Lib: 2.8.1

Steps to reproduce

src/database/seeds directory contents:

  • a.seeder.ts
  • b.seeder.ts
  • c.seeder.ts
  • my.seeder.ts

data-source.ts contents:

import { DataSource, DataSourceOptions } from 'typeorm';
import { SeederOptions } from 'typeorm-extension';

const options: DataSourceOptions & SeederOptions = {
    ...
    seeds: ['src/database/seeds/**/*.ts'],
};

export const dataSource = new DataSource(options);

Command:

ts-node ./node_modules/typeorm-extension/dist/cli/index.js -d data-source.ts --seed MySeeder seed

What is Expected?

Only specified seed is executed.

What is actually happening?

Other seeds being executed before the specified seed.

Bug: No longer able to specify the order of seeds as of 2.1.10

Versions

  • Node: Node 16
  • OS: Windows 10

Reproduction

In order to meet foreign key constraints when seeding, we need to guarantee that our seeds are run in a specific synchronous order. For example, each User must have an Account, so we have to seed Accounts before we can seed Users

We currently do this by listing seeds in the correct order within data-source.ts, but as of 2.1.10 this no longer works as expected

This appears to be caused by these changes, which cause our seeds to be run asynchronously: 3abb417

Steps to reproduce

Example data-source.ts:

import { DataSource, DataSourceOptions } from 'typeorm'
import { SeederOptions } from 'typeorm-extension'

const options: DataSourceOptions & SeederOptions = {
  name: 'default',
  type: 'postgres',
  host: 'localhost',
  port: 5432,
  username: 'admin',
  password: 'password',
  database: 'db',
  synchronize: false,
  logger: 'simple-console',

  entities: [
    'src/entities/*/**.ts'
  ],
  seeds: [
    'src/seeds/create-accounts.ts',
    'src/seeds/create-users.ts',
    'src/seeds/*.ts'
  ]
}

export const dataSource = new DataSource(options)

Example create-accounts.ts seeder:

import { DataSource } from 'typeorm'
import { Seeder } from 'typeorm-extension'

import { Accounts } from '../entities/accounts'
import { accounts } from '../fixtures/accounts'

export default class CreateAccounts implements Seeder {
  public async run (dataSource: DataSource): Promise<void> {
    await dataSource
      .createQueryBuilder()
      .insert()
      .into(Accounts)
      .values(accounts)
      .orIgnore()
      .execute()
  }
}

Example create-users.ts seeder:

import { DataSource } from 'typeorm'
import { Seeder } from 'typeorm-extension'

import { Users } from '../entities/users'
import { users } from '../fixtures/users'

export default class CreateUsers implements Seeder {
  public async run (dataSource: DataSource): Promise<void> {
    await dataSource
      .createQueryBuilder()
      .insert()
      .into(Users)
      .values(users)
      .orIgnore()
      .execute()
  }
}

Seeder command:
ts-node --swc ./node_modules/typeorm-extension/dist/cli/index.js seed

What is Expected?

Seeds are run synchronously in the order listed within data-source.ts. First create-accounts.ts is run, then create-users.ts, then all other seed files

What is actually happening?

create-users.ts runs without waiting for create-accounts.ts to finish, causing create-users.ts to fail with a foreign-key violation because there are no accounts

Feature: Rollback seeders

Hello !!
first, I want to thank you for this amazing library, it has definitively help me so much.

It would be great if there was a way to rollback the seeders, because sometimes the database is changing (specially when the structure of it isn't clear), so having a way to rollback the seeder would be very helpful.

it could be executed with a command similar to this

npm run seed:revert

and it would delete all the entities added to the database added with the seeders.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.