Feature: option to track already executed seeders about typeorm-extension

[tada5hi]

No problem at all. I will create a new branch for the feature. About the name I will then put you in knowledge. The feature is planned for v3.

[faloi]

@tada5hi track sounds great. I also thought about renaming it to runOnce, but track seems more consistent.

Regarding @Playfrog4u question, I insist on treating seeders as migrations. So if you want to add more colors later you can add them by creating a new seeder that may overwrite the previous one, or just add the new colors.

Thinking in my context (we only use seeders for development), I'll try to make seeders that don't destroy the data each developer has created for testing.

[phoenisx]

Correct me if I am wrong:

• Following typeorm-extension option, seedTracking is required to enable Seed tracking, right?

const options: DataSourceOptions & SeederOptions = {
  ...
  seeds: ['path/to/seeds'],
  seedTracking: true,
}

• Once the above option is provided, we need to individually disable seeding per entity, using track property in the Seeder child class

class Foo extends Seeder {
  track = true,
}

[tada5hi]

@faloi This is an awesome idea ☺️but I still would somehow ( maybe class property ) want to give the opportunity to run a seeder more than once.

[faloi]

Yeah, that's why I said it would be great to "have the option" of doing so. If you're seeding with random data it makes a lot of sense to run the same seeder many times.

In the project I'm working, we want to populate the DB with some "real world" data, with relations between entities and so on. So it makes sense for us to do it only once.

Perhaps some attribute can be included in the seeder to mark it as idempotent, persistent, oneTimeOnly or something like that. And then the CLI would ignore it when running db:seed - unless some --force option is included, or something like that.

Do you think I can help with this one?

[tada5hi]

@faloi I completely agree.
That's why I'm also happy about the feature request, because it opens up new application possibilities 😊.

Yes, that sounds like a good way to do it, i like it.
You can fork this repository, create a pull request and make a serve and I would give you feedback on that if that's ok with you ✌️ .
Otherwise I will deal with it. Of course you can also give me feedback than.

[tada5hi]

@faloi are you on it ?

[faloi]

@faloi are you on it ?

No, sorry. I'm with a lot of work and don't have enough time to do it properly.
Perhaps if you (or someone else) starts it, I can contribute with defining the interface or testing it in a real project.

[tada5hi]

Hey, since I'm currently working on the 3.0 release I wanted to inquire how you guys would prefer to identify (executed) seeder. I see 4 possibilities for this at the moment.

• file name The problem with using only the filename could be problematic, because seeder files can be in different folders. Also it is not possible to rename the seeder file.
• file path The problem with just using the filename could be problematic, as it would not be possible to rename the seeder file or move it to a new location.
• class name In the case of the class name, the problem of uniqueness would again be the problem. Different seeder files could have the same class name.
• id attribute The ID attribute would have the advantage that seder files can be moved to new locations and can also be renamed. However, seeder files must be provided with a unique attribute or property.

Another possibility would be to use the id attribute as the primary identifier and if it is not available use the class name or file name/path.

[faloi]

Hi again @tada5hi, sorry for the delay.

We treat seeders the same way as migrations, so we don't rename, delete or move them once they are created. For us, any of the first three approaches would be enough - just for the sake of consistency, I'd use the class name because that's what the migrations use.

Perhaps uniqueness can be guaranteed in the same way migrations do: adding the current timestamp after the given name of the class. For that, it would be very useful to have a seed:create command that behaves like migration:create TypeORM's command.

[tada5hi]

no problem 🙂. I have also taken care of many other things lately.
I like the idea that the behavior should be similar to migration.

[tada5hi]

@faloi I am currently implementing the feature and the following questions have come to my mind to maintain backward compatibility.

1. How do we deal with seeder files that already exist?
2. Should we use a seed:fix cli command or automatically workflow to prepend/append the timestamp to the filename and classname for them?
3. What do we do if the seeder execution is done on built files and the source files can't be changed that way?
4. What do we do with seeds that are referenced via constructor reference instead of via file path as data source option?
5. Should existing seed files not be tracked via a database table?
6. Should seeds only be tracked via the database if the attribute ( idempotent, persistent, oneTimeOnly) is defined?

What is your opinion on this? What do you think?
King regards
Peter

[faloi]

Hi Peter! Sure backwards compatibility is always a challenge, and can take more time than the actual development of the new feature. 😅

(1) For existing seeders, I think backward compatibility would mean always running them - that was the previous behavior. (6) In that case I think the new attribute (oneTimeOnly?) with a false default could be enough.

(2) (5) In case someone wants to migrate to the new style, I think I'd let them rename the seeders to ensure unique names. Adding a timestamp to previously created files would be tricky, you'll have to add some kind of random number and that'd create some inconsistency between new and old seeders.

I owe you an answer for (3) and (4), let me re-read the docs to remember what you mean.

[tada5hi]

Yes, you have a point there.
Do you think one property ( oneTimeOnly) is enough to fulfill your requirements.
In the current alpha (v3.0.0-alpha.6) version, seeders that have the oneTimeOnly property are only executed once. All others are not tracked and are executed over and over again.

export default class UserSeeder implements Seeder {
    oneTimeOnly = true

    public async run(
        dataSource: DataSource,
        factoryManager: SeederFactoryManager
    ): Promise<any> {
         // ...
    }

[faloi]

Yep, sounds great!

A flag to generate the seeder with the oneTimeOnly set to true would be handy too, so you don't have yo manually set it every time.

Thanks for your time! I'm not developing right now but will test this feature ASAP.

[Playfrog4u]

What about if we want to update an existing seed, default values may change in early stages of development so a table for say “colors” may only include some initially then more may come or meta properties may change on existing “colors”

[tada5hi]

Yep, sounds great!

glad to hear.

A flag to generate the seeder with the oneTimeOnly set to true would be handy too, so you don't have yo manually set it every time.

mhhh, one possibility i thought of spontaneously would be to define the flag via the seeder or datasource option e.g. seedTracking.
But in that case i would like to rename the oneTimeOnly property to track for consistency.

Thanks for your time! I'm not developing right now but will test this feature ASAP.

You are very welcome!

[tada5hi]

What about if we want to update an existing seed, default values may change in early stages of development so a table for say “colors” may only include some initially then more may come or meta properties may change on existing “colors”

@Playfrog4u you are thinking of resetting the track status of a specific seed? So that it can be executed again?
However, this would mean that already seeded datasets would still exist and would be partially duplicated after re-execution.

[Playfrog4u]

@Playfrog4u you are thinking of resetting the track status of a specific seed? So that it can be executed again? However, this would mean that already seeded datasets would still exist and would be partially duplicated after re-execution.

Ah yes this makes since. I was thinking resettings but it does make more since to handle with migrations.

Thanks!

[permanar]

Hey, sorry for bump this up again but, is this closed because its implemented or what happened?

There is no doc for this so I'm a bit confused!

[tada5hi]

Sorry ☹️, it is not documented yet, but I will take care of it in a timely manner.
Till then, just set the class property track to true.

export default class UserSeeder implements Seeder {
    track = true

    public async run(
        dataSource: DataSource,
        factoryManager: SeederFactoryManager
    ): Promise<any> {
         // ...
    }