deluan / contentful-migrate

🐎 Schema migration tooling for Contentful, with state management

🎧 I’m currently dedicating my free time to Navidrome

contentful-migrate's Issues

Feature request - typescript ts file support

This would be amazing.

Example: https://github.com/contentful/contentful-migration/blob/master/examples/16-typescript-migration.ts

Bootstrap does not create the right scripts for the field with custom UI extensions.

According to the contentful-migration documentation, the second parameter of the changeFieldControl should be "extension" for the custom UI extensions.

I created a PR to fix the issue.
#191

ctf-migrate bootstrap: Scripts generation contents is undefined

When running ctf-migrate bootstrap I see "Scripts generation : successful" output but when I open any file in the migrations dir the contents is "undefined".

The remainder of the script writes the contentful migration state via the API.

Is there a way to increase verbosity or enable debug logging?

Support for ~/.contentfulrc.json config file

See: https://github.com/contentful/migration-cli#setup

This would make this tool more integrated with the Contentful tooling eco-system

Could we update the contentful-migration dependency?

Hello,

The contentful-migration dependency is an old version and it's causing some vulnerability alerts because of the old axios version that it depends on.

Is there a reason that this library needs to stay on the 1.x version, or do you think it's realistic to update to the latest version? I can help with a PR, but just wanted to check if there's a reason why 1.x was kept while other dependencies were bumped.

Thanks

Bootstrap generates scripts with deprecated changeEditorInterface method

When I run npx ctf-migrate bootstrap -c contentType -s spaceID -e environmentID, the package generates a script that uses the method changeEditorInterface. When I run this script, it emits the following warning:
(deprecated) Method changeEditorInterface is deprecated and will be removed, use changeFieldControl instead

Support new "environments" feature from Contentful

Implement --all for `up` and `list` commands

And also allows -c to receive multiple content-types.

--all and -c must be mutually exclusive

Implement pagination to get all content-types in bootstrap and store

As per: #6 (comment)

See also lib/store.js:10

Allow custom sort function when using as a library

I have been using this tool as a library to run automated migrations and one thing that would be useful would be to allow specifying a custom sorting function that would flow to the node-migrate library see here. This would be difficult to expose via the CLI, but it would be a non-breaking change if it is left unspecified as it is now.

What happens right now is since the default sort is a lexical sort on file name, when using, what I think are, more readable names like 1-migration.js, 2-migration.js, etc. once you hit 10-, the run order is (un)expected if you cross a digit boundary during a run.

I'd be happy to submit a PR showing my idea - it's pretty simple and would not change the tool as it is now and only take effect in more advanced use cases such as my own.

Migration scripts run order independent of content type

Expected Behavior

Run all scripts in date order, regardless of content type.

Actual Behavior

Scripts run in order of content type alphabetical, then by date, causing the migration to fail.

Possible Solution

Instead of going through each folder one by one, get all files under the matching directories and run in order by date.

Steps to Reproduce

1. Create content model A
2. Create content model B
3. Create first file under content model A create some fields
4. Create first file under content model B create some fields
5. Create second file under content model B to create a new field and populate it with data from a field in A in a field in B
6. Create second file under content model A delete field that is now in content model B

Context

We have some spaces that are far behind (or newly created), so need to run through our migration scripts. We have 1 that relies on some field being present in another content type, but another script that deletes that field.

Environment

• Node Version: 14.17.3
• Package Manager Version: yarn 1.18.0
• Operating System: Mac OSX 11.4
• Package Version: v4.0.10

Bootstrap script fails. eslint config not found

  🚨  Failed to perform bootstrap : Error: No ESLint configuration found.
Error: No ESLint configuration found.
    at Config.getLocalConfigHierarchy (/Users/sid/dev/kult-contentful/node_modules/contentful-migrate/node_modules/eslint/lib/config.js:256:39)
    at Config.getConfigHierarchy (/Users/sid/dev/kult-contentful/node_modules/contentful-migrate/node_modules/eslint/lib/config.js:180:43)
    at Config.getConfigVector (/Users/sid/dev/kult-contentful/node_modules/contentful-migrate/node_modules/eslint/lib/config.js:287:21)
    at Config.getConfig (/Users/sid/dev/kult-contentful/node_modules/contentful-migrate/node_modules/eslint/lib/config.js:330:29)
    at processText (/Users/sid/dev/kult-contentful/node_modules/contentful-migrate/node_modules/eslint/lib/cli-engine.js:162:33)
    at CLIEngine.executeOnText (/Users/sid/dev/kult-contentful/node_modules/contentful-migrate/node_modules/eslint/lib/cli-engine.js:668:26)
    at jsonToScript (/Users/sid/dev/kult-contentful/node_modules/contentful-migrate/lib/bootstrap/jsonToScript.js:70:17)
    at /Users/sid/dev/kult-contentful/node_modules/contentful-migrate/lib/bootstrap/generateScripts.js:30:11
    at processTicksAndRejections (internal/process/task_queues.js:89:5) {
  messageTemplate: 'no-config-found',
  messageData: {
    directory: '/Users/sid/dev/kult-contentful',
    filesExamined: Object [Generator] {}
  }
}

Importing Compose migrations: 'A field can't have "relationshipType" as a validation.'

I am setting up new migration scripts to be used with environment deployments and want to include Compose.

I followed this guide to export an initial migration script for Compose: Page and Compose: SEO. But when I try and run the migration script to import the migrations into a new environment I get the following error

Error: A field can't have "relationshipType" as a validation.

I am using the Continuous Intergration tutorial to import the migrations which does not use the contentful-cli but uses the npm package contentful-migrate instead.

How can I import the file generated via the alpha-features flag in the CLI with the contentful-migrate npm package?

Bug when adding `defaultValue` to a field and widget to the sidebar

Hi,

Running migrations with this package throws errors and causes bugs that started a couple of days ago. Namely, doing this

const model = migration.editContentType("contentTypeId");

model.createField("fieldId")
  .name("Field name")
  .type("Integer")
  .defaultValue({
    [locale]: value,
  })

causes scripts to fail with the following error:
"defaultValue" is not a valid property name for a field.

When adding a widget to the sidebar with the following code, it adds the widget, but it removes all of the other (native) widgets

const model = migration.editContentType("contentTypeId");

model.addSidebarWidget(
  "app",
  "widgetId",
  {},
  "publication-widget"
);

Using the native contentful-migration package it executes both scripts without any issues, but when using ctf-migrate to do the migrations, the aforementioned bugs and errors occur.

Package versions:
"contentful": "^9.1.18",
"contentful-migrate": "^0.16.0",
"contentful-cli": "^1.12.8",
"contentful-migration": "^4.0.0"

Updating the packages still doesn't resolve the issue and if anything, it further breaks the whole set up.
I appreciate any help resolving this issue.

P.S. #224 might be the reason why it started failing only recently.

Thanks!

Implement semantic versioning with auto publish to NPM

Maybe using this package: https://github.com/ericclemmons/github-semantic-version

NOTE: This can only be implemented after we put a proper test suite in place

store.js uses a hardcoded locale (en-CA)

This means that right now, for the tool to work, it requires the space to have en-CA as one of its locales, so it can store the migrations metadata.

The locale used by the tool has to be an specified by an option or should be the space's default locale.

Update bootstrap to use changeFieldControl

The bootstrap is using the deprecated version changeEditorInterface, new version is called changeFieldControl and takes 3 parameters.

https://github.com/contentful/contentful-migration/blob/master/README.md#changefieldcontrol-fieldid-widgetnamespace-widgetid-settings--void

field.changeEditorInterface('colors', 'radio')

// widgetNamespace : string - "builtin" | "extension"

field.changeFieldControl('colors', 'builtin', 'radio')

Using lib directly

I'm trying to use the libs directly, and am having stale data returned. I think it's from the cachedState. I'm running ups and downs fine, but when I go to list after doing an up it's stale data.

contentful-migrate/lib/store.js

Line 54 in f97410a

if (typeof cachedState !== 'undefined') {

The cachedState never gets cleared because I'm not using cli, so the node process never finishes etc. Can we add a flag to ignore the cachedState? Thanks!

Support newer version of `contentful-migration`

Hi there @deluan,

We've been using and loving this tool to help organize and run our Contentful migrations. I attempted to update content-migration today to 0.16.6. (I need a new feature they've released) and discovered that the dependency is set to: ^0.15.3.

What are the chances contentful-migrate can be updated to support the newer version? Is there anything I can do to help?

Thanks for the great project!

Getting failed dependency from BaseConfig

Error: Failed to load config "eslint-config-standard" to extend from.
Referenced from: BaseConfig

migration created via bootstrap needs to account for dash in content type id

If the content type id is in the form of foo-bar, the migration src created would have defined constants in the same fashion, i.e:
const foo-bar = migration.createContentType('foo-bar') which obviously isn't semantically correct

I blame Contentful since obviously they allow dashes in id even though their web UI says otherwise:

SS/case 1: special chars not allowed, and rightfully so, notice error message says only letters numbers and underscores allowed

SS/case 2: dashes are accepted

Bootstrap issue

With the new ESLint CLIEngine update - if I have any rules in my eslintrc, the output is undefined. I have to have an empty module.exports = {} in order to produce results.

Typescript migrations

It looks like tj/node-migrate#144 needs to be merged in before we can get typed migrations working.

Validation errors not returning exit code 1

I am getting an error but the exit code is 0 which mean the build passes incorrectly. I was using an invalid character in the contentType name which I fixed it but I think when any error occurs it should fail the build i.e. exit 1

Migration unsuccessful: 
Batch failed

Error: {"status":"Unprocessable Entity","message":"Validation error","details":{"errors":[{"name":"regexp","details":"Does not match /^[a-zA-Z][a-zA-Z0-9_]*$/","value":"inline-link-group","pattern":"^[a-zA-Z][a-zA-Z0-9_]*$","path":["fields",3,"id"]}]},"url":"https://api.contentful.com:443/spaces/qwerty/environments/dev/content_types/topic"}


Error: {"status":"Conflict","message":"","details":{},"url":"https://api.contentful.com:443/spaces/qwerty/environments/dev/content_types/topic/published"}


Please check the errors log for more details: /app/errors-1534393589930.log
  error : Error: Batch failed

🚨  Error applying migrations in dev! See above for error message

Refactor and standardize progress and log messages

Right now it is a mess, with a mix of console.log, migrate's log, listr and other methods. It is hard to follow what is happening through the messages displayed in the console.

Add a flag to just warn about missing script files

When devs are working in parallel in the same dev space, when one dev applies a script to the space, other devs can gets an error, because the script is not in their local workspace yet.

  error : Error: Missing migration file: 20180227004930-add-field-to-block-promo.js

There should be a way to make this error a warning instead, so it does not break any dev's workflow

Support regular contentful-migrations migration syntax

This tool should be able to run scripts meant to be used directly by Contentful's migration tool, i.e. without up and down, only exporting a function.

It could auto-detect that the migrations file is exporting a single function, and consider that as a up, creating a dynamic no-op down, and a auto-generated description (based on the file name.

As an example, a script like this (from Contentful's migration tool README:

module.exports = function (migration, context) {
  const dog = migration.createContentType('dog');
  const name = dog.createField('name');
  name.type('Symbol').required(true);
};

would be behave as if it was something like this:

module.exports.description = <filename>;

module.exports.up = (migration, context) => {
  const dog = migration.createContentType('dog');
  const name = dog.createField('name');
  name.type('Symbol').required(true);
};

module.exports.down = () => throw new Error('down migration is not available");