azu / monorepo-utils

A collection of utilities for monorepo/lerna. Tools for TypeScript project references etc..

monorepo-utils

This repository is utilities for monorepo. Also, this repository is a monorepo.

Packages

@monorepo-utils/package-utils

Collect package file path in the monorepo.

Supports following package manager's workspaces.

• Lerna(lerna.json)
• Yarn's workspaces
• npm v7+'s workspaces

@monorepo-utils/workspaces-to-typescript-project-references

This tool converts lerna/npm workspaces/yarn workspaces to TypeScript's Project References.

You can keep package dependencies synchronized between lerna/npm/yarn workspaces and TypeScript.

@monorepo-utils/collect-changelog

@monorepo-utils/collect-changelog get change from each package's CHANGELOG.md. It help to collect changelog in lerna's Independent mode.

@monorepo-utils/get-workspaces-cli

@monorepo-utils/get-workspaces-cli is a simple CLI for getting workspace list. It allows get file path of packages.

$ npx @monorepo-utils/get-workspaces-cli

Deprecated Packages

Deprecated @monorepo-utils/publish

@monorepo-utils/publish help npm publish.

This script split lerna publish(lerna 2) into versioning and publishing.

⚠️ Notes:

lerna 3 support lerna version and lerna publish. You should use lerna 3 directly.

Release Flow

1. Create Release PR via dispatching "Create Release" action
2. Review and Merge release PR
3. Published

See also:

• azu/monorepo-github-releases: monorepo release flow: lerna + GitHub Release's Automatically generated release notes

monorepo-utils's Issues

Add version to publish

yarn run publish
yarn run v1.7.0
$ monorepo-utils-publish
? Do you publish these packages?
@monorepo-utils/collect-changelog <= HERE
 Yes
✓ @monorepo-utils/[email protected]

Publish successful!


✨  Done in 13.44s.

Allow root tsconfig.json paths generation

Thanks for making this tool! I think some users may want to also add the references to the root tsconfig.json file. In that way, they can run tsc --build from the root directory to build all referenced packages.

Error: Not found supported plugin

I am experiencing an issue similar to #31, but on macOS.

package.json

{
  "name": "panama",
  "packageManager": "[email protected]",
  "private": true,
  "workspaces": [
    "./packages/*"
  ],
  "scripts": {
    "update-references": "yarn node ./scripts/updateReferences.js"
  },
  "devDependencies": {
    "@monorepo-utils/workspaces-to-typescript-project-references": "^2.8.2",
    "typescript": "^4.7.4"
  }
}

/Users/oliveryasuna/.nvm/versions/node/v16.16.0/bin/node /Users/oliveryasuna/.nvm/versions/node/v16.16.0/lib/node_modules/yarn/bin/yarn.js run update-references
/Users/oliveryasuna/Dropbox/NEWNEW/Personal/Programming/Projects/panama/.yarn/cache/@monorepo-utils-workspaces-to-typescript-project-references-npm-2.8.2-ed34f206b0-749ab95089.zip/node_modules/@monorepo-utils/workspaces-to-typescript-project-references/lib/index.js:56
        throw new Error("Not found supported plugin");
              ^

Error: Not found supported plugin
    at Object.toProjectReferences (/Users/oliveryasuna/Dropbox/NEWNEW/Personal/Programming/Projects/panama/.yarn/cache/@monorepo-utils-workspaces-to-typescript-project-references-npm-2.8.2-ed34f206b0-749ab95089.zip/node_modules/@monorepo-utils/workspaces-to-typescript-project-references/lib/index.js:56:15)
    at Object.<anonymous> (/Users/oliveryasuna/Dropbox/NEWNEW/Personal/Programming/Projects/panama/scripts/updateReferences.js:24:45)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Object.require$$0.Module._extensions..js (/Users/oliveryasuna/Dropbox/NEWNEW/Personal/Programming/Projects/panama/.pnp.cjs:9955:33)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.require$$0.Module._load (/Users/oliveryasuna/Dropbox/NEWNEW/Personal/Programming/Projects/panama/.pnp.cjs:9778:22)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:170:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:198:25)
    at async Promise.all (index 0)

Environment:
Computer: 2021 MacBook Pro M1
Node interpreter: Node 16.16.0
Package manager: Yarn 3.2.

Ability to reference sub-directories of referenced projects

Problem

We have a project with the following layout:

packages/
  a/
    tsconfig.json
    src/
      tsconfig.json # extends ../tsconfig.json
  b/
    tsconfig.json
    src/
      tsconfig.json # extends ../tsconfig.json

We use --tsconfigPath src/tsconfig.json to target the correct tsconfig.json file, as this is where we want the references, but we want the references for example for package b to be like:

{
  "references": [
    {
      "path": "../../a/src"
    }
  ]
}

Instead of:

{
  "references": [
    {
      "path": "../../a"
    }
  ]
}

Solution

Would it be safe to assume that the directory structure given in --tsconfigPath can be used for referenced projects?

I will make a PR for this on my fork and see how it goes - let me know what you think about this solution. I guess the alternative is to add another CLI option for it.

project-refereneces: sort and write

monorepo-utils/packages/@monorepo-utils/workspaces-to-typescript-project-references/src/index.ts

Line 89 in e81ccef

assert.deepStrictEqual(cleanCurrentProjectReferences, newProjectReferences);

currently, assert references without sorting.
This behavior may cause false positives.
We need to sort and assert.

Another option:

Write the sorted result to tsconfig.json.
It will be breaking change.

https://github.com/eBayClassifiedsGroup/update-ts-references/blob/0bd3d5ed586d748677f6d9a9ad3f133ae3dcd6c2/src/update-ts-references.js#L99

release workflow is broken

https://github.com/azu/monorepo-utils/actions/runs/6818441573

lerna WARN notice Package failed to publish: @monorepo-utils/package-utils
lerna ERR! E404 Not found
lerna ERR! errno "undefined" is not a valid exit code - exiting with code 1
Error: Process completed with exit code 1.

Same error with

• lerna/lerna#3812

Support pnpm! Couldn't get bessonovs' version to work

Just tried https://github.com/Bessonov/set-project-references but it didn't work for me.

pnpm shouldn't be so different to yarn, you have just to get for values with workspace and check the hard link or not?

btw, is npm 7's workspace support any good?

project references: --tsconfigPath and --includeRoot root paths

When using
workspaces-to-typescript-project-references --tsconfigPath tsconfig.prod.json --includesRoot

The workspace tsconfig.prod.json reference paths include the custom tsconfigPath
"references": [ { "path": foo/bar/tsconfig.prod.json" } ]
but the root tsconfig.prod.json reference paths do not
"references: [ { "path": "foo/bar" }]

It'd be great if the tool used the tsconfigPath for the reference paths in the root too, or if there was an option to enable this.

Thanks for the awesome tool.

publish: support --tag option

npm publish support [--tag <tag>] option.

• publish | npm Documentation

@monorepo-utils/publish should support it too.

publish: support --otp option

npm publish support [--otp <otpcode>].

• publish | npm Documentation

Support lerna@3 and converting file-specifiers in package.json before publication

I like the the new version command in lerna 3.0. I also like the lean monorepo approach (aka golden path) based on file specifiers that is demonstrated in zillow/javascript and lerna/lerna.

However, it is still not clear how to achieve the release flow that your script facilitates -- without using some script such as yours. In particular, I want the dev to bump the version in their branch, and have CI publish the changes once they are merged to master.

One of the features of lerna is that it changes file-specifiers in package.json to version numbers before publication. I don't think your script does that now.

Do not write tsconfig.json if the contents are the same

Problem

Using TS project references, if I build a package, it will build its dependencies in the same monorepo if there were no changes since the last build.
It does this by comparing the oldest modified timestamp under dist/ to the newest under src/. It does not care if the contents of the files are the same.

This presents a problem if using workspaces-to-project-references on a pre-commit hook, as we do, as tsconfig.json will always be "touched" on every commit. This means that the typescript compiler will always think that src is ahead of dist, even if there were no changes

Solution

Only write the file if the contents will be different.
I will chuck up a PR shortly

lerna.json exists, but use package.json's workspaces

npm 7+ support workspaces.
recently, lerna does not use lerna.json's workspaces properties.

So, We need to treat package.json's workspaces rather than lerna.json.

workspaces-to-typescript-project-references: Workspace ranges (workspace:) support

We want to support Yarn v2 Workspace ranges (workspace:) in @monorepo-utils/workspaces-to-typescript-project-references.

It looks like an experimental feature.

project-references: only if there's a tsconfig

Some of our packages aren't typescript and just plain js. But they get inserted as references anyway. It should check if there's a tsconfig before adding.

Don't write project reference if the dependency's version does not match

Example

@scope/[email protected] depends on @scope/[email protected], but the latest version of package-b in the monorepo is 2.0.0.

packages/
  package-a/
    package.json # "version": "1.0.0"
    node_modules/
      @scope/
        package-b/
          package.json # "version": "1.0.0"
  package-b/
    package.json # "version": "2.0.0"

Expected behaviour

The project reference for package-b is omitted from package-a as the version does not match.

Actual behaviour

The project reference for project-b is written into project-a regardless. When using Yarn Workspaces (and probably others), even though TS will have built the project reference, importing it actually uses the older version from node_modules/, so building is pointless.

Suggestion

When filtering the project references of project-a, check the package.json for both projects and ensure the version of project-b matches what is in project-a's package.json

I am working on a PR for this now

`workspaces-to-typescript-project-references` does not handle references to config files correctly

The path property of each reference in a TypeScript project can point to a directory containing a tsconfig.json file or directly to the config file itself (which may have any name).

The path property of each reference can point to a directory containing a tsconfig.json file, or to the config file itself (which may have any name).
-- TypeScript Handbook

I am reporting an issue where workspaces-to-typescript-project-references does not work as expected in this context.

Consider a monorepo with the following structure:

.
├── package.json
└── packages
    ├── a
    │   ├── package.json
    │   ├── tsconfig.json
    │   └── tsconfig.types.json
    ├── b
    │   ├── package.json
    │   ├── tsconfig.json
    │   └── tsconfig.types.json
    └── c
        ├── package.json
        ├── tsconfig.json
        └── tsconfig.types.json

Focusing on the a package:

packages/a/tsconfig.json:

{
  "compilerOptions": {
    "rootDir": "src",
    "outDir": "lib"
  },
  "references": [
    {
      "path": "./tsconfig.types.json"
    }
  ]
}

This configuration file emits JavaScript code to the lib directory.

packages/a/tsconfig.types.json:

{
  "compilerOptions": {
    "rootDir": "src",
    "outDir": "types",
    "composite": true,
    "emitDeclarationOnly": true
  }
}

This configuration file emits TypeScript declaration files to the types directory.

The relationship between these two files is expressed using project references:

graph LR;
tsconfig.json -->|references| tsconfig.types.json;

Loading

Therefore, running tsc -b in the a package should compile both tsconfig.json and the referenced tsconfig.types.json, emitting files to the lib and types directories, respectively.

While specifying the config file in the path property may not be common, tools like moonrepo consider this approach as a valid use case, as described here.

For the entire monorepo, the relationship between the config files is as follows:

graph TB;
subgraph /packages/a
a/tsconfig.json("tsconfig.json")
a/tsconfig.types.json("tsconfig.types.json")
end
subgraph /packages/b
b/tsconfig.json("tsconfig.json")
b/tsconfig.types.json("tsconfig.types.json")
end
subgraph /packages/c
c/tsconfig.json("tsconfig.json")
c/tsconfig.types.json("tsconfig.types.json")
end
a/tsconfig.json -->|references| a/tsconfig.types.json;
b/tsconfig.json -->|references| /packages/a;
b/tsconfig.json -->|references| b/tsconfig.types.json;
c/tsconfig.json -->|references| /packages/a;
c/tsconfig.json -->|references| /packages/b;
c/tsconfig.json -->|references| c/tsconfig.types.json;

Loading

For the complete code, please refer to this repository:
https://github.com/suinplayground/monorepo-utils-1/tree/master/packages/%40monorepo-utils/workspaces-to-typescript-project-references/test/fixtures/yarn-workspaces-pointing-to-config-file

In this monorepo structure, I found that using workspaces-to-typescript-project-references results in references to tsconfig.types.json being removed:

 Error: [packages/a] Expected values to be strictly deep-equal:
 + actual - expected
+ [
+   {
+     path: './tsconfig.types.json'
+   }
+ ]

 Error: [packages/b] Expected values to be strictly deep-equal:
 + actual - expected
    [
    {
        path: '../a'
+   },
+   {
+     path: './tsconfig.types.json'
    }
    ]

 Error: [packages/c] Expected values to be strictly deep-equal:
 + actual - expected ... Lines skipped
    [
    {
...
    {
        path: '../b'
+   },
+   {
+     path: './tsconfig.types.json'
    }

 workspaces-to-typescript-project-references found 3 errors.
 Please update your tsconfig.json via following command.
 $ workspaces-to-typescript-project-references

See the full error log here:
https://github.com/suinplayground/monorepo-utils-1/actions/runs/9474724661/job/26104870429#step:6:60

The expected outcome is that references to tsconfig.types.json within the same package remain intact, while only inter-package references are managed.

I would appreciate it if this issue could be resolved. Thank you for your attention to this matter.

workspaces-to-typescript-project-references: add option to sort references with dependent-upon logic

Hi guys, I was wondering if it wouldn't be that complicated to add an option to sort references using a dependent-upon logic instead of an alphabetical one (especially in the root tsconfig file).

By dependent-upon logic I mean for packages that are dependencies for other packages to appear first in the lists.

I assume you already have the required information since you need to resolve the dependency graph to create the references.

My proposal is motivated by this issue: typescript-eslint/typescript-eslint#2094 (comment)

I know that it is some sort of edge case, but I think that this sorting logic could be of more value than the alphabetical one (which can be easily obtained at a latter time).

My intention is to import my tsconfig file and use the references list. This way I don't need to keep a separate list of dependencies with this particular order elsewhere.

Cheers.

workspaces-to-typescript-project-references: Don't add `references` if there are none

Currently a references: [] is added, which is redundant, as --check passes without it being there.

workspaces-to-typescript-project-references: Dependency resolution logic doesn't match yarn

Based on

monorepo-utils/packages/@monorepo-utils/workspaces-to-typescript-project-references/src/manager/workspaces.ts

Lines 26 to 30 in 8ccf1c7

	resolve({ name, version }): string | null {
	const matchPkg = monorepoPackages.find((info) => {
	if (info.packageJSON.name !== name) return false;
	if (version.startsWith("workspace:")) return true;
	return !!semverMatch(version, [info.packageJSON.version]);

, the dependency resolution logic is:

• package name has to match AND
• if dependency version starts with workspace: OR
• dependency version matches semver of package version

This does not match the Yarn resolution logic in at least the following scenarios:

• Dependency version starts with workspace and semver doesn't match (which does not resolve as per https://yarnpkg.com/features/workspaces#workspace-ranges-workspace, specifially):

if a semver range, it will select the workspace matching the specified version.

• If a module has a non-compatible dependency with the rest of the monorepo, yarn 1 will resolve this inside a node_modules folder inside the module folder - but only for this module. Whereas the current logic will use this to resolve dependencies for other modules

I recommend avoiding re-implementing the dependency resolution logic in this library and instead re-using the resolution logic of the build tool chosen by the plugin

Ignore empty changelog

"@monorepo-utils/collect-changelog" collect all changelogs.
But, we want to ignore empty log on each changelogs.

Tasks

• --skip-empty

expose findPackages from '@monorepo-utils/package-utils'

It will be useful in specific situaltion.

Use tsconfigPath filename in reference, if not default tsconfig.json

Problem

In our projects we have two tsconfigs, the default tsconfig.json, which builds as commonJS, and tsconfig-esm.json.
If I run the following:

yarn workspaces-to-typescript-project-references --tsconfigPath src/tsconfig-esm.json

it will work, however the path that gets written will point to ../../my-package/src, where really I need it to be ../../my-packages/src/tsconfig-esm.json.

Potential solution

Use the filename provided in --tsconfigPath, in this case tsconfig-esm.json and append this to the path only if it is not tsconfig.json (the default).
This makes the assumption that the same filename would be used in the referenced projects. This similar assumption was made as part of #39, where we assume that if the --tsconfigPath is src/tsconfig.json, then all referenced paths should also have src on the end, so I think this is a fine assumption to make.

Have created a PR for it here 🚀 #47

workspaces-to-typescript-project-references doesn't work on Windows

Hello.

Thanks for the great project.

Unfortunately, I've faced the issue with the workspaces-to-typescript-project-references on Windows machine:

PS D:\dev\repos\devops\yaft> npm run lint
> [email protected] lint D:\dev\repos\devops\yaft
> npm run ts:references:check && npm run lint:dependecies && npm run lint:es && npm run lint:prettier
> [email protected] ts:references:check D:\dev\repos\devops\yaft
> workspaces-to-typescript-project-references --check
Error: Not found supported plugin
    at Object.exports.toProjectReferences (D:\dev\repos\devops\yaft\node_modules\@monorepo-utils\workspaces-to-typescript-project-references\lib\index.js:20:15)
    at D:\dev\repos\devops\yaft\node_modules\@monorepo-utils\workspaces-to-typescript-project-references\lib\cli.js:77:28
    at Generator.next (<anonymous>)
    at D:\dev\repos\devops\yaft\node_modules\@monorepo-utils\workspaces-to-typescript-project-references\lib\cli.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (D:\dev\repos\devops\yaft\node_modules\@monorepo-utils\workspaces-to-typescript-project-references\lib\cli.js:4:12)
    at Object.exports.run (D:\dev\repos\devops\yaft\node_modules\@monorepo-utils\workspaces-to-typescript-project-references\lib\cli.js:64:74)
    at Object.<anonymous> (D:\dev\repos\devops\yaft\node_modules\@monorepo-utils\workspaces-to-typescript-project-references\bin\cmd.js:3:6)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)

Looks like the problem is caused by the globby package:

Note that glob patterns can only contain forward-slashes, not backward-slashes, so if you want to construct a glob pattern from path components, you need to use path.posix.join() instead of path.join().

Seems that the simplest fix is to use upath instead of path.

I can send a PR with these changes if you don't mind.

workspaces-to-typescript-project-references: More compact JSON formatting

The workspaces-to-typescript-project-references tool works great but it does this kind of formatting in tsconfig:

{
  "references": [
    {
      "path": "../../shared/package1"
    },
    {
      "path": "../../app-a/package2"
    },
    {
      "path": "../../app-b/package3"
    }
  ]
}

With many references this takes up a lot of lines and is hard to get an overview over. I would prefer this formatting:

{
  "references": [
    { "path": "../../shared/package1" },
    { "path": "../../app-a/package2" },
    { "path": "../../app-b/package3" }
  ]
}

I can fix this manually but workspaces-to-typescript-project-references will revert back to the more verbose formatting with every update so it is a bit tedious. I tried using prettier to auto-format it but prettier will respect multi-line json objects and keep them as-is.

project references: deduplicate dependencies

Hello,

Thanks for a great util, it's really helpful for us.

Noticed something today. We had a package which was falsely marked as a dependency and devDependency. This created 2 references in tsconfig.json.

While I do think having duplicate dependencies is an error, maybe deduplicating is a feature worth considering.

Looking at the code, it might be easier to do once the #44 refactor is completed.

Just a suggestion. Thanks!