Issues with New Child Project Release Feature and Duplicate Projen Package Installation about projen

[dkershner6]

In the case of child projects, I think it would generally be preferable to only trigger the release workflow for a child project if there are changes to the child project directory using the push.paths option.

#3252 handled this in a different way, basically the release workflow always runs, but stops short if no changes are present. I think if your idea works, it would be even better.

[mrgrain]

Trigger Path Configuration

This seems like a good additional thing we should do! In more traditional monorepo setup, I tend to not use this because I find that only looking at the workspace directory often isn't enough to identify changes to a workspace. However here with the quite strictly separated projects, it seems like a good fit. Definitely as a configurable option, probably even as a default. (If some one is going to look at this, please do two PRs: One for supporting it optionally and one to change the default).

[mrgrain]

Install Directory Issues

These are great findings, thank you!

#3208 forced the Install dependencies step to be executed in the parent project directory and didn't provide a configuration to override that behavior. @dkershner6 I'm curious about the reason for that decision and if you have any additional context about the manner you envision that working in application? When a child node project is created in a sub-directory with projen, it initializes the sub-directory with its own package.json and dependencies. So before the child project can be compiled, the install command needs to be run in the child project directory - not the parent directory. Or am I missing something about how this is supposed to be setup?

TBH we might need both. I thought the root install is required so we can run projen properly.

Furthermore, by default projen automatically adds itself as a dependency to the child project (in addition to the parent project) - leading to two different copies of the projen package being present.
Is there a strong reason why projen defaults to installing itself in child project directories? In the event that release: true is set on a child project, can we change the default behavior or at least call out that gotcha in the docs somehow?

Yeah that's likely not needed and can be fixed.

This can lead to some weird behavior such as instanceof checks returning false when true is expected. e.g. #3208 added this line:

100% There's a work around to use symbols instead, which we probably need to implement on all components.

[dkershner6]

Fwiw, I think the way I originally setup my repo with multiple projects was not quite right; I was not using workspaces, so the sub-project dependencies were not installed unless yarn install was run inside the sub-project directory. However, using workspaces, running yarn install inside the root directory now installs dependencies for both the root project and all sub-projects (workspaces). So installing at the root only, may still make the most sense.

This was my thought process, that near everyone would be using monorepos or workspaces (yarn, npm, pnpm all work this way), so it made sense to be opinionated.

[timothymathison]

Closing since I think I've effectively addressed the discussed issues in my two linked PRs

[dkershner6]

@dkershner6 I'm curious about the reason for that decision and if you have any additional context about the manner you envision that working in application?

I don't think I actually changed anything about how root projects interact with subprojects, I just made it so that they can be released.

I re-thought about this, and you likely meant in the release workflow. The reason for that is because installing dependencies from the root is usually required in monorepos. You can even see discussion about deleting install entirely from subprojects in other discussions. Putting in an option wouldn't be hard, but release is already very option heavy.

So before the child project can be compiled, the install command needs to be run in the child project directory - not the parent directory. Or am I missing something about how this is supposed to be setup?

Just to add, I haven't experienced this, but perhaps it is a package manager or monorepo thing. I use pnpm with NX.

[timothymathison]

I re-thought about this, and you likely meant in the release workflow. The reason for that is because installing dependencies from the root is usually required in monorepos. You can even see discussion about deleting install entirely from subprojects in other discussions. Putting in an option wouldn't be hard, but release is already very option heavy.

Perhaps you are referring to #450. I see what you mean. I'll be interested to see the direction that projen goes for the monorepo use case.
In the meantime, thoughts on making the determineInstallWorkingDirectory protected instead of private so its logic can be overridden?

[timothymathison]

This can lead to some weird behavior such as instanceof checks returning false when true is expected. e.g. #3208 added this line:

100% There's a work around to use symbols instead, which we probably need to implement on all components.

That's probably a good idea as it was very frustrating to debug until I realized why it was happening.

[dkershner6]

I re-thought about this, and you likely meant in the release workflow. The reason for that is because installing dependencies from the root is usually required in monorepos. You can even see discussion about deleting install entirely from subprojects in other discussions. Putting in an option wouldn't be hard, but release is already very option heavy.

Perhaps you are referring to #450. I see what you mean. I'll be interested to see the direction that projen goes for the monorepo use case. In the meantime, thoughts on making the determineInstallWorkingDirectory protected instead of private so its logic can be overridden?

Works for me.

[mrgrain]

In the meantime, thoughts on making the determineInstallWorkingDirectory protected instead of private so its logic can be overridden?

This is not a temporary compromise. If it can be overriden by inheritance, it's effectively also part of the public API. I'd rather iterate on the proper public API.

If your main goal is to just make this work for now, you can always access the private property and use @ts-ignore.

Plus you can use the low level object file APIs to manipulate the workflow file directly.

[mrgrain]

Maybe installing at the root was conceptually the wrong approach.

We have two problems to solve here:

• Bootstrapping so projen can run
• Install everything that is needed to release the subproject

I'm not super sure about the former. However for the latter, the responsibility probably should be with the subproject, which totally can decide to just delegate the installation to the root project.

[dkershner6]

However for the latter, the responsibility probably should be with the subproject, which totally can decide to just delegate the installation to the root project.

For my curiosity, what would the best path to go about this? I am assuming there is a better way than exec("cd ../../../"...

[timothymathison]

Fwiw, I think the way I originally setup my repo with multiple projects was not quite right; I was not using workspaces, so the sub-project dependencies were not installed unless yarn install was run inside the sub-project directory. However, using workspaces, running yarn install inside the root directory now installs dependencies for both the root project and all sub-projects (workspaces).
So installing at the root only, may still make the most sense.

[mrgrain]

However for the latter, the responsibility probably should be with the subproject, which totally can decide to just delegate the installation to the root project.

For my curiosity, what would the best path to go about this? I am assuming there is a better way than exec("cd ../../../"...

I'd use the cwd option for that. But essentially, yes that's it.

I think personally would as this anyway, since with yarn workspaces you normally don't want yarn install to be run in the workspace directory.

[mrgrain]

This was my thought process, that near everyone would be using monorepos or workspaces (yarn, npm, pnpm all work this way), so it made sense to be opinionated.

And that's probably still okay. But given that this doesn't happen automatically at the moment, we'll need to look at making this more flexible.

[so0k]

I'd use the cwd option for that. But essentially, yes that's it.

I had to use Typescript projects in monorepos that didn't have a build tool, in that case we just used

const gitRoot = execSync("git rev-parse --show-toplevel").toString().trimEnd()

and wherever needed:

foo.writeTo(path.join(gitRoot,".github","workflows", ...)

altho this fails when the directory hasn't initialised git yet.