Gitlab support about semantic-release

[andreasolund]

👍 for GitLab and Bitbucket support!

[armand1m]

You guys could reuse some of the code in semantic-release-gitlab package into the next branch. I'm actually using it on gitlab and it works like a charm.

[gr2m]

👋 Hey everyone, I'm not a gitlab/bitbucket user myself so I can’t help, but would love for it to happen.

What we need though is ideally at least two people who would be willing to maintain the project for each platform once it’s done. We can help with overall updates but when it comes to bugs and new features, we’ll need you :)

[gr2m]

Hmm I see that you already published https://www.npmjs.com/package/semantic-release-gitlab, maybe we can make it "official" and move it to the semantic-release GitHub organization, stating in the README that you are the maintainer? I don’t know how it’s currently implemented, but ideally it would depend directly on semantic-release but configure custom plugins for gitlab compatibility. What do you think?

[boennemann]

Sorry for the late reply @johann-sonntagbauer,

I'm totally up for that and the good news is that it's already kind of pluggable, because all the post script currently does is the "publishing to github" part. This means writing another post hook just for gitlab is all it takes :) I'm currently running this module through a bigger rewrite, but I could reping you once that's done, so you could contribute that :)

[johann-sonntagbauer]

Perfect, when you need any help with the rewrite let me know.

[boennemann]

@johann-sonntagbauer The rewrite is done on the next branch. Maybe you can look into it again now?

[johann-sonntagbauer]

Thx for the reminder, I will take a look at it tomorrow!

[boennemann]

Thinking about this more it would probably make sense to create a "createGitTag" plugin for this. It has the usual plugin signature and additionally gets passed the object the github.createRelease function receives right now. That way we could implement a bitbucket, gitlab, raw git strategy, etc.

[ariporad]

Actually, what if we just did a `release` plugin? That would solve applications. Or, even better make two, a `publish` plugin (defaults to NPM), and a `release` plugin (defaults to GitHub). That would make it much more flexible for non npm modules.

[johann-sonntagbauer]

Hi, sorry had no time to dig into this right away. @ariporad I like your idea.

[boennemann]

@ariporad I don't get what's different from your proposal and the current plan?
"getLastRelease" is your proposed "publish" plugin, while "createGitTag" is "release".

We can discuss the "createGitTag" name though :)

[ariporad]

Sorry, I phrased that poorly.

What I'm suggesting is that in package.json, we change semantic-release from 'semantic-release pre && npm publish && semantic-release post' to 'semantic-release run'.

Then, semantic-release run runs pre, runs the publish plugin (defaults to npm), then runs post (which runs the release plugin).

That would make semantic-release completely compatible with non-npm packages (ie. Apps)

[johann-sonntagbauer]

Hi @boennemann
I started to investigate that topic today, but I face some problems with the npm dependencies. The dependencies are now managed through github notation? Is that intended?

We use internal build infrastructure which is not allowed to access the internet which is quite common in enterprises. Would be very happy when we can bundle the plugins with semantic relase together.

[johann-sonntagbauer]

also the repository.url verification should be a little relaxed. For now only github repos are supported here (because of parsing with parse-github-repo-url). Maybe moving the verification into the post step is sufficient. The property is only used when publishing the release.

[johann-sonntagbauer]

another question :) NPM Token handling. I think its a litte strange why already present _auth tokens in npmrc configuration has to be respecified with environment variables. Maybe adjust the verify.js check a litte so that it checks the npm config if _auth or _authToken are given and afterwards check the env properties. When absolutly no data is available throw the error.

[boennemann]

@johann-sonntagbauer
What do you mean by "github notation"? Can you point me to the code you're talking about?

Totally up for relaxing the URL thing!

When it comes to the NPM_TOKEN – this a security feature. npm does not automatically ignore .npmrc files, which means if it's already in there in clear text it will publish that to the registry, which I just want to restrict in any case. If you're already using the environment variable notation you might as well change the variable name to NPM_TOKEN (but we could also talk about how to make name of the env variable dynamic if that's a concern).

[johann-sonntagbauer]

dependencies

The dependencies to semantic-release artefacts are handled in github notation. E.g.
"@semantic-release/commit-analyzer": "^2.0.0"
NPM fetches the dependency directly from the github releases. That causes trouble when you use semantic-release in a restricted environment (closed build server without internet connection).

Maybe we can treat the dependencies as devDependencies and bundle the code into the release of the semantic-release npm distribution? In that way you still have the isolation, but also a complete npm package.

repository url

The repository url is only needed for publishing the release to github. So in that case I moved the verification into the post.js but don't like that approach. I like the plugin mechanism and think of extra steps to provide the validation upfront. In that way the github integration itself could be modeled as a set of plugins without any "hardcoded" options and properties in the core. Still need to think a little more about the additional plugin hooks.

NPM_TOKEN

Thx for clearing that for me. Didn't know that npm stores credentials in clear text into configuration files :/ Looked into mine and saw some Hash. Never mind with that background the implementation is perfectly fine.

[johann-sonntagbauer]

So after reading through the code i came up with following proposal for the hooks.
The semantic-release itself should only be a runtime for calling plugins in sequence. Out of the box it does nothing.
The functionality is registered with a set of plugins called bundle. For example git bundle for getting commits; travis bundle for travis support; github bundle which depends on git bundle for github support and so on.

• index
  • applyDefaultConfiguration
  • verifyConfiguration
• pre
  • beforePre
  • getLastRelease
  • getCommits
  • getTyp
  • verifyRelease
  • writeRelease
  • afterPre
• post
  • beforePost
  • generateNotes
  • createRelease
  • afterPost

all hooks should support the array notation for defining more than one hook.

Please let me know if you are ok with that strategy before i start the work ;)

[boennemann]

@johann-sonntagbauer

dependencies:
These aren't loaded from GitHub, they're scoped packages: https://docs.npmjs.com/misc/scope

repository url
I think we could make the repository url another verifyConditions plugin and just set travis and that in series as a default.

runtime for calling plugins in sequence
I like the idea and that's definitely where we're already headed, but I'd like to avoid making changes to this extent in one go and rather implement them one after another, as soon as the need for each individual hook arises.

[johann-sonntagbauer]

@boennemann

dependencies:

Yes, the problem i am facing is that i can't resolve the scoped dependencies. I can't find them in the npm registry. Do you know how scoped dependencies are handled by the registry? Haven't found any good resource about that.

runtime changes

+1 for the incremental implementation. The current implementation is good, but two main problems for now:

1. Some plugins can be defined as Sequences (e.g. verifyConditions), some don't (verifyRelease)
2. Some checks are hardcoded outside a plugin (e.g. all checks in verify.js).
   • To solve that I would also prefer to move the checks in a seperate verifyConditions plugin

planned actions

1. move verification code into own verifyConditions plugin
2. provide default plugin configuration that enables github and travis
3. provide configuration for gitlab as documentation.

[zetxx]

+1 for resolution for those issues(@johann-sonntagbauer i see strong logic in your posts and seems reasonable), i work in corporate env too, and 80% of the developers i think.
this problem should be nailed down until module is still not well known..

[johann-sonntagbauer]

@boennemann maybe when you find some time to take a look at pr #77. I fixed the test coverage issue and the current implementation is in use.

[hutson]

@johann-sonntagbauer it seems you cannot search for scoped npm packages using the search box on npmjs.org.

However, the package you provided as an example can be accessed here: https://www.npmjs.com/package/@semantic-release/commit-analyzer

Scope support was introduced in npm version 2 (Though, for those using Artifactory within corporate environments, the recommended version if 2.1.9 or higher).

Issue related to searching scoped packages: npm/newww#929

[hutson]

Also meant to say 👍 as I would like to take advantage of semantic-release for packages on a GitLab instance.

[mfilotto]

+1

[jimitndiaye]

Looking forward to generic (including private) git repository support rather than just GitHub.

[KeithPepin]

Also looking forward to this possibility. We're developing some new things at work right now and could really use this for our process!

[stalinkay]

+1 gitlab and bitbucket

[andrei-cacio]

+1 gitlab

[freeman29]

+1 for Bitbucket

[vnpnlz]

+1 for Bitbucket

[armand1m]

Someone willing to implement this? The ability to use other services than github seems to be the a key thing that is missing in this package :P

[armand1m]

@gr2m What do you think about creating some kind of pluggable system? Would be fair easier to maintain this and an official github plugin. People can develop their own plugins and maintain those themselves. The most popular one's would probably be adopted by the community.

[hutson]

What we need though is ideally at least two people who would be willing to maintain the project for each platform

If you need any assistance with GitLab, please let me know.

[gr2m]

@armand1m @hbetts I’m very open to any suggestions. I’m happy to make the github-specific parts pluggable. In fact, I think they already are :) semantic-release already has a plugin system.

Next step would be to investigate where we are today and whats holds us back to use semantic-release on GitLab, for example. Would any of you be up for doing that research task?

[hutson]

whats holds us back to use semantic-release on GitLab

A non-blocker, but definitely a burden, is the fact that semantic-release is hard-coded to enforce Travis-CI.

Travis-CI does not integrate with GitLab.

GitLab users would need to always disable the Travis-CI plugin before they could use semantic-release.

Is there a way we can make the default experience better for GitLab users?

[hutson]

I don’t know how it’s currently implemented

So semantic-release-gitlab was a from-scratch implementation of a semantic-release like tool for GitLab. While in spirit it's similar to semantic-release, there are a few core differences that would need to be considered before it could become the official semantic-release of GitLab.

Does Not Publish To NPM Registry

semantic-release-gitlab does not publish to the npm registry, nor does it interact in any way with the npm registry - https://gitlab.com/hyper-expanse/semantic-release-gitlab/commit/f359a6c45f7181de30fcb955c05549a3eb2818da

That functionality was remove, leaving semantic-release-gitlab as a tool that interacts solely with GitLab for the purpose of auto-versioning.

Previous tag information is fetched from the local environment instead of the npm registry.

No CI Limitation

As hinted at in my comment above, semantic-release-gitlab does not depend on a specific CI environment, so it's capable of working on GitLab CI just as easily as Jenkins (The company I work for uses Jenkins, while I personally use GitLab CI. Cross CI compatibility was a core requirement)

Node LTS Support

semantic-release-gitlab is written to run on all Node LTS versions.

Our Node Support Policy - https://gitlab.com/hyper-expanse/semantic-release-gitlab#node-support-policy

Many enterprises, including the one I work for, have a policy of only allowing Node LTS versions for production processes, including the release process for production code.

[gr2m]

Does Not Publish To NPM Registry

I would be open to make the currently built-in release to npm pluggable, so other folks can set other deploy / publish targets

No CI Limitation

I think we have that already

Node LTS Support

There is quite a simple workaround using npx which you can install on older node versions and it lets you run single commands with newer node versions.

Besides, Node 8 becomes LTS next month :)

[hutson]

I would be open to make the currently built-in release to npm pluggable, so other folks can set other deploy / publish targets

You would need to fetch version information from the project's repository instead of npm. Would y'all consider that?

I’m happy to make the github-specific parts pluggable. In fact, I think they already are :) semantic-release already has a plugin system.

To my knowledge the GitHub integration is not part of a plugin. My attempt in #150 to make GitHub a plugin never received much input.

---

The rest is more about policy and business requirements.

There is quite a simple workaround using npx which you can install on older node versions and it lets you run single commands with newer node versions.

But that technically runs semantic-release on a non-LTS version of Node.

Besides, Node 8 becomes LTS next month :)

And Node 9 becomes Current. What's to say you won't want to switch to Node 9 ( It will have native support for import/export, which is a pretty compelling feature. )

Also, our corporate policy is that developers may use any LTS version (Node 4 or 6, so long as they migrate towards newer releases), and upgrade as resources become available. This ensures developers have room to chosen when to upgrade, while still having access to actively supported software.

[gr2m]

You would need to fetch version information from the project's repository instead of npm. Would y'all consider that?

yes :)

My attempt in #150 to make GitHub a plugin never received much input.

It’s a good time to revive that discussion

Also, our corporate policy is that developers may use any LTS version (Node 4 or 6, so long as they migrate towards newer releases)

Node 4 is not LTS and Node 6 stops being LTS in April 2018, which could be a good target for us to ship first class GitLab support by then :)

[hutson]

Node 4 is LTS. It's in Maintenance mode, but it's still LTS. Please see the Release Plan section of the Release WG README - https://github.com/nodejs/Release/blob/master/README.md#release-plan As for the GitHub plugin, should we just re-open that pull request? To be honest, I won't have the time to contribute such a significant refactor any time soon.

…

On Sep 26, 2017 1:43 PM, "Gregor Martynus" ***@***.***> wrote: You would need to fetch version information from the project's repository instead of npm. Would y'all consider that? yes :) My attempt in #150 <#150> to make GitHub a plugin never received much input. It’s a good time to revive that discussion Also, our corporate policy is that developers may use any LTS version (Node 4 or 6, so long as they migrate towards newer releases) Node 4 is not LTS and Node 6 stops being LTS in April 2018, which could be a good target for us to ship first class GitLab support by then :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGY_5h-Q7_SQ2u9yHaYgvf4u2lSHan8Jks5smUXhgaJpZM4E7vvz> .

[gr2m]

I can’t drive it myself, if you are interested in merging the two projects, you’d need to be the one driving it :) There is no rush

So I filed #485 to discuss a Node LTS policy.

It looks like y'all are already discussing removing npm from semantic-release in #465.

Lastly, I've filed #486 to discuss removing the Travis-CI plugin.

[pvdlg]

Closing in favor of #565

[pvdlg]

Duplicate of #565

[Veitor]

Is support Gogs? or can support any self-build private repository via developing plugin by self.