semantic-rs / semantic-rs

:rocket: Automatic crate publishing done right

semantic-rs

The purpose of this tool is to help people to publish crates following the semver specification.

Right now if you're building a new crate publishing new versions includes a high amount of work. You need to decide if the new version will be either a new Major, Minor or Patch version. If that decision is made, the next step is to write a changelog with all the things changed. Then increase the version in Cargo.toml. Make a commit and a new tag for the new version and lastly publish it to crates.io. If you need to repeat these steps every time, chances are high you make mistakes. semantic-rs automates all these steps for you so you can focus more on developing new features instead.

Workflow

• Install semantic-rs on your machine.
• Follow the Angular.js commit message conventions when you commit changes to your repository
• When you're done with development, run semantic-rs
• Based on your changes it determines the next version number, generates a changelog, commits it and creates a new tag
• It also increases the version number in Cargo.toml (also committed)
• Runs cargo package for you
• Creates a release on GitHub
• Publishes the new version to crates.io
• Done 🚀

Usage

Installation via crates.io

Ubuntu:

$ sudo apt-get install -y cmake libssl-dev pkg-config zlib1g-dev
$ cargo install semantic-rs

Prerequisites

You need the following data beforehand:

• A GitHub application token Get it here
• Your crates.io API key Get it here

Run it

semantic-rs depends on some data being passed in via environment variables. In our examples we specify those variables explicitly but if you run semantic-rs frequently you may want to configure those in your shell's configuration file.

Setting GIT_COMITTER_NAME and GIT_COMMITTER_EMAIL is optional. If you omit those, we default to the settings from your (global) git configuration.

If you run semantic-rs without any arguments, it operates on your current working directory:

$ export GH_TOKEN=<GHTOKEN>
$ export CARGO_TOKEN=<CARGOTOKEN>
$ export GIT_COMMITTER_NAME=<Your name>
$ export GIT_COMMITTER_EMAIL=<Your email>
$ semantic-rs
#...

By default it runs in dry-run mode. This means it doesn't perform changes automatically. You see which steps would be performed and also the resulting changelog.

To perform the changes, pass -w as an argument:

$ semantic-rs -w=yes

This performs the following operations:

• Create or update Changelog.md containing everything that changed
• Create a new commit containing the following changes:
  • Changelog.md
  • An updated Cargo.toml with the new version number
• Create a new annotated git tag pointing to the last commit created recently and including the Changelog for the new version
• A new version published to crates.io
• A new release on GitHub
• Push the new commit and tag to GitHub

Development

Requirements:

• cmake
• OpenSSL development package
  • Ubuntu: libssl-dev, pkg-config, zlib1g-dev
  • Mac Homebrew: openssl
• Rust 1.15 or later

For OS X > 10.10

Note that since OS X 10.11 Apple doesn't ship development headers for OpenSSL anymore. In order to get it working, you need to run cargo with these variables configured:

OPENSSL_INCLUDE_DIR=`brew --prefix openssl`/include \
OPENSSL_LIB_DIR=`brew --prefix openssl`/lib \
cargo build

Build locally

Clone this project:

$ git clone [email protected]:semantic-rs/semantic-rs.git

As a test project you can use this one: https://github.com/badboy/test-project.

Clone it as well:

$ git clone https://github.com/badboy/test-project.git

In your top level directory there should be now the following two folders:

$ ls -l
semantic-rs
test-project

Change into the semantic-rs folder and run cargo build. Then you can run semantic-rs against the test project:

$ cargo run -- -p ../test-project
   Compiling semantic-rs v0.1.0 (file:///Users/janschulte/projects/semantic-rs/semantic-rs)
     Running `target/debug/semantic-rs -p ../test-project`
semantic.rs 🚀
Analyzing your repository
Current version: 2.0.3
Analyzing commits
Commits analyzed. Bump would be Minor
New version would be: 2.1.0
Would write the following Changelog:
====================================
## v2.1.0 (2016-07-03)


#### Features

*   Math mode ([24afa46f](24afa46f))

#### Bug Fixes

*   Into the void ([9e54f4bf](9e54f4bf))

====================================
Would create annotated git tag

Since -w yes was not passed, it only prints out what it would do. Note that if you run it on your local machine the output may differ.

Running the tests

To run semantic-rs's tests:

• unit tests: cargo test.
• integration tests: ./tests/integration/run-locally.sh.

Run semantic-rs in CI environment

Make sure to set the CI=true environment variable to disable dry-run mode.

Contributing

Bug reports and pull requests are welcome on GitHub. You can find more information about contributing in the CONTRIBUTING.md. This project is intended to be a safe, welcoming space for collaboration and discussion, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

This project is licensed under the MIT license.

semantic-rs's Issues

Test on Windows

We should build it on Windows

Mark automatically created commits and tags

Right now semantic-rs uses user.email and user.name from the user's gitconfig. When browsing the log it is not clear which commits where made by the user themselves and which by semantic-rs.
These commits and tags should be marked either by a comment that they were created in an automated fashion or we use a different author for that.

Make sure there are changes before writing the changelog

Rename to `cargo-semantic`

This is a great tool that helps improve the project publishing lifecycle.
Have you considered making it a cargo extension?

cargo install cargo-semantic
cargo semantic

Better user information about missing data and actions to be performed

It would be nice to know which steps semantic-rs will perform when it runs. Right now it looks like this:

janschulte@barad-dur ‹ master ● › : ~/projects/avm
$ GH_TOKEN=<TOKEN> CARGO_TOKEN=<TOKEN> semantic-rs -w
semantic.rs 🚀
Analyzing your repository
Current version: 1.0.0
Analyzing commits
Commits analyzed. Bump will be Patch
New version: 1.0.1
Writing Changelog
Creating annotated git tag

But it doesn't tell if everything is in place to:

• Create a GitHub release
• Publish to crates.io

Therefore it would be nice if semantic-rs would perform a preflight check and tells the user what is there and what not. If the user has passed -w then it immediately proceeds with all steps. Otherwise it just prints what it would do.

Document available and mandatory environment variables

semantic-rs uses a bunch of environment variables. These need to be documented with a short explanation in the README.md so users know what to configure.

Add support for non library projects

Right now, we only support libraries by ignoring Cargo.lock when performing version upgrades. This is fine for libraries, since they have Cargo.lock ignored.
For Commandline applications, we need to run a cargo build after increasing the version number to ensure that a new version number is written in Cargo.lock as well.
We need to figure out if Cargo.lock needs to be committed afterwards or if it should be skipped.

Cargo complains about dirty files

Since a few days Cargo won't package when dirty files are around.
We need to check if we have the right order: https://travis-ci.org/semantic-rs/semantic-rs/jobs/138855161

Build new version number

The commits read from the project's git log need to be analyzed to build a new version number.

Enhance ssh support

Right now we allow ssh only if the user does not host their project on GitHub. When the user hosts on GitHub we use https without exception.
As a user I want to be able to pick if I either want to user ssh or https as my preferred protocol.

refactor main function

Right now the main function is over 150 lines long. Adding new features gets more and more tedious so this needs to be refactored.

A first step could be to move all big blocks of functionality into their own function. The problem here though is that right now we do have some sort of global state. This circumstance prevents us from moving stuff into functions easily. For example the config object gets assembled over at least 50 LoC before it actually can be used.

Write changelog in tag

The new tag's message should contain the changelog for the new version.

User Setup

Right now as it runs local only there's no setup step at all.
At some point when we fully integrate with GitHub and Travis we might want to assist the user in setting everything up, setting environment variables and keys and such.

Suggest names for tags to allow easy marketing to humans

While version numbers like 1.17.4 are nice for machines, humans sometimes need things that are easy to remember. It would be pretty cool to automatically suggest names for versions, e.g. using fnichol/names or nishanths/rust-haikunator.

cc @badboy who suggested this at the Cologne meetup this week 😃

Support for other languages

We should decide if we want to support other environments as well in the future.
Possible candidates:

• Ruby
• Elixir
• Objective-C

Integration test error in OS X

The script that runs the integration is not compatible with OS X.
The tests fails on commands readlink -f and cp -ar, since those have different options interface on that platform.

• readlink
  fails with readlink: illegal option -- f
• cp -ar
  fails with cp: the -R and -r options may not be specified together. The flag -a expands to -pPR in OS X, there the conflict.

Pass a repository as commandline argument

As a user I want to be able to pass a repository path as commandline argument.

$ semantic-rs ../other-repo

If I omit the argument, it uses the current directory.

Pull in dependencies again

We can probably pull in commit-walker and commit-analyzer directly. Makes development a bit easier and I don't see a need for them being its own crate.

(also: 2 less crates to maintain on crates.io once semantic-rs gets published)

Automatic API diff

Elm does that already: https://gist.github.com/badboy/a302dd0c9020e5759240

This does need a deeper look into the Rust parser (or a Rust parser) to find changes.
From this we can then deduce version changes or give hints.

RFC 1105 already defines version bump levels for some code changes we should adopt

dry run mode

Binary build

Once this is can be used by users and on a CI server we need binary builds to avoid additional compilation time

Make sure release runs after all tests are green

Using https://github.com/badboy/travis-after-all-rs

What happens if repository has a remote under different name?

Right now we check for the remote origin. But in case the remote has a different name it's not possible to run semantic-rs right now.
There should be a mechanism in place that allows users to run semantic-rs with a different remote than origin.

Read from history

We need to read all commits since the last tag from the project's git history.

• When there is no tag yet, read all comits.

Projects which don't follow the expected commit message format

When I run semantic-rs on a project where the commit message format does not follow the Angular message style format, I get the following output:

[0] % cargo run -- -p ../Sokan
     Running `target/debug/semantic-rs -p ../Sokan`
semantic.rs 🚀
Analyzing your repository
Current version: 0.1.0
Analyzing commits
Commits analyzed. Bump would be Major
New version would be: 1.0.0
Would write the following Changelog:
====================================
## v1.0.0 (2016-01-30)


====================================
Would create annotated git tag

We need to decide what should happen here. Do we generate a changelog or do we skip it?

semantic-rs fails if origin is configured with ssh url

Given the case I configure the remote origin with a ssh url it fails with

thread '<main>' panicked at 'Remote URL can't be parsed: RelativeUrlWithoutBase', ../src/libcore/result.rs:760
Process didn't exit successfully: `target/debug/semantic-rs -p /Users/janschulte/projects/semantic-rs/Sokan -w` (exit code: 101)

Link to Contributor Covenant Version?

Dear all, as discussed with @schultyy I'd like to suggest to link to a specific version of the contributor covenant in your README file, e.g. http://contributor-covenant.org/version/1/4/ (latest version). What do you think?

After you have done this, I would love to add your project here, if that is okay?
https://github.com/1000miles/coc-collections

What should happen if repository has no remote?

Right now if a repository has no remote semantic-rs fails. I'm currently thinking about what should happen here. In case there's no remote we can't publish any tags and can't do a GitHub release but it still would be possible to set a new version number and tag the new version. So should that be allowed? WDYT?

Expand relative project path

When a user passes a relative path to -p like this:

$ semantic-rs -p ../test -w

the path needs to be expanded to ensure that all functionalities work correctly. Because right now cargo package and cargo fetch do not work correctly with relative paths passed

Run fails when project doesn't follow commit guidelines

Given a project where the commit message format does not follow the Angular.js commit message format.
Right now execution fails with

[1] % cargo run -- -p ../test-project
     Running `target/debug/semantic-rs -p ../test-project`
semantic.rs 🚀
Analyzing your repository
Current version: 0.6.0
Analyzing commits
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: ParseError("Parse error")', ../src/libcore/result.rs:736
An unknown error occurred

To learn more, run the command again with --verbose.

Instead it should print out a meaningful error message.

Fails when no Cargo.lock available

Right now it tries to commit both the Cargo.toml and Cargo.lock after a version change.
According to the documentation:

If you’re building a library, put Cargo.lock in your .gitignore.
If you’re building an executable, check Cargo.lock into git.

So it might very well the case that it shouldn't end up in the commit at all. Also it doesn't make sense to commit it without changes, so at least a cargo update would be necessary, which in turn means we would need another test run to be sure no of the dependencies broke (if Cargo.lock is tracked as well, that is)

Publish to crates.io

When a new version of a lib was tagged, that one should be published to crates.io

CLI output

Right now our printed output is a bit adhoc.
We might want to support both quiet and debug output as well as coloring (because SHINY COLORS 🌈)

The logging/printing stuff from Cargo is a bit more complicated, but seems nicely abstracted and its output is always quite readable and orgainzed.
Maybe we can adopt similar maintainable code.

Necessary tools

Exist:

• Changelog generator: clog
• git2-rs + glue code
• Command line parsing: docopt

We might want to have

• GitHub: github-rust might be a starting point. Though no code for things like releases yet
• Travis: We want to read the status of builds.

Build is failing on OS X 10.11

It looks like there are issues when semantic-rs gets built on OS X 10.11 and up because of openssl. Apparently OS X doesn't ship OpenSSL headers anymore.

Read and modify Cargo.toml

• The current version number needs to be fetched from the project's Cargo.toml
• The new version number needs to be written to project's Cargo.toml

Provide better error when repository was not found

When I try to run the following command:

$ cargo run -- -p ../test-project

and the folder ../test-project does not exist, the user gets in-descriptive error message right now. Instead we should provide a well formatted error which explains what went wrong.

semantic-rs fails if git user.name and user.email aren't configured globally

Given the case there's no user.name and user.email configured globally just for the current repository.
Then semantic-rs fails with the following error:

vagrant@vagrant-ubuntu-trusty-64:~/avm$ semantic-rs -w
semantic.rs 🚀
Analyzing your repository
Current version: 0.6.0
Analyzing commits
Commits analyzed. Bump will be Minor
New version: 0.7.0
Writing Changelog
Committing `Cargo.toml` and `Changelog.md` failed: "Config value \'user.name\' was not found"

default behavior if no args passed

I'm currently thinking of what should happen if the user passes no args. Right now semantic-rs takes the current directory as working directory into account. But when installed on a system and not run via cargo I guess this won't work anymore because the current dir is then something like /usr/local/ or ~/.bin. My proposal here is that the user just gets the help text printed and they need to pass a path to a repository explicitly.

Write tests for semantic-rs

I have an idea how to do proper integration tests (started here) using bats.

Far from usable right now

Document write option in a better way

Right now it's not 100% obvious what needs to be passed to -w as an argument. The help text says (default: yes if CI is set, otherwise no). but it took me some time to figure that out after looking at it after some time.
My proposal would be to explicitly say what needs to be passed for the W param here:

  -w W, --write=yes/no        Write changes to files (default: yes if CI is set, otherwise no).

We need to check what docopt actually allows here as valid values.

This should also be mentioned in the README as well.

Place changelog in repository's folder

Right now the changelog is placed in the same directory, semantic-rs is running. But instead the file should be placed in the passed repository path itself.

Disable write mode when CI=true

Add section about how to test semantic-rs locally

The README.md should contain a section which explains how to run both the unit tests and the integration tests.

Unit tests: cargo test
Integration tests: ./tests/integration/run-locally.sh

Optimize CLI output

This may be related to #44

When semantic-rs runs the output looks like this:

We should supress the output of cargo package and only show the results here or make sure that the user can easily distinguish the output from cargo package and semantic-rs. Because right now it's just the small message at the bottom which indicates that the publish was successful

Repository not hosted on GitHub

If the user does not host their source code on GitHub but somewhere else we cannot create a release there. Therefor we need to check that first before we create a release.

Create a release on GitHub

We need to publish a new version on GitHub after successful build.
This requires to write a new version to Cargo.toml with a new version bump commit.

HTTP remotes require authentication

Given the case a repository was cloned via HTTP remote. semantic-rs runs and tries to push changes back it currently fails with:

#...
Creating annotated git tag
Pushing new commit and tag
thread '<main>' panicked at 'called `Option::unwrap()` on a `None` value', ../src/libcore/option.rs:330
Process didn't exit successfully: `target/debug/semantic-rs -p /Users/janschulte/projects/semantic-rs/test-rs -w` (exit code: 101)

This happens because in this case the remote wants username and password from the user but semantic-rs doesn't show the prompt. When I cd into the repository's directory and git push I get asked for username and password.

The remote in this case is not GitHub so working with the GitHub API Token is not an option.

Get clog changes upstream do reduce dependencies

Right now we're using clog's latest release and our own fork for a minimal change. Bring that back to upstream and use only their version

Run tests from previous version

When there's a tests/ folder containing integration tests we check out the folder from the previous tag and run them against the current implementation.
If they fail and the user didn't specify a breaking change, the build fails.