GitLab support about scala-steward

[daddykotex]

I'm on my way to finish the Gitlab integration. I will test it a little bit here, in-house, and if everything operates smoothly, I'll open multiple PR one by one and try to split my changes to ease review.

[fthomas]

👍

@tjheslin1 is working on BitBucket support. I expect that there is quite a bit of overlap in your efforts.

[daddykotex]

In fact, the changes in master...tjheslin1:bitbucket look very similar to mine. The naming is different but that's it. I think vcs, as @tjheslin1 use is better than what I had, so I'll rebase on his work and see where it leads me.

[tjheslin1]

Hi @daddykotex. Glad my branch is able to help you. I look forward to seeing a PR!

I have gotten a bit distracted in the last week or two regarding this contribution, I'm glad you're working on it too! When trying to test locally I ran into the issue documented in #48. I was waiting for this to be closed as I saw something was in progress on a fork for this.

For BitBucket the reading up on the BitBucket api is done, unless there's a chance that changes to authentication needs to be made, which I was trying to run scala-steward locally for.

[kubukoz]

This would be totally awesome, and we'd benefit greatly from having scala-steward work in self-hosted GL. I want to help, so @daddykotex feel free to let me know if you need some help (even if only in the review).

[daddykotex]

So there you go, with a little bit of pressure :)

https://github.com/daddykotex/scala-steward/tree/refactor/vcsapialg

This is a WIP for Gitlab support. I tried it on an internal repository but also on Gitlab public services. I don't think my tests covered the whole API so I suggest a few Gitlab users take an hour to try it on their own Gitlab instances (or their public repositories) to make sure it works fine.

Here is how to proceed:

1. Create a user on your Gitlab installation.
2. In this user's settings add a new Access Token with permissions to access the API.
3. Write a bash program that print the password, like that:

#!/bin/bash
echo "ACCESS_KEY"

4. Check out the code shared above
5. In the scala-steward directory, generate the docker image: sbt docker:publishLocal
6. Make a temporary directory where scala-steward will work, here is what I did:

mkdir tmprunning
cd tmprunning
mkdir workspace
touch pass.sh # copy the pass.sh file
touch repos.md # see the current repos.md in scala-steward
tree .
.
├── pass.sh
├── repos.md
└── workspace [directory]

7. Run the container and mount the tmprunning folder in it as a volume and pass all the parameters:

docker run -it \
    --rm \
    -v /tmp/scala-steward/tmprunning:/data scala-steward:0.1.0-SNAPSHOT \
    --workspace "/data/workspace/" --repos-file "/data/repos.md" --git-author-name "Scala Steward @ daddykotex" --git-author-email "[email protected]" --vcs-type "gitlab" --vcs-api-host "https://gitlab.com/api/v4/" --vcs-login "daddykotex-scalasteward" --git-ask-pass "/data/pass.sh"

8. Enjoy

Notes: since it runs in a container, there is nothing cached so the execution is long. To speed it up, I suggest you mount a few additional volumes where sbt will be able to place all of it's stuff. That way it does not have to do it everytime. Something like -v /tmp/scala-steward/tmprunning/sbt:/root/.sbt -v /tmp/scala-steward/tmprunning/ivy2:/root/.ivy2

You can also add -d to see all the logs.

Here is the output of a successful run:

2019-05-20 06:25:07,009 INFO  Add global sbt plugins
2019-05-20 06:25:07,135 INFO  Clean workspace /data/workspace
2019-05-20 06:25:09,116 INFO  Check dependencies of daddykotex/scala-steward-test
2019-05-20 06:25:11,154 INFO  Refresh dependencies of daddykotex/scala-steward-test
2019-05-20 06:27:47,241 INFO  Found 3 updates:
  com.github.daddykotex:courier : 1.0.0-RC1 -> 1.0.0
  org.typelevel:cats-core : 1.0.0 -> 1.0.1 -> 1.6.0
  org.typelevel:cats-effect : 1.0.0 -> 1.3.0
Repo(daddykotex,scala-steward-test) List(Single(com.github.daddykotex,courier,1.0.0-RC1,NonEmptyList(1.0.0),None), Single(org.typelevel,cats-core,1.0.0,NonEmptyList(1.0.1, 1.6.0),None), Single(org.typelevel,cats-effect,1.0.0,NonEmptyList(1.3.0),None))
daddykotex/scala-steward-test
1
2019-05-20 06:27:47,462 INFO  Nurture daddykotex/scala-steward-test
2019-05-20 06:27:47,462 INFO  Clone and synchronize daddykotex/scala-steward-test
2019-05-20 06:27:51,498 INFO  Find updates for daddykotex/scala-steward-test
2019-05-20 06:28:33,553 INFO  Ignore org.scala-lang:scala-library:provided : 2.12.7 -> 2.12.8
2019-05-20 06:28:33,632 INFO  Found 3 updates:
  com.github.daddykotex:courier : 1.0.0-RC1 -> 1.0.0
  org.typelevel:cats-core : 1.0.0 -> 1.0.1 -> 1.6.0
  org.typelevel:cats-effect : 1.0.0 -> 1.3.0
2019-05-20 06:28:33,716 INFO  Process update com.github.daddykotex:courier : 1.0.0-RC1 -> 1.0.0
2019-05-20 06:28:33,951 INFO  Trying update strategy 'replaceAllInStrict'
2019-05-20 06:28:35,868 INFO  Create branch update/courier-1.0.0
2019-05-20 06:28:38,443 INFO  Create PR update/courier-1.0.0
2019-05-20 06:28:39,357 INFO  Created PR https://gitlab.com/daddykotex/scala-steward-test/merge_requests/4
2019-05-20 06:28:39,650 INFO  Process update org.typelevel:cats-core : 1.0.0 -> 1.0.1 -> 1.6.0
2019-05-20 06:28:39,898 INFO  Found PR https://gitlab.com/daddykotex/scala-steward-test/merge_requests/3
2019-05-20 06:28:40,372 INFO  PR is behind master
2019-05-20 06:28:40,376 INFO  Reset and update update/cats-core-1.0.1
2019-05-20 06:28:40,580 INFO  Trying update strategy 'replaceAllInStrict'
2019-05-20 06:28:44,127 INFO  Process update org.typelevel:cats-effect : 1.0.0 -> 1.3.0
2019-05-20 06:28:44,347 INFO  Found PR https://gitlab.com/daddykotex/scala-steward-test/merge_requests/2
2019-05-20 06:28:44,852 INFO  PR is behind master
2019-05-20 06:28:44,852 INFO  Reset and update update/cats-effect-1.3.0
2019-05-20 06:28:45,069 INFO  Trying update strategy 'replaceAllInStrict'
2019-05-20 06:28:49,845 INFO  ──────────── Total time: daddykotex/scala-steward-test: 1m 2s 375ms ────────────
2019-05-20 06:28:49,848 INFO  ──────────── Total time: run: 3m 42s 895ms ────────────
2019-05-20 06:28:49,850 INFO  Shutting down connection pool: curAllocated=1 idleQueues.size=1 waitQueue.size=0 maxWaitQueueLimit=256 closed=false

And the PR are there: https://gitlab.com/daddykotex/scala-steward-test/merge_requests

[fthomas]

Fixed by #524 and #645.