GithubHelp home page GithubHelp logo

drone / go-scm Goto Github PK

View Code? Open in Web Editor NEW
167.0 12.0 231.0 1.54 MB

Package scm provides a unified interface to multiple source code management systems.

Home Page: https://godoc.org/github.com/drone/go-scm/scm#pkg-examples

License: Other

Go 100.00%
github github-enterprise gitlab bitbucket bitbucket-server gogs gitea vcs scm

go-scm's Introduction

go-scm

Go Doc

Package scm provides a unified interface to multiple source code management systems including GitHub, GitHub Enterprise, Bitbucket, Bitbucket Server, Gitee, Gitea and Gogs.

Getting Started

Create a GitHub client:

package main

import (
  "github.com/drone/go-scm/scm"
  "github.com/drone/go-scm/scm/driver/github"
)

func main() {
client := github.NewDefault()
}

Create a GitHub Enterprise client:

import (
  "github.com/drone/go-scm/scm"
  "github.com/drone/go-scm/scm/driver/github"
)

func main() {
    client, err := github.New("https://github.company.com/api/v3")
}

Create a Bitbucket client:

import (
  "github.com/drone/go-scm/scm"
  "github.com/drone/go-scm/scm/driver/bitbucket"
)

func main() {
    client, err := bitbucket.New()
}

Create a Bitbucket Server (Stash) client:

import (
  "github.com/drone/go-scm/scm"
  "github.com/drone/go-scm/scm/driver/stash"
)

func main() {
    client, err := stash.New("https://stash.company.com")
}

Create a Gitea client:

import (
  "github.com/drone/go-scm/scm"
  "github.com/drone/go-scm/scm/driver/gitea"
)

func main() {
  client, err := gitea.New("https://gitea.company.com")
}

Create a Gitee client:

import (
  "github.com/drone/go-scm/scm"
  "github.com/drone/go-scm/scm/driver/gitee"
)

func main() {
  client, err := gitee.New("https://gitee.com/api/v5")
}

Authentication

The scm client does not directly handle authentication. Instead, when creating a new client, provide an http.Client that can handle authentication for you. For convenience, this library includes oauth1 and oauth2 implementations that can be used to authenticate requests.

package main

import (
  "github.com/drone/go-scm/scm"
  "github.com/drone/go-scm/scm/driver/github"
  "github.com/drone/go-scm/scm/transport"
  "github.com/drone/go-scm/scm/transport/oauth2"
)

func main() {
  client := github.NewDefault()

  // provide a custom http.Client with a transport
  // that injects the oauth2 token.
  client.Client = &http.Client{
    Transport: &oauth2.Transport{
      Source: oauth2.StaticTokenSource(
        &scm.Token{
          Token: "ecf4c1f9869f59758e679ab54b4",
        },
      ),
    },
  }

  // provide a custom http.Client with a transport
  // that injects the private GitLab token through
  // the PRIVATE_TOKEN header variable.
  client.Client = &http.Client{
    Transport: &transport.PrivateToken{
      Token: "ecf4c1f9869f59758e679ab54b4",
    },
  }
}

Usage

The scm client exposes dozens of endpoints for working with repositories, issues, comments, files and more. Please see the godocs to learn more.

Example code to get an issue:

issue, _, err := client.Issues.Find(ctx, "octocat/Hello-World", 1)

Example code to get a list of issues:

opts := scm.IssueListOptions{
  Page:   1,
  Size:   30,
  Open:   true,
  Closed: false,
}

issues, _, err := client.Issues.List(ctx, "octocat/Hello-World", opts)

Example code to create an issue comment:

in := &scm.CommentInput{
  Body: "Found a bug",
}

comment, _, err := client.Issues.CreateComment(ctx, "octocat/Hello-World", 1, in)

Useful links

Here are some useful links to providers API documentation:

Release procedure

Run the changelog generator.

docker run -it --rm -v "$(pwd)":/usr/local/src/your-app githubchangeloggenerator/github-changelog-generator -u drone -p go-scm -t <secret github token>

You can generate a token by logging into your GitHub account and going to Settings -> Personal access tokens.

Next we tag the PR's with the fixes or enhancements labels. If the PR does not fufil the requirements, do not add a label.

Run the changelog generator again with the future version according to semver.

docker run -it --rm -v "$(pwd)":/usr/local/src/your-app githubchangeloggenerator/github-changelog-generator -u drone -p go-scm -t <secret token> --future-release v1.15.2

Create your pull request for the release. Get it merged then tag the release.

go-scm's People

Contributors

abhinav-harness avatar adivishy1 avatar aman-harness avatar aradisavljevic avatar bakito avatar bhavya181 avatar bradrydzewski avatar christianruhstaller avatar cirnot avatar deepakpatankar avatar devkimittal avatar eoinmcafee00 avatar kit101 avatar lunny avatar marcotuna avatar marko-gacesa avatar mohitg0795 avatar nathan-fps avatar ogarcia avatar raghavharness avatar rajatharanganath avatar rathodmeetsatish avatar rutvijmehta-harness avatar senjucanon2 avatar shalini-agr avatar shubhag avatar shubham149 avatar takirala avatar tboerger avatar techknowlogick avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-scm's Issues

DRONE_COMMIT_BRANCH refers to target, not source branch in merge requests

Type:
Bug report. I have a single drone instance running hooked up to a gitlab server via webhooks.

What happened:
Both https://docs.drone.io/reference/environ/drone-commit-branch/ and https://docs.drone.io/reference/environ/drone-branch/ refer to the target branch in builds triggered by merge requests.

Expected:
Both https://docs.drone.io/reference/environ/drone-commit-branch/ and https://docs.drone.io/reference/environ/drone-branch/ should refer to the source branch, while https://docs.drone.io/reference/environ/drone-target-branch/ should refer to the target.

I suspect this is a gitlab issue, and is probably not reproducible on github.

When attempting to clone my git repo from GitLab drone hangs on git fetch.

There are nearly zero logs that are relevant from the runtime itself as it just hangs after the following text. It started happening 15 days ago and has not been able to clone since.

Initialized empty Git repository in /drone/src/.git/
+ git fetch origin +refs/heads/master:

Gitlab version 14.8.2
Drone CI: 2.11
Drone CI Runner: 1.0.0-rc.3

The only log from the primary drone server is a warning relating to the status, but I don't believe that's causing it to fail. It successfully publishes the pending status and stands up the required containers to run the whole pipeline.

{"build.id":830,"build.number":570,"error":"Cannot transition status via :enqueue from :pending (Reason(s): Status cannot transition via \"enqueue\")","level":"warning","msg":"manager: cannot publish status","repo.id":2,"stage.id":1308,"time":"2022-03-24T03:32:04Z"}

After much time later this gets logged

{"build.id":827,"build.number":568,"error":"Optimistic Lock Error","level":"warning","msg":"manager: cannot persist the step","repo.id":2,"stage.id":1302,"stage.status":"killed","step.id":4344,"step.name":"clone","time":"2022-03-24T03:46:06Z"}
{"build.id":827,"build.number":568,"error":"Optimistic Lock Error","level":"warning","msg":"manager: cannot persist the step","repo.id":2,"stage.id":1303,"stage.status":"killed","step.id":4345,"step.name":"clone","time":"2022-03-24T03:46:06Z"}

I've done the following so far:

  • remounted the data drive
  • changed the underlying build steps to use a completely different image
  • tested that the runner has network access to the git repo

I've inspected all of the running containers and there are no logs as to where it is actually stuck. There seems to be no reason other than something potentially changing in drone/git:latest, which I believe is this repo.

Runner logs

time="2022-03-24T04:31:35Z" level=debug msg="stage received" stage.id=1311 stage.name=my-app stage.number=1 thread=20
time="2022-03-24T04:31:35Z" level=debug msg="stage accepted" stage.id=1311 stage.name=my-app stage.number=1 thread=20
time="2022-03-24T04:31:35Z" level=debug msg="stage received" stage.id=1312 stage.name=my-app-2 stage.number=2 thread=88
time="2022-03-24T04:31:35Z" level=debug msg="stage accepted" stage.id=1312 stage.name=my-app-2 stage.number=2 thread=88
time="2022-03-24T04:31:35Z" level=debug msg="stage details fetched" build.id=832 build.number=572 repo.id=2 repo.name=my-app-repo repo.namespace=SelfHostedGitlab stage.id=1311 stage.name=my-app stage.number=1 thread=20
time="2022-03-24T04:31:35Z" level=debug msg="stage details fetched" build.id=832 build.number=572 repo.id=2 repo.name=my-app-repo repo.namespace=SelfHostedGitlab stage.id=1312 stage.name=my-app-2 stage.number=2 thread=88
time="2022-03-24T04:31:36Z" level=debug msg="updated stage to running" build.id=832 build.number=572 repo.id=2 repo.name=my-app-repo repo.namespace=SelfHostedGitlab stage.id=1311 stage.name=my-app stage.number=1 thread=20
time="2022-03-24T04:31:36Z" level=debug msg="updated stage to running" build.id=832 build.number=572 repo.id=2 repo.name=my-app-repo repo.namespace=SelfHostedGitlab stage.id=1312 stage.name=my-app-2 stage.number=2 thread=88
time="2022-03-24T04:31:36Z" level=debug msg="Engine: Starting step" build.id=832 build.number=572 container=drone-7qqtxno66e4hnx0jngrh image="drone/git:latest" namespace=drone placeholder="drone/placeholder:1" pod=drone-64sl8zmy5x3e3vwteu6f repo.id=2 repo.name=my-app-repo repo.namespace=SelfHostedGitlab stage.id=1311 stage.name=my-app stage.number=1 step=clone step.name=clone thread=20
time="2022-03-24T04:31:36Z" level=debug msg="Engine: Starting step" build.id=832 build.number=572 container=drone-b3bkjqswcpdc2nucfsxy image="drone/git:latest" namespace=drone placeholder="drone/placeholder:1" pod=drone-0fib21m23uy5xft9ddnj repo.id=2 repo.name=my-app-repo repo.namespace=SelfHostedGitlab stage.id=1312 stage.name=my-app-2 stage.number=2 step=clone step.name=clone thread=88
time="2022-03-24T04:31:36Z" level=debug msg="Launched containers. Duration=0.13s" count=1 failed=0 success=1
time="2022-03-24T04:31:36Z" level=debug msg="PodWatcher: Waiting..." container=drone-7qqtxno66e4hnx0jngrh pod=drone-64sl8zmy5x3e3vwteu6f stepState=running
time="2022-03-24T04:31:36Z" level=debug msg="Launched containers. Duration=0.07s" count=1 failed=0 success=1
time="2022-03-24T04:31:36Z" level=debug msg="PodWatcher: Waiting..." container=drone-b3bkjqswcpdc2nucfsxy pod=drone-0fib21m23uy5xft9ddnj stepState=running
time="2022-03-24T04:31:38Z" level=debug msg="PodWatcher: Container state changed" container=drone-b3bkjqswcpdc2nucfsxy image="docker.io/drone/git:latest" pod=drone-0fib21m23uy5xft9ddnj restartCount=1 state=RUNNING stepState=running
time="2022-03-24T04:31:38Z" level=debug msg="PodWatcher: Wait finished. Duration=2.17s" container=drone-b3bkjqswcpdc2nucfsxy error="<nil>" pod=drone-0fib21m23uy5xft9ddnj stepState=running
time="2022-03-24T04:31:39Z" level=debug msg="PodWatcher: Container state changed" container=drone-7qqtxno66e4hnx0jngrh image="docker.io/drone/git:latest" pod=drone-64sl8zmy5x3e3vwteu6f restartCount=1 state=RUNNING stepState=running
time="2022-03-24T04:31:39Z" level=debug msg="PodWatcher: Wait finished. Duration=3.27s" container=drone-7qqtxno66e4hnx0jngrh error="<nil>" pod=drone-64sl8zmy5x3e3vwteu6f stepState=running

Bitbucket Stash Pull Request Diff

Implement support for Bitbucket Stash's pull request diff endpoint

rest/api/1.0/projects/KEY/repos/slug/pull-requests/N/changes?limit=1

This would be for Bitbucket Stash only

gitee client pagination bug

gitee 分页逻辑有问题,依旧只能查询到前 100 个仓库

Page is default 0

func (s *service) List(ctx context.Context, user *core.User) ([]*core.Repository, error) {
	err := s.renew.Renew(ctx, user, false)
	if err != nil {
		return nil, err
	}

	ctx = context.WithValue(ctx, scm.TokenKey{}, &scm.Token{
		Token:   user.Token,
		Refresh: user.Refresh,
	})
	repos := []*core.Repository{}
	opts := scm.ListOptions{Size: 100}
	for {
		result, meta, err := s.client.Repositories.List(ctx, opts)
		if err != nil {
			return nil, err
		}
		for _, src := range result {
			if src != nil {
				repos = append(repos, convertRepository(src, s.visibility, s.trusted))
			}
		}
		opts.Page = meta.Page.Next
		opts.URL = meta.Page.NextURL

		if opts.Page == 0 && opts.URL == "" {
			break
		}
	}
	return repos, nil
}
func encodeListOptions(opts scm.ListOptions) string {
	params := url.Values{}
	if opts.Page != 0 {
		params.Set("page", strconv.Itoa(opts.Page))
	}
	if opts.Size != 0 {
		params.Set("per_page", strconv.Itoa(opts.Size))
	}
	return params.Encode()
}

current always default value 0

So can't paginate the query

func populatePageValues(req *scm.Request, resp *scm.Response) {
	last, totalError := strconv.Atoi(resp.Header.Get("total_page"))
	reqURL, err := url.Parse(req.Path)
	if err != nil {
		return
	}
	current, currentError := strconv.Atoi(reqURL.Query().Get("page"))
	if totalError != nil && currentError != nil {
		return
	}
	resp.Page.First = 1
	if last != 0 {
		resp.Page.Last = last
	}
	if current != 0 {
		if current < resp.Page.Last {
			resp.Page.Next = current + 1
		} else {
			resp.Page.Next = resp.Page.Last
		}
		if current > resp.Page.First {
			resp.Page.Prev = current - 1
		} else {
			resp.Page.Prev = resp.Page.First
		}
	}
}

Bitbucket Server (Stash) Write Access

Find method to determine if user has write access


First I want to document how we handled this in 0.8

In 0.8 we use the following webhook to check and see if a user has access to the repository webhooks. If yes, the user is granted admin and write access:

var pathHook = "/rest/api/1.0/projects/%s/repos/%s/settings/hooks/%s"

func (c *Client) FindRepoPerms(owner string, repo string) (*model.Perm, error) {
	perms := new(model.Perm)
	// If you don't have access return none right away
	_, err := c.FindRepo(owner, repo)
	if err != nil {
		return perms, err
	}
	// Must have admin to be able to list hooks. If have access the enable perms
	resp, err := c.client.Get(fmt.Sprintf(pathHook, c.base, owner, repo, hookName))
	if resp != nil {
		defer resp.Body.Close()
	}
	if err == nil {
		perms.Push = true
		perms.Admin = true
	}
	perms.Pull = true
	return perms, nil
}

The source code can be found here:
https://github.com/drone/drone/blob/v0.8.8/remote/bitbucketserver/internal/client.go#L123:L140

The 1.0 version of Drone uses this exact same logic which can be found here:
https://github.com/drone/go-scm/blob/master/scm/driver/stash/repo.go#L118:L150

ASSESSMENT

There is a regression in the Drone UI that disables the cancel button for Drone system administrators if the admin does not have write/admin access to the repository. This only impacts the user interface and does not impact the ability for a system administrator to cancel a build via the API or the CLI.

Perhaps this UI regression is the difference you are noticing between 0.8 and 1.0? Because there has been no change in underlying permissions enforced by the Drone server to cancel a build, nor have there been any changes to how we determine write/admin access?

Rebuild Drone 1.0.0 RC6(?)

It seems that the patch for Gitea referenced over at #7 has not been built into a Docker image. How would I go about including this patch for the drone container I am currently running? @bradrydzewski, I noticed you also locked the original thread.

[Proposal] add search interface for go-scm

we know go-scm can work well with Drone, and it can use repo.Find interface to find a specific project.

In our scenario, we hope can use string to search the project list, so whether we can add a search interface for go-scm,

interface

# https://github.com/go-atomci/go-scm/blob/master/scm/search.go#L16
	SearchService interface {
		// Find returns a repository by name.
		FindProjects(context.Context, string) ([]*Repository, *Response, error)
	}

gitlab sample

# gitlab
# https://github.com/go-atomci/go-scm/blob/master/scm/driver/gitlab/search.go#L18
func (s *searchService) FindProjects(ctx context.Context, projectName string) ([]*scm.Repository, *scm.Response, error) {
	path := fmt.Sprintf("api/v4/search?scope=projects&search=%s", encode(projectName))
	out := []*repository{}
	res, err := s.client.do(ctx, "GET", path, nil, &out)
	return convertRepositoryList(out), res, err
}

file naming conventions

Update files to follow Go naming conventions and use snake case instead of camel case. Example testSettings.go in the Stash and Azure integration directories.

Support issue & PR comment events in the webhook service.

The goal of this issue is to discuss technical details about how to implement issue & PR comment events.

In some drivers, e.g., github, we might need a context for its webhook service to reach out to the API endpoint to retrieve additional information. For example, when processing an issue comment event from a pull request, we might need to issue another http request to get the pull request information which is not in the event payload.

So I'm suggesting to change the signature of the Parse function to:

Parse(ctx context.Context, req *http.Request, fn SecretFunc) (Webhook, error)

Since this is a breaking change, I'd like to raise some discussion first.

paginate Gitea repository list

Gitea implemented pagination for the repository list which results in only 10 repositories being returned. We have mechanisms in place for handling pagination. Here is a note from our Discourse forum:

Pagination would need to be added to the code here:
https://github.com/drone/go-scm/blob/master/scm/driver/gitea/repo.go#L42

For reference you can see how we do this with GitHub:
https://github.com/drone/go-scm/blob/master/scm/river/github/repo.go#L85

Some additional code may be required to extract the pagination data from the response (i.e. current page, next page, last page, results per page). However, if Gitea uses the same approach as GitHub (example below) than this would not require any additional code.

Link: <https://api.github.com/user/repos?page=3&per_page=100>; rel="next",
 <https://api.github.com/user/repos?page=50&per_page=100>; rel="last"

Source: https://discourse.drone.io/t/drone-only-shows-10-repositories-from-gitea/6922/16

Gitlab push with multiple commits (wrong information)

When gitlab triggers drone via the webhook with multiple commits the message gets wrong.
Here's an example of an incoming webhook:

{
  ...
  "commits": [
    {
      "id": "8c1107102080db6dbfc81adc9fcebddaff73c446",
      "message": "Merge branch 'develop' into 'REDACTED'\n\nDevelop\n\nSee merge request REDACTED",
      "timestamp": "2019-10-08T15:08:32Z",
      "url": "http://REDACTED/commit/8c1107102080db6dbfc81adc9fcebddaff73c446",
      "author": {
        "name": "Marco Santos",
        "email": "REDACTED"
      },
      "added": [
        ...
      ],
      "modified": [
        ...
      ],
      "removed": [

      ]
    },
    {
      "id": "85ca066d823d3e572fc706b8ca1ae5b8c980f1a4",
      "message": "Merge branch 'develop' into 'REDACTED'\n\nDevelop\n\nSee merge request REDACTED",
      "timestamp": "2019-10-10T16:26:51Z",
      "url": "http://REDACTED/commit/85ca066d823d3e572fc706b8ca1ae5b8c980f1a4",
      "author": {
        "name": "Marco Santos",
        "email": "REDACTED"
      },
      "added": [
        ...
      ],
      "modified": [
        ...
      ],
      "removed": [
        ...
      ]
    },
    {
      "id": "f5959a562efd643152303377fd0ed89b34f2452a",
      "message": "REDACTED\n",
      "timestamp": "2019-10-23T14:58:38Z",
      "url": "http://REDACTED/commit/f5959a562efd643152303377fd0ed89b34f2452a",
      "author": {
        "name": "REDACTED",
        "email": "REDACTED"
      },
      "added": [
      	...
      ],
      "modified": [
        ...
      ],
      "removed": [

      ]
    },
    {
      "id": "568299b5fc8d63c96d6401a551c14d3017adae82",
      "message": "Merge branch 'REDACTED' into 'REDACTED'\n\nREDACTED\n\nSee merge request REDACTED",
      "timestamp": "2019-10-23T15:15:45Z",
      "url": "http://REDACTED/commit/568299b5fc8d63c96d6401a551c14d3017adae82",
      "author": {
        "name": "Marco Santos",
        "email": "REDACTED"
      },
      "added": [

      ],
      "modified": [
        ...
      ],
      "removed": [

      ]
    },
    {
      "id": "a09757196b130e6a744b296ec0c2d809ddaf1228",
      "message": "fix bug in REDACTED\n",
      "timestamp": "2019-10-29T15:00:28Z",
      "url": "http://REDACTED/commit/a09757196b130e6a744b296ec0c2d809ddaf1228",
      "author": {
        "name": "REDACTED",
        "email": "REDACTED"
      },
      "added": [

      ],
      "modified": [
        ...
      ],
      "removed": [

      ]
    }
  ],
  "total_commits_count": 5,
  ...
}

This is the latest commits on the project in Gitlab
alt text

The project on Drone
alt text

The commit is arriving with the right user, sha ... But not the Message and URL.

According to the file: https://github.com/drone/go-scm/blob/master/scm/driver/gitlab/webhook.go (line 144)

if len(src.Commits) > 0 {
	dst.Commit.Message = src.Commits[0].Message
	dst.Commit.Link = src.Commits[0].URL
}

Was this intended to be as it is? I can open a PR with the fix, I just wanted to get some feedback regarding this.

Inconsistency in representing events in HookInput and Hook

Right now, hooks are created with parameters specified by HookInput, which contains a HookEvents to indicate which abstract event should be listened to. But the Hook struct returned by FindHook() contains Events []string, which is a list of native event names. This makes it hard to implement a "reconciliation" logic, where the program needs to check if an existing webhook matches a HookInput to decide if it should recreate the webhook. The workaround is to only use HookInput.NativeEvents and don't rely on HookInput.Events at all.

It seems reasonable to change Hook.Events from []string to HookEvents, and introduce a HookEvents.NativeEvents field to make the API more consistent. However, these are all breaking changes.

Would it make sense to deprecate HookInput.Events, HookInput.NativeEvents and Hook.Events, and instead having a new field name for HookEvents in both HookInput and Hook?

Enable pluggable drivers via RPC

It would be interesting to have an RPC driver that lets you implement your own scm client. This could be useful when you need to test / implement a driver that is not currently supported, or when you need to customize or override how an existing driver behaves.

I have started an example branch here:
https://github.com/drone/go-scm/tree/rpc

Some challenges:

  1. the rpc client will need to pass oauth1/2 credentials from the client to server. How do we do this in a generic and customizable way?
  2. some of the abstractions in this library currently leak. Specifically we need to know how to create a netrc file for cloning (in Drone) and we use the driver name to determine the exact format. We would need to work around this. Also we use the driver name to decide whether or not to ignore certain webhook types. For example, GitHub may use a push webhook for tags, and ignore tag_create webhooks because we get more information from the push.
  3. we will need to split parsing the webhook and validating the signature into separate methods. Doing this in a single function with a callback is not conducive to this rpc proposal.
  4. ideally we would do simple REST rpc (I'd prefer not to pull in a giant dependency like grpc). We would probably have to come up with some sort of convention for how we want to handle routing, passing input parameters, etc. Maybe just simple jsonrpc?

Potential import collision: import path should be "gopkg.in/h2non/gock.v1", not "github.com/h2non/gock".

Background

The h2non/gock has already renamed it’s import path from "github.com/h2non/gock" to "gopkg.in/h2non/gock.v1".
As README of h2non/gock v1.0.9 said, downstream repos should use "gopkg.in/h2non/gock.v1" to get or import h2non/gock.

**Installation**
> go get -u gopkg.in/h2non/gock.v1

**Examples**
See examples directory for more featured use cases.
Simple mocking via tests
package test
import (
  "github.com/nbio/st"
  "gopkg.in/h2non/gock.v1"
  "io/ioutil"
  "net/http"
  "testing"
)
…

But drone/go-scm still used the old path:
https://github.com/drone/go-scm/blob/master/go.mod#L5

github.com/h2non/gock v1.0.9

When you use the old path "github.com/h2non/gock" to import the h2non/gock, it will be very easy to reintroduce h2non/gock through the import statements "import gopkg.in/h2non/gock.v1" in the go source file of h2non/gock`.
https://github.com/h2non/gock/blob/v1.0.9/_examples/custom_matcher/matcher.go#L5

package main
import (
	"fmt"
	"gopkg.in/h2non/gock.v1"
	"net/http"
)
…

The "gopkg.in/h2non/gock.v1" and "github.com/h2non/gock" are the same repos. This will work in isolation, bring about potential risks and problems.

So, why not get rid of the old import path "github.com/h2non/gock ", use "gopkg.in/h2non/gock.v1" instead.

Solution

Replace all the old import paths, change "github.com/h2non/gock" to "gopkg.in/h2non/gock.v1".
Where did you import it: https://github.com/drone/go-scm/search?q=github.com%2Fh2non%2Fgock&unscoped_q=github.com%2Fh2non%2Fgock

Broken pagination function of Stash provider

func encodeListOptions(opts scm.ListOptions) string {
params := url.Values{}
if opts.Page > 1 {
params.Set("start", strconv.Itoa(
(opts.Page-1)*opts.Size),
)
}
if opts.Size != 0 {
params.Set("limit", strconv.Itoa(opts.Size))
}
return params.Encode()
}

Reference to docs https://docs.atlassian.com/bitbucket-server/rest/5.16.0/bitbucket-rest.html

Important: If more than one page exists (i.e. the response contains "isLastPage": false), the response object will also contain a nextPageStart attribute which must be used by the client as the start parameter on the next request. Identifiers of adjacent objects in a page may not be contiguous, so the start of the next page is not necessarily the start of the last page plus the last page's size. A client should always use nextPageStart to avoid unexpected results from a paged API.

The deprecation of Bitbucket API endpoint /2.0/teams breaks user registration

Hello,

I failed to register a new user with Bitbucket account with the error message below:

This endpoint has been removed. See our documentation for more information. https://developer.atlassian.com/cloud/bitbucket/rest/api-group-teams/

It looks like Bitbucket has deprecated the API endpoint /2.0/teams ​which Drone used to check whether a user is a member of a team. Is there plan to upgrade from /2.0/teams to /2.0/workspaces?

Thank you

Bitbucket Stash driver doesn't handle event `pr:from_ref_updated` (new commits / force push)

It seems to me that Drone ignores changes to from ref, i.e., new commits to PR and thus a new build is not started for the PR.

If I have understood correctly, this part is handled in stash/webhook.go and possibly just adding pr:from_ref_updated might fix this.

case "pr:opened", "pr:declined", "pr:merged":
hook, err = s.parsePullRequest(data)
}

Bitbucket Build Status Empty

Hello, I found a small issue with the way the Bitbucket build statuses are being posted. The issue, I think, is that the "name" field isn't being supplied to the Bitbucket API. The effect is that although the build is green in the PR UI, there's no link to Drone. This only affects the new (beta) PR UI, which can be enabled in a user's settings (Bitbucket Labs).

Expected behavior is that a link shows up next to the build status in the new PR UI. Example from a Jenkins build:
image

The observed behavior is that the build status is shown, but there's no link:
image

The string I see from our Jenkins builds is from the optional "name" field, so I think that's all that needs to be added. As far as I can tell, this is in the CreateStatus function in https://github.com/drone/go-scm/blob/master/scm/driver/bitbucket/repo.go. Adding Name: input.Label to the status struct should do the trick, although it might be nicer to have "name" set to the label plus the build number.

Cheers!

GitHub CommitListOptions Parameter is Out of Date

It seems to me that one of Git.ListCommits arguments CommitListOptions requires updating.
It uses ref to query commits from branch, but the GitHub official documentation (List Commits Parameters) says the query parameter should be sha

func encodeCommitListOptions(opts scm.CommitListOptions) string {
	params := url.Values{}
	if opts.Page != 0 {
		params.Set("page", strconv.Itoa(opts.Page))
	}
	if opts.Size != 0 {
		params.Set("per_page", strconv.Itoa(opts.Size))
	}
	if opts.Ref != "" {
		params.Set("ref", opts.Ref)
	}
	return params.Encode()
}

Using ref to query branch commits will always return commits from default (master) branch.
Please let me know if I am wrong. I could also create PR to update it if needed.

Support for listing the contents of a directory

Feature request
I would like to call Client.Contents.Find to discover which files and directories exist within a directory.

Use case
This is an issue for the implementation of drone config plugins that require traversing the source repository, such as drone-tree-config. See bitsbeats/drone-tree-config#2

Details
The GitHub API call to repos/%s/contents/path accepts directories, but the source code of go-scm assumes that you're calling it on a file, causing it to crash because the response has a different format than the one expected. See https://developer.github.com/v3/repos/contents/#get-contents.

Is this a feature that you could see making it into go-scm, or should Drone plugins preferably implement their own git integration logic?

Drone and Bitbucket broken for write permission detection for drone build restart permission.

Since this commit:
802e265

Changing of
path := fmt.Sprintf("rest/api/1.0/repos?size=1000&permission=REPO_WRITE&project=%s&name=%s", namespace, name)
to
path := fmt.Sprintf("rest/api/1.0/repos?size=1000&permission=REPO_WRITE&projectname=%s&name=%s", namespace, name)

Has resulted in drone not allowing users with write permission to restart drone jobs or do drone deploys. As it is not detecting the user has write permission.

Admins still work using the webhook lookup but users with only write permission can not.

PullRequest.Fork field is unintended content

It seems that the webhook payload is wrong when a Pull Request or Merge Request is issued within GitHub and GitLab.The PullRequest.Fork is always set to the repository name, even when the PR comes from a branch or fork.

I think the string should only be specified when the PR is issued from the fork repository, otherwise it should be empty.

github driver: https://github.com/drone/go-scm/blob/v1.15.2/scm/driver/github/pr.go#L153
gitlab driver: https://github.com/drone/go-scm/blob/v1.15.2/scm/driver/gitlab/webhook.go#L267
bitbucket driver: https://github.com/drone/go-scm/blob/v1.15.2/scm/driver/bitbucket/webhook.go#L585

It seems that the code that uses this library is also expected to specify an empty string if it is not forked repository.
https://github.com/drone/drone/blob/v2.4.0/plugin/secret/encrypted.go#L57

Gitea API ref causing build failure

Gitea has changed their API for getting commit refs in their recent version 1.15.0+dev-183-gec69f3472.
Now the reference of commit head should be fetched using
image

The /commit/ has somehow been removed from the URL. Meanwhile, the drone container lists the commits with

path := fmt.Sprintf("api/v1/repos/%s/git/commits/%s", repo, url.PathEscape(ref))

Could you please confirm if this is a bug or it's possibly because of other issues?

Request to support creating a webhook with the scope all

In some cases, users need to add a webhook with the scope all. For example, users want to receive all events. On a GitHub repository, the option is Send me everything.

Possible solution

Define a special event name: all. Add all events if there is a all event.

Bitbucket Author Username empty because of API changes

Atlassian changed the API and it removed the "username" field from the user object. As documented here:
https://developer.atlassian.com/cloud/bitbucket/bitbucket-api-changes-gdpr/#removal-of-usernames-from-user-referencing-apis

This leads to an empty Username field and as a consequence the author field in drone will be empty. For example if you use the slack notification plugin the build.author field will be empty.

I think this issue has the same root: https://discourse.drone.io/t/branch-author-not-works-in-slack-plugin/4624

I think the a good replacement would be to use the nickname as a replacement for username.

This line is affected:

Username string `json:"username"`

gitea find commit

gitea

  • api/v1/repos/${autour}/${repo}/git/commits/${sha}
    +autour 仓库所有者
  • repo 仓库
  • sha 提交码

return json contain files

没有返回 files 给 commitInfo

Namespace in GitLab is wrong

Hello. I have an issue in GitLab repository.

Some repositories in my GitLab belong to sub-group, such as foo/bar/baz/sample.
In this case, sample is project name and foo/bar/baz is namespace in my opinion.
But actually only baz is parsed as namespace via this code.

Related codes are below.
https://github.com/drone/go-scm/blob/master/scm/driver/gitlab/repo.go#L172

func convertRepository(from *repository) *scm.Repository {
	to := &scm.Repository{
		ID:        strconv.Itoa(from.ID),
		Namespace: from.Namespace.Path,
		Name:      from.Path,
		Branch:    from.DefaultBranch,
		Private:   convertPrivate(from.Visibility),
		Clone:     from.HTTPURL,
		CloneSSH:  from.SSHURL,
		Perm: &scm.Perm{
			Pull:  true,
			Push:  canPush(from),
			Admin: canAdmin(from),
		},
	}
	if to.Namespace == "" {
		if parts := strings.SplitN(from.PathNamespace, "/", 2); len(parts) == 2 {
			to.Namespace = parts[1]
		}
	}
	return to
}

As I know, Drone also uses this drone/scm project to communicate various kinds of repositories. And the above issue makes some errors to sync GitLab repository.

GitLab response of foo/bar/baz/sample project is as below.

[
  {
    "id": 1000,
    "name": "sample",
    "path": "sample",
    "path_with_namespace": "foo/bar/baz/sample",
    "namespace": {
      "id": 256,
      "name": "baz",
      "path": "baz",
      "kind": "group",
      "full_path": "foo/bar/baz",
      "parent_id": 14
    },

I suggest that namespace.full_path is used as namespace property.

Reference: https://discourse.drone.io/t/drone-cannot-parse-sub-group-project-path/6361

State not set for Gitea commit status

I haven't been able to get Drone's status to be reflected back in Gitea. When I look at the commit status via the Gitea API, I see that statuses have been created from drone with almost all the right information, but the status field is an empty string.

The webhook to gitea's status API seems to be based on Github's Status API, which has state instead of status for the JSON object being posted.

Manually editing the HTTP request Drone makes back to gitea to change the object key from status to state seems to do what is wanted.

Looks like this line should have json:"state" instead.

Making this an issue instead of a PR because I'm not entirely sure if I've got it all right in my head and have only done a little bit of research on this so far.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.