google / go-github Goto Github PK

Text match metadata returns context on what matched in code search.

It's the same API that GitHub is using when you search on the site (or so I suspect by how they wrote their documentation).

To get this metadata, we just have to supply the text-match media type in the Accept Header.

application/vnd.github.v3.text-match+json

This also provides another field in the search results, a text_matches array.

I have blatantly stolen from their own documentation for this issue, check it out for more details.

There's no rush on this, I'm just leaving a note for whoever ends up implementing the watcher and star APIs, since I just noticed this. These APIs are going through a 3 phase transition, during which the meaning of the term "watcher" is changing.

I haven't actually read though the transition plan closely enough to know which phase we're currently in, but my point here was to make sure that we use the terms "watch" and "star" correctly in the function names. I'm not too concerned which endpoint we hit (it looks like /subscribers is a temporary stopgap) or if we use the beta mime type... whichever way we do it, we should just make sure we pick the function names that will be stable, even if we switch to use a different API URL at some point.

• Repos -> List Branches can be marked as "Done" - merged PR #86

Also the following are deprecated API's:

• Downloads (replaced with Releases, not on spreadsheet) -> Replace w/ Releases?
• Legacy search (replaced with completed "Search" spreadsheet group) -> Remove?

Hi,

I'm trying to get at the login attribute within author that's nestled between commit and parents. I don't see any of the structs supporting that. I'm more than happy to submit a patch to enable this, I'd just like a bit of input before I go hacking like crazy. If you happen to know a method by which you can access it, I'd be very grateful.

Here's what github's API specs say is returned: https://developer.github.com/v3/repos/#response-10

At a minimum, I'll need to add the ability to list PRs for a repository, update a PR, list comments on a PR, and add a new comment.

The github API v3 declares that blank fields are included with a null as the value instead of being omitted.

For a user which did not specify an organisation, blog or bio the Get returns null as the field values.
In go-github this leads to an unmarshalling failure as type string is defined in the struct type User. Unfortunately, null can not be unmarshalled/mapped to string.

A potential solution may be to change the type from string to pointers, allowing nil.
But then this should probably be implemented as a general concept across the library, as the API docs state that generally empty fields are returned as null values - and does not seem to note which fields can be empty/null in the end.

see comment at https://github.com/google/go-github/pull/4/files#r4714862

There is at least ListMembers() and ListPublicMembers() as mentioned in that comment, but there may be others. These should be updated to use the calling style @sqs has.

The github api documentation shows that the /issues api endpoints are paginated.

The pagination is information is returned in the header like this:

Link:[<https://api.github.com/issues?direc
tion=desc&filter=all&labels=&sort=updated&state=open&page=2>; rel="next",

Unfortunately as far as I can tell the go-github package do not expose this information.
listIssues does not return the Headers nor the response. What is in your opinion the best approach to fix this issue ?

• Changing the method signature to return the response.Headers.
• Changing the method signature to return the response.
• Changing the method signature to return a new struct that exposes some of the information returned in the response.Header

To put it in a nutshell I can't think of an easy way to make a backward compatible changes to fix this bug.

starting work on the new search API

The GitHub API docs are now CC-BY licensed, so we can safely copy method and parameter descriptions. The LICENSE file should be updated to reflect the license.

http://developer.github.com/v3/users/followers/

Depending on how our types are defined, we run into various interesting problems with setting empty values.

Option 1

This is the current behavior throughout the library.

As currently setup, we can't set empty values at all on Create or Update methods. For example, this won't work to remove the description from an existing repository:

// remove the repository description
r := &github.Repository{Description:""}
client.Repositories.Edit("user", "repo", r)

That is actually a no-op because the empty Description field gets dropped, since it is currently defined with omitempty. Fortunately, there are only a handful of mutable values where the zero value is actually meaningful, most notably bool values.

Option 2

We could then instead drop the omitempty, but that has potentially really bad side-effects, particularly on edit methods. Take the above code sample. Without omitempty on any of our fields, this would work as intended. However, it would also have the unintended side-effect of wiping out the repo homepage, [and once we add the additional fields...] making it public, and disabling issues and the wiki for the repo, since all of those fields would be passing their zero values. The solution there is to first call Get, update the response, and then call Edit with the updated object:

// remove the repository description
r, _ := client.Repositories.Get("user", "repo")
r.Description = ""
client.Repositories.Edit("user", "repo", r)

If you forget to follow that flow, you're gonna have a bad time.

Option 3

The third option is to do what goprotobuf does and actually use pointers for all non-repeated fields, since that allows a clear distinction between "unset" (nil) and "empty" (non-nil, zero value). That would result in types like:

type Repository struct {
    ID          *int        `json:"id,omitempty"`
    Owner       *User       `json:"owner,omitempty"`
    Name        *string     `json:"name,omitempty"`
    Description *string     `json:"description,omitempty"`
    CreatedAt   *time.Time  `json:"created_at,omitempty"`
    PushedAt    *time.Time  `json:"pushed_at,omitempty"`
    UpdatedAt   *time.Time  `json:"updated_at,omitempty"`
}

This is by far the safest approach, but does make working with the library a bit cumbersome, since creating pointers to primitive types takes a little extra work (multiplied by the number of fields you are setting). For example, the above code sample now becomes:

// remove the repository description
d = ""
r := &github.Repository{Description:&d}
client.Repositories.Edit("user", "repo", r)

The goprotobuf library makes this a little simpler by providing helper functions for creating pointers to primitives. Using those methods would result in:

// remove the repository description
r := &github.Repository{Description:proto.String("")}
client.Repositories.Edit("user", "repo", r)

Additionally, when working with these values, developers will always have to remember to dereference them where appropriate. While this is common for anyone used to working with protocol buffers in go, I'm not sure how unexpected it will be for the general community.

The required change is trivial:

diff --git a/github/gists.go b/github/gists.go
index ec662b5..48a0eec 100644
--- a/github/gists.go
+++ b/github/gists.go
@@ -56,6 +56,8 @@ func (g GistFile) String() string {
 type GistListOptions struct {
        // Since filters Gists by time.
        Since time.Time `url:"since,omitempty"`
+
+       ListOptions
 }

 // List gists for a user. Passing the empty string will list

Even though it's not strictly part of the v3 API, it would be nice to define structs for GitHub's post-receive webooks, so that those messages can be easily parsed and acted on.

opening this bug to track implementation of using URI templates to simplify construction of request URLs. Original description at jtacoma/uritemplates#3

The gist of it is that I'd like to go from this:

type ListOptions struct {
        Page int
}

func (s *OrganizationsService) List(user string, opt *ListOptions) ([]Organization, error) {
        u := fmt.Sprintf("users/%v/orgs", user)
        if opt != nil {
                params := url.Values{
                        "page": []string{strconv.Itoa(opt.Page)},
                }
                u += "?" + params.Encode()
        }
        ...
}

to something more like this:

type ListOptions struct {
        Page int `url:"page"`
}

func (s *OrganizationsService) List(user string, opt *ListOptions) ([]Organization, error) {
        u := fmt.Sprintf("users/%v/orgs", user)
        u += uritemplates.Parse("{?page}").Expand(ListOptions)
        ...
}

or possibly something even simpler with a little more work.

I've seen a couple of other GitHub libraries do this, and it's almost certainly worth adding here as well. github.Client should add a Rate field which is updated after each request using the rate-related HTTP headers in the response. This can all be done inside the Do() func, so it should be a pretty minimal change.

If the OAuth credentials are being shared between multiple clients then this rate my actually get out of date, so the documentation for the field should also point to the RateLimit() method to retrieve the most up-to-date limit for the client.

As discussed in issue #34, the convention that has been previously used to name functions that check if something exists in a given state has been Check*(). For example, CheckAssignee().

In order to make these more consistent to the style of the go stdlib functions (see math and os), these should be changed to Is*() (i.e. CheckAssignee() -> IsAssignee()).

Some of these functions are not always in the Check*() form, such as Starred().

A good place to start would be to search for parseBoolResponse(). Most of these functions use this to determine their result.

Might as well make this one large issue and cross things off as we go instead of making 1 issue per API. There are still a few APIs missing for the repository service:

• Repository
• Collaborators
• Comments : @wlynch92
• Commits: @ktoso
• Contents
• Forks
• Keys: @willnorris
• Hooks
• Merging
• Releases
• Statistics
• Statuses

I'll edit this post as the APIs are assigned/implemented. If any one wants to help in implementing any of these APIs feel free to comment saying which you'll be working on.

( original discussion on sqs/go-github@515d5d7 )

Resources in the GitHub API are often truncated when returned from certain API endpoints. For example, fetching a single repository returns the full representation including properties like source and parent, but fetching a list of repos returns a truncated view. This is even more apparent when comparing the view of a single user to a list of org members. The latter truncates about half of the resource properties; and yet even more properties are included if you make an authenticated request for your own account.

The question then becomes, how should we represent this in the library? The two basic options are:

1. Define a type for each permutation of a resource and use embedded structs to build up the more defined types. Quinn has a more complete example of this in sqs/go-github@515d5d7, but it would amount to something like:

   type Repository struct {
       // common set of fields that every repository API endpoint returns
       ID          int        `json:"id,omitempty"`
       Owner       *User      `json:"owner,omitempty"`
       Name        string     `json:"name,omitempty"`
       // etc
   }
   
   type DetailedRepository struct {
       // fields returned when getting a user's or org's repositories
       Repository
       FullName     string     `json:"full_name,omitempty"`
       Private      bool       `json:"private"`
       Fork         bool       `json:"fork"`
       // etc
   }
   
   type FullRepository struct {
       // fields returned when getting a single repository
       DetailedRepository
       Parent Repository `json:"parent,omitempty"`
       Source Repository `json:"source,omitempty"`
   }

   Each function would returned the relevant type, based on how much data the GitHub API returns, making it very explicit how much information to expect. This comes at the cost of more types to deal with and to understand the hierarchy of.

2. Define a single type which includes all possible fields, and document under which circumstances a truncated resource will be returned. This simplifies the library surface and interaction between different functions, at the cost of potential confusion as to why certain fields are missing in different cases.

It's worth pointing out that the source code used to generate the GitHub API documentation uses the former approach, whereas the Objective-C Octokit library appears to use the latter. For what it's worth, the ruby Octokit library punts entirely be returning untyped hashes.

Given an Issue, I want to comment or patch on it. However, to do that I need owner, repo and number, which are buried in the *issue.URL. It would be nice if either issues had CreateComment and Edit methods which received a *Client, or if they had a method to obtain those three fields easily.

I would like to write tests for an app that uses a lot go-github.

For instance I would like to mock some methods in the client.*Services structs. Is this possible with the current usage of structs ? Wouldn't it be preferable that the client struct holds some interfaces rather than some structs ? With interfaces, mocking is very easy with struct embedding.

see 5f1c20a for more details

Hi, I have made a simple library for webapp to get access_token to then be able
to be used with your library: https://github.com/vvo/go-ghoauth

Reading your https://github.com/google/go-github#authentication it seems that we need
to create an http client etc.. Do you think it would be a good idea to be able to simply do client.AccessToken = "token"

?

This just saves some code. The thing is that in the pull request object, there is a field called statuses_url, which can be directly used to create new commit statuses. The current API implements CreateStatus, which internally generates the same URL, the user just has to take care of getting the right ref, which is, however, already supplied in statuses_url. So it could make sense to implement PostStatus or so, which would take the whole URL path, e.g. octocat/Hello-World/statuses/6dcb09b5b57875f334f61aebed695e2e4193db5e. So the API could be PostStatus(url string, status *RepoStatus). What do you think?

I will implement this anyway since I need it. Perhaps just let me know what do you think about the method name...

Not much to say, http://developer.github.com/v3/git/refs/

I am already working on this and I will create a pull request once I push some tests.

I noticed that there is issue #51 open for tags support. We may define the same stucts, namepy GitObject, so I guess the one being merged later will have to modify his code...

http://developer.github.com/v3/issues/milestones/

current use case needs basic issues methods, and support for comments:

• http://developer.github.com/v3/issues/
• http://developer.github.com/v3/issues/comments/

GitHub API endpoints accept conditional requests using ETag and/or If-Modified-Since (http://developer.github.com/v3/#conditional-requests).

This library could support conditional requests by modifying API methods to take a struct parameter containing those fields (or the fields could be added to, e.g., ListOptions, as in https://github.com/sqs/go-github/compare/b01324...User_events__xpr).

Or there could be a more serious refactor where the current API becomes the high-level API and there is a new lower level API that separates the construction of abstract API requests from the HTTP headers and client. You would first create the abstract API request (and each existing API method would correspond to a helper method for creating the abstract API request), and then pass it to an executor along with any arbitrary set of HTTP headers you want to use for that particular request.

I'm interested in support for conditional requests and wanted to get feedback on the right approach. (Let me know if it's outside the scope of what you want in the library.)

@wlynch92's comment:

I was thinking about adopting some sort of naming scheme that would easily allow us to separate and distinguish source files and the APIs they provide. Instead of looking through repos.go (which after adding all 12 APIs, might be very lengthy) for the ListHook function, why not separate the hook API into something like repos_hooks.go. This lets us sort APIs into separate files while still keeping everything in a single package and making a clear relation to the service they are a part of. It also is a close mapping to the overall layout of the Github API (i.e. http://developer.github.com/v3/repos/ -> repos.go and http://developer.github.com/v3/repos/hooks/ -> repos_hooks.go).

I think this is something we should definitely consider for a large services such as RepositoryService, and apply to smaller services as a matter of consistency.

I'm just curious, even for bool, int and string, is there any specific reason for doing this?

Thanks!

On latest code on master

G15491-2:go-github sgaddipati$ g show head --name-only
3a1d485 - (HEAD, origin/master, origin/HEAD, master) Fix PushEvent JSON representation of size attribute (4 weeks ago) <Tobias Schmidt>
github/activity_events.go

I get test failures

G15491-2:go-github sgaddipati$ go test -v ./github
# go-github/github
github/orgs_members_test.go:42: cannot use "StatusBadRequest" (type string) as type *testing.T in function argument
github/orgs_members_test.go:42: not enough arguments in call to testURLParseError
FAIL    go-github/github [build failed]

If an issue has a pull request, a pull request JSON payload is returned. That JSON payload should be parsed into a PullRequest struct and attached to the Issue.

as part of resolving #19, all data structures now use pointer field values. To ease the burden on developers to constantly perform nil checks, we should generate Get*() funcs on all our data structures, similar to what goprotobuf does. If a field is non-nil, return it. Otherwise, return the zero value for that type. So for example,

func (m *User) GetLogin() string {
    if m != nil && m.Login != nil {
        return *m.Login
    }
    return ""
}

Go provides good capabilities to read and manipulate go source code, so this can be completely generated. We might need to move all these types into a single data.go file, I'm not sure.

http://developer.github.com/v3/users/emails

The Issue struct type is missing the milestone and labels fields the github API v3 docs defines.

• http://developer.github.com/v3/issues/#create-an-issue
• http://developer.github.com/v3/issues/#get-a-single-issue

I am going to start adding the Repository APIs that are currently not implemented. First up is Collaborators.

I'll be making separate issues for these APIs so others can hop in on other Repository APIs if/when they want.

take for example:

repo, _, err := client.Repositories.Get("foo", "bar")

If foo/bar is not a valid repository, I would expect repo to be nil. Currently, it's set to an empty (but non-nil) Repository. I guess I just didn't really think about this when setting things up initially. This should be fixed throughout the library. It won't require changing any method signatures, just some behavior.

When trying to create a private repo for an organization I got the following error form github:

POST https://api.github.com/orgs/<organization_name>/repos: 422 Validation Failed [{Resource:Repo Field:team_id Code:invalid}]

While it isn't documented as required for organizations the error message seams to indicate that it is. https://developer.github.com/v3/repos/#create

I have a PR ready if this is indeed a bug and I am not missing something.

https://github.com/mattlandis/go-github/compare/add-team-id

https://developer.github.com/v3/oauth_authorizations/

One tricky part is that it only accepts basic auth, so I think it'd require users of the library to implement a simple RoundTripper that added this header (and then use that as the HTTP client transport for go-github).

The Assignee field in the create request is of type 'string', while the Issue struct has a full User struct. If you do something like this (note: sp() creates a string pointer):

issue := &github.Issue{
Title: sp("My new bug."),
Body: sp("Some text describing that bug..."),
Assignee: &github.User{Login: sp("fluffle")},
}

when this is serialized to JSON the 'assignee' field will contain {'login': 'fluffle'}.

I suspect (but haven't yet tested) that similar things will happen with Labels too. I don't think you can get away with using the same types for both API input and output; you may well need a type IssueCreate struct {} &c unless you want to teach all your types how to serialize themselves for specific uses.

I can send you a pull req for IssueCreate if you're interested. Nice API otherwise! Also, you can find me at who/abramley if you need to :-)

Cheers,
--alex

Gists API doc: http://developer.github.com/v3/gists/

@ttacon: I found the time type I was thinking of in #94 ... it's our own Timestamp type... it supports unmarshaling from either unix or RFC timestamps. Could you look into updating the repo stats to use that where appropriate?

https://developer.github.com/changes/2014-02-28-issue-and-pull-query-enhancements/

This is particularly important if you want to process issue events coming from GitHub webhooks. There is no other way how to detect the source repository other than parsing the issue URL.

I will personally take care of this if there are no issues with simply adding URL field to the Issue struct.

This is an ongoing work item to add integration tests for the entire library. Unlike the existing unit tests that spin up a local HTTP server that mimics certain behavior of the API, these integration tests will call against the live GitHub API. As such, they will likely take a lot longer to execute and will not be run near as often. I've set things up with aff0a13 and will continue working from there.

I'm not 100% settled yet on how extensive these tests need to be. For example, we don't need to test the value of every field in a response. The biggest goal is to have a test suite that can be run when things like the default media type changes to make sure the library is still working properly. I also have a separate test suite that I'm working on to identify fields in responses that we aren't mapping into our Go structs.

When i try to search against the Github api using the github example i get an 422 error.

In go code this looks like:

opt := &github.SearchOptions{Sort: "updated"}
client.Search.Code("addClass+in:file+language:js+repo:jquery/jquery", opt)

The error looks like:

error: GET https://api.github.com/search/code?q=addClass%2Bin%3Afile%2Blanguage%3Ajs%2Brepo%3Ajquery%2Fjquery&sort=updated: 422 Validation Failed [{Resource:Search Field:q Code:invalid}]

The Problem is that go encodes the query string and not only the params.
The Github Api isn't able to deal with such queries.

Go generated url:

$ curl https://api.github.com/search/code?q=addClass%2Bin%3Afile%2Blanguage%3Ajs%2Brepo%3Ajquery%2Fjquery&sort=updated

=> does not work

Same curl without encoding:

$ curl https://api.github.com/search/code?addClass+in:file+language:js+repo:jquery/jquery&sort=updated

=> works

We will prepare a pullrequest for this issue.

A handful of API methods (like issues) support the use of custom mime types to return additional fields in the response, using different formats.

As currently designed, go-github doesn't have a simple way to expose this, since most HTTP-specific details are hidden from the caller. We could probably just add it to the Options parameter for the given method, but that feels a little weird. Additionally, I don't yet have a real use-case for needing to use these custom mime-types. Therefore, I'm currently leaning toward punting on this until someone actually needs it.

So if you need it, please comment here, and we can look at how to work that into the library.

From http://developer.github.com/v3/repos/commits/:

A special note on pagination: Due to the way Git works, commits are paginated based on SHA instead of page number. Please follow the link headers as outlined in the pagination overview instead of constructing page links yourself.

go-github currently assumes index based pagination using the page URL parameter.

With the new fields tool, it's now quite easy to identify which JSON fields are not being mapped into our resource structs. Most of these are API URLs, but there are also a handful of other fields that got overlooked.

Originally, I had pushed back a bit on adding the API URLs since this library doesn't use them at all. I've long since given up on that since a few use-cases have been mentioned, so we might as well go ahead and add them all. I also have a few ideas for how the library could actually use these URLs or ones like them.

Related, the API Versions page documents some changes in API resources that we should make sure are addressed.

Hi,
I'm writing a similar library to this one for a different project. I'm curious about how you'd choose to implement paging via channels, if you wanted to do this.

Particularly I am wondering about how you'd pass errors through. Consider if the API goes down or exceeds the timeout while you are paging. It would be bad to not handle this case - I suppose you could just close the channel in the event of failure, but that doesn't seem good either.

I was thinking about something like this

type OrganizationPagingResult struct {
    Orgs []Organization
    // or some object here to hold errors.
    Err Error
}

for pagingResult := range myChannel {
    if pagingResult.Err != nil {
        return nil, pagingResult.Err
    }
    for i := 0; i < len(pagingResult.Orgs); i++ {
        org := pagingResult.Orgs[i];
        fmt.Println(org.ID);
    }
}

Then the channel returns one page at a time. Compared with the channel yielding one organization at a time, this simplifies the logic in the channel as the channel only has to fetch the next page, instead of sometimes yielding results and sometimes fetching and yielding a new page. Also the error is likely to come from the paging request failing, not the iterating over individual results.

You have more experience with Go and you've also used the library internally, so I was wondering if you had feedback on this or a proposed implementation for this library.