ubisoft / ngitlab Goto Github PK

Since the version 6.11.0, there seems to be an issue where doing multiple requests (for functional tests for example) times out when getting the response.

Versions before 6.11.0 worked perfectly fine.

GitLap API doc: https://docs.gitlab.com/ee/api/merge_requests.html#response

Couldn't work out in this client implementation if POST /projects/:id/statuses/:sha was implemented within this library.

Came to this library from a previous library which seems to not getting the latest API endpoints also nmklotas/GitLabApiClient#219

Cheers

Currently, the library only handles project hooks, and we haven't extended support to group level hooks just yet.

It's something we're considering, especially since it's important to account for group level hooks when retrieving a project's hooks.

Our main focus right now is on reading hooks rather than creating or editing them.

Additional Information

• GitLab API Documentation

https://docs.gitlab.com/ee/api/job_artifacts.html#download-a-single-artifact-file-by-job-id
https://docs.gitlab.com/ee/api/job_artifacts.html#download-a-single-artifact-file-from-specific-tag-or-branch

When downloading a single artifact file that does not exist, GitLab will return an empty 404 response with a zero-length body.

Maybe this behavior only appeared in 16.0+, I upgraded from 15.6 to 16.3 yesterday and then met this problem I've never seen before.

Anyway, we can reproduce it from gitlab.com, eg.

• 200: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/jobs/5527580160/artifacts/coverage-frontend/lcov.info
• 404: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/jobs/5527580160/artifacts/foo/bar

For the 404 empty response, NGitLab will throw a System.IO.IOException from GitLabRequest.HandleWebException at https://github.com/ubisoft/NGitLab/blob/6.42.2/NGitLab/Impl/HttpRequestor.GitLabRequest.cs#L133

Stacktrace:

System.IO.IOException: The response ended prematurely, with at least 2 additional bytes expected.
   at System.Net.Http.HttpConnection.ContentLengthReadStream.Read(Span`1 buffer)
   at System.IO.StreamReader.ReadBuffer()
   at System.IO.StreamReader.ReadToEnd()
   at NGitLab.Impl.HttpRequestor.GitLabRequest.HandleWebException(WebException ex) in /_/NGitLab/Impl/HttpRequestor.GitLabRequest.cs:line 133
   at NGitLab.Impl.HttpRequestor.GitLabRequest.GetResponseImpl(RequestOptions options) in /_/NGitLab/Impl/HttpRequestor.GitLabRequest.cs:line 106
   at NGitLab.Extensions.FunctionRetryExtensions.Retry[T](Func`1 action, Func`3 predicate, TimeSpan waitTime, Int32 maxRetryCount, Boolean useExponentialBackoff) in /_/NGitLab/Extensions/FunctionRetryExtensions.cs:line 41
   at NGitLab.Impl.HttpRequestor.Stream(String tailAPIUrl, Action`1 parser) in /_/NGitLab/Impl/HttpRequestor.cs:line 118
   at NGitLab.Impl.JobClient.GetJobArtifact(Int32 jobId, String path)

NGitLab.Impl.Json.Serializer is an internal class for now.

I have 2 use cases where I want to reuse the serializer in my own code:

1. For a special purpose, we want the Get job token’s job API, it's not a common use case and it requires a special authentication mechanism that is different from other API. So, I don't think it falls under NGitLab. I made my own client for this API, and I reused some models from NGitLab. But as you know, we need some JSON converters to serialize or deserialize with GitLab. If we can't reuse the NGitLab.Impl.Json.Serializer we have to reimplement them.

2. We plan to carry out secondary development of GitLab in the future and may add some customized API. At this point, we have to implement our own clients, but they can still reuse some models from NGitLab, and they also follow GitLab's serialization rules which need the serializer.

To sum up, I request to make NGitLab.Impl.Json.Serializer public.

We should add Microsoft.CodeAnalysis.PublicApiAnalyzers to ensure we do not introduce breaking changes to NGitLab and NGitLab.Mock

When fetching an image that was stored in LFS, the Repository.GetFile method returns an empty image.

Hi,
Could the license information be added to the nuget information?
it is good to add it so it will be shown in Visual Studio nuget management.

for example,I use the following <PackageLicenseExpression>MIT</PackageLicenseExpression> in my nugets:

<PropertyGroup>
	<PublishRepositoryUrl>true</PublishRepositoryUrl>
	<IncludeSymbols>true</IncludeSymbols>
	<SymbolPackageFormat>snupkg</SymbolPackageFormat>
	<ContinuousIntegrationBuild>true</ContinuousIntegrationBuild>
	<PackageLicenseExpression>MIT</PackageLicenseExpression>
	<RepositoryType>git</RepositoryType>
	<Nullable>enable</Nullable>
	<LangVersion>latest</LangVersion>
</PropertyGroup>

Also some tools can create auto generated report like (dotnet-project-licenses) but right now those create missing entry:

In a GitLab server, IDs and IiDs are different, but mocked merge requests do not mimic that behavior.

Say you're creeating a mocked MR:

• The id is set using mergeRequest.Id = Server: Interlocked.Increment(ref _lastMergeRequestId);
  -> ID = 1
• The IiD however uses a different logic: mergeRequest.Id = MergeRequestsCollection: Select(mr => mr.Iid).DefaultIfEmpty().Max() + 1;
  -> IiD = 1

As a solution, we could just keep incrementing the ID.

The Job IDs are at "4681067651" in my GitLab.com instance. However, this may impact all IDs.

Running the following code throws an exception:

var projects = client.Projects.Get(new ProjectQuery() { });

foreach (var project in projects)
{
    var pipelines = client.GetPipelines(project.Id);

    foreach (var pipeline in pipelines.AllJobs)
    {

    }
}

System.Text.Json.JsonException
  HResult=0x80131500
  Message=The JSON value could not be converted to System.Int32. Path: $[0].id | LineNumber: 0 | BytePositionInLine: 17.
  Source=System.Text.Json
  StackTrace:
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at NGitLab.Impl.Json.Serializer.Deserialize[T](String json) in /_/NGitLab/Impl/Json/Serializer.cs:line 29
   at NGitLab.Impl.HttpRequestor.Enumerable`1.<GetEnumerator>d__5.MoveNext() in /_/NGitLab/Impl/HttpRequestor.cs:line 193
   at GitLabTool.Worker.<ExecuteAsync>d__4.MoveNext() in C:\Repos\GitLabTool\Worker.cs:line 31
   at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>d__12.MoveNext()
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.<RunAsync>d__4.MoveNext()
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.<RunAsync>d__4.MoveNext()
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
   at Program.<Main>$(String[] args) in C:\Repos\GitLabTool\Program.cs:line 10

Inner Exception 1:
FormatException: Either the JSON value is not in a supported format, or is out of bounds for an Int32.

Hey!

Is it possible to monitor the progress when downloading big artifact files?
If not, where would be a good place to implement this kind of behavior?

Thank you very much for this awesome library!

What is the recommended way to go from a Project URL to a Project Id, to use many of the methods that take an int parameter?

Let's say I have a GitLab Project at https://gitlab.com/gep13/grm-test, how can I find the Project Id?

I am sure that this is obvious, but couldn't find anything immediately showing how to do this.

Thanks!

Hi,
Is there a way to get all related issues for a specific issues?
I normaly uses primary issues with "sub issues" linked to that one so this option to get it will be usefull to me for reporting.

In the recent versions of GitLab (15.7+) a new 'raw' flag has been introduced along side of 'masked' and 'protected'.
The purpose of this flag is to be able to signal whether or not the variable value should be expanded or used raw.
When working with variables it would be helpful to have the possibility to set this, so we do not have to rely on the default value.

Docs:
https://about.gitlab.com/releases/2022/12/22/gitlab-15-7-released/#support-the--character-in-cicd-variables
https://gitlab.com/gitlab-org/gitlab/-/issues/17069
https://docs.gitlab.com/ee/api/project_level_variables.html
https://docs.gitlab.com/ee/api/group_level_variables.html

I don't think we can specify a due date when creating or editing an issue, is there a reason this is omitted or is it something I could look at adding?

Hi,

how can I trigger a pipeline when the property variable in the PipelineCreate object is readonly.
https://github.com/ubisoft/NGitLab/blob/main/NGitLab/Models/PipelineCreate.cs

Shouldn't that be assignable?

In a project I am working on, we heavily use GitLab webhooks to automate our workflow. Now when writing tests, we wanted to incorporate these things as well.

I could not find a way to use NGitLab.Mock to listen for changes to manually call our webhook with the relevant data. Am I missing something? Does this exist, is there existing API that could be leveraged to make something like this?

Essentially what I would need is some stream or callback handler thing that allows me to get notified when an issue or a merge request or something alike changes and then call a certain function.

Commit 947927b causes a breaking change.
Some string that had the '' character previously replaced by a '-' keep their '' char.

For example, before this commit, a string like the following: 'my_name' was replaced by: 'my-name'.

Now it stays : 'my_name'.

Hi,

Just wondering if there's a way to retrieve a single issue using its ID (with no project ID) as shown in the GitLab API docs here - I might be just missing something, in which case apologies!

Thanks for maintaining this, it's been really useful in our project ⭐ .

Reset approvals of a merge request
https://docs.gitlab.com/ee/api/merge_request_approvals.html#reset-approvals-of-a-merge-request

Hi everyone and thank you for this awesome library!

Currently it is possible to access time stats on issues:

var timestats = await _gitLabClient.Issues.TimeStatsAsync(
  issue.ProjectId,
  issue.IssueId,
  CancellationToken.None);

The same does not work with merge requests as the TimeStatsAsync is missing:

var timestats = await _gitLabClient.MergeRequests.TimeStatsAsync(
  mergeRequest.ProjectId,
  mergeRequest.Id,
  CancellationToken.None);

This API does not seem to exist. According to GitLab documentation the time stats are provided along with merge requests themselves.

Can the merge request model be extended with a TimeStats property or the client be extended with TimeStatsAsync method?

Hi, is it possible to create an empty commit (git commit --allow-empty) using this API Client?

thx

Hi/Bonjour!

I have with the FileClient.Update method of the API because it writes the file content directly in the URL query parameters. I think the parameter should be passed in the body instead.

I can gladly do a PR for that but before I would like an active contributor to confirm if there's an actual reason for this behavior.

Thanks!

Hi,
First off, thanks so much for maintaining this managed API. I'm coming from the v3 API and going v4, but having a hard time with a tool we use to auto merge requests and publish them to our filesystem. This is the v3 API code which worked before, and I've got 90% of the code refactored for v4, but struggling with the filesClient and fileUpsert object. I started looking into the repoClient to see if I need to use that, but wanted some help before I go down the wrong path. The PublishAction member is just a simple local struct (defined below the the TakeActions method. The main issue seems to be that the FileData and other properties are null. Thanks for any help or direction.

private static List<string> TakeActions(string basePath, IFilesClient fileClient, List<PublishAction> actions) { List<string> actionLog = new List<string>(); List<PublishAction> deferredActions = new List<PublishAction>();
        foreach (PublishAction action in actions)
        {
            if (action.Type == FilePublishActionType.Ignore)
            {
                actionLog.Add(string.Format(&quot;{0}: {1} ({2})&quot;, action.Type.ToString().ToUpper(), action.File.Path, action.Reason));
            }
            else
            {
                string publishPath = string.Format(
                    @&quot;{0}\{1}&quot;,
                    basePath,
                    action.File.Path.Replace(&#39;/&#39;, &#39;\\&#39;)
                );

                string directory = Path.GetDirectoryName(publishPath);

                switch (action.Type)
                {
                    case FilePublishActionType.Create:
                    case FilePublishActionType.Update:
                        if (!Directory.Exists(directory))
                            Directory.CreateDirectory(directory);

                        FileData file = fileClient.Get(action.File);
                        byte[] bContent = Convert.FromBase64String(file.Content);
                        File.WriteAllBytes(publishPath, bContent);

                        actionLog.Add(string.Format(&quot;{0}: {1}&quot;, action.Type.ToString().ToUpper(), action.File.Path));
                        break;
                    case FilePublishActionType.Delete:
                        if (File.Exists(publishPath))
                        {
                            File.Delete(publishPath);
                            actionLog.Add(string.Format(&quot;{0}: {1}&quot;, action.Type.ToString().ToUpper(), action.File.Path));

                            actionLog.AddRange(DeleteEmptyDirectories(basePath, directory));
                        }
                        break;
                }
            }
        }

        return actionLog;
    }

    private struct PublishAction
    {
        public FilePublishActionType Type;
        public FileData File;
        public string Reason;
    }

    private enum FilePublishActionType
    {
        Create,
        Update,
        Delete,
        Ignore
    }

As tests are pretty slow to run, it's quite frustrating to have to relaunch them because of some styling issues.

Ex: https://github.com/ubisoft/NGitLab/pull/32/checks?check_run_id=2914801455

I miss the api request to get job artifacts by ref and file path.

see
https://docs.gitlab.com/ee/api/job_artifacts.html#download-a-single-artifact-file-from-specific-tag-or-branch

I've been trying to get the subgroups for a group, however I don't see any other way than to get all groups and then filter by parent id. This seems a bit roundabout, seeing how gitlab does offer a list subgroup endpoint:

https://docs.gitlab.com/ee/api/groups.html#list-a-groups-subgroups

This endpoint is basically a scoped groups endpoint.
This plus the list descendant groups would be nice additions. We heavily structure our projects in subgroups within subgroups so having something to work with that would be nice.

I tryied this:

Project p = client.Projects.GetAsync("SubgropProject").Result;
p.Branches // error
p.Repository // error

My GitLab tree:

Group1
 - Group2
 - - Group3
 - - - Group4 
 - - - - Group5
 - - - - Group6
 - - - - Group7
 - - - - Group8
 - - - - - Group9
 - - - - - Project1
 - - - - - Project2
 ...

Target subgroup is Group8.

The GitLab v4 api provides information about suggested approvers (based on approval rules) in the MergeRequestApproval API, using the "approved_by" and "suggested_approvers" properties. It also has an "approved" boolean. Those should be made available in the NGitLab.Models.MergeRequestApprovals object.

Hello, is there a way to perform a multi-file commit?
I need to create an initial commit with files from a zip file.

Q: How to filter commits?

What I have Tried

I find GetCommitsRequest model, but it doesn't contain all filter params!

public class GetCommitsRequest
{
    internal const uint DefaultPerPage = 100;

    public string RefName { get; set; }

    public string Path { get; set; }

    public bool? FirstParent { get; set; }

    public int MaxResults { get; set; }

    public uint PerPage { get; set; } = DefaultPerPage;
}

GitLab API Document:

List repository commits

Just like picture above, there are many request params which I need!

We want to track when milestones are changed in an issue, for which we need the timestamp of the comment changing it.

Unfortunately, the call for comments (called notes in the api) in an issue does not return system comments for milestones.

For that, we need this call: https://docs.gitlab.com/ee/api/resource_milestone_events.html

This is very similar to how ResourceLabelEvents is implemented.

The current PipelineClient can get the test reports using this endpoint,

public TestReport GetTestReports(int pipelineId)
{
    return _api.Get().To<TestReport>($"{_projectPath}/pipelines/{pipelineId.ToStringInvariant()}/test_report");
}

but not the summaries, which would be the faster call, if the full details are not needed.

var client = new GitLabClient("https://git.xxx.yy", "XXX");

var groupQueryAllAvailable = new GroupQuery
{
  AllAvailable = true,
};

foreach (var group in client.Groups.GetAsync(groupQueryAllAvailable))
  Console.WriteLine(group.Name);

csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <PublishAot>true</PublishAot>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="NGitLab" Version="6.50.2" />
  </ItemGroup>
</Project>

Error occurs:

System.InvalidOperationException: 
'Reflection-based serialization has been disabled for this application. 
Either use the source generator APIs or explicitly 
configure the 'JsonSerializerOptions.TypeInfoResolver' property.'

Stack-trace:

   at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_JsonSerializerIsReflectionDisabled()
   at System.Text.Json.JsonSerializerOptions.ConfigureForJsonSerializer()
   at System.Text.Json.JsonSerializer.GetTypeInfo(JsonSerializerOptions options, Type inputType)
   at System.Text.Json.JsonSerializer.GetTypeInfo[T](JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at NGitLab.Impl.Json.Serializer.Deserialize[T](String json) <-- HERE
   at NGitLab.Impl.HttpRequestor.Enumerable`1.<GetEnumerator>d__5.MoveNext() <-- HERE
   at Program.<Main>$(String[] args) in D:\imba.sandbox\gitlab-roles\gitlab-roles\Program.cs:line 12

• As the title says, implement the keyset pagination support
• https://docs.gitlab.com/ee/api/repositories.html#list-repository-tree:~:text=with%20keyset%20pagination.-,pagination,-string
• As the comment says, This endpoint changed to keyset-based pagination in GitLab 15.0. Iterating pages of results with a number (?page=2) is unsupported.

• Add a test to search for user by email. GitLab changed its behavior and recent versions require an admin client.
• Update the GitLab versions we test the library against

As I see it we are missing endpoints to fetch feature flags & environments, I don't know if there are more missing?

Hi,

I came across the method GetJobsAsync() in the PipelineClient which from the naming seems to be async, but the implementation doesn't really look like it supports async operations?

https://github.com/ubisoft/NGitLab/blob/main/NGitLab.Mock/Clients/PipelineClient.cs#L273