kubernetes-client / csharp Goto Github PK

Hi there Everyone,

I'm on OSX targeting netcoreapp2.0 and trying to make any API call:

    class Program {
        static void Main(string[] args)
        {
            var config = KubernetesClientConfiguration.BuildConfigFromConfigFile();
            var client = new Kubernetes(config);
            
            foreach (var item in client.ListPodForAllNamespaces().Items) {
                Console.WriteLine(item.Metadata.Name);
            }
        }
    }

Generates this error:

System.PlatformNotSupportedException: The handler does not support custom handling of certificates with this combination of libcurl (7.54.0) and its SSL backend ("LibreSSL/2.0.20").

It looks like it might get sorted in .NET Core 2.1 according to this. In the meantime, I ended up creating a custom DelegatingHandler and passing it in while instantiating a new Kubernetes object:

    class Program {
        static void Main(string[] args)
        {
            var config = KubernetesClientConfiguration.BuildConfigFromConfigFile();
            var client = new Kubernetes(config, new InSecureHandler());
            
            foreach (var item in client.ListPodForAllNamespaces().Items) {
                Console.WriteLine(item.Metadata.Name);
            }
        }
    }

    class InSecureHandler : DelegatingHandler {
        protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) {
            var handler = new HttpClientHandler {
                ServerCertificateCustomValidationCallback =
                    HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
            };
            
            InnerHandler = handler;
            return await base.SendAsync(request, cancellationToken);
        }
    }

...and that works for me:

nginx-4217019353-5g8b7
heapster-2574232661-5xsbr
kube-dns-v20-2253765213-fpr2b
kube-dns-v20-2253765213-t4d95
kube-proxy-611c4
kube-svc-redirect-nkp5g
kubernetes-dashboard-2898242510-qr2bp
tunnelfront-2213086069-w59fx

Just thought I'd share in case others were running into similar issues. Anyone aware of other ways of managing this problem for local development use cases ?

Hi,
My need is to get something like this How to get pods under the service with client-go the client library of kubernetes? using C#.

Can you please let me know the equivalent API in C# for those used in the above solution?

Hi,
I'm using the Azure AKS cluster for Kubernetes. I find that using the SDK when I connect I get the below error

System.Net.Http.HttpRequestException
HResult=0x80072EE7
Message=An error occurred while sending the request.
Source=System.Private.CoreLib
StackTrace:
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at System.Net.Http.DiagnosticsHandler.<SendAsync>d__2.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at k8s.WatcherDelegatingHandler.d__0.MoveNext() in D:\Venkatesh\ASPNETCore\2.0\KubernetesSDK\ReadKubeLogs\lib\WatcherDelegatingHandler.cs:line 20
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass11_0.<b__1>d.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Rest.RetryDelegatingHandler.d__11.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at System.Net.Http.HttpClient.<FinishSendAsyncUnbuffered>d__59.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()
at k8s.Kubernetes.d__88.MoveNext() in D:\Venkatesh\ASPNETCore\2.0\KubernetesSDK\ReadKubeLogs\lib\generated\Kubernetes.cs:line 13409
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at k8s.KubernetesExtensions.<ReadNamespacedPodLogAsync>d__125.MoveNext() in D:\Venkatesh\ASPNETCore\2.0\KubernetesSDK\ReadKubeLogs\lib\generated\KubernetesExtensions.cs:line 5599 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at ReadKubeLogs.Service.Controllers.KubeController.d__2.MoveNext() in D:\Venkatesh\ASPNETCore\2.0\KubernetesSDK\ReadKubeLogs\src\ReadKubeLogs.Service\Controllers\KubeController.cs:line 24
Inner Exception 1:
WinHttpException: The server name or address could not be resolved

Version of Kubernetes in AKS is 1.8.1.
I'm using the API ReadNamespacedPodLogAsync
I'm able to get response to the same by hitting the uri provided in the server tag through browser (though I get 401 unauthorized as certificates are not present ) which proves that server is reachable.

I'm able to connect to the same using kubectl command also which proves that my AKS cluster does respond to Kubernetes commands

please assist me on the same

Hi folks,

I was just testing out the .NET Client and got stuck just at the beginning. A YamlException is thrown when I use KubernetesClientConfiguration.BuildConfigFromConfigFile():

Message:
Property 'as-user-extra' not found on type 'k8s.KubeConfigModels.UserCredentials'.

StackTrace:

   at YamlDotNet.Serialization.TypeInspectors.TypeInspectorSkeleton.GetProperty(Type type, Object container, String name, Boolean ignoreUnmatched)
   at YamlDotNet.Serialization.NodeDeserializers.ObjectNodeDeserializer.YamlDotNet.Serialization.INodeDeserializer.Deserialize(IParser parser, Type expectedType, Func`3 nestedObjectDeserializer, Object& value)
   at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)

I am using the client on a Mac and I am trying to connect to my local Minikube instance.

./kube/config:

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /Users/mgernand/.minikube/ca.crt
    server: https://192.168.99.100:8443
  name: minikube
contexts:
- context:
    cluster: minikube
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    client-certificate: /Users/mgernand/.minikube/client.crt
    client-key: /Users/mgernand/.minikube/client.key

Package Version: 0.4.0-beta

Any ideas?
Cheers, Matt

Hi,

I see currently this library depends on an OpenSSL executable to do some key format conversion. We are building a web service so opening a new process seems a bit heavy. Do you have any plan to drop this dependency? Some libraries such as BouncyCastle might help.

Thanks.

I try to use this client inside CronJob

It is work in minikube, and does not work in real cluster with RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  name: tool-dockercheck-serviceaccount
  namespace: dev

---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: dev
  name: tool-dockercheck-role
rules:
    - apiGroups: [""] 
      # "" indicates the core API group
      resources: ["deployments", "namespaces"]
      verbs: ["get", "list", "update", "patch"]
  

  
---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tool-dockercheck-binding
  namespace: dev
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: tool-dockercheck-role
subjects:
  - kind: ServiceAccount
    name: tool-dockercheck-serviceaccount
    namespace: dev

---


apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: tool-dockercheck-cron
  namespace: dev
  labels:
    app: tool-dockercheck
spec:
  schedule: "*/3 * * * *"
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 2
  failedJobsHistoryLimit: 3  
  
  jobTemplate:
      spec:                  
          template:
            metadata:
              namespace: dev
              labels:
                app: tool-dockercheck   
                ci: someDataForOverride
            spec:
              serviceAccountName: tool-dockercheck-serviceaccount
              restartPolicy: Never               
              containers:
              - name: tool-dockercheck
                image: roured/tdtb-dockercheck:dev
                imagePullPolicy: Always
                env:
                - name: KUBERNETES_NAMESPACE
                  value: "dev"
                - name: LABEL
                  value: "ci"
                

This code

var config = KubernetesClientConfiguration.InClusterConfig();
IKubernetes client = new Kubernetes(config);
client.ListNamespacedDeployment(k8Namespace)

throw

Unhandled Exception: Microsoft.Rest.HttpOperationException: Operation returned an invalid status code 'Forbidden'

I think that there is a mistake in the configuration or a bug in the client itself.

It would be good to make the message about the error more informative. For example, what kind of role is needed.

I was able to use the solution on my Windows 10 client without problems.

Steps:

1. I copied my config file to: "C:\Users\myusername\.kube"
2. After that, I installed openssl from: http://gnuwin32.sourceforge.net/packages/openssl.htm
3. Added "C:\Program Files (x86)\GnuWin32\bin" in my PATH variable
4. Run example. Success :)

Shouldn't we update the README.md documentation to share this Windows support?

Hi - hoping to start a discussion here :)

I've found the use of IObservable<T> to be really useful when exposing streaming or "push"-like behaviour from API clients (especially when combined with Rx / System.Reactive). I appreciate that this client is maturing fast and probably already has existing users, but do you folks see any room in the design for this (beyond the existing pure-event-handler mechanism for watches)?

Here's an example:

using System.Reactive;
using System.Reactive.Linq;

static class HttpClientExtensions
{
    const char CR = '\r';
    const char LF = '\n';

    public static IObservable<string> ObserveLines(this HttpClient httpClient, Uri requestUri, int bufferSize = 2048)
        => httpClient.ObserveLines(HttpMethod.Get, requestUri, bufferSize);

    public static IObservable<string> ObserveLines(this HttpClient client, HttpMethod method, Uri requestUri, int bufferSize = 2048)
    {
        if (client == null)
            throw new ArgumentNullException(nameof(client));

        if (method == null)
            throw new ArgumentNullException(nameof(method));

        return Observable.Create<string>(async (subscriber, cancellationToken) => // cancellationToken will be canceled when the subscription is disposed
        {
            try
            {
                using (HttpRequestMessage requestMessage = new HttpRequestMessage(method, requestUri))
                using (HttpResponseMessage responseMessage = await client.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead, cancellationToken))
                {
                    responseMessage.EnsureSuccessStatusCode();

                    MediaTypeHeaderValue contentTypeHeader = responseMessage.Content.Headers.ContentType;
                    if (contentTypeHeader == null)
                        throw new HttpRequestException("Response is missing 'Content-Type' header.");

                    Encoding encoding =
                        !String.IsNullOrWhiteSpace(contentTypeHeader.CharSet)
                            ? Encoding.GetEncoding(contentTypeHeader.CharSet)
                            : Encoding.UTF8;

                    Decoder decoder = encoding.GetDecoder(); // Decoder is stateful, and can handle characters whose byte sequence spans buffer boundaries.

                    using (Stream responseStream = await responseMessage.Content.ReadAsStreamAsync())
                    {
                        StringBuilder lineBuilder = new StringBuilder();

                        byte[] buffer = new byte[bufferSize];
                        int bytesRead = await responseStream.ReadAsync(buffer, 0, buffer.Length, cancellationToken);
                        while (bytesRead > 0)
                        {
                            // AF: Slightly inefficient because we wind up scanning the buffer twice.
                            char[] decodedCharacters = new char[decoder.GetCharCount(buffer, 0, bytesRead)];
                            int charactersDecoded = decoder.GetChars(buffer, 0, bytesRead, decodedCharacters, 0);
                            for (int charIndex = 0; charIndex < charactersDecoded; charIndex++)
                            {
                                char decodedCharacter = decodedCharacters[charIndex];
                                switch (decodedCharacter)
                                {
                                    case CR:
                                        {
                                            if (charIndex < charactersDecoded - 1 && decodedCharacters[charIndex + 1] == LF)
                                            {
                                                charIndex++;

                                                goto case LF;
                                            }

                                            break;
                                        }
                                    case LF:
                                        {
                                            string line = lineBuilder.ToString();
                                            lineBuilder.Clear();

                                            subscriber.OnNext(line);

                                            break;
                                        }
                                    default:
                                        {
                                            lineBuilder.Append(decodedCharacter);

                                            break;
                                        }
                                }
                            }

                            bytesRead = await responseStream.ReadAsync(buffer, 0, buffer.Length, cancellationToken);
                        }

                        // If stream doesn't end with a line-terminator sequence, publish trailing characters as the last line.
                        if (lineBuilder.Length > 0)
                        {
                            subscriber.OnNext(
                                lineBuilder.ToString()
                            );
                        }
                    }
                }
            }
            catch (OperationCanceledException operationCanceled) when (operationCanceled.CancellationToken != cancellationToken)
            {
                if (!cancellationToken.IsCancellationRequested) // Don't bother publishing if subscriber has already disconnected.
                    subscriber.OnError(operationCanceled);
            }
            catch (Exception exception)
            {
                if (!cancellationToken.IsCancellationRequested) // Don't bother publishing if subscriber has already disconnected.
                    subscriber.OnError(exception);
            }
            finally
            {
                if (!cancellationToken.IsCancellationRequested) // Don't bother publishing if subscriber has already disconnected.
                    subscriber.OnCompleted();
            }
        });
    }
}

A more complete usage example here:

https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient/ResourceClients/KubeResourceClient.cs#L147
which is used by:
https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient/ResourceClients/KubeResourceClient.cs#L122
which is used by:
https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient/ResourceClients/PodClientV1.cs#L73
and:
https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient/ResourceClients/PodClientV1.cs#L184

Using IObservable in this way has some advantages over raw event handlers (specially when it comes to composability and scheduling).

My team and I was trying to create a namespace using the following code:

[...]
var KubeNamespaceBody = new V1Namespace
{
	Metadata = new V1ObjectMeta
	{
		Name = nameSpace
	}
};
var KubeNamespace = await _kubernetes.CreateNamespaceAsync(KubeNamespaceBody);
[...]

We're using Kubernetes 1.7.7, hosted on Azure (ACS).
The Kubernetes API is able to create the namespace with success, returning a HTTP status of 202 (CREATED).

Unfortunately, on line 2235 of Kubernetes.cs, it contains the following piece, expecting a HTTP 200:

[...]
if ((int)_statusCode != 200 && (int)_statusCode != 401)
{
	var ex = new HttpOperationException(string.Format("Operation returned an invalid status code '{0}'", _statusCode));

and on line 2262:

// Deserialize Response
if ((int)_statusCode == 200)
{

The new reference is shown here: https://kubernetes.io/docs/api-reference/v1.9/#namespace-v1-core

The old one here: https://v1-7.docs.kubernetes.io/docs/api-reference/v1.7/#namespace-v1-core

As a temporary alternative, we've changed the mentioned code for the following, and it's working perfectly.

[...]
if ((int)_statusCode != 200 && (int)_statusCode != 201 && (int)_statusCode != 202 && (int)_statusCode != 401)
{

I believe this behavior is due to AutoRest was used with an older version of Kubernetes API.
Should we sent a PR? Or you have plans to generated that code with a more recently version?

which is insecure and not supported by the Kubernetes server, we should use SslProtocols.None instead to pick up the system defaults when the user doesn't specify anything...

The Kubernetes Swagger is incomplete, and the AutoRest code is very precise. It's a bad combination.

Stack trace looks like:

Unhandled Exception: Microsoft.Rest.HttpOperationException: Operation returned an invalid status code 'Created'
   at k8s.Kubernetes.<CreateNamespaceWithHttpMessagesAsync>d__35.MoveNext() in /mnt/c/Users/bburns/kubernetes-client/csharp/src/generated/Kubernetes.cs:line 1927
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at k8s.KubernetesExtensions.<CreateNamespaceAsync>d__19.MoveNext() in /mnt/c/Users/bburns/kubernetes-client/csharp/src/generated/KubernetesExtensions.cs:line 624
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at k8s.KubernetesExtensions.CreateNamespace(IKubernetes operations, V1Namespace body, String pretty) in /mnt/c/Users/bburns/kubernetes-client/csharp/src/generated/KubernetesExtensions.cs:line 605
   at simple.PodList.Main(String[] args) in /mnt/c/Users/bburns/kubernetes-client/csharp/examples/simple/PodList.cs:line 26

On Windows, deleting the Kubernetes configuration file immediately after KubernetesClientConfiguration.BuildFromConfigFile(FileInfo) will likely result in an IOException with error message:

The process cannot access the file '<file>' because it is being used by another process.

The issue is due to KubernetesClientConfiguration.LoadKubeConfig(FileInfo) opening a Stream from the FileInfo for deserialization but never disposing of it. This leaves the file in an open state long enough for subsequent file operations on that file to fail (again, on Windows).

I've realized that trying to generate all my Kubernetes objects in C# code may be a bit too cumbersome and loading them from YAML instead may be a better way.

So I ended up with something like this:

var deserializer = 
    new DeserializerBuilder()
    .WithNamingConvention(new CamelCaseNamingConvention())
    .Build();

var deployment = deserializer.Deserialize<Apiappsv1beta1Deployment>(File.ReadAllText("deployment.yml"));

While this works, it would be neat if there was an easier way to do this; say:

var deployment = Apiappsv1beta1Deployment.FromYamlFile("deployment.yml");

Does anybody have any objections to me moving the library project into a subdirectory of /src?

As mentioned to @brendanburns, I'd like to add a second library that adds support for a Microsoft.Extensions.Configuration provider targeting Secrets and ConfigMaps. This enables you to write .NET Core apps that natively retrieve their configuration (and can automatically reload that configuration if the underlying resource changes).

Here's an example using an implementation I did elsewhere if that helps:

https://github.com/tintoy/dotnet-kube-client/blob/dc323cb27938d417202b9217d3750fe7f05f39b8/samples/ConfigFromConfigMap/Program.cs#L42-L117

With a basic unit testing framework working, we definitely need a code coverage tool to guide us during the test process.

Hey all,

I have found an interesting issue that I'm not sure is an incorrect use of the library or a bug.

When I call DeleteNamespacedDeployment() the V1Status object that is returned has null for all of the attributes if the action was successful. If the action was a failure, you get a Microsoft.Rest.HttpOperationException with a response that looks like it contains a V1Status object.

It seems, from other implementations of the same functions in other client libraries (python, go, etc) and from the kubectl verbose output that return statuses should have an attribute status of "Success" on successful executions.

Thanks!
Chris

As the library assemblies are unsigned, they cannot be referenced by assemblies that are (required to be) signed, which then excludes its use in certain applications.

It'd be great to offer a signed flavor of the assemblies.

(Alternatively, one could fork the repo, build, and sign a forked version, but that's a good deal of overhead to take on.)

Hi.

I was wondering if it might be useful to have a shared base class for models representing Kubernetes resources / resource-lists (the script I wrote to generate models from Swagger does this, and I imagine autorest can do something similar).

My existing client library does this and I've found that it makes it easier to deal with resources in a generic fashion:

https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient/Models/PodV1.cs
https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient/Models/PodListV1.cs
https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient/ResourceClients/KubeResourceClient.cs#L90
https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient/ResourceClients/KubeResourceClient.cs#L122

TBH given that the official client is mostly generated code, I don't know how useful it will be within the client itself, but it's almost certainly going to be useful to some of its consumers.

Again, I get that there are existing users and this might be a breaking change (at the binary-compatibility level at least) but I still think it's something worth considering.

My team and I are working in k8s automation solution, where we will administrate several different k8s clusters using k8s API.

For that, we store every config yaml file in our databases in a text field (due to limitations on store it as an object).
When instantiating K8SConfiguration, it would be much better load it directly as a string into the constructor.

Running an Azure Function locally will result in the kubernetes client successfully communicating with the agents a given k8s cluster on azure.

When running the function on the azure functions environment I receiving the follow stack trace

Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: NodeDestroyer ---> System.Net.Http.HttpRequestException : An error occurred while sending the request. ---> System.Net.Http.WinHttpException : The client certificate credentials were not recognized
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state) 
   End of inner exception
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async k8s.WatcherDelegatingHandler.SendAsync(HttpRequestMessage request,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass11_0.<SendAsync>b__1(??)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Rest.RetryDelegatingHandler.SendAsync(HttpRequestMessage request,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask,HttpRequestMessage request,CancellationTokenSource cts,Boolean disposeCts)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async k8s.Kubernetes.ListNamespacedServiceWithHttpMessagesAsync(String namespaceParameter,String continueParameter,String fieldSelector,Nullable`1 includeUninitialized,String labelSelector,Nullable`1 limit,String resourceVersion,Nullable`1 timeoutSeconds,Nullable`1 watch,String pretty,Dictionary`2 customHeaders,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async k8s.KubernetesExtensions.ListNamespacedServiceAsync(IKubernetes operations,String namespaceParameter,String continueParameter,String fieldSelector,Nullable`1 includeUninitialized,String labelSelector,Nullable`1 limit,String resourceVersion,Nullable`1 timeoutSeconds,Nullable`1 watch,String pretty,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at k8s.KubernetesExtensions.ListNamespacedService(IKubernetes operations,String namespaceParameter,String continueParameter,String fieldSelector,Nullable`1 includeUninitialized,String labelSelector,Nullable`1 limit,String resourceVersion,Nullable`1 timeoutSeconds,Nullable`1 watch,String pretty)
   at InfrastructureManager.NodeDestroyer.TakeAvailableService(String id,TraceWriter log,ExecutionContext context) at C:\GitHub\WebContent\InfrastructureManager\InfrastructureManager\InfrastructureManager.cs : 129
   at InfrastructureManager.NodeDestroyer.CreateAvailableNodes(TraceWriter log,ExecutionContext context) at C:\GitHub\WebContent\InfrastructureManager\InfrastructureManager\InfrastructureManager.cs : 57
   at InfrastructureManager.NodeDestroyer.Run(TimerInfo myTimer,TraceWriter log,ExecutionContext context) at C:\GitHub\WebContent\InfrastructureManager\InfrastructureManager\InfrastructureManager.cs : 38
   at lambda_method(Closure ,NodeDestroyer ,Object[] )
   at Microsoft.Azure.WebJobs.Host.Executors.VoidMethodInvoker`2.InvokeAsync(TReflected instance,Object[] arguments) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\VoidMethodInvoker.cs : 20
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync[TReflected,TReturnValue](Object instance,Object[] arguments) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs : 63
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeAsync(IFunctionInvoker invoker,ParameterHelper parameterHelper,CancellationTokenSource timeoutTokenSource,CancellationTokenSource functionCancellationTokenSource,Boolean throwOnTimeout,TimeSpan timerInterval,IFunctionInstance instance) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 583
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstance instance,ParameterHelper parameterHelper,TraceWriter traceWriter,CancellationTokenSource functionCancellationTokenSource) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 534
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 477
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 264 
   End of inner exception

The only difference from azure function local dev to production is how the environment variables container the certs are ingested.

I have hardcoded the Client credentials to what would be available in azure, and the error+stacktrace persists when ran on azure, but successful when ran locally.

Please let me know what data I can provide to help debug.

Raised an issue with Azure Functions team Azure/azure-functions-host#2286

I see that #98 adds a utility object for loading from Yaml. Does this mean I will be able to load a deployment.yaml file as a V1Deployment object, and pass that in to CreateNamespacedDeployment?

If so, I was wondering when a new Nuget package is planning on being released with this addition. I know this change is recent but it would be nice to pull it in without having to copy over the code into my project.

Thanks!

EDIT: I copied the code into my own project. Will keep this open to hopefully track when a new package is released so I can migrate to that.

For example, this is a valid config file that kubectl accepts, but this client can not parse into KubernetesClientCredentials:

apiVersion: v1
kind: Config
clusters:
- name: colemick1
  cluster:
    server: https://colemick1-api.westus.cloudapp.azure.com:443
    certificate-authority-data: <redacted>
users:
- name: kubelet
  user:
    client-certificate-data: <redacted>
    client-key-data: <redacted>
contexts:
- context:
    cluster: colemick1
    user: kubelet

I had to:

1. name the context
2. actually set the active context in the kubeconfig

I'm not sure it's technically a well-formed kubeconfig, but I figure if kubectl accepts it, that this client should too.

The package on NuGet is about two months old and quite some fixes went into master.

Time to release 0.4.0-beta to NuGet? 😄

Hello, I'm trying to run code copied from examples but got Unhandled Exception: k8s.Exceptions.KubeConfigException: User: X does not have appropriate auth credentials in kubeconfig

Can not find instructions in readme how should we run code locally agains google cloud kubernetes

I do have ~/.kube and commands like kubectl get pods are working as expected

When attempting to add the package in a .NET Framework 4.5.2 project, nuget reports the following error:

Could not install package 'KubernetesClient 0.3.0-beta'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.5.2', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.

I believe this is because the package specifically targets only .NET Standard 1.4.

This is a general problem with our current Kubernetes Swagger, Create and Delete can return an object or a Status, the Swagger isn't set up to handle that properly. It tries to parse and it dies:

Unhandled Exception: System.AggregateException: One or more errors occurred. (Unable to deserialize the response.) ---> Microsoft.Rest.SerializationException: Unable to deserialize the respo
nse. ---> Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: {. Path 'status', line 1, position 305.
   at Newtonsoft.Json.JsonTextReader.ReadStringValue(ReadType readType)
   at Newtonsoft.Json.JsonTextReader.ReadAsString()
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ReadForType(JsonReader reader, JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerC
ontract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Microsoft.Rest.Serialization.SafeJsonConvert.DeserializeObject[T](String json, JsonSerializerSettings settings)
   at k8s.Kubernetes.<DeleteNamespaceWithHttpMessagesAsync>d__173.MoveNext() in /mnt/c/Users/bburns/kubernetes-client/csharp/src/generated/Kubernetes.cs:line 29720
   --- End of inner exception stack trace ---
   at k8s.Kubernetes.<DeleteNamespaceWithHttpMessagesAsync>d__173.MoveNext() in /mnt/c/Users/bburns/kubernetes-client/csharp/src/generated/Kubernetes.cs:line 29729
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at simple.NamespaceExample.Main(String[] args) in /mnt/c/Users/bburns/kubernetes-client/csharp/examples/namespace/Namespace.cs:line 38

This is similar to: kubernetes-client/java#86

I've tried to install KubernetesClient 0.2.0-beta via nuget but I get the following error:
Starling.NR -> KubernetesClient 0.2.0-beta -> Microsoft.Rest.ClientRuntime (>= 3.0.3)
Starling.NR -> Microsoft.AspNetCore.All 2.0.3 -> Microsoft.Extensions.Configuration.AzureKeyVault 2.0.0 -> Microsoft.Azure.KeyVault 2.3.2 -> Microsoft.Rest.ClientRuntime (>= 2.3.8 && < 3.0.0).

I'm currently running version 2.3.8 of the Microsoft.Rest.ClientRuntime. I went looking for the 3.0.3 version but it seems that it has been unlisted: https://www.nuget.org/packages/Microsoft.Rest.ClientRuntime/3.0.3

I did manually install it and everything seems to still work. I haven't currently tried anything that was dependent on the old version though.

It would be nice if this library targeted the latest 2.x release (at the moment 2.3.10) instead of the currently unlisted 3.x version.

I am very pleased to be able to see c# version of the kubernetes client.

But I do not know what version of the corresponds.

Can I use this client when my kubernetes version is 1.6.2?

I have found some issues. The status code of the return interface does not seem to be too right.

Hi

Is there a plan that this library would support lower dotnet runtime? like:

• netstandard 1.0
• netframework 4.0

See #129 for details, but custom/client certificate handling on macOS seems to be broken unless using the SocketsHttpHandler.

In .NET Core 2.1 Preview 1, you can enable this using an environment variable. In Preview 2, you should be able to control this in code by running:

new HttpClient(new SocketsHttpHandler())

This issue is to track us making that change when .NET Core 2.1 Preview 2 ships.

When requests are rejected by Kubernetes, the server responds with HTTP 400. The body contains a JSON object with additional error information. This information is lost in the C# client.

Here's an example:

await client.CreateNamespacedPodAsync(
    new Corev1Pod() 
    {
         // Pod configuration
    }).ConfigureAwait(false);

got me:

Microsoft.Rest.HttpOperationException: 'Operation returned an invalid status code 'BadRequest''

which doesn't help me debug my problem.

Fiddler reveals the server did return an error message:

{
  "kind":"Status",
  "apiVersion":"v1",
  "metadata":{},
  "status":"Failure",
  "message":"Deployment in version \"v1\" cannot be handled as a Pod: no kind \"Deployment\" is registered for version \"v1\"",
  "reason":"BadRequest",
  "code":400
}

It would be great if the C# API could surface that error message.

There is a small mistake in README.md. Specifically where it says:

$ Where REPO_DIR points to the root of the csharp repository
cd ${REPO_DIR}/csharp/src
${GEN_DIR}/openapi/csharp.sh generated csharp.settings

csharp.settings is located in $REPO_DIR/csharp, but in the example we are in $REPO_DIR/csharp/src. The last line of this block should instead say:

${GEN_DIR}/openapi/csharp.sh generated ../csharp.settings

Also:

• it might be helpful to list dependencies as: git, nodejs >= 7.10.0, docker
• it might be helpful to state that the instructions are intended to be run on a Linux system (since it's a C# library, people might assume they are Windows instructions)

@brendanburns as formal report about the issue found in PR #11

During unit testing found that severals tests, related with KubernetesUserCredentials, where broken reporting a 'Could not load file or assembly System.Net.Http'. Initially thought it was related with my computer, but after adding Travis-CI support found that the problem is also present in trusty environment with dotnet SDK 1.0.4.

Here is all information I found so far:

1. It looks like minimum version present for System.Net.Http in netcoreapp1.1 (SDK 1.0.4) is 4.3.2. Trying a hard reference to downgrade the library through NuGet throws an error:

$ dotnet build
/Users/Sergio/Repositories/kubernetes/cli/csharp/src/KubernetesClient.csproj : error NU1605: Detected package downgrade: System.Net.Http from 4.3.2 to 4.1.1. Reference the package directly from the project to select a different version.  [/Users/Sergio/Repositories/kubernetes/cli/csharp/tests/tests.csproj]
/Users/Sergio/Repositories/kubernetes/cli/csharp/src/KubernetesClient.csproj : error NU1605:  KubernetesClient (>= 1.0.0) -> Microsoft.NETCore.App (>= 1.1.2) -> System.Net.Http (>= 4.3.2)  [/Users/Sergio/Repositories/kubernetes/cli/csharp/tests/tests.csproj]
/Users/Sergio/Repositories/kubernetes/cli/csharp/src/KubernetesClient.csproj : error NU1605:  KubernetesClient (>= 1.0.0) -> System.Net.Http (>= 4.1.1) [/Users/Sergio/Repositories/kubernetes/cli/csharp/tests/tests.csproj]

2. IMO, System.Net.Http 4.1.1.1 dependency comes from Microsoft.Rest.ClientRuntime. Furthermore, note that 3.0.3 been unlisted, but we are still referencing 3.0.3.

• A downgrade to Microsoft.Rest.ClientRuntime 2.3.8 do not fix the issue.

3. Made a full battery test using all available SDKs with Travis-CI, with no luck. Either the solution do not build or we get the 'Could not load file or assembly System.Net.Http' exception.

Full log output as registered in Build#2

    k8s.Tests.KubernetesClientCredentialsTests.UsernameNull [FAIL]
      Assert.Throws() Failure
      Expected: typeof(System.ArgumentNullException)
      Actual:   typeof(System.IO.FileNotFoundException): Could not load file or assembly 'System.Net.Http, Version=4.1.1.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

      Stack Trace:
           at k8s.Tests.KubernetesClientCredentialsTests.<>c.<UsernameNull>b__1_0()
    k8s.Tests.KubernetesClientCredentialsTests.PasswordNull [FAIL]
      Assert.Throws() Failure
      Expected: typeof(System.ArgumentNullException)
      Actual:   typeof(System.IO.FileNotFoundException): Could not load file or assembly 'System.Net.Http, Version=4.1.1.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

      Stack Trace:
           at k8s.Tests.KubernetesClientCredentialsTests.<>c.<PasswordNull>b__2_0()
    k8s.Tests.KubernetesClientCredentialsTests.ValidUserPasswordIsSet [FAIL]
      System.IO.FileNotFoundException : Could not load file or assembly 'System.Net.Http, Version=4.1.1.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

    k8s.Tests.KubernetesClientCredentialsTests.ValidTokenIsSet [FAIL]
      System.IO.FileNotFoundException : Could not load file or assembly 'System.Net.Http, Version=4.1.1.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

    k8s.Tests.KubernetesClientCredentialsTests.TokenNull [FAIL]
      Assert.Throws() Failure
      Expected: typeof(System.ArgumentNullException)
      Actual:   typeof(System.IO.FileNotFoundException): Could not load file or assembly 'System.Net.Http, Version=4.1.1.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

I am getting an HTTP 422 (Unprocessable Entity) error for an API call I am making to submit a deployment (json attached, submitting the same json through kubectl works fine), the API call is
k8sClient.CreateNamespacedDeploymentWithHttpMessagesAsync

The only relevant line in the APIServer logs is the following:
I1104 00:08:00.847824 1 wrap.go:42] POST /apis/apps/v1beta1/namespaces/default/deployments?pretty=true: (45.627034ms) 422 [[FxVersion/4.7.2114.0 OSName/WindowsServer2012R2Datacenter OSVersion/6.3.9600 k8s.Kubernetes/0.0.0.0] 167.220.1.198:34012]

deployment-gateway.json.txt

Hi,
At present when we read the pod logs from SDK, we are getting a stream. The stream data does not have date and time details as to when the log was written. How would I get the date and time when the log was written ?

The generated methods for the proxy-related REST calls (e.g. ProxyGETNamespacedServiceWithHttpMessagesAsync()) expect responses to be JSON-serialized strings; this is often not the case (e.g. when a web site is being proxied and raw HTML, CSS, and JavaScript are returned).

Those proxy-related methods should probably just pass through the raw response content.

Placeholder to split off the design conversation from #63: what API should the C# client expose for interacting with the exec, attach & port forward endpoints?

I have some config

apiVersion: apps/v1beta2 
kind: Deployment
metadata:
  name: web-deployment
  labels:
    app: web
spec:
  template:
    metadata:
      labels:
        app: web
        ci: somedataForOverride        

And I write this code, dokerize it and deploy to cluster with env KUBERNETES_SERVICE_HOST="kubernetes.default.svc.cluster.local" KUBERNETES_SERVICE_PORT="443"

            var config = KubernetesClientConfiguration.InClusterConfig();
            IKubernetes client = new Kubernetes(config);
            Console.WriteLine("Starting Request!");

            var list = client.ListNamespacedPod("default");
            foreach (var item in list.Items)
            {
                Console.WriteLine(item.Metadata.Name);
            }
            if (list.Items.Count == 0)
            {
                Console.WriteLine("Empty!");
            }

            Console.WriteLine("ListNamespacedDeployment!");
            foreach (var item in client.ListNamespacedDeployment("default").Items)
            {
                Console.WriteLine($"Name {item.Metadata.Name}");
                foreach (var label in item.Spec.Template.Metadata.Labels)
                {
                    Console.WriteLine($"Label {label.Key} = {label.Value}");
                    if (label.Key == "ci")
                    {
                        var newVal = "newdata";
                        Console.WriteLine($"\t\t\tFound! try set to {newVal}");
                        item.Spec.Template.Metadata.Labels.Add("ci2", "newVal");
                        client.PatchNamespacedDeployment(new
                        {
                            op = "replace",
                            path = "/spec/template/metadata/labels/"+ label.Key,
                            value = newVal
                        },item.Metadata.Name, "default"); //fail!
                    }
                }
                foreach (var container in item.Spec.Template.Spec.Containers)
                {
                    Console.WriteLine($"Image {container.Image}");
                }
            }

And I get

2018-01-07T21:03:26.886868069Z Starting Request!
2018-01-07T21:03:27.398606591Z tool-deployment-664848d469-fdfqp
2018-01-07T21:03:27.398636889Z web-deployment-7d87b9fb8-r24c2
2018-01-07T21:03:27.398642596Z web-deployment-7d87b9fb8-s96d6
2018-01-07T21:03:27.398647321Z ListNamespacedDeployment!
2018-01-07T21:03:27.464030335Z Name tool-deployment
2018-01-07T21:03:27.464060924Z Label app = tool
2018-01-07T21:03:27.464066364Z Label ddd = dd6rty6
2018-01-07T21:03:27.464430225Z Image skipeddata/skipeddata:dev
2018-01-07T21:03:27.464444583Z Name web-deployment
2018-01-07T21:03:27.46444943Z Label app = web
2018-01-07T21:03:27.464453986Z Label ci = somedataForOverride
2018-01-07T21:03:27.464458516Z Found! try set to newdata
2018-01-07T21:03:47.549478912Z
2018-01-07T21:03:47.577526422Z Unhandled Exception: Microsoft.Rest.HttpOperationException: Operation returned an invalid status code 'InternalServerError'
2018-01-07T21:03:47.57769817Z at k8s.Kubernetes.<PatchNamespacedDeploymentWithHttpMessagesAsync>d__321.MoveNext()
2018-01-07T21:03:47.577706886Z --- End of stack trace from previous location where exception was thrown ---

I've found the following error executing the tests locally, with MacOS 10.12.6 (High Sierra) and latest .NET Core SDK (2.1.300-preview1-008174):

Failed   k8s.Tests.AuthTests.Cert
Error Message:
 Interop+AppleCrypto+AppleCommonCryptoCryptographicException : MAC verification failed during PKCS12 import (wrong password?)
Stack Trace:
   at Interop.AppleCrypto.X509ImportCertificate(Byte[] bytes, X509ContentType contentType, SafePasswordHandle importPassword, SafeKeychainHandle keychain, Boolean exportable, SafeSecIdentityHandle& identityHandle)
   at Internal.Cryptography.Pal.CertificatePal.FromBlob(Byte[] rawData, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] data)
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData)
   at k8s.Tests.AuthTests.Cert() in /Users/Sergio/Repositories/kubernetes/kubernetes-client-csharp/tests/AuthTests.cs:line 171

Accordingly to @qmfrederik in corefx/24225 "If you have a PKCS#12 file which is not protected with a password, and which does have a MAC entry, opening the file will work on Windows and Linux but fails on Mac (which use CommonCrypto)."

Maybe our test certificates can be regenerated with password protection.

I'm working on a small POC and would like access to entire kube config. Any reason why we couldn't have a public method that uses this?

public static K8SConfiguration LoadKubeConfig(string kubeconfigPath = null)
{
    var fileInfo = new FileInfo(kubeconfigPath ?? KubeConfigDefaultLocation);
    
    return LoadKubeConfig(fileInfo);
}

private static K8SConfiguration LoadKubeConfig(FileInfo kubeconfig) { ... }

This would be similar to how the kubectl go package gets the config using this and this

Sadly, C# did not have a go fmt yet.
A check tool before checking in is wanted.

the tool should include, but not limit to, those features:

• naming convention
• line coding (crlf / lf)
• utf8 bom
• run on all platform (linux, mac, windows, ...)

some candidates:
https://github.com/dotnet/codeformatter
https://github.com/dotnet/corefx/blob/master/Documentation/coding-guidelines/coding-style.md

I was trying to use the Yaml class in version 0.5.0-beta this morning, and ran into this error when calling Yaml.LoadFromFileAsync:

Unhandled Exception: System.AggregateException: One or more errors occurred. (Stream was not readable.) ---> System.ArgumentException: Stream was not readable.
   at System.IO.StreamReader..ctor(Stream stream, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean leaveOpen)
   at System.IO.StreamReader..ctor(Stream stream)
   at k8s.Yaml.<LoadFromStreamAsync>d__0`1.MoveNext()

It looks like changing

using (FileStream fs = File.Open(file, FileMode.Open, FileAccess.Write, FileShare.None))

to

using (FileStream fs = File.Open(file, FileMode.Open, FileAccess.Read, FileShare.None))

fixes the problem, since in the first one the file is being opened for writing and not reading.

Would you like me to open a PR to fix this?

Hey, @brendanburns, as previously discussed in tintoy/dotnet-kube-client#6 I'm happy to contribute my WebSocket functionality (which includes support for custom verification of the server certificate).

But I figured it'd be worth discussing here first :)

The big gotcha (currently) is that it has a dependency on netcoreapp2.1 because in previous versions of corefx WebSockets used WinHTTP on Windows (which didn't honour a subset of the certificate options on ClientWebSocketOptions) and something else on OSX / Linux (which didn't honour a different subset of the certificate options). In 2.1, the WebSockets implementation is entirely implemented in managed code and fully supports customisation of SSL certificate usage / validation.

As it stands, here's how I implemented it:

https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient.Extensions.WebSockets/K8sWebSocketOptions.cs#L13
https://github.com/tintoy/dotnet-kube-client/blob/develop/src/KubeClient.Extensions.WebSockets/K8sWebSocket.cs#L31

I borrowed some of the original ClientWebSocket code from .NET Core 2.0 and modified it to work with 2.1-preview1 (but from 2.1-preview2 onwards, the ManagedWebSocket types will be publicly available and much of this code can be eliminated).

The latest code from corefx is a fair bit simpler because the required managed types are available internally:

https://github.com/dotnet/corefx/blob/f1f3b3f8af21c0f782f7f0699d505a0cdb9026ba/src/System.Net.WebSockets.Client/src/System/Net/WebSockets/WebSocketHandle.Managed.cs#L71

I'm trying to use the C# client to execute a command in a container in a pod.
I assumed that ConnectGetNamespacedPodExec would allow me to execute a command and return the standard output and error output.

That doesn't seem to work, though: calls to ConnectGetNamespacedPodExec and ConnectPostNamespacedPodExecAsync both fail with an exception:

Microsoft.Rest.HttpOperationException: 'Operation returned an invalid status code 'BadRequest''

The server returns the following data:

{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Upgrade request required","reason":"BadRequest","code":400}

Any plans to publish a package on Nuget?