The request failed with unexpected status code 'Forbidden' when connecting to GKE about dotnet-kube-client

[felixfbecker]

I can confirm that it works with kubectl proxy on stable WiFi :)

[tintoy]

Hi :)
What sort of authentication is specified in your kube config? There's a chance that GKE may be using the new "script"-style authentication mechanism (which I haven't added support for yet).

[felixfbecker]

I posted the relevant section of the yaml in the issue, seems like it uses what you mean. Is support for it planned?

[tintoy]

It is now :)

[tintoy]

Oh, and sorry I've been reading this on my phone which is why I missed the config yaml before :)

[tintoy]

Ok - I'm going to see what's required to implement it this morning; I was hoping to get some ideas from the official client but it looks like they don't support exec-based authentication either yet. Hopefully it should be relatively easy to implement though 😄

[tintoy]

GCP authentication seems to be something specific but there's a Go implementation I think I can reverse-engineer:

https://github.com/kubernetes/client-go/blob/fb6075f2e0b094320f3d2c3e99add90a4661fcff/plugin/pkg/client/auth/gcp/gcp.go

[tintoy]

The official K8s client for C# currently suggests using kubectl proxy but I think we might be able to do better in the long run. I don't currently have a GCP cluster running so I can't test this yet but if you have a moment would you mind trying the linked workaround just so I know if the authentication is the only thing stopping us from connecting to GCP?

[tintoy]

In the meanwhile I'll start sketching out a mechanism for authentication plugins (and implementing the GCP authentication plugin which doesn't seem all that complex, except perhaps for their use of Go-style field selector syntax such as {.token} which we may be able to fudge).

[felixfbecker]

Yeah I was hoping we could do better than that - I'm trying to reimplement kubectl in PowerShell, so it would be nice if kubectl didn't have to be used in the background to get it working.

When I try to connect to GKE through kubectl proxy, I get the error "Empty response body". However, when I try to use Postman or curl, the response body is in fact empty, so I don't think this is a bug in KubeClient. kubectl proxy logs this every time:

I0818 00:48:45.712346   67980 logs.go:41] http: proxy error: net/http: TLS handshake timeout

It might just be because I'm tethering over LTE.

[tintoy]

I like the PS idea :)

That empty response body seems vaguely familiar - I might try digging through my mailbox to see if I can find anything.

[tintoy]

I assume kubectl get pods still works against gcp?

[tintoy]

Agh, again sorry you did say - I need to stop reading issues on my phone!

[tintoy]

What's the output of kubectl version?

[tintoy]

Ok, as long as kubectl uses the command-style token provider (which from the look of your config YAML it already does), it should be pretty simple to support this (the Google-specific authentication mechanism may take a little longer so I'll probably focus on the simple case for now).

[tintoy]

I'll have a go at building out something tomorrow (Sunday AEST) and I'll post updates here in case you want to try a preview build :)

[tintoy]

BTW, if you're doing async stuff in a Cmdlet, you might find this useful:

https://github.com/DimensionDataResearch/cloudcontrol-powershell-core/blob/develop/src/DD.CloudControl.Powershell/AsyncCmdlet.cs
https://github.com/DimensionDataResearch/cloudcontrol-powershell-core/blob/develop/src/DD.CloudControl.Powershell/Utilities/ThreadAffinitiveSynchronizationContext.cs

(apologies if you've already worked this out, but it took me a while to get it right)

[felixfbecker]

I looked at a similar thing: https://github.com/ttrider/PowerShellAsync/blob/master/PowerShellAsync/AsyncCmdlet.cs
But then I saw the issues and I was afraid to use it. So far I’ve just been using .GetAwaiter().GetResult() and a CancellationToken that gets cancelled in StopProcessing(). For Observables, I used GetEnumerable() and foreach. any reason why that wouldn’t work?

I’m a C# beginner so how threading works in C# and what a SynchronisationContext does is quite alien to me.

[felixfbecker]

I wish Microsoft supported async cmdlets natively or at least had some docs on it. Maybe you should publish that class as a nuget package?

[felixfbecker]

The thing I can’t figure out is how to hook the LoggerFactory up to WriteVerbose/Information/Debug/Warning, since those methods are only allowed to be called on the main thread from within Process methods

[tintoy]

Yeah, slightly painful that one - the short answer is that if you expose the ThreadAffinitiveSynchronizationContext (or equivalent) used while running the async method, then anything from any thread can call SynchronizationContext.Post with a delegate and that delegate will be executed on the main thread

[tintoy]

(you can get at it inside the Cmdlet via SychronizationContext.Current.

[felixfbecker]

Ah, so I could use that to make sure the Write calls of the logger get executed on the main thread, and then I only need to ignore them while no Process method is active.

[tintoy]

Yep - just check if SynchonizationContext.Current is ThreadAffinitiveSynchronizationContext

[tintoy]

@felixfbecker I'm about to start work on this - would you be up for trying it out once I've got something working?

[tintoy]

Ok - I've tried the new code in the feature/auth-cmd branch against GKE and it works! If you have a moment, would you mind trying it for yourself to see if it works for you?

In the meanwhile I'll see what I can do about using the existing cached token, if present in ~/.kube/config.

[tintoy]

Ok, this should be working now in the latest release :)

Feel free to reopen if it doesn't work for you.