Comments (78)
We are aiming to bring the certificates API to beta in v1.6.
from enhancements.
GA docs PR here:
from enhancements.
multiple identities obtained from a single shared credential
I'm not convinced of this. In fact I think this would work equally well if the CSR API was open (didn't require auth[n/z]). The shared secret is a mechanism that prevents a specific dos attack where a user could spam the CSR API with CSRs that would not be approved. @gtank can you weigh in on this?
from enhancements.
PR documenting certificates API is here kubernetes/website#2939, the kubelet flags are still experimental in 1.6.
from enhancements.
cc @kubernetes/sig-node FYI about this feature for Kubelet TLS bootstrap
from enhancements.
I think we should consider this pre-alpha until we complete:
- kubectl integration
- kubelet integration
- e2e testing
Is anyone currently working on this?
from enhancements.
@mikedanese @gtank should be working on all of those things to get the feature done done.
from enhancements.
Current status of what needs to be done:
- kubelet needs to use the Certificates API. This will involve one new flag, provisionally
--bootstrap-auth-token
which will take a long random string. The kubelet uses this token to authenticate its requests to the Certificates API. Cluster-level access control should ensure that it can only be used for certificates requests. When this token exists, if the kubelet can't find TLS assets locally it will generate a fresh keypair and a Certificate Signing Request (CSR) to submit to the API. It will then watch the CSR object for the appearance of an issued certificate before proceeding with registration. It will use the certificate for client cert auth in subsequent API requests. - kubectl needs to fully support CSR objects. This is currently blocked on a problem with the swagger generation and base64-encoded []byte fields. We also need to document the best way of handling the manual approval flow.
- Still need to write tests and update all the docs.
from enhancements.
Status update:
PR for kubelet TLS bootstrap is up: kubernetes/kubernetes#30094
from enhancements.
Kubectl support tracked in kubernetes/kubernetes#30163 with some basic support prs out
from enhancements.
Status update for TLS work:
In flight:
- Implement TLS bootstrap for kubelet using
--bootstrap-auth-token
kubernetes/kubernetes#30094 - Add an option to controller-manager to auto approve all CSRs kubernetes/kubernetes#30153
- Add a certificate signing request resource printer in kubectl kubernetes/kubernetes#30166
- Implement kubectl procelain csr commands kubernetes/kubernetes#30237
- csr: add approval to the typed client kubernetes/kubernetes#30236
- add shortname for certificate signing request in kubectl kubernetes/kubernetes#30165
- swagger: report golang type []byte as json type string emicklei/go-restful#311
Issues:
- certificatesigningrequests should overwrite spec.user info in createstrategy kubernetes/kubernetes#30239
- kubectl should support CertificateSigningRequest kubernetes/kubernetes#30163
Merged:
- add delete strategy to certificates registry kubernetes/kubernetes#30181
from enhancements.
It will use the certificate for client cert auth in subsequent API requests.
The scope of kubernetes/kubernetes#20439 was around obtaining TLS serving certs. The kubelet's use of this API should probably be limited to obtaining serving certs initially. Since the kubelet already needs API credentials to submit the initial CSR, obtaining client credentials this way doesn't buy us a whole lot from a bootstrapping perspective.
from enhancements.
It allows us to bootstrap individual kubelet identities from a single cluster wide shared secret (bearer token for a kubelet-bootstrap user authorized to submit CSRs).
from enhancements.
If the goal is to identify kubelets individually so we can partition node permissions (which I am in favor of), multiple identities obtained from a single shared credential aren't meaningful from an auth perspective.
If we just want to identify requests from different nodes for debugging purposes, we can do that with user-agent strings (the same way controllers set up their own client user-agent string)
from enhancements.
cc @kubernetes/sig-cluster-lifecycle
from enhancements.
@mikedanese @liggitt can we please move the technical discussion about the design of the system to the https://groups.google.com/forum/#!forum/kubernetes-sig-node mailing list.
from enhancements.
@liggitt @mikedanese The nodes derive individual identities from the shared token. The token only exists to 1) filter requests meant for other clusters and 2) allow us restrict the nodes (via groups or ABAC) from accessing the rest of the API before they have client certs.
from enhancements.
@philips Is this on target for the 1.4 feature freeze this Friday (Aug 19)?
from enhancements.
@pwittrock: based on kubernetes/kubernetes#30094 this feature suddenly started being redesigned in flight after 5 months of work... Any suggestions on how to avoid the major redesign three days before feature freeze are welcome
from enhancements.
Occasionally we will realize gaps in designed features as they are implemented. We definitely need a process that handles that - rushing features to delivery without all the proper technical due diligence being done is going to cause more issues than occasionally features missing a release. I think everyone involved is trying to find the best and most secure option for the platform here.
from enhancements.
@smarterclayton based on PR discussions it would be great if @liggitt and @deads2k could help with parts of the feature, wdyt? any coding / review help is appreciated to address raised concerns.
from enhancements.
I believe both of them had been representing sig-auth on the security aspects here - I had not yet seen an update on the remaining issues though. That's probably appropriate on the linked issue.
from enhancements.
Update: kubelet support merged in kubernetes/kubernetes#30922
from enhancements.
To be more clear, kubernetes/kubernetes#30922 only requests the kubelet client cert from API server. We will need follow up work on getting the kubelet serving cert.
from enhancements.
Could you create another issue to clarify that?
On Wed, 24 Aug 2016, 00:18 Yifan Gu, [email protected] wrote:
To be more clear, kubernetes/kubernetes#30922
kubernetes/kubernetes#30922 only requests the
kubelet client cert from API server. We will need follow up work on getting
the kubelet serving cert.—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
#43 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPWS-UriOVAXjKfqJAoP-mRahm9LOxcks5qi3_TgaJpZM4JS7ez
.
from enhancements.
@errordeveloper Basically we need to think about:
- How this continues to combine with the work of dynamic kubelet settings kubernetes/kubernetes#27980
- How we get the serving cert for kubelet
- How to do the cert rotation
Probably need to sync with @kubernetes/sig-auth to get a better idea before creating the issue.
from enhancements.
@gtank @yifan-gu Are the docs ready? Please update the docs in https://github.com/kubernetes/kubernetes.github.io, and then add PR numbers and check the docs box in the issue description
from enhancements.
@janetkuo, I'll be tracking our progress on creating the docs. Is there anything besides high level "this is how you turn it on" + known issues that you're expecting for the PR? Also sorry for the delayed response!
from enhancements.
@calebamiles That should be fine, and if you could add a link to the PR in this issue that would help me track it. Thanks!
from enhancements.
@gtank @yifan-gu @calebamiles Another ping on docs. Any PRs you can point me to?
from enhancements.
I believe we also still need to figure out next steps for including kubelet server certificates as part of the TLS bootstrap. Based on the kubelet authN proposal - this would greatly simplify the deployment of secure clusters.
Initially discussed in: kubernetes/kubernetes#30094 it sounded like something along these lines might be reasonable?
- kubelet uses bootstrap-kubeconfig to post client-cert CSR
- kubelet retrieves signed certificate and is now a "full-client" of api
- kubelet uses client-cert to post a server-cert CSR
- kubelet starts api
/cc @gtank @yifan-gu @philips @liggitt
from enhancements.
@gtank @yifan-gu @calebamiles Another ping on docs. Any PRs you can point me to?
I can take the item, but what's expected in the doc?
What I can think about is:
- Introduce this new flag
--experimental-bootstrap-kubeconfig
, why we need it. - Explain what it does step-by-step
- Give a successful step-by-step guide on how to set up?
Others?
from enhancements.
- Was an auto-approver in the controller-manager merged? If so, probably help to document that functionality also.
- Also maybe "adding your own approver section"?
from enhancements.
Yeah, the auto-approver is in. FYI, this repo is meant to be low on
traffic. Let's migrate the discussions elsewhere.
On Tue, 13 Sep 2016, 00:54 Aaron Levy, [email protected] wrote:
- Was an auto-approver in the controller-manager merged? If so,
probably help to document that functionality also.- Also maybe "adding your own approver section"?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#43 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPWSzol3PTrv47n-xvFZx3JBQjRvMtYks5qpeYggaJpZM4JS7ez
.
from enhancements.
@jaredbhatti @janetkuo, we will hopefully have a docs PR ready today
from enhancements.
@calebamiles , Any word on that Docs PR? Please reference if it's up.
from enhancements.
@devin-donnelly, kubernetes/website#1246
from enhancements.
Great, thanks!
from enhancements.
As part of the documentation effort, here's a flow chart for how this works as of the 1.4 release.
from enhancements.
@gtank @mikedanese @yifan-gu please, provide us with the release notes and documentation PR or link at https://docs.google.com/spreadsheets/d/1nspIeRVNjAQHRslHQD1-6gPv99OcYZLMezrBe3Pfhhg/edit#gid=0
Also, please, select the valid checkpoints at the Progress Tracker.
from enhancements.
@idvoretskyi Added Introduces an API for the Kubelet to request TLS certificates from the API server.
What is the valid checkpoints
?
from enhancements.
So, certificates API will be beta in 1.6, I'm working on a docs PR for that but also trying to unblock upgrade testing first. Will post in the spreadsheet once it's ready.
from enhancements.
@yifan-gu I'm speaking about the "Progress Tracker" checkpoints at the heading message.
from enhancements.
@kubernetes/sig-auth-feature-requests @kubernetes/sig-node-feature-requests
Is this going fully to GA in v1.8 or just a subset of it?
What I can see there are different sub-features in this feature:
- the csrapprover and csrsigner in the controller-manager
- the certificates API group
- the CSR post functionality in the kubelet (the
--bootstrap-kubeconfig
flag)
Which of these features are being worked on in the v1.8 timeframe?
from enhancements.
@luxas this depends on the certificates API being GA which I don't see happening in 1.8. There's a lot in early design phase that will either depend on the certificates API in some form (e.g. pod identity, spiffy, istio) or redo the API. I see greater value in getting feedback on the API as these PKI efforts mature than moving to GA in 1.8.
from enhancements.
@christianhuening this is a feature development progress tracking issue, not intended for solving user-facing questions.
Please, open an issue at https://github.com/kubernetes/kubernetes/issues (or check if one has been opened before).
Thanks
from enhancements.
@mikedanese @kubernetes/sig-auth-feature-requests are you planning to graduate this to GA soon?
from enhancements.
@roberthbailey still beta for 1.10?
from enhancements.
This is going to GA next.
from enhancements.
@mikedanese GA in 1.10, right?
from enhancements.
@mikedanese thanks.
from enhancements.
@mikedanese it looks as though the docs could use some updating for GA:
- built: https://kubernetes.io/docs/admin/kubelet-tls-bootstrapping/
- repo: https://github.com/kubernetes/website/blob/master/docs/admin/kubelet-tls-bootstrapping.md
Could you also please update the spreadsheet for 1.10? Thanks!
from enhancements.
This didn't make the cut.
from enhancements.
This didn't make the cut.
cc @nickchase
from enhancements.
@mikedanese can you please update the OP with the new issue template? It will simplify the feature tracking a lot. Thanks
@kubernetes/sig-auth-feature-requests @kubernetes/sig-cluster-lifecycle-feature-requests @kubernetes/sig-node-feature-requests
from enhancements.
@mikedanese @kubernetes/sig-auth-feature-requests @kubernetes/sig-cluster-lifecycle-feature-requests @kubernetes/sig-node-feature-requests
Any plans for this in 1.11?
If so, can you please ensure the feature is up-to-date with the appropriate:
- Description
- Milestone
- Assignee(s)
- Labels:
stage/{alpha,beta,stable}
sig/*
kind/feature
cc @idvoretskyi
from enhancements.
@mikedanese @kubernetes/sig-auth-feature-requests @kubernetes/sig-cluster-lifecycle-feature-requests @kubernetes/sig-node-feature-requests
Any plans for this in 1.11?
from enhancements.
@mikedanese please fill out the appropriate line item of the
1.11 feature tracking spreadsheet
and open a placeholder docs PR against the
release-1.11
branch
by 5/25/2018 (tomorrow as I write this) if new docs or docs changes are
needed and a relevant PR has not yet been opened.
from enhancements.
@mikedanese -- We're doing one more sweep of the 1.11 Features tracking spreadsheet.
Would you mind filling in any incomplete / blank fields for this feature's line item?
from enhancements.
No updates. Bumped to next milestone.
from enhancements.
Thanks for the update, @mikedanese!
from enhancements.
@mikedanese @kubernetes/sig-auth-feature-requests @kubernetes/sig-cluster-lifecycle-feature-requests @kubernetes/sig-node-feature-requests --
This feature was removed from the previous milestone, so we'd like to check in and see if there are any plans for this in Kubernetes 1.12.
If so, please ensure that this issue is up-to-date with ALL of the following information:
- One-line feature description (can be used as a release note):
- Primary contact (assignee):
- Responsible SIGs:
- Design proposal link (community repo):
- Link to e2e and/or unit tests:
- Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
- Approver (likely from SIG/area to which feature belongs):
- Feature target (which target equals to which milestone):
- Alpha release target (x.y)
- Beta release target (x.y)
- Stable release target (x.y)
Set the following:
- Description
- Assignee(s)
- Labels:
- stage/{alpha,beta,stable}
- sig/*
- kind/feature
Please note that the Features Freeze is July 31st, after which any incomplete Feature issues will require an Exception request to be accepted into the milestone.
In addition, please be aware of the following relevant deadlines:
- Docs deadline (open placeholder PRs): 8/21
- Test case freeze: 8/28
Please make sure all PRs for features have relevant release notes included as well.
Happy shipping!
/cc @justaugustus @kacole2 @robertsandoval @rajendar38
from enhancements.
@kubernetes/sig-auth-feature-requests can we graduate this to stable in v1.12?
from enhancements.
@mikedanese @kubernetes/sig-auth-feature-requests @kubernetes/sig-auth-misc --
Feature Freeze is today. Are we planning on graduating this feature in Kubernetes 1.12?
If so, can you make sure everything is up-to-date, so I can include it on the 1.12 Feature tracking spreadsheet?
from enhancements.
I'm ok with this going to GA as long as the certificates API remains beta.
from enhancements.
Can a GA feature depend on a beta API? What does that mean, support-wise?
from enhancements.
I've added this to the 1.12 sheet as stable, but let me know if we need to walk that back.
cc: @kacole2 @wadadli @robertsandoval @rajendar38
from enhancements.
I'm ok with this going to GA as long as the certificates API remains beta.
Why not graduate the as-is API to GA and do a v2 in case we need additional features or whatever?
from enhancements.
We discussed this in the sig-auth meeting on 8/8, and agreed the bootstrap feature can be promoted to stable, but not the CSR API yet. That means the externally-facing portions of the bootstrap mechanism (bootstrap kubeconfig, etc) will continue to be supported.
from enhancements.
Hey there! @philips I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?
from enhancements.
from enhancements.
from enhancements.
@zparnold please use the assignee of the feature issue rather than the original creator (which is immutable) of the issue for this kind of stuff. I can do this.
from enhancements.
from enhancements.
@mikedanese Will you be able to handle docs?
from enhancements.
@mikedanese --
Any update on docs status for this feature? Are we still planning to land it for 1.12?
At this point, code freeze is upon us, and docs are due on 9/7 (2 days).
If we don't here anything back regarding this feature ASAP, we'll need to remove it from the milestone.
cc: @zparnold @jimangel @tfogo
from enhancements.
It looks like this has graduated to GA so I'm going to clean up and close the issue. nice job everyone!
/close
from enhancements.
@kacole2: Closing this issue.
In response to this:
It looks like this has graduated to GA so I'm going to clean up and close the issue. nice job everyone!
/close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
from enhancements.
Related Issues (20)
- VolumeSource: OCI Artifact and/or Image HOT 29
- CRI Logging Docs & Spec Inaccurate Example for Partial Logs HOT 5
- Cluster Feature Gate in etcd HOT 1
- StatefulSet Support for Updating Volume Claim Template HOT 4
- Add kubelet instance configuration to configure CRI socket for each node HOT 2
- Guarantee PodDisruptionBudget When Preemption Happens HOT 2
- Gang Scheduling Support in Kubernetes HOT 23
- misspell: add a misspell ignore words HOT 8
- Add Resource Health Status to the Pod Status for Device Plugin and DRA HOT 17
- Deprecate and remove kustomize from kubectl HOT 24
- relationship between --image-gc-high-threshold and imagefs.available HOT 1
- Expose Node labels via downward API HOT 1
- Kubernetes-etcd interface HOT 1
- KEP-4753: Expose `ownerReferences` via `valueFrom` and downward API HOT 1
- Introduce load balancer for client-go HOT 5
- Introduce finalizerProbe for the pods like we have livenessProbe HOT 6
- [WIP]Keep container ready state after kubelet restarted HOT 1
- Allows setting any FQDN as the pod's hostname HOT 3
- kubectl supports WatchList to list resources HOT 1
- KEP 4770: EndpointSlice Controller Flexibility HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from enhancements.