Override load-balancer healthcheck target via annotations about cloud-provider-aws

[vvbogdanov87]

Kuberntes 1.14 uses SSL health checks for ELBs when backend protocol is SSL/HTTPS
But for the health check pIng target k8s selects first port in the listeners list
https://github.com/2rs2ts/kubernetes/blob/72895a84a9670d6f6f8921681c5bbe4b2745319e/pkg/cloudprovider/providers/aws/aws.go#L3602
Without annotation described above, you MUST set https/SSL port to be first in the spec.ports
I had a service spec:

apiVersion: v1
kind: Service
metadata:
  name: myapp
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
spec:
  type: LoadBalancer
  selector:
    app: myapp
  ports:
    - name: http
      protocol: TCP
      port: 80
    - name: https
      protocol: TCP
      port: 443

After I upgraded to k8s 1.14 I got OutOfService for all instances in AWS LB. I had to change the spec to

  ports:
    - name: https
      protocol: TCP
      port: 443
    - name: http
      protocol: TCP
      port: 80

[steerben]

I think that is indeed a very valuable feature.

Considering subsequent use case:

HTTPS (ELB - ACM SSL termination) -> HTTPS (Worker Self signed SSL termination)

In that case the healthcheck also turns into HTTPS. However most health check endpoints run on separate ports and do not support SSL termination like e.g. the one of the istio ingress gateway which renders the mentioned use case not feasible at the moment.

[2rs2ts]

/remove-lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[nicolasbelanger]

Got burned by this too today. Consequence: no more ingress, useless cluster. We're using ambassador as the gateway, and the default values for the service is exactly as @vvbogdanov87 described above, http first, https second. SSL ping target was done on the http port, which failed.

[2rs2ts]

Use case 2 in the original post is pretty common at my company, so I'd love to have this. We do not use the standalone provider at my company though so I don't think even if I could find the time to implement this that I would know how to test it.

[afnanenayet]

We ran into this today as well at my company. I believe that in GKE the health check from the deployment propagates to the google cloud ELB, so you don't even need to set annotations if you want parity between the external load balancer and your internal kube service.

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[bordeuax]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[bordeuax]

/remove-lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[bordeuax]

/remove-lifecycle rotten

[kishorj]

Support available for specifying https protocol for NLB instance mode health check via annotatoin service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol kubernetes/kubernetes#92321. Fix will be included in 1.20.

NLB IP mode also supports the annotation, and is available in the AWS Load Balancer controller

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[nckturner]

/close

Ref:
kubernetes/kubernetes#92321
kubernetes/kubernetes#94546

This fix is included in the external cloud controller manager (this repository) as well.

[k8s-ci-robot]

@nckturner: Closing this issue.

In response to this:

/close

Ref:
kubernetes/kubernetes#92321
kubernetes/kubernetes#94546

This fix is included in the external cloud controller manager (this repository) as well.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.