openfaas / ingress-operator Goto Github PK
View Code? Open in Web Editor NEWCustom domains, paths and TLS for your OpenFaaS Functions
Home Page: https://docs.openfaas.com/reference/ssl/kubernetes-with-cert-manager/
License: MIT License
Custom domains, paths and TLS for your OpenFaaS Functions
Home Page: https://docs.openfaas.com/reference/ssl/kubernetes-with-cert-manager/
License: MIT License
We could use Google's distroless container image for the runtime for this operator
It's using scratch and then we manually copy in things like certs and user files.
See also: https://github.com/alexellis/registry-creds/blob/master/Dockerfile#L28
The function deployed in the steps below should be invokable at http://example.kube/api/v1/test/
.
The function deployed is not reachable at http://example.kube/api/v1/test/
but rather a 404 (by Nginx) is returned. Visiting http://example.kube/api/v1/test/(.*)
also returns a 404 (OpenFaaS). While the function works as intended at http://gw.example.kube/function/test-fn
Check at: https://github.com/sponsors/openfaas
The only workarounds I can think of right now. Are to either call the gateway path (i.e., not using the ingress-operator) or manually compose and maintain ingresses.
I'd prefer it if I got ingress-operators working as intended.
127.0.0.1 example.kube
and 127.0.0.1 gw.example.kube
to your hosts file.GW_HOST="http://gw.example.kube"
FN_NAME="test-fn"
# Create namespaces
kubectl create namespace ingress-nginx
kubectl apply -f https://raw.githubusercontent.com/openfaas/faas-netes/master/namespaces.yml
# Update chart repos
helm repo add nginx-stable https://helm.nginx.com/stable
helm repo add openfaas https://openfaas.github.io/faas-netes/
helm repo update
# Apply charts
helm install example-ingress nginx-stable/nginx-ingress \
-n ingress-nginx \
--set rbac.create=true
helm install openfaas openfaas/openfaas \
-n openfaas \
--values openfaas.yml
# Wait for OpenFaaS to start
sleep 60
# Login to gateway
echo $(kubectl -n openfaas get secret basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode) | faas-cli login -g $GW_HOST -u admin --password-stdin
# Get and host base node14 template
faas-cli template store pull node14
faas-cli new $FN_NAME --lang node14 -g $GW_HOST
faas-cli build -f "$FN_NAME.yml"
faas-cli deploy -f "$FN_NAME.yml"
# Apply FunctionIngress for test-fn
kubectl apply -f test-fi.yml
with the following files:
# openfaas.yml
functionNamespace: openfaas-fn
generateBasicAuth: true
ingressOperator:
create: true
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx
hosts:
- host: gw.example.kube
serviceName: gateway
servicePort: 8080
path: /
faasnetes:
imagePullPolicy: IfNotPresent
# test-fi.yml
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: test-fn
namespace: openfaas
spec:
domain: "example.kube"
function: "test-fn"
ingressType: "nginx"
path: "/api/v1/test/(.*)"
http://example.kube/api/v1/test/
I'm trying to create a more REST-like path for my api. As well as, replace existing apis with functions.
faas-cli version
): ___ _____ ____
/ _ \ _ __ ___ _ __ | ___|_ _ __ _/ ___|
| | | | '_ \ / _ \ '_ \| |_ / _` |/ _` \___ \
| |_| | |_) | __/ | | | _| (_| | (_| |___) |
\___/| .__/ \___|_| |_|_| \__,_|\__,_|____/
|_|
CLI:
commit: 72816d486cf76c3089b915dfb0b66b85cf096634
version: 0.13.13
docker version
(e.g. Docker 17.0.05 ):kubectl version
This controller is a good candidate for a multi-arch build given that it's built as a static binary.
v1alpha2
cert-manager
API referred to in the README is getting deprecated.
Update the example configs to use the latest v1
API.
It should deploy artifacts/operator-rbac.yaml
It deploys, but kubectl logs
come back with
Failed to list *v1beta1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:openfaas:ingress-operator" cannot list resource "ingresses" in API group "networking.k8s.io" in the namespace "openfaas"
modify artifacts/operator-rbac.yaml
to
Trying to setup custom domain for my functions
Docker version docker version
(e.g. Docker 17.0.05 ):
What version and distriubtion of Kubernetes are you using? kubectl version
1.6
Operating System and version (e.g. Linux, Windows, MacOS):
Link to your project or a code example to reproduce issue:
What network driver are you using and what CIDR? i.e. Weave net / Flannel
K8s < 1.22 and > 1.22 should work without changes.
Only < 1.22 is supported due to the API group being used going away.
Have an if statement based upon the kube API server version returned.
Must fix to continue being used on latest K8s versions.
We currently let the user choose the kind
of cert-manager issuer, the two available options are Issuer
and ClusterIssuer
. However, we don't use the value of that field when we set the annotations.
I would expect that when I set the kind
field inside issuerRef
that change would be reflected on the ingress's annotations.
We can set the kind
field inside issuerRef
but it's a no-op.
I see the following two options:
issuer
kind and remove the kind
field so that we are not misleading the userkind
to set the cert-manager's annotationsI've implemented the second one and test it out on my cluster. It works OK and it was a very simple change, but I don't know if this is what we want. Lets discuss ๐
kind
in issuerRef
to ClusterIssuer
and notice that the annotation continues to be for an issuer
.I was trying to create a new function ingress using the existing ClusterIssuer
in my cluster, this issuer is created by ofc-bootstrap when installing ofc in the cluster.
Docker version docker version
(e.g. Docker 17.0.05 ):
What version and distriubtion of Kubernetes are you using? kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"66049e3b21efe110454d67df4fa62b08ea79a19b", GitTreeState:"clean", BuildDate:"2019-05-16T16:23:09Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.7-gke.8", GitCommit:"7d3d6f113e933ed1b44b78dff4baf649258415e5", GitTreeState:"clean", BuildDate:"2019-06-19T16:37:16Z", GoVersion:"go1.11.5b4", Compiler:"gc", Platform:"linux/amd64"}
Operating System and version (e.g. Linux, Windows, MacOS):
Windows
Link to your project or a code example to reproduce issue:
What network driver are you using and what CIDR? i.e. Weave net / Flannel
Calico on GKE
Create cert using domain instead of function name
If we have three FunctionIngress entries all on the same domain but with differing paths, we should create 1 cert only.
Three get created since the "name" is the function's name instead of the domain name.
Create cert using domain instead of function name. This will need a change in the annotation and / or secret name.
See this example for repro - https://docs.openfaas.com/reference/ssl/kubernetes-with-cert-manager/#30-rest-style-api-mapping-for-your-functions
Update CI and Dockerfile to use Go 1.18
Still using Go 1.16 to build this project
https://github.com/openfaas/ingress-operator/blob/master/go.mod#L3
https://github.com/openfaas/ingress-operator/blob/master/Dockerfile
https://github.com/openfaas/ingress-operator/tree/master/.github/workflows
When defining the FunctionIngress
you should be able to customise the host path, e.g. in a RESTful API you may want to map functions more logically (or if you were porting an existing application).
Path maps and routes correctly
FunctionIngress
only ever maps to /(.*)
for each domain specified
Currently, the generated ingress looks like;
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
cert-manager.io/issuer: letsencrypt-openfaas-io-prod
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /function/openfaas-insiders/$1
name: insiders-openfaas-io-tls
spec:
rules:
- host: insiders.openfaas.io
http:
paths:
- backend:
serviceName: gateway
servicePort: 8080
path: /(.*)
tls:
- hosts:
- insiders.openfaas.io
secretName: insiders-openfaas-io-tls-cert
However, for this particular use case the end result should look like;
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: api-app-profiles-create
annotations:
nginx.ingress.kubernetes.io/rewrite-target: "/function/profiles-view/$1"
kubernetes.io/ingress.class: "ingress-api"
namespace: openfaas
spec:
tls:
- hosts:
- api.example.dev
secretName: example-tls
rules:
- host: api.example.dev
http:
paths:
- path: /v1/profiles/view/(.*)
backend:
serviceName: gateway
servicePort: 8080
Through some form of annotation on the CRD filter up the; spec.rules.host.http.paths.backend.path
Provides RESTful like URI's through OF without the need for manually defining the ingress(s)
Currently only the Nginx ingress controller is supported. It'd be good to provide support for others, in this case Skipper.
Example manual ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-openfaas
namespace: openfaas
annotations:
kubernetes.io/ingress.class: skipper
zalando.org/skipper-filter: setPath("/function/hello-openfaas")
spec:
rules:
- host: hello-openfaas.example.com
http:
paths:
- path: /
backend:
serviceName: gateway
servicePort: 8080
We're not using Nginx controller in our clusters so wouldn't be able to leverage this controller. Currently we're manually creating ingress resources using the above example.
What version and distriubtion of Kubernetes are you using: 1.12
What network driver are you using and what CIDR: Weave net
spec.ingressClassName needs to be set on Ingress records created by the operator
The kubernetes.io/ingress.class annotation is still being used, and is deprecated
Update reconciliation behaviour and test out the change with Traefik and ingress-nginx separately.
At some point the warning will become an error.
go run . --kubeconfig ~/.kube/config
I1020 14:20:08.391934 3319312 main.go:51] Starting FunctionIngress controller version: latest-dev commit: local-dev
I1020 14:20:08.394413 3319312 controller.go:70] Setting up event handlers
I1020 14:20:08.394456 3319312 core.go:72] Waiting for informer caches to sync
I1020 14:20:08.494815 3319312 core.go:77] Starting workers
I1020 14:20:08.494842 3319312 core.go:83] Started workers
I1020 14:20:26.381036 3319312 controller.go:116] Creating Ingress for: nodeinfo
W1020 14:20:26.452110 3319312 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
When adding/updating annotations to the FunctionIngress
, these should be added/updated on the underlying Ingress
.
If the FunctionIngress
and the underlying Ingress
already exist, changes to the FunctionIngress
annotations aren't passed to the underlying Ingress
.
There's probably a validation on whether the ingress already exists that is bypassing the expected behaviour.
FunctionIngress
with no custom annotationskubectl
nginx.ingress.kubernetes.io/limit-connections: 10
kubectl
All target functions are assumed to be within the openfaas-fn namespace, and the spec only takes a function as an input:
apiVersion: openfaas.com/v1
kind: FunctionIngress
metadata:
name: nodeinfo
namespace: openfaas
spec:
domain: "nodeinfo.myfaas.club"
function: "nodeinfo"
ingressType: "nginx"
path: "/v1/profiles/(.*)"
Add a functionNamespace field into the spec:
apiVersion: openfaas.com/v1
kind: FunctionIngress
metadata:
name: nodeinfo
namespace: openfaas
spec:
domain: "nodeinfo.myfaas.club"
function: "nodeinfo"
+ functionNamespace: "staging-fn"
ingressType: "nginx"
path: "/v1/profiles/(.*)"
If we really want to do things properly and make a breaking change, this could be a functionRef
:
spec:
domain: "nodeinfo.myfaas.club"
functionRef:
name: "nodeinfo"
namespace: "staging-fn"
But I think that might be more work than is warranted for now.
Kubiya asked for this in #63
All FunctionIngress objects and their eventual Ingress objects must be created in the openfaas namespace since they all point at the gateway.
But that said, the namespace is just appended to the path that comes out on the Ingress:
http://gateway.openfaas:8080/function/FUNCTION.NAMESPACE
Convert CI to GitHub Actions
We use Travis and need to move away
See the approach in https://github.com/openfaas/faas-netes/tree/master/.github
$ kubectl apply -f examples/tls-nodeinfo-ingress.yaml
functioningress.openfaas.com/nodeinfo-tls created
$ kubectl apply -f examples/tls-nodeinfo-ingress.yaml
The FunctionIngress "nodeinfo-tls" is invalid: spec.path: Required value
Add path: "/"
to each of the examples and remove the comment in the Readme that path
is optional https://github.com/openfaas-incubator/ingress-operator#schema
Trying to test my last PR by using the example files
The badge in the readme should reflect the status of the CI build for this project
It shows the status of a Travis build which is unknown
Update with the GitHub Actions generated badge
Adding nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
annotations to FunctionIngress resources has no effect.
My function is a website that uses the dockerfile template and is served with Nginx. I am wondering if this Nginx is causing the annotation to not be applied
Expect annotations to be applied.
Adding from-to-www-redirect
annotation.
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: example
namespace: openfaas
annotations:
nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
...
kubectl apply -f fn_ingress.yml
Note that I can see the annotation on functioningress resource.
$ kubectl describe functioningress.openfaas.com/example -n openfaas
Name: example
Namespace: openfaas
Labels: <none>
Annotations: nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
And also in the ingress resource.
$ kubectl describe ing -n openfaas
Name: example
Namespace: openfaas
...
Annotations: cert-manager.io/issuer: letsencrypt-prod
com.openfaas.spec: {"metadata":{"name":"example","namespace":"openfaas"...
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/from-to-www-redirect: true
nginx.ingress.kubernetes.io/rewrite-target: /function/example/$1
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-21T14:51:23Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.5", GitCommit:"e0fccafd69541e3750d460ba0f9743b90336f24f", GitTreeState:"clean", BuildDate:"2020-04-16T11:35:47Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Per the security announcement https://groups.google.com/forum/#!topic/golang-announce/65QixT3tcmg we should update the docker build layers to at least Go 1.11.13
Because the docker build layers already use golang:1.11
we should only need to rebuild and tag a new release to ensure that it is using the latest 1.11.13 base image
openfaas/faas#1291
openfaas/templates#170
openfaas/faas-netes#494
openfaas/nats-queue-worker#66
openfaas/of-watchdog#78
https://github.com/openfaas-incubator/faas-idler/issues/32
openfaas/golang-http-template#28
openfaas-incubator/faas-federation#4
openfaas-incubator/vcenter-connector#27
openfaas-incubator/faas-memory#3
openfaas-incubator/faas-rancher#8
openfaas/faas-swarm#56
#10
https://github.com/openfaas-incubator/openfaas-operator/issues/87
The YAML spec file should contain a field for Path for validation, it's missing.
The Path field from FunctionIngress' Spec is missing from the YAML CRD
Add it, or regenerate the spec.
There is no bug, it still works, but it's better to have the schema correct.
Reported in #31
The current instructions in the readme https://github.com/openfaas-incubator/ingress-operator#ingresscontroller-nginx
$ helm install nginxingress stable/nginx-ingress --set rbac.create=true
WARNING: This chart is deprecated
NAME: nginxingress
LAST DEPLOYED: Sat Oct 31 17:34:38 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
*******************************************************************************************************
* DEPRECATED, please use https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx *
*******************************************************************************************************
The nginx-ingress controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w nginxingress-nginx-ingress-controller'
An example Ingress that makes use of the controller:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: example
namespace: foo
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
serviceName: exampleService
servicePort: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
$ helm install stable/nginx-ingress --name nginxingress --set rbac.create=true
Error: unknown flag: --name
Update the helm install instructions, we should also update which repo it is using.
$ helm version
version.BuildInfo{Version:"v3.2.4", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"dirty", GoVersion:"go1.14.3"}
When using Traefik and the FunctionIngress object, we will get an Ingress record created, however I am not sure we will get functioning TLS with cert-manager. We do get that with Nginx presently, so this issue should be partly figuring out which additional annotations are needed and applying them.
Add cert-manager should be supported with Traefik
Untested, it may be but I suspect it is not due to the additional annotations I found in a (French) blog post.
Currently only the annotations for cert-manager are added: https://github.com/openfaas-incubator/ingress-operator/blob/master/pkg/controller/controller.go#L538
Edit the following code: https://github.com/openfaas-incubator/ingress-operator/blob/master/pkg/controller/controller.go#L514
Add these annotations:
traefik.ingress.kubernetes.io/redirect-entry-point: https
traefik.ingress.kubernetes.io/redirect-permanent: "true"
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/ssl-temporary-redirect: "false"
As see here: https://www.cerenit.fr/blog/kubernetes-ovh-traefik-cert-manager-secrets/
You can rebuild the controller with make
and specify a TAG
env-var, then tag this as your own username before pushing it. You can also run the cluster as a local Go binary.
K3s ships with Traefik running in "host mode", so this would be a very fast way of testing the fix. A VM on DO or a similar cloud will come with a public IP for the DNS certificate challenge.
You'll also need your own domain name.
traefik
This is an important piece of the puzzle for Traefik users.
Instead of using the custom spec.tls.*
definition to generate certificates we can use the default cert-manager annotations (certmanager.k8s.io/issuer
or certmanager.k8s.io/cluster-issuer
), since #7 was implemented.
We need to use a custom spec.tls.*
definition to enable the certificate creation.
Even though we are copying all the annotations to the Ingress, the current implementation of the controller doesn't support the default cert-manager
ones.
Remove the special case for TLS annotations inside makeAnnotations and change the makeTLS functions to look into the annotations instead of function.Spec.UseTLS()
.
I didn't try these changes, just an idea by taking a quick look into the code. It may be more complicated than that.
Using the same Ingress
behaviour to create TLS certificates in the FunctionIngress
would require a lower learning curve, less docs and make configurations more standard across the cluster.
Since #7 was merged, we can use ExternalDNS to automatically create DNS entries for the FunctionIngress
.
We could document this behaviour, including:
What do you think? Any more ideas on what do document?
Instead of using spec.ingressType
to select an Ingress class, we should use the default annotation kubernetes.io/ingress.class
, since #7 was implemented.
In the version 0.4.0
, by the docs, we need to define the Ingress class using the spec.ingressType
configuration.
BUT
After merging #7 we actually don't need that. I tested two cases:
Deploying the following FunctionIngress
without spec.ingressType
and without kubernetes.io/ingress.class
annotation defaults the Ingress
class to nginx
and works. I believe this should be considered a bug and specifying an Ingress class should be mandatory.
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: nodeinfo
namespace: openfaas
spec:
domain: "nodeinfo.deployeveryday.com"
function: "nodeinfo"
# ingressType: "nginx" # disable ingressType
Deploying the following FunctionIngress
without spec.ingressType
but using the kubernetes.io/ingress.class
works like a charm. I believe this should be the default way of doing things.
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: nodeinfo
namespace: openfaas
annotations:
# Uses default K8S annotation
kubernetes.io/ingress.class: "nginx"
spec:
domain: "nodeinfo.deployeveryday.com"
function: "nodeinfo"
# ingressType: "nginx" # disable ingressType
function.Spec.IngressType
.Using the default K8S annotation to select an Ingress class requires lower learning curve and it's more a standard across other K8S controllers.
It will be useful if we can pass on the ingress operator the secretName
on config.yaml
.
At the moment it only creates a new cert from cert issuer like the cert-manager
provider.
We have a case where we are using only wildcard domain and we have to edit manually each Ingress to use the same secretName
like an example below.
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: nodeinfo
namespace: openfaas
spec:
domain: "nodeinfo.myfaas.club"
function: "nodeinfo"
ingressType: "nginx"
tls:
enabled: true
secretName: same-cert-secret
If it is a workaround like to automate this bit, it will very useful to know.
What do you think about it?
Test multiple paths for the same host
i.e. two function ingress entries as per the new feature in #20, but with the same domain and differing paths.
I.e. can we write two FunctionIngress records that would serve two separate back-end functions from the same sub-domain and TLS cert:
api.dx.no/v1/customers
api.dx.no/v1/tickets
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: api.dx.no
namespace: openfaas
spec:
domain: "api.dx.no"
function: "list-customers"
ingressType: "nginx"
path: "/v1/customers/(*.)"
tls:
enabled: true
issuerRef:
name: "letsencrypt-staging"
kind: "Issuer"
---
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: api.dx.no
namespace: openfaas
spec:
domain: "api.dx.no"
function: "list-tickets"
ingressType: "nginx"
path: "/v1/tickets/(*.)"
tls:
enabled: true
issuerRef:
name: "letsencrypt-staging"
kind: "Issuer"
It should deploy the operator
The deployment never gets ready, CrashLoopBack on pod
Modify ingress-operator/artifacts/operator-amd64.yaml
command:
- ./ingress-operator
- -logtostderr
- -v=2
remove --v=2
After that it worked for me.
Docker version docker version
(e.g. Docker 17.0.05 ):
What version and distriubtion of Kubernetes are you using? kubectl version
1.6
Operating System and version (e.g. Linux, Windows, MacOS):
Link to your project or a code example to reproduce issue:
What network driver are you using and what CIDR? i.e. Weave net / Flannel
It's still needed to manually create a DNS entry when using FunctionIngress
. Implementing support to ExternalDNS would allow FunctionIngress
to create, delete and update DNS entries automatically.
The need to create manually DNS entries each time a new FunctionIngress
is deployed.
Implement support for ExternalDNS to manage DNS entries automatically.
This tutorial shows how to use ExternalDNS with NGINX Ingress. The behaviour is basically synchronising the Ingress domains names with the DNS.
Also, if using with Services, ExternalDNS has annotations to indicate that the DNS entry should be manager, as shown in this example. We could try to enable/disable the ExternalDNS in FunctionIngress
with these annotations.
Would be amazing to eliminate one more manual process to get TLS/custom handlers to our functions โก๏ธ
All traffic destined to a domain correctly forwarded to a function regardless of the path requested.
Depending on installed version of the Traefik, you may observe different behaviors.
If your Traefik version is below v1.7.21, then your function will receive incorrect (duplicated) path, e.g. nodeinfo.example.com/version
will be forwarded to gateway/function/nodeinfo/versionversion
and your function will get /versionversion
as the path. All requests to root path (/) will be forwarded correctly to a function.
If you have Traefik v.1.7.21 or above then you'll get infinite redirect in attempt to request any path (even root)!
The behaviors are different because of this issue that has been addressed in release v1.7.21 of Traefik
Traefik's rewrite target annotation should be changed from
traefik.ingress.kubernetes.io/rewrite-target: /function/FUNC_NAME/$1
to
traefik.ingress.kubernetes.io/rewrite-target: /function/FUNC_NAME
where FUNC_NAME
is the name of the function.
I've created PR which fixes this issue.
(Nice to have) Document (where?) that Traefik v1.7.21 or above is required
Disclaimer. I tested all of the following on linux, so not really sure that this would work on Mac or Windows.
minikube start
helm upgrade -i traefik \
--version 1.87.1 \
--namespace kube-system \
--set imageTag=1.7.20 \
--set rbac.enabled=true \
--set dashboard.enabled=true \
--set dashboard.serviceType=NodePort \
--set dashboard.domain=traefik.local.dev \
--set dashboard.ingress.annotations."kubernetes\.io\/ingress\.class=traefik" \
--set deployment.hostPort.httpEnabled=true \
--set deployment.hostPort.dashboardEnabled=true \
--set serviceType=NodePort \
--set containerSecurityContext.capabilities.drop[0]=ALL \
--set containerSecurityContext.capabilities.add[0]=NET_BIND_SERVICE \
--set deploymentStrategy.type=Recreate \
stable/traefik
kubectl create ns dev-openfaas
kubectl create ns dev-openfaas-fn
helm upgrade -i openfaas \
--version 5.8.5 \
--namespace dev-openfaas \
--set functionNamespace=dev-openfaas-fn \
--set basic_auth=false \
--set generateBasicAuth=false \
--set ingressOperator.create=true \
--set ingress.enabled=true \
--set ingress.hosts[0].host=gateway.local.dev \
--set ingress.hosts[0].serviceName=gateway \
--set ingress.hosts[0].servicePort=8080 \
--set ingress.hosts[0].path=/ \
--set ingress.annotations."kubernetes\.io\/ingress\.class=traefik" \
openfaas/openfaas
echo "$(minikube ip) traefik.local.dev" | sudo tee -a /etc/hosts
echo "$(minikube ip) gateway.local.dev" | sudo tee -a /etc/hosts
or alternatively you can get their services urls to access their UIs
minikube -n kube-system service traefik-dashboard --url
minikube -n dev-openfaas service gateway-external --url
faas-cli deploy \
--image stefanprodan/podinfo:3.1.0 \
--namespace dev-openfaas-fn \
--name podinfo \
--annotation com.openfaas.health.http.initialDelay="2s" \
--annotation com.openfaas.health.http.path="/healthz" \
--env PODINFO_PORT="8080" \
--gateway $(minikube -n dev-openfaas service gateway-external --url) \
--tls-no-verify
cat <<EOF | kubectl apply -f -
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: podinfo
namespace: dev-openfaas
spec:
function: podinfo
domain: podinfo.local.dev
ingressType: "traefik"
bypassGateway: false
tls:
enabled: false
EOF
Now the interesting part begins
curl -s $(minikube -n kube-system service traefik-dashboard --url)/api/providers/kubernetes/frontends | jq -r '.["podinfo.local.dev/"].routes'
Output:
{
"/": {
"rule": "PathPrefix:/;ReplacePathRegex: ^/(.*) /function/podinfo/$1$1"
},
"podinfo.local.dev": {
"rule": "Host:podinfo.local.dev"
}
}
Note the two $1
at the end of the string with the rule. This is why our function gets duplicated path with this Traefik version.
echo "$(minikube ip) podinfo.local.dev" | sudo tee -a /etc/hosts
curl -s http://podinfo.local.dev
Output:
{
"hostname": "podinfo-5569bb569c-9wvdw",
"version": "3.1.0",
"revision": "7b6f11780ab1ce8c7399da32ec6966215b8e43aa",
"color": "cyan",
"logo": "https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif",
"message": "greetings from podinfo v3.1.0",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.13.1",
"num_goroutine": "6",
"num_cpu": "2"
}
And let's also request the other path
curl -s http://podinfo.local.dev/version
Output:
404 page not found
Function actually gets /versionversion
as the path.
Let's also make sure that this path is handled by requesting it through gateway:
curl -s http://gateway.local.dev/function/podinfo/version
or
curl -s $(minikube -n dev-openfaas service gateway-external --url)/function/podinfo/version
Output:
{
"commit": "7b6f11780ab1ce8c7399da32ec6966215b8e43aa",
"version": "3.1.0"
}
kubectl -n dev-openfaas annotate --overwrite ing podinfo 'traefik.ingress.kubernetes.io/rewrite-target=/function/podinfo'
curl -s http://podinfo.local.dev/version
Output:
error finding function podinfoenv.dev-openfaas-fn: server returned non-200 status code (404) for function, podinfoenv
kubectl -n dev-openfaas annotate --overwrite ing podinfo 'traefik.ingress.kubernetes.io/rewrite-target=/function/podinfo/$1'
helm upgrade traefik \
--namespace kube-system \
--reuse-values \
--set imageTag=1.7.23 \
stable/traefik
kubectl -n kube-system rollout status deploy/traefik
curl -sLv --max-redirs 1 http://podinfo.local.dev
Output:
* Rebuilt URL to: http://podinfo.local.dev/
* Trying 192.168.39.39...
* TCP_NODELAY set
* Connected to podinfo.local.dev (192.168.39.39) port 80 (#0)
> GET / HTTP/1.1
> Host: podinfo.local.dev
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-Length: 0
< Date: Thu, 18 Jun 2020 11:18:59 GMT
< Location: /function/podinfo/
< Vary: Accept-Encoding
<
* Connection #0 to host podinfo.local.dev left intact
* Issue another request to this URL: 'http://podinfo.local.dev/function/podinfo/'
* Found bundle for host podinfo.local.dev: 0x5566c09a19d0 [can pipeline]
* Re-using existing connection! (#0) with host podinfo.local.dev
* Connected to podinfo.local.dev (192.168.39.39) port 80 (#0)
> GET /function/podinfo/ HTTP/1.1
> Host: podinfo.local.dev
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-Length: 0
< Date: Thu, 18 Jun 2020 11:18:59 GMT
< Location: /function/podinfo/function/podinfo/function/podinfo/
< Vary: Accept-Encoding
<
* Connection #0 to host podinfo.local.dev left intact
* Maximum (1) redirects followed
curl -sLv --max-redirs 1 http://podinfo.local.dev/version
Output:
* Trying 192.168.39.39...
* TCP_NODELAY set
* Connected to podinfo.local.dev (192.168.39.39) port 80 (#0)
> GET /version HTTP/1.1
> Host: podinfo.local.dev
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Thu, 18 Jun 2020 11:22:08 GMT
< Location: /function/podinfo/version/version
< Vary: Accept-Encoding
<
* Connection #0 to host podinfo.local.dev left intact
* Issue another request to this URL: 'http://podinfo.local.dev/function/podinfo/version/version'
* Found bundle for host podinfo.local.dev: 0x5584618379d0 [can pipeline]
* Re-using existing connection! (#0) with host podinfo.local.dev
* Connected to podinfo.local.dev (192.168.39.39) port 80 (#0)
> GET /function/podinfo/version/version HTTP/1.1
> Host: podinfo.local.dev
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-Length: 0
< Date: Thu, 18 Jun 2020 11:22:08 GMT
< Location: /function/podinfo/function/podinfo/version/version/function/podinfo/version/version
< Vary: Accept-Encoding
<
* Connection #0 to host podinfo.local.dev left intact
* Maximum (1) redirects followed
kubectl -n dev-openfaas annotate --overwrite ing podinfo 'traefik.ingress.kubernetes.io/rewrite-target=/function/podinfo'
curl -s http://podinfo.local.dev
Output:
{
"hostname": "podinfo-5569bb569c-9wvdw",
"version": "3.1.0",
"revision": "7b6f11780ab1ce8c7399da32ec6966215b8e43aa",
"color": "cyan",
"logo": "https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif",
"message": "greetings from podinfo v3.1.0",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.13.1",
"num_goroutine": "6",
"num_cpu": "2"
}
curl -s http://podinfo.local.dev/version
Output:
{
"commit": "7b6f11780ab1ce8c7399da32ec6966215b8e43aa",
"version": "3.1.0"
}
Those who made this far: Thanks for reading and don't forget to clean your hosts file at /etc/hosts
I'm trying to use ingress operator with traefik to forward all traffic from specific domain to function.
Docker version docker version
(e.g. Docker 17.0.05 ):
Docker version 19.03.11, build 42e35e61f3
What version and distriubtion of Kubernetes are you using? kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-21T20:58:41Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:48:36Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Operating System and version (e.g. Linux, Windows, MacOS):
Ubuntu 18.04.4 LTS
Link to your project or a code example to reproduce issue:
n/a
What network driver are you using and what CIDR? i.e. Weave net / Flannel
n/a
In Kubernetes 1.18 the Ingress definitions have changed slightly.
In particular the Kubernetes release notes [1] mention the change about ingressClass becoming part of the spec, instead of being an annotation.
If we can add both, perhaps that will help keep backwards compatibility for the time being?
Alex
This feature would allow an Ingress record to be created that bypasses the gateway, so that an endpoint can be accessed with absolute minimal latency, by removing an additional hop.
Today the IngressOperator, FNI definitions are created in the openfaas
namespace which causes an Ingress record to be created in the openfaas
namespace pointing at the gateway, which is also deployed in that namespace.
For a bypass mode we will have to deploy all resources a second time into the openfaas-fn
namespace because Ingress cannot point into a different namespace - i.e. having an Ingress in openfaas
which connects to a service in openfaas-fn
. It only works in the first example because the gateway acts as a proxy.
See also: kubernetes/kubernetes#17088
I expect the ingress-operator to be able to watch multiple namespaces, all namespaces, or namespaces specified by annotation. This would also include the appropriate RBAC permissions for the ingress-operator.
Currently, the ingress-operator is set to watch a specific namespace as defined in the 'function_namespace' and 'ingress_namespace' environment variables. If I want to use a different namespace, I have to manually override these values and adjust the RBAC permissions.
Adding an option to watch all namespaces in nothing was set.
Allowing the definition of multiple namespaces to watch.
Allowing the use of namespace annotations to determine which namespaces to watch.
I would recommend implementing the option to watch namespaces by labels(as with OpenFaas pro), as this would offer the most flexibility to users.
It's more a feature request than bug.
Anyway the steps to reproduce are simple: deploy the stack.yaml to namespace associated with openfaas.
I am trying to deploy the ingress-operator in a different namespace ('demo' in my case).
However, I need to override the 'function_namespace', 'ingress_namespace' values and adjust the RBAC permissions time.
Less related to the issue.
faas-cli version
):Docker version docker version
(e.g. Docker 17.0.05 ):
What version and distriubtion of Kubernetes are you using? kubectl version
Server Version: v1.21.14-eks-0a21954
Operating System and version (e.g. Linux, Windows, MacOS):
AmazonLinux2
version: 1.0
provider:
name: openfaas
gateway: https://openfaas-gw.dev.local/
functions:
next-gen:
lang: [email protected]
handler: .
image: my-custom-image:tag
secrets: []
environment:
fprocess: python function/main.py
labels:
com.openfaas.scale.min: "1"
com.openfaas.scale.max: "5"
com.openfaas.scale.target: "5"
com.openfaas.scale.type: rps
com.openfaas.scale.target-proporation: "0.80"
function-ingress:
next-gen-ing:
domain: "gen2.dev.local"
function: "next-gen"
ingressType: "nginx-general"
annotations:
external-dns.alpha.kubernetes.io/hostname: "gen2.dev.local"
If I have created a function ingress resource like so:
apiVersion: openfaas.com/v1alpha2
kind: FunctionIngress
metadata:
name: pets
namespace: openfaas
spec:
domain: api.petstore.com
function: pets
ingressType: nginx
path: /pets/(.*)
How would I retrieve the (.*)
part of the path from within the function handler? I am using the golang-http
template btw.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.