client->VIP BFE->upstream server ？

ingress-bfe开源的功能太少，很多基础功能都不支持：

1. 负载均衡算法只支持默认的WRR；没有开源其他算法支持，如least_conn(WLC)等

2. 权限控制页面（mod_auth/mod_access等）也不支持

TestBfeRouteConfigBuilder_Build_CasePriorityHost4 occasionally fail.

see log:
#7
--- FAIL: TestBfeRouteConfigBuilder_Build_CasePriorityHost4 (0.01s)

TroubleShooting:
Route rule is supposed to be built based on global-sorted rules, but not host-specified-sorted rules.

背景

BFE 支持对流量进行URL重写，希望 bfe-ingress-controller 实现该功能。

配置方式

Ingress 资源内

• spec.rules 定义路由规则
• metadata.annotations 定义对符合路由规则的流量，修改 URL的行为
  • 定义修改覆盖的流量范围
  • 定位修改 URL 的行为

设计方案

#54 (comment)

参考格式：

重写时机

• 默认确认目标后端实例后，执行重写

确认目标后端前

确认目标后端后

重写 Host

通过特定 annotation 配置Host重写，可设置为以下模式其中之一：

• 静态 Host
• 动态 Host

静态 Host

对应示例

• 重写前: http://host/path?query-key=value
• 重写后: http://www.baidu.com/path?query-key=value

动态 Host

对应示例

• 重写前: https://old-host/new-host/path?query-key=value
• 重写后: https://new-host/path?query-key=value

重写 Path

通过特定 annotation 配置Path重写，可设置为以下模式其中之一：

• 静态 Path
• 动态 Path

静态 Path

对应示例

• 重写前: http://host/path?query-key=value
• 重写后: http://host/new-path?query-key=value

动态 Path

支持：

• PATH_PREFIX_ADD：添加Path前缀
• PATH_PREFIX_TRIM：删除Path前缀

添加前缀

对应示例

• 重写前: https://host/path?query-key=value
• 重写后: https://host/prefix/path?query-key=value

删除前缀

对应示例

• 重写前: https://host/prefix/path?query-key=value
• 重写后: https://host/path?query-key=value

重写 Query

支持：

• RAW_QUERY_ADD：添加指定Query
• RAW_QUERY_RENAME：重命名指定Query
• RAW_QUERY_DEL：删除指定Query
• RAW_QUERY_DEL_ALL_EXCEPT：仅保留指定Query，删除指定Query外的其他 Query

新增

对应示例

• 重写前: https://host/path?a=value
• 重写后: https://host/path?a=value&b=2

重命名

对应示例

• 重写前: https://host/path?a=value
• 重写后: https://host/path?b=value

删除

对应示例

• 重写前: https://host/path?a=value&b=2&c=3
• 重写后: https://host/path?a=value

仅保留

对应示例

• 重写前: https://host/path?a=value&b=2&c=3
• 重写后: https://host/path?a=value

BFE rewrite
Nginx ingress rewrite

背景

BFE 支持对流量进行重定向，希望 bfe-ingress-controller 实现该功能。

配置方式

Ingress 资源内

• spec.rules 定义路由规则
• metadata.annotations 定义对符合路由规则的流量，重定向响应的行为

参考格式：

metadata:
    annotations:
        bfe.ingress.kubernetes.io/redirect.target: "https://www.baidu.com"
        bfe.ingress.kubernetes.io/redirect.status: 302
spec:
  rules:
  - ...

metadata:
    annotations:
        bfe.ingress.kubernetes.io/redirect.cmd: SCHEME_SET
        bfe.ingress.kubernetes.io/redirect.params: '["http"]'
        bfe.ingress.kubernetes.io/redirect.status: 302
spec:
  rules:
  - ...

重定向类型

通过设置重定向返回状态码 bfe.ingress.kubernetes.io/redirect.status ，配置重定向类型

• 默认为 302 临时重定向

永久重定向

bfe.ingress.kubernetes.io/redirect.status: 301

bfe.ingress.kubernetes.io/redirect.status: 308

临时重定向

bfe.ingress.kubernetes.io/redirect.status: 302

重定向目标

通过特定 annotation 配置重定向目标，可设置为以下模式其中之一：

• 静态 URL
• 动态 URL

静态 URL

通过设置 bfe.ingress.kubernetes.io/redirect.target ，配置静态重定向目标URL

bfe.ingress.kubernetes.io/redirect.target: "https://www.baidu.com"

对应示例

• request: http://host/path?query-key=value
• response: https://www.baidu.com

动态 URL

通过设置 bfe.ingress.kubernetes.io/redirect.cmd 、 bfe.ingress.kubernetes.io/redirect.params ，配置静态重定向目标URL

设置 Scheme

bfe.ingress.kubernetes.io/redirect.cmd: SCHEME_SET
bfe.ingress.kubernetes.io/redirect.params: '["http"]'

对应示例

• request: https://host/path?query-key=value
• response: http://host/path?query-key=value

从Query中获得URL

bfe.ingress.kubernetes.io/redirect.cmd: URL_FROM_QUERY
bfe.ingress.kubernetes.io/redirect.params: '["url"]'

对应示例

• request: https://host/path?url=https%3A%2F%2Fwww.baidu.com
• response: https://www.baidu.com

添加前缀

bfe.ingress.kubernetes.io/redirect.cmd: URL_PREFIX_ADD
bfe.ingress.kubernetes.io/redirect.params: '["http://www.baidu.com/redirect"]'

对应示例

• request: https://host/path?query-key=value
• response: http://www.baidu.com/redirect/path?query-key=value

https://www.bfe-networks.net/en_us/modules/mod_redirect/mod_redirect/

背景

K8s 环境中已通过 Openshift Route 配置部分路由，希望 bfe-ingress-controller 兼容。

• Route CRD

配置示例

apiVersion: route.openshift.io/v1
kind: Route
metadate:
  name: app
  namespace: default
spec:
  alternateBackends:
  - kind: Service
    name: app1
    weight: 20
  host: example.com
  path: /
  port:
    targetPort: 8080
  to:
    kind: Service
    name: app
    weight: 80
  wildcardPolicy: None

Hi community!

I just found that the Deployment bfe-ingress-controller in the charts has both list and get verbs for the secrets resource (rbac.yaml). However, after reading the source code of ingress-bfe, I didn't find any Kubernetes API usages that require list secrets permissions. If a malicious user gets the service account token, they can list all the names of the secrets, and with the name, they can get the details of all the secrets objects (since this is declared in a ClusterRole). Therefore, for security reasons, I recommend removing this unnecessary permission.

`
ok github.com/bfenetworks/ingress-bfe/cmd/bfe_ingress_controller 0.065s coverage: 3.4% of statements
ok github.com/bfenetworks/ingress-bfe/internal/bfe_ingress 0.065s coverage: 3.6% of statements
--- FAIL: TestBfeRouteConfigBuilder_Build_CasePriorityHost4 (0.01s)
route_builder_test.go:847: routeTableFile: {
"Version": "0",
"BasicRule": {
".bar.host.com": [
{
"Hostname": [
".bar.host.com"
],
"Path": [
"/foo/"
],
"ClusterName": "production_service5"
}
],
".baz.host.com": [
{
"Hostname": [
".baz.host.com"
],
"Path": [
"/foo/"
],
"ClusterName": "production_service6"
}
],
".foo.host.com": [
{
"Hostname": [
".foo.host.com"
],
"Path": [
"/foo/"
],
"ClusterName": "ADVANCED_MODE"
}
]
},
"ProductRule": {
".foo.host.com": [
{
"Cond": "req_host_suffix_in(".foo.host.com") \u0026\u0026 (req_path_in("/foo", false) || req_path_prefix_in("/foo/", false))",
"ClusterName": "production_service4"
}
],
"bar.foo.host.com": [
{
"Cond": "req_host_in("bar.foo.host.com") \u0026\u0026 (req_path_in("/foo", false) || req_path_prefix_in("/foo/", false)) \u0026\u0026 req_header_value_in("key", "value", false)",
"ClusterName": "production_service1"
}
],
"bar.host.com": [
{
"Cond": "req_host_in("bar.host.com") \u0026\u0026 (req_path_in("/foo", false) || req_path_prefix_in("/foo/", false)) \u0026\u0026 req_header_value_in("key", "value", false)",
"ClusterName": "production_service2"
}
],
"foo.bar.baz.host.com": [
{
"Cond": "req_host_in("foo.bar.baz.host.com") \u0026\u0026 (req_path_in("/foo", false) || req_path_prefix_in("/foo/", false)) \u0026\u0026 req_header_value_in("key", "value", false)",
"ClusterName": "production_service3"
}
]
}
}
route_builder_test.go:854:
Error Trace: route_builder_test.go:854
Error: Not equal:
expected: "ADVANCED_MODE"
actual : "production_service6"

    	            	Diff:
    	            	--- Expected
    	            	+++ Actual
    	            	@@ -1 +1 @@
    	            	-ADVANCED_MODE
    	            	+production_service6
    	Test:       	TestBfeRouteConfigBuilder_Build_CasePriorityHost4
route_builder_test.go:858: 
    	Error Trace:	route_builder_test.go:858
    	Error:      	Not equal: 
    	            	expected: 5
    	            	actual  : 4
    	Test:       	TestBfeRouteConfigBuilder_Build_CasePriorityHost4

panic: runtime error: index out of range [0] with length 0 [recovered]
panic: runtime error: index out of range [0] with length 0

goroutine 180 [running]:
testing.tRunner.func1.2({0x1a63f00, 0xc000254438})
/usr/local/go/src/testing/testing.go:1209 +0x36c
testing.tRunner.func1()
/usr/local/go/src/testing/testing.go:1212 +0x3b6
panic({0x1a63f00, 0xc000254438})
/usr/local/go/src/runtime/panic.go:1047 +0x266
github.com/bfenetworks/ingress-bfe/internal/builder.TestBfeRouteConfigBuilder_Build_CasePriorityHost4(0xc000682d00)
/root/bfe-ingress-controller/ingress-bfe-main/internal/builder/route_builder_test.go:863 +0x985
testing.tRunner(0xc000682d00, 0x1bb1268)
/usr/local/go/src/testing/testing.go:1259 +0x230
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1306 +0x727
FAIL github.com/bfenetworks/ingress-bfe/internal/builder 0.186s
? github.com/bfenetworks/ingress-bfe/internal/kubernetes_client [no test files]
? github.com/bfenetworks/ingress-bfe/internal/utils [no test files]
FAIL
make: *** [test-case] Error 1
`

first commit for BFE ingress controller, enable BFE in Kubernetes by Ingress

背景

BFE 支持对流量进行Header重写，希望 bfe-ingress-controller 实现该功能。

配置方式

Ingress 资源内

• spec.rules 定义路由规则
• metadata.annotations 定义对符合路由规则的流量，修改 Header 的行为
  • 定义修改覆盖的流量范围
  • 定位修改 Header 的行为

参考格式：

metadata:
    annotations:
        bfe.ingress.kubernetes.io/rewrite-header.actions: '[{}]'
spec:
  rules:
  - ...

重写请求 Header

通过设置 bfe.ingress.kubernetes.io/rewrite-header.actions ，配置请求 Header 修改规则。
支持：

• REQ_HEADER_ADD：添加指定 Header
• REQ_HEADER_SET：修改指定 Header
• REQ_HEADER_DEL：删除指定 Header

添加

bfe.ingress.kubernetes.io/rewrite-header.actions: '[{"cmd": "REQ_HEADER_ADD", "params": ["key", "value"]}]'

修改

bfe.ingress.kubernetes.io/rewrite-header.actions: '[{"cmd": "REQ_HEADER_SET", "params": ["key", "value"]}]'

删除

bfe.ingress.kubernetes.io/rewrite-header.actions: '[{"cmd": "REQ_HEADER_DEL", "params": ["key"]}]'

重写响应 Header

通过设置 bfe.ingress.kubernetes.io/rewrite-header.actions ，配置响应 Header 修改规则。
支持：

• RSP_HEADER_ADD：添加指定 Header
• RSP_HEADER_SET：修改指定 Header
• RSP_HEADER_DEL：删除指定 Header

添加

bfe.ingress.kubernetes.io/rewrite-header.actions: '[{"cmd": "RSP_HEADER_ADD", "params": ["key", "value"]}]'

修改

bfe.ingress.kubernetes.io/rewrite-header.actions: '[{"cmd": "RSP_HEADER_SET", "params": ["key", "value"]}]'

删除

bfe.ingress.kubernetes.io/rewrite-header.actions: '[{"cmd": "RSP_HEADER_DEL", "params": ["key"]}]'

https://www.bfe-networks.net/en_us/modules/mod_header/mod_header/

1. 使用Annotation方式支持URL重写配置
2. 添加URL重写功能的相关e2e测试

fix #54

Signed-off-by: Tao Meng [email protected]

At present, it is very convenient to deploy ingress-nginx using helm. Really, it is very convenient to use helm. Therefore, we look forward to providing ingress-bfe chart package.

What is the relationship between Ingress-bfe and the three components just open-sourced?