salrashid123 / envoy_ext_proc

Envoy External Processing Filter

Envoy External Processing Filter

A really basic implementation of envoy External Processing Filter. This capability allows you to define an external gRPC server which can selectively process headers and payload/body of requests (see External Processing Filter PRD. Basically, your own unrestricted filter.

          ext_proc 
             ^
             |
client ->  envoy -> upstream

NOTE, this filter is really early and has a lot of features to implement!

• Source: ext_proc.cc

---

All we will demonstrate in this repo is the most basic functionality: manipulate headers and body-content on the request/response. I know, there are countless other ways to do this with envoy but just as a demonstration of writing the external gRPC server that this functionality uses. If interested, pls read on:

The scenario is like this

A) Manipulate outbound headers and body

          ext_proc   (delete specific header from client to upstream; append body content sent to upstream)
             ^
             |
client ->  envoy -> upstream

B) Manipulate response headers and body

          ext_proc   (delete specific header from upstream to client; append body content sent to client)
             ^
             |
client <-  envoy <- upstream

---

Specifically for (A), if a header key= "user" is sent by the client AND if the request is a POST, the external processing filter will

• redact that header
• append 'foo' to the body and send that to httpbin.org/post

If A is triggered, the couple of headers from httpbin are removed and the content type is set to text. Finally, the response body has the text qux appended to it.

If the request type is GET or if the header 'user' is not present no modifications are made

---

First this code was just committed in PR 14385 so we will need envoy from the dev branch that was just committed

docker cp `docker create envoyproxy/envoy:v1.29.2`:/usr/local/bin/envoy /tmp/

Now start the external gRPC server

go run grpc_server.go

This will start the gRPC server which will receive the requests from envoy.

I'm not sure if i've implemented the server correctly but the following does redact the user header from upstream

As more features are implemented, you can handle new processing request types.

Now start envoy

./envoy -c server.yaml -l debug

Note, the external processing filter is by default configured to ONLY ask for the inbound request headers. What we're going to do in code is first check if the header contains the specific value we're interested in (i.,e header has a 'user' in it), if so, then we will ask for the request body, which will ask for the response headers which inturn will override and ask for the response body

          http_filters:
          - name: envoy.filters.http.ext_proc
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3alpha.ExternalProcessor
              failure_mode_allow: false
              async_mode: false              
              request_attributes:
              - user
              response_attributes:
              - server
              processing_mode:
                request_header_mode: "SEND"
                response_header_mode: "SKIP"
                request_body_mode: "NONE"
                response_body_mode: "NONE"
                request_trailer_mode: "SKIP"
                response_trailer_mode: "SKIP"
              grpc_service:
                envoy_grpc:                  
                  cluster_name: ext_proc_cluster

Send in some requests

note, in each of tests, the upstream is httpbin.org which will just echo back the inbound request to the caller and display the headers and body it got back as the json response (i.,e the json response below is what httpbin saw)

1. GET Request

In this case, we should not expect any modifications to take place.

$ curl -v -H "host: http.domain.com"  --resolve  http.domain.com:8080:127.0.0.1  http://http.domain.com:8080/get

> GET /get HTTP/1.1
> Host: http.domain.com
> User-Agent: curl/7.74.0
> Accept: */*


< HTTP/1.1 200 OK
< date: Wed, 31 Mar 2021 16:59:50 GMT
< content-type: application/json
< content-length: 311
< server: envoy
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 31

{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "http.domain.com", 
    "User-Agent": "curl/7.74.0", 
    "X-Amzn-Trace-Id": "Root=1-6064aa86-1e8e99652e2c7ee003a2750f", 
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000"
  }, 
  "origin": "108.51.98.171", 
  "url": "https://http.domain.com/get"
}

2. GET Request with user header

Here we're also not expecting changes

$ curl -v -H "host: http.domain.com"  --resolve  http.domain.com:8080:127.0.0.1  -H "user: sal" http://http.domain.com:8080/get

> GET /get HTTP/1.1
> Host: http.domain.com
> User-Agent: curl/7.74.0
> Accept: */*
> user: sal

< HTTP/1.1 200 OK
< date: Wed, 31 Mar 2021 17:00:37 GMT
< content-type: application/json
< content-length: 331
< server: envoy
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 24

{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "http.domain.com", 
    "User": "sal", 
    "User-Agent": "curl/7.74.0", 
    "X-Amzn-Trace-Id": "Root=1-6064aab5-1c5e1204091c69600d45b6ba", 
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000"
  }, 
  "origin": "108.51.98.171", 
  "url": "https://http.domain.com/get"
}

3. POST Request with user header

In this case,we send in a POST but no user header so also no difference

$ curl -v -H "host: http.domain.com" -H "content-type: text/plain" --resolve  http.domain.com:8080:127.0.0.1  -d 'foo' http://http.domain.com:8080/post

> POST /post HTTP/1.1
> Host: http.domain.com
> User-Agent: curl/7.74.0
> Accept: */*
> content-type: text/plain
> Content-Length: 3

< HTTP/1.1 200 OK
< date: Wed, 31 Mar 2021 17:03:06 GMT
< content-type: application/json
< content-length: 441
< server: envoy
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 8

{
  "args": {}, 
  "data": "foo", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "3", 
    "Content-Type": "text/plain", 
    "Host": "http.domain.com", 
    "User-Agent": "curl/7.74.0", 
    "X-Amzn-Trace-Id": "Root=1-6064ab4a-6df7e0d437a8ad2637c35fce", 
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000"
  }, 
  "json": null, 
  "origin": "108.51.98.171", 
  "url": "https://http.domain.com/post"
}

4. Finally,

We send a post request and the 'user' header below

What happens is that the external processing filter will

1. In *pb.ProcessingRequest_RequestHeaders,

• detect and remove user header
• instruct further processing of the request body

2. In *pb.ProcessingRequest_RequestBody,

• append bar to the inbound request body
• update the content-length header (since thats just what we did here by appending)

3. In *pb.ProcessingRequest_ResponseHeaders,

• remove the following headers sent by httpbin: "access-control-allow-origin", "access-control-allow-credentials"
• update the content-length value by addin in the byte-length contained in the data we're going to later add to the body (i.e, add by #bytes in qux)

4. In *pb.ProcessingRequest_ResponseBody

• Append qux to the response body sent by httpbin

$ curl -v -H "host: http.domain.com" -H "content-type: text/plain" \
  --resolve  http.domain.com:8080:127.0.0.1 \
   -H "user: sal" -d 'foo' http://http.domain.com:8080/post

> POST /post HTTP/1.1
> Host: http.domain.com
> User-Agent: curl/7.74.0
> Accept: */*
> content-type: text/plain
> user: sal
> Content-Length: 3

< HTTP/1.1 200 OK
< date: Wed, 31 Mar 2021 17:05:01 GMT
< server: envoy
< x-envoy-upstream-service-time: 24
< content-type: text/plain
< content-length: 453

{
  "args": {}, 
  "data": "foo baaar ", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "10", 
    "Content-Type": "text/plain", 
    "Host": "http.domain.com", 
    "User-Agent": "curl/7.74.0", 
    "X-Amzn-Trace-Id": "Root=1-6064abbd-1a54289105d3cec56cec7c9c", 
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000"
  }, 
  "json": null, 
  "origin": "108.51.98.171", 
  "url": "https://http.domain.com/post"
}

 qux

---

Thats it, i'll be adding on more features as they become available to this repo.

---

Building docker image

docker build --tag extproc .
docker run -it -p 18080:18080 extproc

Other links

Other reference envoy samples

• Envoy WASM and LUA filters for Certificate Bound Tokens

• Envoy mTLS

• Envoy control plane "hello world"

• Envoy for Google Cloud Identity Aware Proxy

• Envoy External Authorization server (envoy.ext_authz) with OPA HelloWorld

• Envoy RBAC

• Envoy Global rate limiting helloworld

• Envoy EDS "hello world"

• Envoy WASM with external gRPC server

• Redis AUTH and mTLS with Envoy

• gRPC per method observability with envoy, Istio, OpenCensus and GKE

• gRPC XDS

• gRPC ALTS

envoy_ext_proc's Issues

Add Dockerfile

Currently we don't have a dockerfile.
Having a docker file would help when experimenting in local Istio clusters for example.

wdyt? I can raise a PR if you think this will be help.

Stream is not reaching EOF

I just tested the project, i couldn't see the context is getting Done and EOF condition is satisfied. Do you face similar issue?

How's Progress

I've been following but noticed there hasn't been activity for a couple of months.

Do you have thoughts around where development is and is there any structured way of understanding remaining development required?

I've been watching this closely and think it's great!