In order to learn and understand gRPC, I decided to
implement my wildly successful FaaS API as a
gRPC service. The service is a simple unary remote procedure call with one
method to get some number of fucks, which are returned by the server. The
service definition is in the api
directory and
reproduced below.
syntax = "proto3";
package api;
service NgFaaS {
rpc GetFucks(FuckNumber) returns (FuckBox) {}
}
message FuckNumber {
int64 number = 1;
}
message FuckBox {
repeated string contents = 1;
}
This protocol defines a service, NgFaaS
which has one method, GetFucks
and
takes a FuckNumber
message as a request, returning a FuckBox
message as a
response. The FuckNumber
message contains one 64 bit integer, while the
FuckBox
message contains an array of strings.
Although there are many examples of writing simple unsecured servers and clients, I wanted to learn how to build a real service, with error responses, TLS encryption, and authentication. I did that in a series of steps using the single service defined above.
I write the server in Go and have examples of clients written in both python and Go. I have a total of five samples. They are:
-
Clients and server with no encryption or authentication
- python client (python_client_one)
- Go client (client_one)
- Go server (server_one)
This server runs without any encryption, but does include error responses for requesting a negative amount of fucks, or for requesting too many fucks (which I set as more than 500 fucks). Unlike the RESTful FaaS I wrote previously, there is no need to validate that I receive an integer as that is part of the protocol definition, and will be validated by the generated server code.
-
Clients with TLS, and a server behind a Cloud Run proxy
- python client (python_client_two)
- Go client (client_two)
- Go server (server_one) - Note this is unchanged from the previous sample.
This is a small detour, where I take the same server defined above and deploy it using Cloud Run. Cloud Run can run unary gRPC services, and will require an encrypted connection, acting as a TLS terminating proxy. The certificate is automatically generated by Cloud Run and signed by Let's Encrypt. The only changes I have to make is to require the python and Go clients to set up encrypted connections. The python client implicitly chooses default root certificates with the
grpc.ssl_channel_credentials()
call, while for the Go client the system default certificates are explicitly selected with thex509.SystemCertPool()
call. -
Clients and server use TLS with a private CA
- python client (python_client_three)
- Go client (client_three)
- Go server (server_two)
Typically, you might expect to use a private CA for an internal API. These clients and server use a common root certificate authority (generated by minica) and the server uses a signed certificate for "localhost". The clients now explicitly add only the common root certificate, and will not recognize any endpoints that are signed by any other root certificates.
-
Client and server use TLS with a private CA, and clients authenticate with a certificate during TLS negotiation
- python client (python_client_four)
- Go client (client_four)
- Go server (server_three)
This is the first go at client authorization using mutual TLS. The server requires that the client identify itself with a signed certificate. I let the clients use a certificate generated by minica for 127.0.0.1, while the server continues to use the localhost certificate generated above. Both the certificates are signed by the same root certificate, so the client and server can both verify the signatures against that same root.
-
Client and server use TLS with a private CA, and clients authenticate with a token.
- python client (python_client_five)
- Go client (client_five)
- Go server (server_four)
In this final example, I send an authorization bearer token from the client with the value "HelloWorld". The server is now somewhat more sophisticated, with an interceptor that logs the connection from the peer and validates the bearer token. It then attaches a value to the context indicating if the client is authorized or not. Ultimately the handler can determine if it wants to respond or not. Typically one might use the context variable to store information about the user or the role of the client. Then the handler could use that information to authorize the scope of the user's request. In this server, I optionally allow the client to use a client certificate. If a client certificate is passed, the server will validate that certificate and will not let a client with an invalid certificate connect. The interceptor would also be useful for rate limiting and any other cross-cutting functions across the service.
I use Go modules in this code, which means I use a go.mod
file. Given this
file, when you build the client and server, the google.golang.org/grpc
library should automatically be installed. See the gRPC
page for more information about this.
To begin, install protocol buffers v3 from the github project release
page. You will then need to
install the protoc
plugin for Go using the command:
$ go get -u github.com/golang/protobuf/protoc-gen-go
Make sure the protoc-gen-go
binary is within your PATH
.
For the python side, set up a virtual environment with the command:
$ python -m venv venv
Enter the environment and install the gRPC libraries with the commands:
$ source venv/bin/activate
$ pip install grpcio
$ pip install grpcio-tools
In the root directory of the repository, you can generate the required Go code with the command:
$ protoc api/ngfucks.proto -I api/ --go_out=plugins=grpc:api
This will write the appropriate go file in the api
directory where it can be
loaded. For the python client, I found it easier to run the following command
multiple times from within each python client directory:
$ python -m grpc_tools.protoc -I ../api --python_out=. --grpc_python_out=. ../api/ngfucks.proto
This will generate the appropriate python files in each directory where they can be easily imported by the client software.
The server can be built with the command:
$ go build -o ngfaas_server server_one/main.go
The Go client can be built with the command:
$ go build -o ngfaas_client client_one/main.go
From the root directory, run the client using the command
$ ./ngfaas_server
Then in another window or shell you can run the clients. The clients take one
optional argument -n
followed by a number of fucks to get. By default they
will request 5 fucks.
To request 20 fucks with the Go client run
$ ./ngfaas_client -n 20
from the root directory of the repository.
To use the python client to request 80 fucks, from the root of the repository run the command:
$ python python_client_one/client.py -n 80
You can try very large or negative numbers too in order to see what an error response looks like.
I have put up a server at ngfaas.unnecessary.tech:443
running on Cloud Run.
The clients are automatically set up to query that server. Hopefully the
clients will find and load the appropriate client root certificates for your
system, which should include the Let's Encrypt root certificate. If you
experience any issues, the system certificates may not have loaded correctly.
For all the private certificate authority examples, the clients and servers
are hardcoded to look for a file called minica.pem
as the root certificate
in the current directory. The clients uses a certificate in
127.0.0.1/cert.pem
and a key in 127.0.0.1/key.pem
while the server uses a
certificate in localhost/cert.pem
and a key in localhost/key.pem
. All the
certificates should be signed by the root certificate.
This can easily be set up using the minica mini certificate authority, which is also available via homebrew on the Mac. Minica is a good certificate authority for testing TLS enabled services and clients, but in production I would recommend using something like HashiCorp Vault which can create short-lived certificates on the fly in a secure manner.
To set up the needed development certificates, run the following commands:
$ minica -domains localhost
$ minica -ip-addresses 127.0.0.1
This will create your root certificate and the server and client certificates.