aws-samples / aws-kms-xks-proxy

AWS KMS External Keystore (XKS) Proxy reference implementation

aws-kms-xks-proxy

This package provides a sample implementation of the AWS KMS External Keystore (XKS) Proxy API for reference by external customers against any Hardware Security Module (HSM) that supports PKCS#11 v2.40.

The current implementation is written in Rust and is known to work with the following types of HSM:

• LunaSA 7.4.0 (via Thales eLab)
• Luna 7 Network HSM (hardware device)
• Entrust nShield XC HSM
• SoftHSMv2
• AWS CloudHSM

Quick Start

Build xks-proxy into a Docker image to serve http traffic, with SoftHSMv2 pre-installed and pre-generated with a single AES key named foo:

docker build -t xks-proxy:latest .

Run xks-proxy as a Docker container:

docker run --rm --name xks-proxy -d -p 0.0.0.0:80:80 xks-proxy:latest

Verify the server is up and running:

curl localhost/ping

You should see the response:

pong from xks-proxy v3.1.2-unknown

which indicates the server is now up and running. You can monitor the logs with:

docker logs -f xks-proxy

Using another terminal, download a curl based client and cd to it:

git clone https://github.com/aws-samples/aws-kms-xksproxy-test-client.git
cd aws-kms-xksproxy-test-client

(Note you'd need to have curl 7.75+ installed for the following.)

Run tests locally against the xks-proxy:

export XKS_PROXY_HOST="localhost"
export URI_PREFIX="example/uri/path/prefix"
# Access key ID must have between 20 and 30 characters. Valid characters are uppercase A-Z and 2-7
export SIGV4_ACCESS_KEY_ID="BETWEEN2TENAND3TENCHARACTERS"
# Secret access key must have between 43 and 64 characters. Valid characters are a-z, A-Z, 0-9, /, +, and =
export SIGV4_SECRET_ACCESS_KEY="PleaseReplaceThisWithSomeSecretOfLength43To64"
export KEY_ID="foo"
export SCHEME=
./test-xks-proxy

You should see All 35 tests PASSED at the end.

To stop the Docker container running xks-proxy:

docker container stop xks-proxy

More information about the use of docker can be found at docker/README.md.

Details

The XKS Proxy API spec assumes you have already set up the vendor specific HSM with the necessary cryptographic keys created. Once that is done, set up a configuration file at /var/local/xks-proxy/.secret/settings.toml. A number of sample configuration files for different types of HSM's can be found under the xks-axum/configuration folder.

Then, to run the XKS Proxy server for debugging purposes:

# Under the xks-axum directory
XKS_PROXY_SETTINGS_TOML=/var/local/xks-proxy/.secret/settings.toml cargo run

Unit tests

To run the unit tests, type

# Under the xks-axum directory
cargo test

Generate RPM for Centos Linux

To generate the RPM for xks-proxy for installation on a Centos Linux x86_64 platform, type:

# Under the root directory
make

Install via RPM

To install the generated RPM on a Centos Linux x86_64 platform:

# For example
sudo yum install -y xks-proxy-3.1.2-0.el7.x86_64.rpm

Configuration

Specify the configuration file path, such as /var/local/xks-proxy/.secret/settings.toml, by setting the environment variable XKS_PROXY_SETTINGS_TOML.

See the xks-axum/configuration folders for some sample configuraions for various HSM's such as Thales Luna HSM, Entrust nShield HSM, AWS CloudHSM and SoftHSMv2.

Manage xks-proxy as a systemd unit

Once the xks-proxy RPM has been installed, you can manage it as a systemd unit. For example

# Show the backing file
sudo systemctl cat xks-proxy

# Display the status
systemctl status xks-proxy

# The default systemd unit configuration for xks-proxy can be found at /etc/systemd/system/xks-proxy.service.
# To override the configuration:
sudo systemctl edit xks-proxy

# To start xks-proxy
sudo systemctl start xks-proxy

# To stop
sudo systemctl stop xks-proxy

# Display the system journal log
sudo journalctl -xef --unit xks-proxy

Clean logs via xks-proxy_cleanlogs.timer

If you choose to output logging of xks-proxy to a file (with is_file_writer_enabled = true in settings.toml), you can have the log files automatically removed perodically via a systemd timer.

# Show the backing file for the timer
sudo systemctl cat xks-proxy_cleanlogs.timer

# Display the status of the cleaning service driven by the timer
systemctl status xks-proxy_cleanlogs

# Display the status of the timer
systemctl status xks-proxy_cleanlogs.timer

# The default systemd unit configuration for the timer can be found at /etc/systemd/system/xks-proxy_cleanlogs.timer.
# To override the (default hourly timer) configuration:
sudo systemctl edit xks-proxy_cleanlogs.timer

# To start the timer
sudo systemctl start xks-proxy_cleanlogs.timer

# Display the system journal log of the cleaning service
sudo journalctl -xef --unit xks-proxy_cleanlogs

# Display the system journal log of the timer
sudo journalctl -xef --unit xks-proxy_cleanlogs.timer

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

aws-kms-xks-proxy's Issues

Provide docker files or rpm files to support installation for AWS CloudHSM and other HSM solutions

Security issue notifications

No potential security issues.

Problem:

The current Docker file supports only SoftHSM2 installation, when the sample settings files are provided for different platforms.
The RPM script is for CentOS.

Solution:

It will simplify the effort to use aws-kms-xks-proxy if Dockerfiles and rpm scripts were provided for other HSM, like AWS CloudHSM.
For example, OS -Amazon Linux 2023, HSM - AWS Cloud HSM

Out of scope:

Is there anything the solution will intentionally NOT address?

Timeout on many requests

Hello,

When we start generating thousands of KMS requests to XKS proxy, the following logs start appearing in xks-proxy.log and timeout message saying the request can't be executed:
2024-02-23T11:06:54.285096Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: 742: error shutting down IO: Transport endpoint is not connected (os error 107) 2024-02-23T11:06:54.285134Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: 742: error shutting down IO: Transport endpoint is not connected (os error 107) 2024-02-23T11:06:54.285168Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: 742: error shutting down IO: Transport endpoint is not connected (os error 107) 2024-02-23T11:06:54.285202Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: 742: error shutting down IO: Transport endpoint is not connected (os error 107) 2024-02-23T11:06:54.285233Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: 742: error shutting down IO: Transport endpoint is not connected (os error 107) 2024-02-23T11:06:54.285266Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: 742: error shutting down IO: Transport endpoint is not connected (os error 107)

We are running xks-proxy on a EC2 instance which is powerful enough and when we check the monitoring it's not loaded at all. When we load balance the requests behind a NLB and many xks-proxies behind it, this problem doesn't exist. Keep in mind we start many docker containers on the same EC2 instance and load balance between them, which proves the issue is not related with overload of the ec2 itself.

Do you have any ideas what could be the reason for that. Here's the configuration we use in settings.toml:
`[server]
ip = "0.0.0.0"
port = 8000
region = "eu-west-1"
service = "kms-xks-proxy"

[server.tcp_keepalive]
tcp_keepalive_secs = 60
tcp_keepalive_retries = 3
tcp_keepalive_interval_secs = 1

[tracing]
is_stdout_writer_enabled = true
is_file_writer_enabled = true
level = "DEBUG"
directory = "/var/local/xks-proxy/logs"
file_prefix = "xks-proxy.log"
rotation_kind = "HOURLY"

[security]
is_sigv4_auth_enabled = true
is_tls_enabled = false
is_mtls_enabled = false

[tls]
tls_cert_pem = "/var/local/xks-proxy/tls/server_cert.pem"
tls_key_pem = "/var/local/xks-proxy/tls/server_key.pem"

[[external_key_stores]]
uri_path_prefix = "/keys"
sigv4_access_key_id = "XXXXXXXXXXXXXX"
sigv4_secret_access_key = "XXXXXXXXXXX"
xks_key_id_set = ["abc123", "xyz123"]

[pkcs11]
session_pool_max_size = 300
session_pool_timeout_milli = 45
session_eager_close = false
user_pin = ""
PKCS11_HSM_MODULE = "/usr/local/lib/libvault-pkcs11.so"
context_read_timeout_milli = 100

[limits]
max_plaintext_in_base64 = 8192
max_aad_in_base64 = 16384

[hsm_capabilities]
can_generate_iv = false
is_zero_iv_required = false`

Please let me know if you need more information.

Thank you!

Can this be used for production environments?

Security issue notifications

---

Problem:

We are currently evaluating options of storing our KMS keys outside of AWS. The only options for XKS seem to be to either use an external service partner, implement the XKS proxy ourselves or use this implementation.

Writing the XKS proxy ourselves would be out of scope for us so we stumbled upon this sample integration and were curious if this has been tested in a production system or if it is stable enough to use in one. Right now we would probably use the SoftHSMv2 module as this is just a PoC and we would later switch to a proper HSM.

Thank you for any input regarding this. It is very appreciated!

Solution:

---

Out of scope:

---