netflix / weep

The ConsoleMe CLI utility

weep

Weep is a CLI utility for retreiving AWS credentials from ConsoleMe. Weep can run a local instance metadata service proxy, or export credentials as environment variables for your AWS needs.

Documentation

This README contains developer documentation. Weep user documentation can be found on GitBook.

Configuration

Weep can be compiled with an embedded configuration (See the Building section below), or it can get its configuration from a YAML-formatted file. We've included an example config file in example-config.yaml.

Weep searches for a configuration in the following locations:

• embedded configuration (see below)
• /etc/weep/weep.yaml
• ~/.weep/weep.yaml
• ./weep.yaml

Multiple configurations in these locations will be merged in the order listed above (e.g. entries in ./weep.yaml will take precedence over ~/.weep/weep.yaml.

You can also specify a config file as a CLI arg. This configuration will be used exclusively and will not be merged with other configurations:

weep --config somethingdifferent.yaml list

Weep supports authenticating to ConsoleMe in either a standalone challenge mode (ConsoleMe will authenticate the user according to its settings), or mutual TLS (ConsoleMe has to be configured to accept mutual TLS).

In challenge mode, Weep will prompt the user for their username the first time they authenticate, and then attempt to derive their username from their valid/expired jwt on subsequent attempts. You can also specify the desired username in weep's configuration under the challenge_settings.user setting as seen in example-config.yaml.

Pre-Commit Setup

Weep uses pre-commit to run unit tests and Go linting. Pre-commit documentation can be found on pre-commit

Installation

You can install pre-commit using the following steps:

Using pip:

pip install pre-commit

Using homebrew:

brew install pre-commit

Using Conda:

conda install -c conda-forge pre-commit

Validate your installation with the following:

$ pre-commit --version
pre-commit 2.9.3

Configuration

Set up the git hook scripts to run automatically with git commit

$ pre-commit install
pre-commit installed at .git/hooks/pre-commit

Building

In most cases, weep can be built by running the make command in the repository root. make release (requires upx) will build and compress the binary for distribution.

Embedded configuration

weep binaries can be shipped with an embedded configuration to allow shipping an "all-in-one" binary. An example of such a configuration is included in example-config.yaml.

To compile with an embedded config, set the EMBEDDED_CONFIG_FILE environment variable at build time. The value of this variable MUST be the absolute path of the configuration file relative to the root of the module:

EMBEDDED_CONFIG_FILE=/example-config.yaml make

Note that the embedded configuration can be overridden by a configuration file in the locations listed above.

Docker

Building and Running

make build-docker
docker run -v ~</optional/path/to/your/mtls/certs>:</optional/path/to/your/mtls/certs> --rm weep --meta-data --role <roleArn>

Releasing

Weep uses goreleaser in Github Actions for releases. Check their install docs if you would like to experiment with the release process locally.

To create a new release, create and push a tag using the release script (requires svu):

./scripts/release.sh

Goreleaser will automatically create a release on the Releases page.

Generating docs

Weep has a built-in command to generate command documentation (in the docs/ directory):

weep docs

weep's Issues

IMDS reachability self-test

When Weep is started in IMDS emulation mode, perform a reachability test on 169.254.169.254:80 to verify routing is configured properly. If the self-test fails, print a warning/error message with information on how to configure routing correctly.

Add support for weep to run as Windows service

There's some plumbing that needs to be done to let Weep run as a Windows service. More info can be found in this StackOverflow answer.

ccccccjellghkctdffkelljtunvvdlujrclciiegkjtu

Output configuration information when running `weep serve`

Each time the ECS container credential provider starts, print out the exact value that the AWS_CONTAINER_CREDENTIALS_FULL_URI environment variable should be set to. Optionally, implement a flag to set this environment variable when the local webserver starts up, and unset it when the server stops.

weep binary name in ps

This may be an artifact of how netflix packages weep for debian, but:

$ weep serve &
time="2021-10-02T14:09:04-07:00" level=info msg="starting weep on 127.0.0.1:9091"

$ ps axf | grep weep
 708631 pts/0    Sl     0:00      |   \_ /home/[username]/.weep-cache/weep-binaries/current serve

leads to this less than informative output:

$ sudo netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:9091          0.0.0.0:*               LISTEN      708631/current      

It would be nice if it set a more descriptive name.

README Instructions needed for configuring and running pre-commit

We use pre-commit to run unit tests and Go linting. We want instructions to help new users with getting pre-commit configured locally.

IMDS routing doesn't persist on MacOS Big Sur

Weep includes plist files that are meant to create persistent configurations to route 169.254.169.254:80 (the EC2 IMDS address) to 127.0.0.1:9091. The manual commands for these configurations seem to work correctly, but the plists do not make the configurations persist after a reboot.

(Theoretical) reproduction steps:

1. Follow the documentation steps (i.e. place the plist files in /Library/LaunchDaemons and reboot)
2. Attempt to curl http://169.254.169.254/latest/meta-data/iam/security-credentials

List output goes to stderr

The output of weep list is sent to stderr but should be sent to stdout.

Include account information in `weep list` output

Print the account name as part of the weep list output to help users pick the right account. This will require API changes in ConsoleMe.

Inline command invocation with weep credentials

Add a command to let users call a command using the credentials provided by weep.

Example usage:

# this command would get credentials for test_role then call `aws sts get-caller-identity` 
# with the credentials set as environment variables
weep cmd test_role aws sts get-caller-identity

# this command would get credentials for test_role, get assume role credentials for other_role,
# then call `aws sts get-caller-identity` with the credentials set as environment variables
weep cmd test_role --assume-role arn:aws:iam::123456789012:role/other_role aws sts get-caller-identity

Cache credentials to file

Weep should cache credentials to a database file so we can make fewer calls to ConsoleMe for new creds.

Automatically create ~/.aws directory

When running weep file role_name, or weep credential_process, Weep should create the ~/.aws directory if it doesn't already exist

ec2-user@Mac-mini ~ % weep file role_name Error: open /Users/ec2-user/.aws/credentials: no such file or directory Usage:...

weep incorrectly detects a shell

There is no -g option for the set command in Bash 5.1 on mac:

This is what worked for me when executing from the bash script:

cmd=$(weep export arn:aws:iam::123:/aaa/sss)
cmd=${cmd//"set -gx"/"export"}
cmd=${cmd//" \""/=\"}
eval $cmd

As you see, I changed the output to:

export NAME=VALUE

format. It would be great if we could get such output as well.

Weep isn't honoring SIGINT during challenge validation

When running a weep command with challenge mode configured, weep will not honor SIGINT while it is polling for the challenge status.

I also noticed that when I compiled Weep for a special domain, I provided an embedded configuration with my username. If I take out the username, compile, and use it, Weep asks me what username I want to use. However it doesn't seem to persist this in a local configuration

Powershell completion usage

We don't know how Powershell completion is supposed to work, so anyone with a Windows machine and a bit of patience can help us out by fixing the Powershell help text in cmd/completion.go.

Add `whoami` command

Add a weep whoami command that runs aws sts get-caller-identity as a convenience for folks who don't want to type all that nonsense. Potentially include additional information about the current principal.

weep export to use the profile name instead of ARN

Look at the following example:

aws --profile foo sts get-caller-identity
eval $(weep export arn:aws:iam::123:role/PowerUser)

It would be great if we could use the profile name for weep instead of the ARN. This information is stored in ~/.aws/config, it is easy to parse (configparser style).

The usage could look like:

weep export foo

or, better

weep export --profile foo

Let weep use existing credentials

Certain current and future weep commands could be useful apart from retrieving credentials from ConsoleMe. For example:

• the ecs_credential_provider could accept a profile name and serve credentials from the AWS Shared Credentials file, e.g. AWS_CONTAINER_CREDENTIALS_FULL_URI=http://localhost:9091/ecs/profile/test_profile
• the assume role functionality could be used with credentials in environment variables/shared credentials file, e.g. weep export --from-env -a arn:aws:iam::123456789012:role/other_role or weep export --from-profile test_profile -a arn:aws:iam::123456789012:role/other_role

The UX will need some ironing out to make it more intuitive, but hopefully this captures the spirit well enough.