I think teller may be able to help a use case that I have in fetching hashicorp vault dynamic database secrets. The dynamic database secrets engine creates and returns a username and password for databases. This is useful for security reasons but can be cumbersome when using database clients such as psql or mysql.

I think teller might be helpful by fetching the dynamic username and password from vault, setting the correct environment variables, and runing the database client. However, the data structure returned by vault is slightly different than the kv data structure.

The vault kv data structure containing secrets returns a double data.data{} object. Whereas the vault database secrets engine returns data{}.

If teller supported the database endpoint the following .teller.yml could be used to set the correct environment variables and connect to postgres:

project: postgres_client
providers:
  hashicorp_vault:
    env:
      PGUSER:
        path: database/creds/my-role
        field: username
      PGPASSWORD:
        path: database/creds/my-role
        field: password

Then, one could connect to the database using psql:

teller run psql -h db.example.com -d postgres

I think support could be added to teller by adjusting the double data.data{} assumptions in hashicorp_vault.go#L60 and hashicorp_vault.go#L84.

Any thoughts on this? I can open a PR if their is interest.

Using M1 mac

✦ ❯ brew tap spectralops/tap
==> Tapping spectralops/tap
Cloning into '/opt/homebrew/Library/Taps/spectralops/homebrew-tap'...
remote: Enumerating objects: 53, done.
remote: Counting objects: 100% (53/53), done.
remote: Compressing objects: 100% (47/47), done.
remote: Total 53 (delta 24), reused 12 (delta 1), pack-reused 0
Receiving objects: 100% (53/53), 9.23 KiB | 1.84 MiB/s, done.
Resolving deltas: 100% (24/24), done.
Error: Invalid formula: /opt/homebrew/Library/Taps/spectralops/homebrew-tap/netz.rb
formulae require at least a URL
Error: Invalid formula: /opt/homebrew/Library/Taps/spectralops/homebrew-tap/senv.rb
formulae require at least a URL
Error: Cannot tap spectralops/tap: invalid syntax in tap!

Spec out a design suggestion for self-documenting providers.

Each provider should have a metadata part, where it will declare:

• name (already exists today)
• description
• support matrix (read/write,etc.)
• example usage / use case
• 'more info' (regarding the provider)

Notes:

• Implement this as serializable so it would be easy to dump from a running process
• Developer experience should be as effortless as possible, so explore Golang codegen if necessary, or, try to interpolate the support matrix from actual implementation (can we reflect/observe current implementations?)

New Provider Request

Details

• Name: secret files
• URL: https://docs.docker.com/engine/swarm/secrets/#how-docker-manages-secrets
• Modes: (read)
• GitHub projects:
• API Documentation: https://docs.docker.com/engine/swarm/secrets/#how-docker-manages-secrets

For security reasons secrets should be mounted as files inside docker container rather than passed as environment variable. However only few applications support those files. For easy solving this issue it would be great to have a tool which transforms secrets from files into environment variables, but inside the container.

Add New System Report + Report Management for only Staff By @WIZOX

The teller run function as documented does not pass flags to the child process. The following should return help for psql instead of printing help for teller:

# teller run psql --help                               
Usage: teller run <cmd> ...

Run a command

Arguments:
  <cmd> ...    Command to execute

Flags:
  -h, --help             Show context-sensitive help.
  -c, --config=STRING    Path to teller.yml

The work around is to use -- between run and the child process to exec.

teller run -- your-process arg1 arg2... --switch1 ...

This is expected behavior as referenced in alecthomas/kong#80

In order to protect against unauthorized automation of Teller on a developer machine, we want interactivity to be protected with another layer -- configurable in teller.yaml.

Challenges:

• It's recommended to have it by default ON, and only allow to suppress it via configuration (where people can do that in CI environments, where we assume people don't get physical access to). However changing this semantics is going to break a LOT of existing integrations. We need some smart backward-compat mode.

Implementation:

Lean version - touch ID only on mac:

• https://github.com/mritd/touchid

Full version:

However, there probably is some ubikey/other methods of auth available such that a user can:

1. Identify using their hard-access credentials, or OS credentials
2. Or, by way of standard 2FA method (maybe through one of Google's own libs)

Thanks @eladbash for the idea

tlr is absolutely amazing!
It would be awesome to be able to use teller with 1Password vaults.
There is 1Password CLI: https://support.1password.com/command-line-getting-started/
WDYT?

Hello again,

Teller already supports:

• Put (as create+update; both for single and multiple values)
• Get (get/list) for the providers
• copying (syncing) all secrets from one provider to another one

Did you also think about supporting Delete? It could be a simple teller delete command. What I'm thinking is extending the Provider interface:

type Provider interface {
	// (...)

        Delete(p KeyPath) error
        DeleteMapping(p KeyPath) // probably this would be also useful
}

In a result, Teller would support the full lifecycle of secret management, which could be extremely useful.

We at Capact would be very interested in this feature 🙂 And, we would love to contribute - Initially, in next few days, I could prepare such Delete functionality for dotenv and AWS Secrets Manager providers. At some point, we would also need full lifecycle support for Vault, which means we can also take care of the implementation 🙂

Cheers!

Add New System Report + Report Management for only Staff By @WIZOX

We need to add an application logger into our code for:

Why?

1. Better debugging visibility
2. Gives the user more verbosity on the commands
3. Allow to attache log when users open an issue

How:

Implement logger and cover the essential places with the log message

Note

We need to keep the STDOUT cleanest like we have today. The log will throw to the screen only in errors or when the user asks for with new flag --log-level to any Teller commands.

I would like for a provider to be developed to retrieve and add secrets for CyberArk Conjur Secrets Manager.

You can use the open source version here: https://conjur.org
Or learn more about the Enterprise edition here: https://cyberark.com

I have not found any where I can set environment via cli flag, but I can use seperate files.
How are you using this today?
Or what is your best practise suggstion on this problem?

Just came across this tool, super great. Been in devops for a few years and this definitely solves a need. Love the support for multiple providers within a single invocation.

I'm personally interested in gopass (#11) for some local only api keys. While I fully appreciate first class integration with providers, I'm wondering if #3 #10 #11 (and other future requests) could all be achieved if there was a generic shell provider. Using that provider would just invoke that shell script with a known STDIN api, ex [command] [path] [options].

Could we get azure key vault integration?
https://docs.microsoft.com/en-us/rest/api/keyvault/getsecret/getsecret

in https://tlr.dev/#install it says to install teller like brew install teller but that fails

❯ brew install teller
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> Updated Formulae
Updated 9 formulae.
==> Updated Casks
Updated 2 casks.

==> Searching for similarly named formulae...
Error: No similarly named formulae found.
Error: No available formula or cask with the name "teller".
==> Searching for a previously deleted formula (in the last month)...
Error: No previously deleted formula found.
==> Searching taps on GitHub...
Error: No formulae found in taps.

I guess you meant brew install spectralops/tap/teller or brew tap spectralops/tap && brew install teller like in the readme file.

Retrieve secrets from different vaults/stores and inject them into Kubernete's secrets.

Hey there 👋

I think it would be really excellent to support extending or merging teller config files. As we've begun to use teller more and more, we've found that often we need to have a teller config file for different environment. This would be similar to how chamber works with chaining services. The reason for this is sometimes in deployment context, our environment configuration will come from different providers.

An example of this would be a DATABASE_URL. If I am deploying my application on heroku, and using their database extensions, this variable will be accessible from the heroku provider, and thats where the app should consume it from. In other deployment/development contexts, I would want to access this variable from another provider. (e.g the AWS SSM param store).

For a situation like this, I end up with config files that look like this:

project: hello-world-production
providers:
  aws_ssm:
    env:
      SECRET_KEY:
        path: /{{project}}/{{environment}}/secret_key
        decrypt: true
  heroku:
    env:
      DATABASE_URL:
        path: hello-world-{{environment}}

and:

project: hello-world-production
providers:
  aws_ssm:
    env:
      SECRET_KEY:
        path: /{{project}}/{{environment}}/secret_key
        decrypt: true
      DATABASE_URL:
        path: /{{project}}/{{environment}}/database_url
        decrypt: true

The problem with this is the duplication of env declarations across files. I need to define SECRET_KEY in both files. It would be amazing we could avoid this duplication by merging configs in some way. An example of this could look like

Core configuration shared across environments:

// .teller.yml
project: hello-world-production
providers:
  aws_ssm:
    env:
      SECRET_KEY:
        path: /{{project}}/{{environment}}/secret_key
        decrypt: true

and production specific file:

// .teller.deploy.yml
providers:
  heroku:
    env:
      DATABASE_URL:
        path: /{{project}}/{{environment}}/database_url
        decrypt: true

Run teller:

teller run -c .teller.yml -c .teller.deploy.yml -- my command

This would merge the above files and inject both the SECRET_KEY and DATABASE_URL environment variables.

Hopefully that explains the issue well! Happy to answer any questions about this! Thanks.

See #65

Use case, if you have a frontend and a backend project and the frontend one has the backend url/port as env var and the backend has an env var vor its port also both would need a teller file and the teller file needs to be in the project folder.
Instead, an option for having a central teller file for multiple possibly overlapping project that don't have to be checked in at all and can reside in the home directory would be nice.

New Provider Request

Add bitwarden provider

Details

• Name: bitwarden
• URL: https://bitwarden.com/
• Modes: (read/write/read+write)
• GitHub projects:
• API Documentation: https://bitwarden.com/help/public-api/, https://bitwarden.com/help/api/

New Provider Request

Manage secrets from Mac keychain

Details

• URL: https://support.apple.com/en-il/guide/mac-help/mchlf375f392/mac
• Modes: read+write
• GitHub projects: https://github.com/keybase/go-keychain

New Provider Request

Add enpass provider

Details

• Name: enpass
• URL: https://www.enpass.io/
• Modes: read+write
• GitHub projects: https://github.com/hazcod/enpass-cli

Hi, this looks like an interesting project and our team is investigating whether it's a good fit for our workflow. We're wondering how you would suggest giving developers access to change secrets though, since this tool seems to only read from the secret stores. When using it, do you give developers direct access to the secret store to update values? Thanks!

Add New System Report + Report Management for only Staff By @WIZOX

Hello 👋 !

I was having a look though the teller roadmap and see that there are plans for adding "write only" provider support. I think this is an excellent Idea and would be something would be really useful. In particular for our team, adding support for cloudflare workers secrets would be extremely useful for our workflows. I would be happy to contribute a PR for this.

Has there been any work put into designing/implementing this feature yet?

Cheers

Hi,

Very cool project! I would be interested to use it as Go API for accessing different secret storage backends (Delegated storage feature for https://github.com/capactio/capact open source project), as you prepared great abstractions.

Anyway, while playing around with Teller Go API, I noticed the following behavior for dotenv Provider. When doing Put, it returns an error when a given file doesn't exist.

I would love to prepare a pull request to fix such issue for dotenv provider. In fact, I have the code ready, but I want to confirm first the expected behavior.

Expected behavior

I would expect that the Put method is doing upsert (create or update) operation, like PUT in REST API. By reading the code, this is how I believe the vault, heroku and etcd providers work.

Minimal example to reproduce

package main

import (
	tellerpkg "github.com/spectralops/teller/pkg"
	"github.com/spectralops/teller/pkg/core"
	"log"
)

func main() {
	providers := tellerpkg.BuiltinProviders{}
	provider, err := providers.GetProvider("dotenv")
	if err != nil {
		panic(err)
	}

	key := core.KeyPath{
		Path:  "/tmp/foo.env",
		Field: "data",
	}

	val := `{"key": "true"}`
	err = provider.Put(key, val)
	if err != nil {
		panic(err) // it will panic here
	}

	// Get
	entry, err := provider.Get(key)
	if err != nil {
		panic(err)
	}

	log.Printf("%+v\n", entry)
}

Result:

panic: open /tmp/foo.env: no such file or directory

Hi guys! I really like this tool, I just have one concern and that is that it would be great with multiple formats for teller env
Some flag to be able to set what shall be outputed.

F.ex, google cloud functions have a way to set envs via file, but they expect a yml file.
You can set via an array, but again, this format only outputs as an .env file.

Has there been any thoughts around this or exist some undocumented code?

Some documentation, search for --env-vars-file in https://cloud.google.com/sdk/gcloud/reference/functions/deploy

New Provider Request

Details

• Name: Ansible Vault
• URL: https://docs.ansible.com/ansible/latest/user_guide/vault.html
• Modes: read+write
• GitHub projects: https://github.com/ansible/ansible
• API Documentation: https://docs.ansible.com/ansible/latest/user_guide/vault.html

It would be great if Ansible Vault could be supported. That way it would be easy to re-use Ansible Vault secrets as environment variables. My example use case would be a setup where Ansible talks to an API and thus that API key is in Ansible Vault, and Terraform needs to talk to this API as well.

Time for shaping the content. Build a docs website based on the following TOC:

• Quickstart
• Providers (let's also do something smart. Each provider should also specify declaratively what it supports, so when we run something like "teller providers --info" we get a JSON that we can bake into the docs). In other words - we want the providers to be self-documenting so that:

1. maintainers can contribute very easily
2. syncing the docs provider section is effortless or automatic

• Usage (list all use cases)
• FAQ
• Design principles
• Contributing (developer's guide)

Taking a look into docs, there is an secret obfuscation possibility

1. Does it do secret obfuscation based on only known secrets?
2. Can it be used for any data like history, where someone might accidentally paste a password or etc in clear text?
3. teller --in dirty.csv --out clean.csv
   On MacOS

➜ teller-test teller version Teller 1.0.0 Revision 684b421ad8f96acb3de9bcb0f75339d5b4912030, date: 2021-04-14T12:43:07Z ➜ teller-test teller --in file-t.txt --out clean.txt teller: error: unknown flag --in

SO you can give it a config, that give it some defaults for the new config it will create ?

I'll preface this by saying I'm pretty new to Vault in general, and odds are I'm just doing something incorrectly. For some reason Teller is unable to read from my Vault cluster. I've done several different combinations of secret locations to try and get it to read, for example:

• secret/data/{{stage}}/test/teller
• secret/data/{{stage}}/test
• secrets/{{stage}}/test/teller
• secrets/{{stage}}/test
• {{stage}}/test/teller
• {{stage}}/test

Regardless of what I use I get back the error that data not found at <insert_vault_string>. However, I am able to read it just fine by running the vault kv get developement/test/teller directive. Any insight into what I may be doing wrong here?

Teller Config:

project: teller-test
confirm: Are you sure you want to run on {{stage}}?
opts:
  region: env:AWS_REGION
  stage: development
providers:
  hashicorp_vault:
    env_sync:
      path: 'secret/data/{{stage}}/test/teller'
    env:
      TELLER_TEST_SECRET:
        path: '{{stage}}/test'
        field: teller

Screenshot of Vault store:

Screenshot of terminal call:

Hi. I've recently discovered your project and fascinated with capabilities it provides. However, there is some things that are not completely clear for me. As you might guessed from a title, I am figuring out how does Teller authenticates against Hashicorp Vault.
Readme says:

if for example you connect to your organization's Hashicorp Vault, we assume you already have a secure way to do that, which is "blessed" by the organization.

Does this mean that Teller inherits all auth methods Vault provides, or it is up to me how to acquire and feed Teller with VAULT_TOKEN? I am specifically interested in AWS Auth method, since Teller might perfectly fit to my ECS runtime as secret injection tool.

The repository currently associates with topics reflecting what is support. Given the recent support for CyberArk Conjur, it is requested that the topics are also added onto the repo to reflect the support that was just contributed.

Thanks!

It would be great if it would integrate with an existing gopass store

Would be neat to support passwords and other secrets stored in password managers via this tool, such as https://github.com/lastpass/lastpass-cli .

New Provider Request

Add KeePass provider

Details

• Name: KeePass
• URL: https://keepass.info/
• Modes: (read/write/read+write)
• GitHub projects: https://github.com/tobischo/gokeepasslib, https://github.com/tobischo/gokeepasslib/tree/master/examples

The README.md is missing the default parameter variable for teller templates rendered with teller template and .Teller.EnvByKey

Error: cannot render template 'connet.tmpl': template: t:1:21: executing "t" at <.Teller.EnvByKey>: wrong number of args for EnvByKey: want 2 got 1

Hello, I stumbled upon this project while looking for a way to sync my secrets from AWS Secrets Manager to my application server. Originally we are using dotenv (.env) with nodejs to set the secrets for the app. We're looking to move to AWS Secrets Manger to host the secrets and then import them into the application server into a dotenv file. How can I achieve this with teller?

Not to water your wine
This is a 'Teller' in Germany 🍽️

Default value for opts

I use this alot

  opts:
  #region: env:AWS_REGION    # you can get env variables with the 'env:' prefix
  prefix: env:PREFIX

But it would be great with a way to say that something was default?

dotenv not required

I also use this alot

dotenv:
    env_sync:
      path: ./.env.local

It would be great to have an option to mark this as not required meaning that it does not crash when there is not file found. Because this particular example is only used for local override of config