square / keywhiz

A system for distributing and managing secrets

Deprecated

As of 9/18/23 this project is now deprecated and no longer maintained; we recommend using HashiCorp Vault as a more robust and actively supported alternative.

Keywhiz

Keywhiz is a system for distributing and managing secrets. For more information, see the website.

Our Protecting infrastructure secrets with Keywhiz blog post is worth reading, as it provides some useful context.

Develop

Keywhiz requires Java 11 and MySQL 5.7 or higher.

See CONTRIBUTING for details on submitting patches.

Build Keywhiz:

mvn install

Run Keywhiz:

java -jar server/target/keywhiz-server-*-shaded.jar [COMMAND] [OPTIONS]

Useful commands to get started are migrate, add-user and server. Use with --help for a list of all available commands. Use with [COMMAND] --help to get help on a particular command.

For example, to run Keywhiz with a mysql database in development mode:

SERVER_JAR="server/target/keywhiz-server-*-shaded.jar"
KEYWHIZ_CONFIG="server/target/classes/keywhiz-development.yaml"

# Initialize dev database
java -jar $SERVER_JAR migrate $KEYWHIZ_CONFIG

# Add an administrative user
java -jar $SERVER_JAR add-user $KEYWHIZ_CONFIG

# Run server
java -jar $SERVER_JAR server $KEYWHIZ_CONFIG

To connect to a running Keywhiz instance, you will need to use the CLI.

An example helper shell script that wraps the keywhiz-cli and sets some default parameters:

#!/bin/sh

# Set the path to a compiled, shaded keywhiz-cli JAR file
KEYWHIZ_CLI_JAR="/path/to/keywhiz-cli-shaded.jar"
KEYWHIZ_SERVER_URL="https://$(hostname):4444"

# Use these flags if you want to specify a non-standard CA trust store.
# Alternatively, in development and testing specify the --devTrustStore 
# flag to use the default truststore (DO NOT use this in production, as
# the truststore is checked into Keywhiz' code).
TRUSTSTORE="-Djavax.net.ssl.trustStore=/path/to/ca-bundle.jceks"
TRUSTTYPE="-Djavax.net.ssl.trustStoreType=JCEKS"

java "$TRUSTSTORE" "$TRUSTTYPE" -jar "$KEYWHIZ_CLI_JAR" -U "$KEYWHIZ_SERVER_URL" "$@"

Keywhiz uses jOOQ to talk to its database.

If you made changes to the database model and want to regenerate sources:

mvn install -pl model/ -Pgenerate-jooq-sources

We recommend IntelliJ IDEA for development.

IntelliJ IDEA

To enable auto-completion, code navigation, etc., open the keywhiz repository in IDEA, right click pom.xml in the repository root, and select "Add as Maven Project".

Clients & API

Square also maintains a Keywhiz client implementation called Keysync.

Docker

We ship a Dockerfile for building a Docker container for Keywhiz. Please see the Dockerfile for extra instructions.

License

Keywhiz is under the Apache 2.0 license. See the LICENSE file for details.

keywhiz's Issues

cli doesn't build on windows

My guess is that the entire project isn't compatible with Windows, but at least from the start, when I run mvn clean verify from the parent, I get an error in the cli module. Going in to that module specifically and running mvn clean verify produces the following output on my machine (Win7, jdk1.8.0_60+JCE):

C:\Users\Me\dev\java\keywhiz\cli [master ≡]
λ mvn clean verify
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Keywhiz CLI 0.7.9-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ keywhiz-cli ---
[INFO] Deleting C:\Users\Me\dev\java\keywhiz\cli\target
[INFO]
[INFO] --- maven-enforcer-plugin:1.2:enforce (enforce-unlimited-crypto-policy) @ keywhiz-cli ---
[INFO]
[INFO] --- maven-enforcer-plugin:1.2:enforce (enforce-maven) @ keywhiz-cli ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ keywhiz-cli ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ keywhiz-cli ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 22 source files to C:\Users\Me\dev\java\keywhiz\cli\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ keywhiz-cli ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ keywhiz-cli ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 9 source files to C:\Users\Me\dev\java\keywhiz\cli\target\test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ keywhiz-cli ---
[INFO] Surefire report directory: C:\Users\Me\dev\java\keywhiz\cli\target\surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running keywhiz.cli.commands.AddActionTest
2015-11-04 21:05:01,799 - Creating secret 'newSecret' with version '1576915baeb123e0'.
2015-11-04 21:05:01,837 - Creating group 'newGroup'.
2015-11-04 21:05:01,844 - Creating secret 'newSecret' with version '1576915bc725a9a1'.
2015-11-04 21:05:01,850 - Creating client 'newClient'.
2015-11-04 21:05:01,868 - Creating secret 'newSecret' with version '1576915bcdfd8e14'.
2015-11-04 21:05:01,880 - Creating secret 'newSecret' with version ''.
2015-11-04 21:05:01,905 - Creating secret 'newSecret' with version '1576915bdc6dc6b4'.
2015-11-04 21:05:01,906 - Allowing group 'newGroup' access to secret 'newSecret'.
2015-11-04 21:05:01,914 - Creating secret 'newSecret' with version '1576915bea18e06a'.
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.397 sec
Running keywhiz.cli.commands.AssignActionTest
2015-11-04 21:05:01,943 - Creating client 'non-existent-client-name'.
2015-11-04 21:05:01,944 - Enrolling client 'non-existent-client-name' in group 'group'.
2015-11-04 21:05:01,947 - Enrolling client 'existing-client-name' in group 'group'.
2015-11-04 21:05:01,951 - Allowing group 'group' access to secret 'secret'.
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
Running keywhiz.cli.commands.DeleteActionTest
Please confirm deletion of secret 'secret': Y/N
Please confirm deletion of secret 'secret': Y/N
Please confirm deletion of secret 'secret': Y/N
2015-11-04 21:05:01,965 - Deleting group 'Web'.
2015-11-04 21:05:01,976 - Deleting client 'newClient'.
Please confirm deletion of secret 'secret': Y/N
2015-11-04 21:05:01,978 - Deleting secret 'secret'.
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 sec
Running keywhiz.cli.commands.DescribeActionTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec
Running keywhiz.cli.commands.ListActionTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
Running keywhiz.cli.commands.UnassignActionTest
2015-11-04 21:05:02,079 - Evicting client 'client-name' from group 'group-name'.
2015-11-04 21:05:02,081 - Revoke group 'group-name' access to secret 'secret-name..1576915c401e87fb'.
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec
Running keywhiz.cli.JsonCookieTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.129 sec
Running keywhiz.cli.UtilitiesTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running keywhiz.client.ClientUtilsTest
Tests run: 6, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 0.869 sec <<< FAILURE!
testLoadCookiesWithFile(keywhiz.client.ClientUtilsTest)  Time elapsed: 0.022 sec  <<< ERROR!
java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/Me/dev/java/keywhiz/cli/target/test-classes/fixtures/cookies.json
        at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
        at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
        at java.nio.file.Paths.get(Paths.java:84)
        at keywhiz.client.ClientUtilsTest.testLoadCookiesWithFile(ClientUtilsTest.java:129)

testSaveAndLoadCookies(keywhiz.client.ClientUtilsTest)  Time elapsed: 0.054 sec  <<< ERROR!
java.lang.UnsupportedOperationException: null
        at java.nio.file.Files.setPosixFilePermissions(Files.java:2044)
        at keywhiz.cli.ClientUtils.saveCookies(ClientUtils.java:131)
        at keywhiz.client.ClientUtilsTest.testSaveAndLoadCookies(ClientUtilsTest.java:138)

testSaveCookies(keywhiz.client.ClientUtilsTest)  Time elapsed: 0.001 sec  <<< ERROR!
java.lang.UnsupportedOperationException: null
        at java.nio.file.Files.setPosixFilePermissions(Files.java:2044)
        at keywhiz.cli.ClientUtils.saveCookies(ClientUtils.java:131)
        at keywhiz.client.ClientUtilsTest.testSaveCookies(ClientUtilsTest.java:113)


Results :

Tests in error:
  ClientUtilsTest.testLoadCookiesWithFile:129 » InvalidPath Illegal char <:> at ...
  ClientUtilsTest.testSaveAndLoadCookies:138 » UnsupportedOperation
  ClientUtilsTest.testSaveCookies:113 » UnsupportedOperation

Tests run: 70, Failures: 0, Errors: 3, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.749 s
[INFO] Finished at: 2015-11-04T15:05:03-06:00
[INFO] Final Memory: 46M/369M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.14.1:test (default-test) on project keywhiz-cli: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Users\Me\dev\java\keywhiz\cli\target\surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Keywhiz CLI instructions for specifying truststore

Keywhiz CLI uses the JRE default truststore. In a deployed environment, this is generally sufficient. However, in development, a test certificate authority is used.

Website instructions could be updated to use -Djavax.net.ssl.trustStore et. al. Or, Keywhiz CLI could take a --cacerts parameter.

Secure file transmission

Hi,
Is it possible to transmitting the secret JCEKS file from server to deployed machines directly using keywhiz ?

Test to check that the api (v2) generates the expected output

It seems there are no tests that check the output of the v2 api. We should probably add a few tests to avoid accidentally breaking the API in the future.

Is it not possible consume a secret(with version) by name ?

Hi,
I came to know that if we create a secret with version then we can not consume it by name.
Am i correct ? if not ,then how to do using keywhiz api in java,.

Keywhiz CLI supports --username parameter

Keywhiz CLI uses the current user as the username when authenticated to Keywhiz Server. This generally works, but the development environment comes with user keywhizAdmin out-of-the-box.

improve travis + testing against different database setup

I manually ran the unittests against mysql and discovered an issue related to cascading deletion. We should make sure that the tests run against all three, h2/mysql/postgres databases in travis.

Display secret metadata in UI

Each secret content has associated metadata (typically empty). This is visible via the CLI but is not exposed in the UI currently.

CC @xaviershay

I am not able to create secret using keywhiz api.

I used below code to create secret,

public static void testCreateNewSecrete() throws IOException{
        KeywhizClient keywhizClient = keywhizClient();
        keywhizClient.login("keywhizAdmin", "adminPass".toCharArray());
        String str = FileUtils.readFileToString(new File("path/to/somefile/location/DefaultBundle.jceks"),"UTF-8");
        ImmutableMap<String,String> myMap = ImmutableMap.<String, String>builder().put("client", "client").put("group", "Security").build();
        SecretDetailResponse createSecret = keywhizClient.createSecret("key_store","key store", str, true, myMap);
        System.out.println(createSecret);
    }

Then i getting below exception,

Exception in thread "main" keywhiz.client.KeywhizClient$ValidationException: Malformed request semantics from client (422)
at keywhiz.client.KeywhizClient.throwOnCommonError(KeywhizClient.java:263)
at keywhiz.client.KeywhizClient.httpPost(KeywhizClient.java:290)
at keywhiz.client.KeywhizClient.createSecret(KeywhizClient.java:148)
at com.cybermoney.keywhiz.Application.testCreateNewSecrete(Application.java:45)
at com.cybermoney.keywhiz.Application.main(Application.java:37)

How to how to convert somesecretFile.jceks to string and make above code working ?

If i do not use file content then it is working.

create a common module for server, model and cli.

We can then put the default db pom.xml thing in there.
We should also cleanup the cli (move keyStoreFromResource in there, put the keystore file, wrap improve sslOkHttpClient's signature, etc.).

Swagger docs don't understand ZonedDateTime fields

Swagger auto-generates the API documentation for Keywhiz. When it generates model JSON, it doesn't seem to understand datetime fields (using ZonedDateTime) and expands the object further and further. What we would like it to do is stop at any ZonedDateTime and treat it as a ISO8691 datetime string.

Automation API style inconsistent for deleting a secret

To delete a secret via automation API, DELETE /automation/secrets/<xyz> takes a secret name as <xyz>. All of the other automation APIs use an id for deletion.

track createdby/updatedby for accessgrants and memberships

We do it for all the other pieces of data.

Dockerize Keywhiz server

create dockerfile to run keywhiz server

How to customize keywhiz server to authenticate with username and password to consume any service ?

I want to consume a service which has to give me a secret by taking a user name and password.

Migrate to JOOQ

JDBI is getting relatively painful to use for keywhiz due to complexity and verbosity (see AclDAO.java).

We've used JOOQ for some DAO tests and it'd be nice to move everything to JOOQ and keep things uniform.

How can i add a client from UI and how to add version to the secret?

Help needed for consuming /secret/{secret name}

Hi @sul3n3t ,
I want to create group,client and assign them each other.Create secret and grant access to previously created client and group,Consume secret it by /secret/{secret name} service.

Related to #82

Everything using java and keywhiz api.

Please do help me.

Exception in thread "main" java.lang.VerifyError: Cannot inherit from final class

Please help me,
I tried with some test classes,but i am unable to retrieve the secret,

Application.java

public class Application {

       public static void main(String[] args) throws IOException {
    OkHttpClient client = mutualSslClient();
    Request.Builder b = new Request.Builder().url("https://localhost:4444");
    Request get = b.get().url("/secret/General_Password").build();
    Response response = client.newCall(get).execute();
    System.out.println(response.code());
}

public static OkHttpClient mutualSslClient() {
    String password = "ponies";
    KeyStore keyStore = keyStoreFromResource("clients/client.p12", password);
    KeyStore trustStore = keyStoreFromResource(
            "dev_and_test_truststore.p12", password);

    return HttpClients
            .builder()
            .withClientCert(keyStore, password)
            .addRequestInterceptors(
                    new AuthHelper.AcceptRequestInterceptor(
                            MediaType.APPLICATION_JSON))
            .build(trustStore, 4445);
}

private static KeyStore keyStoreFromResource(String path, String password) {
    KeyStore keyStore;
    try (InputStream stream = Resources.getResource(path).openStream()) {
        keyStore = KeyStore.getInstance("PKCS12");
        keyStore.load(stream, password.toCharArray());
    } catch (IOException | NoSuchAlgorithmException | CertificateException
            | KeyStoreException e) {
        throw new AssertionError(e);
    }
    return keyStore;
}

}

Then i am getting ,

Exception in thread "main" java.lang.VerifyError: Cannot inherit from final class
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.cybermoney.keywhiz.Application1.mutualSslClient(Application1.java:36)
at com.cybermoney.keywhiz.Application1.main(Application1.java:22)

Can we update the secret using service call.

Hi,

I want to create a secret with filename.jceks with version 1.0.From second time onwards i just to update filename.jceks along with versions 1.1 .then i just want to retrieve filename.jceks with latest version.

Is it possible with keywhiz java api ?

Need better compile instructions

I'm running into the error:
[ERROR] Failed to execute goal org.codehaus.mojo:sql-maven-plugin:1.5:execute (default) on project keywhiz-model: FATAL: password authentication failed for user "root" -> [Help 1]
When trying to compile per the instructions, mvn package -am -pl server. How do I get around this please?

Also, the instructions need to be improved as there are too many assumptions such as you need to do git clone [repo] && cd [repo] and then start the build. Copying once the above issue is fixed the remainder will be smoother.

Regression test all endpoints for authentication

It would be fantastic to have a test ensuring all API endpoints use an @Auth parameter OR explicitly mark unauthenticated access (annotation?).

Add pagination to web UI

When there are a large number of secrets, loading them all on the web UI can take a while and blocks other functionality on the page.

It would be nice to paginate, or change the loading logic somehow.

How to convert encrypted secret into a file

After uploading a file i am unable to see file in the UI after refreshing the page also.

but when i use below
curl --cert client.pem -k -H "Content-Type:application/json" https://localhost:4444/automation/secrets/1

{"id":1,"name":"Test","secret":"zs7OzgAAAAIAAAAFAAAAAwAQY3licS5kYi4yMDE1MDQyNwAAAUz6otwbrO0ABXNyADNjb209yS2V5UHJvdGVjdG9yzVfKWecwu1MCAAB4cgAZamF2YXguY3J5cHRvLlNlYWxlZE9iamVjdD42PabDt1RwAgAEWwANZW5jb2RlZFBhcmFtc3QAAltCWwAQZW5jcnlwdGVkQ29udGVudHEAfgACTAAJcGFyYW1zQWxndAASTGphdmEvbGFuZy9TdHJpbmc7TAAHc2VhbEFsZ3EAfgADeHB1cgACW0Ks8xf.............................","secretLength":2474,"creationDate":"2015-06-03T10:32:17.834Z","groups":[],"isVersioned":false}

Have you used any encryption algorithm,if so how i can convert to file.

Upgrade httpclient version

See dropwizard/dropwizard#1017

We can either wait for dropwizard to merge the PR or do something similar ourselves.

Configure travis to allow failures on non master branches.

It's annoying that everything looks broken when only some experimental branch is not passing travis.

Migrate createdBy/updatedBy/description columns to be NOT NULL

The createdBy, updatedBy, and description columns are useful in different DB tables, but they are only used for display. In code, these values are coerced to empty string when the DB field is NULL. This issue is an enhancement to migrate these columns to be NOT NULL with a default of "". Code will be able to be slightly simpler.

Consider converting permissions related joins to semi-joins

Most of the joins in the codebase are related to permission checks. Expressing these using in or exists clauses might make the code more readable and improve performance.

Related link worth reading: http://roylyseng.blogspot.com/2012/04/semi-join-in-mysql-56.html

how to access the service '/automation/secrets' with name

I gone through the link http://square.github.io/keywhiz/apidocs/#!/secrets-admin/createSecret
and found

It is mentioned as 'Implementation Notes
'Returns a single secret or a set of all secrets'

So i want to access this service by name.

Error using db-seed data

I'm trying to get a basic test setup running, so far, I've built a Docker image, and run the migrations:

docker run -p 4444:4444 square/keywhiz java -jar server/target/keywhiz-server-0.7.5-shaded.jar migrate server/src/main/resources/keywhiz-development.yaml

Which seems to have worked. However when I try and load the seed data, I get:

docker run -p 4444:4444 square/keywhiz java -jar server/target/keywhiz-server-0.7.5-shaded.jar db-seed server/src/main/resources/keywhiz-development.yaml

...

Exception in thread "main" org.jooq.exception.DataAccessException: SQL [insert into GROUPS (ID, NAME, CREATEDAT, UPDATEDAT) values (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp))]; Table "GROUPS" not found; SQL statement:
insert into GROUPS (ID, NAME, CREATEDAT, UPDATEDAT) values (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)) [42102-187]
    at org.jooq.impl.Utils.translate(Utils.java:1645)
    at org.jooq.impl.DefaultExecuteContext.sqlException(DefaultExecuteContext.java:661)
    at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:356)
    at org.jooq.impl.AbstractDelegatingQuery.execute(AbstractDelegatingQuery.java:133)
    at keywhiz.commands.DbSeedCommand.doImport(DbSeedCommand.java:89)
    at keywhiz.commands.DbSeedCommand.run(DbSeedCommand.java:71)
    at keywhiz.commands.DbSeedCommand.run(DbSeedCommand.java:51)
    at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:76)
    at io.dropwizard.cli.Cli.run(Cli.java:70)
    at io.dropwizard.Application.run(Application.java:73)
    at keywhiz.KeywhizService.main(KeywhizService.java:83)
Caused by: org.h2.jdbc.JdbcSQLException: Table "GROUPS" not found; SQL statement:
insert into GROUPS (ID, NAME, CREATEDAT, UPDATEDAT) values (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)), (cast(? as int), cast(? as varchar), cast(? as timestamp), cast(? as timestamp)) [42102-187]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
    at org.h2.message.DbException.get(DbException.java:179)
    at org.h2.message.DbException.get(DbException.java:155)
    at org.h2.command.Parser.readTableOrView(Parser.java:5261)
    at org.h2.command.Parser.readTableOrView(Parser.java:5238)
    at org.h2.command.Parser.parseInsert(Parser.java:1031)
    at org.h2.command.Parser.parsePrepared(Parser.java:401)
    at org.h2.command.Parser.parse(Parser.java:305)
    at org.h2.command.Parser.parse(Parser.java:277)
    at org.h2.command.Parser.prepareCommand(Parser.java:242)
    at org.h2.engine.Session.prepareLocal(Session.java:461)
    at org.h2.engine.Session.prepareCommand(Session.java:403)
    at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1189)
    at org.h2.jdbc.JdbcPreparedStatement.<init>(JdbcPreparedStatement.java:72)
    at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:277)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126)
    at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108)
    at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:81)
    at com.sun.proxy.$Proxy30.prepareStatement(Unknown Source)
    at org.jooq.impl.ProviderEnabledConnection.prepareStatement(ProviderEnabledConnection.java:112)
    at org.jooq.impl.SettingsEnabledConnection.prepareStatement(SettingsEnabledConnection.java:76)
    at org.jooq.impl.AbstractQuery.prepare(AbstractQuery.java:394)
    at org.jooq.impl.AbstractDMLQuery.prepare(AbstractDMLQuery.java:192)
    at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:316)
    ... 8 more

I assume I need this to at least populate the default user for the web ui? It seems like I'm missing a setup step in there somewhere?

When i create secret with one jceks file(by uploading),i can not my file in the UI.

At the time of creating secret i uploaded a file with somefile.jceks but i do not know whether file is uploaded or not into server and how to read this secret along with somefile.jceks from java application.
Is there any certificate required ? how can i contact the server.

remove foreign keys + manually implement cascading deletion

Not having foreign keys makes it easier to do things like schema changes or data migrations.

Dev CLI fails to list groups - Cannot instantiate value of type java.time.OffsetDateTime

I was able to successfully build the server and the CLI using the instructions provided on the homepage, but when I run the CLI to see if things work, I get an error:

$ ./cli/target/keywhiz-cli-*-SNAPSHOT-shaded.jar --devTrustStore --user keywhizAdmin list groups
Server URL not specified (--url flag), assuming https://localhost:4444/
Exception in thread "main" java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Can not instantiate value of type [simple type, class java.time.OffsetDateTime] from String value ('2012-06-21T14:38:09.000Z'); no single-String constructor/factory method
 at [Source: [{"id":916,"name":"Blackops","description":"","createdAt":"2012-06-21T14:38:09.000Z","createdBy":"","updatedAt":"2012-06-21T14:38:09.000Z","updatedBy":""},{"id":917,"name":"Security","description":"","createdAt":"2012-06-21T14:38:09.000Z","createdBy":"","updatedAt":"2012-06-21T14:38:09.000Z","updatedBy":""},{"id":918,"name":"Web","description":"","createdAt":"2012-06-21T14:38:09.000Z","createdBy":"","updatedAt":"2012-06-21T14:38:09.000Z","updatedBy":""},{"id":919,"name":"iOS","description":"","createdAt":"2012-06-21T14:38:09.000Z","createdBy":"","updatedAt":"2012-06-21T14:38:09.000Z","updatedBy":""},{"id":920,"name":"DeprecatedGroup","description":"","createdAt":"2013-03-12T11:23:43.000Z","createdBy":"","updatedAt":"2013-03-12T11:23:43.000Z","updatedBy":""}]; line: 1, column: 46] (through reference chain: java.util.ArrayList[0]->keywhiz.api.model.Group["createdAt"])
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at keywhiz.cli.commands.ListAction.run(ListAction.java:71)
        at keywhiz.cli.CommandExecutor.executeCommand(CommandExecutor.java:137)
        at keywhiz.cli.CliMain.main(CliMain.java:68)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not instantiate value of type [simple type, class java.time.OffsetDateTime] from String value ('2012-06-21T14:38:09.000Z'); no single-String constructor/factory method
 at [Source: [{"id":916,"name":"Blackops","description":"","createdAt":"2012-06-21T14:38:09.000Z","createdBy":"","updatedAt":"2012-06-21T14:38:09.000Z","updatedBy":""},{"id":917,"name":"Security","description":"","createdAt":"2012-06-21T14:38:09.000Z","createdBy":"","updatedAt":"2012-06-21T14:38:09.000Z","updatedBy":""},{"id":918,"name":"Web","description":"","createdAt":"2012-06-21T14:38:09.000Z","createdBy":"","updatedAt":"2012-06-21T14:38:09.000Z","updatedBy":""},{"id":919,"name":"iOS","description":"","createdAt":"2012-06-21T14:38:09.000Z","createdBy":"","updatedAt":"2012-06-21T14:38:09.000Z","updatedBy":""},{"id":920,"name":"DeprecatedGroup","description":"","createdAt":"2013-03-12T11:23:43.000Z","createdBy":"","updatedAt":"2013-03-12T11:23:43.000Z","updatedBy":""}]; line: 1, column: 46] (through reference chain: java.util.ArrayList[0]->keywhiz.api.model.Group["createdAt"])
        at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
        at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:875)
        at com.fasterxml.jackson.databind.deser.ValueInstantiator._createFromStringFallbacks(ValueInstantiator.java:281)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromString(StdValueInstantiator.java:284)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromString(BeanDeserializerBase.java:1176)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:143)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:134)
        at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3674)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1996)
        at com.fasterxml.jackson.core.JsonParser.readValueAs(JsonParser.java:1506)
        at keywhiz.api.ApiDate$ApiDateDeserializer.deserialize(ApiDate.java:50)
        at keywhiz.api.ApiDate$ApiDateDeserializer.deserialize(ApiDate.java:47)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:461)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:376)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1099)
        at com.fasterxml.jackson.module.afterburner.deser.SuperSonicBeanDeserializer.deserializeFromObject(SuperSonicBeanDeserializer.java:196)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:131)
        at com.fasterxml.jackson.module.afterburner.deser.SuperSonicBeanDeserializer.deserialize(SuperSonicBeanDeserializer.java:117)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3702)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2733)
        at keywhiz.client.KeywhizClient.allGroups(KeywhizClient.java:124)
        at keywhiz.cli.commands.ListAction.run(ListAction.java:56)
        ... 2 more

Is it possible to create a secret only using keywhiz api

As per #82,
To consume a service /secret/{secretName}, It should have client and groups,
I do not want to create groups and clients.
Even i can consume service /secret/{id} without having groups and clients to that secret.

Like wise,I just want to create secret and consume it by /secret/{secretName}.

Security headers should also be included in 401 error response

For some reason, the filter doesn't run for 401 error responses. It does run on some error responses (e.g. 404), so something subtle is going on.

To repro: curl -k -v 'https://localhost:4444/admin/me'

Examples don't work out of the box

Keywhiz server and CLI compile fine, and I can log in to the webui with both a user from the development sample SQL and a user I added my self in the database. However I can't seem to use keywhiz.cli to log in.

danny@MacBook-Pro ~/src/keywhiz ±master » ./cli/target/keywhiz-cli-*-SNAPSHOT-shaded.jar -U https://localhost:4444/ login      130 ↵
password for 'danny':
Exception in thread "main" javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1917)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:301)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:295)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1369)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:156)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:925)
    at sun.security.ssl.Handshaker.process_record(Handshaker.java:860)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1043)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1371)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1355)
    at com.squareup.okhttp.Connection.upgradeToTls(Connection.java:241)
    at com.squareup.okhttp.Connection.connect(Connection.java:158)
    at com.squareup.okhttp.Connection.connectAndSetOwner(Connection.java:174)
    at com.squareup.okhttp.OkHttpClient$1.connectAndSetOwner(OkHttpClient.java:120)
    at com.squareup.okhttp.internal.http.RouteSelector.next(RouteSelector.java:131)
    at com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:312)
    at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:235)
    at com.squareup.okhttp.Call.getResponse(Call.java:262)
    at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:219)
    at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:192)
    at com.squareup.okhttp.Call.execute(Call.java:79)
    at keywhiz.client.KeywhizClient.httpPost(KeywhizClient.java:272)
    at keywhiz.client.KeywhizClient.login(KeywhizClient.java:111)
    at keywhiz.cli.CommandExecutor.executeCommand(CommandExecutor.java:121)
    at keywhiz.cli.CliMain.main(CliMain.java:68)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
    at sun.security.validator.Validator.validate(Validator.java:260)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1351)
    ... 22 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:145)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:131)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
    ... 28 more

This is a 100% stock server with just the development user and my "danny" user added. I'm assuming this is because the certificate i'm testing with it not actually signed by a trusted root CA?

It also seems there is little-to-no documentation around actually adding your own CA to the server. I believe this would live in "derivation.jceks" but i'm not sure.

Any guidance as to how one would actually use keywhiz in production would be greatly appreciated. I'm working on secrets management right now, and keywhiz seems to be the 'answer' to that if I can make it work in production.

How to make an initial automation request?

I would like to use the automation endpoints.

The documentation suggests that "The automation API requires a client certificate and automationAllowed=true in the clients DB table." ... but how do I set this up?

From poking around I think if I send a POST to /automation/clients with my client cert it should just work. But how do I make this initial client cert? There are various .p12 and .crt files in the core checkout, but seems to be missing a .key file to make into a CA pem that I can use to sign a new client cert. I tried converting the .p12 to a .pem on the off chance that made sense, but didn't work.

Trying to regenerate key material (which would give me all the private/public keys), I run into this error in the https://github.com/square/keywhiz/wiki/Development-and-test-key-material#create-a-server-certificate step

> openssl pkcs12 -aes128 -in out/localhost.crt -inkey out/localhost.key-export -out out/localhost.p12
81214:error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:/SourceCache/OpenSSL098/OpenSSL098-52.40.1/src/crypto/asn1/tasn_dec.c:1341:
81214:error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:/SourceCache/OpenSSL098/OpenSSL098-52.40.1/src/crypto/asn1/tasn_dec.c:385:Type=PKCS12

So now I'm knee deep in cert things and pretty sure I shouldn't be here :) What did I miss?

Request log missing security context

Currently, authentication is done at the Jersey layer. However, the request log is written using data at the Jetty servlet layer. When data is put into a security context to show a request is authenticated (at Jetty layer), the principal isn't in the request log. This means there's a bunch of extraneous and easy to miss logging statements.

A portion of the authentication should move to the Jetty servlet layer so the principal can be shown in the request log.

Login for development

Hi,

I just loaded the dev data and started the Keywhiz server and tried logging in with keywhizAdmin:adminPass. This does not seem to work. Are these the correct credentials ?

Should we improve our release process?

I don't think we have documented our development process which boils down to:

1. Add new code to master (with all changes visible to the open source community as pull requests and tasks/bugs tracked as issues). These changes are only tested on our local development setup.
2. Cut a release branch, currently numbered 0.7.x
3. Deploy the code on Square's staging environment.
4. If things look good, deploy the code on Square's production environment.

This process is significantly more transparent than alternatives such as internal forks. We however do not have a way to communicate if a release is stable or still undergoing tests. Newly released versions may have bugs or performance regression issues.

I suggest we append a -pre suffix to the release in step 2, adopt an odd/even numbering scheme or some other convention. We can then add a step 5. to create a -stable label (or some other system to communicate that a release is ready for production use).

LdapAuthenticatorFactory should not depend on DB

LdapAuthenticatorFactory.build takes a connection parameter (because it extends UserAuthenticatorFactory) but the parameter is not used.

ui dropdown menu is broken

Repro steps:

1. https://localhost:4444/ui/#/secrets/739
2. focus on "Add Group"
3. type "A"
4. try to click on a group, the typeahead fails to autofill with the value.

Investigate if SecretDAO.deleteSecretsByName should also delete the contents of all the secrets.

deleteSecretByNameAndVersion calls into SecretContentDAO. Should SecretDAO.deleteSecretsByName do the same?

Also, can we leverage cascading deletes in the database to simplify the code?

Build Issues on Ubuntu

Hi,

I'm getting some build errors on my Ubuntu server. Any advice would be appreciated.

 mvn3 package -am -pl server -P h2 -e
 [INFO] Error stacktraces are turned on.
 [INFO] Scanning for projects...
 [INFO] ------------------------------------------------------------------------
 [INFO] Reactor Build Order:
 [INFO] 
 [INFO] Keywhiz (Parent)
 [INFO] Keywhiz Testing
 [INFO] Keywhiz API
 [INFO] Keywhiz Client
 [INFO] Keywhiz HKDF
 [INFO] Keywhiz Model
 [INFO] Keywhiz Server
 [INFO] 
 [INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder      with a thread count of 1
 [INFO]                                                                         
 [INFO] ------------------------------------------------------------------------
 [INFO] Building Keywhiz (Parent) 0.7.9-SNAPSHOT
 [INFO] ------------------------------------------------------------------------
 [INFO] 
 [INFO] --- maven-enforcer-plugin:1.2:enforce (enforce-unlimited-crypto-policy) @ keywhiz-parent ---
 [INFO] 
 [INFO] --- maven-enforcer-plugin:1.2:enforce (enforce-maven) @ keywhiz-parent ---
 [INFO] 
 [INFO] --- maven-dependency-plugin:2.8:copy-dependencies (copy) @ keywhiz-parent ---
 [INFO] 
 [INFO] --- maven-source-plugin:2.4:jar-no-fork (attach-sources) @ keywhiz-parent ---
 [INFO]                                                                         
 [INFO] ------------------------------------------------------------------------
 [INFO] Building Keywhiz Testing 0.7.9-SNAPSHOT
 [INFO] ------------------------------------------------------------------------
 [INFO] 
 [INFO] --- maven-enforcer-plugin:1.2:enforce (enforce-unlimited-crypto-policy) @ keywhiz-testing ---
 [INFO] 
 [INFO] --- maven-enforcer-plugin:1.2:enforce (enforce-maven) @ keywhiz-testing ---
 [INFO] 
 [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ keywhiz-testing ---
 [INFO] Using 'UTF-8' encoding to copy filtered resources.
 [INFO] skip non existing resourceDirectory /home/vagrant/keywhiz/testing/src/main/resources
 [INFO] 
 [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ keywhiz-testing ---
 [INFO] Nothing to compile - all classes are up to date
 [INFO] 
 [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ keywhiz-testing ---
 [INFO] Using 'UTF-8' encoding to copy filtered resources.
 [INFO] skip non existing resourceDirectory /home/vagrant/keywhiz/testing/src/test/resources
 [INFO] 
 [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ keywhiz-testing ---
 [INFO] No sources to compile
 [INFO] 
 [INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ keywhiz-testing ---
 [INFO] No tests to run.
 [INFO] 
 [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ keywhiz-testing ---
 [INFO] 
 [INFO] --- maven-dependency-plugin:2.8:copy-dependencies (copy) @ keywhiz-testing ---
 [INFO] 
 [INFO] --- maven-source-plugin:2.4:jar-no-fork (attach-sources) @ keywhiz-testing ---
 [INFO]                                                                         
 [INFO] ------------------------------------------------------------------------
 [INFO] Building Keywhiz API 0.7.9-SNAPSHOT
 [INFO] ------------------------------------------------------------------------
 [INFO] 
 [INFO] --- maven-enforcer-plugin:1.2:enforce (enforce-unlimited-crypto-policy) @ keywhiz-api ---
 [INFO] 
 [INFO] --- maven-enforcer-plugin:1.2:enforce (enforce-maven) @ keywhiz-api ---
 [INFO] 
 [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ keywhiz-api ---
 [INFO] Using 'UTF-8' encoding to copy filtered resources.
 [INFO] skip non existing resourceDirectory /home/vagrant/keywhiz/api/src/main/resources
 [INFO] 
 [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ keywhiz-api ---
 [INFO] Changes detected - recompiling the module!
 [INFO] Compiling 33 source files to /home/vagrant/keywhiz/api/target/classes
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SecretSeries.java:23: error: package java.time      does not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SecretSeries.java:25: error: cannot find symbol
 import java.util.Optional;
                 ^
   symbol:   class Optional
   location: package java.util
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SecretContent.java:25: error: package java.time      does not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SecretContent.java:26: error: cannot find symbol
 import java.util.Optional;
                 ^
   symbol:   class Optional
   location: package java.util
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/ApiDate.java:29: error: package java.time does not      exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/ApiDate.java:30: error: package java.time.format does      not exist
 import java.time.format.DateTimeFormatter;
                        ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SecretSeries.java:61: error: cannot find symbol
   public abstract Optional<String> type();
                   ^
   symbol:   class Optional
   location: class SecretSeries
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SecretContent.java:48: error: cannot find symbol
   public abstract Optional<String> version();
                   ^
   symbol:   class Optional
   location: class SecretContent
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/ApiDate.java:80: error: cannot find symbol
   public ApiDate(OffsetDateTime odt) {
                  ^
   symbol:   class OffsetDateTime
   location: class ApiDate
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/ApiDate.java:84: error: cannot find symbol
   public OffsetDateTime offsetDateTime;
          ^
   symbol:   class OffsetDateTime
   location: class ApiDate
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/GroupDetailResponse.java:22: error: package java.time      does not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SanitizedSecret.java:26: error: package java.     time does not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SanitizedSecret.java:29: error: cannot find      symbol
 import java.util.Optional;
                 ^
   symbol:   class Optional
   location: package java.util
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/Client.java:24: error: package java.time does      not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/Group.java:23: error: package java.time does not      exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/Secret.java:24: error: package java.time does      not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/Secret.java:26: error: cannot find symbol
 import java.util.Optional;
                 ^
   symbol:   class Optional
   location: package java.util
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SanitizedSecret.java:121: error: cannot find      symbol
   @JsonProperty public abstract Optional<String> type();
                                 ^
   symbol:   class Optional
   location: class SanitizedSecret
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/Secret.java:149: error: cannot find symbol
   public Optional<String> getType() {
          ^
   symbol:   class Optional
   location: class Secret
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/SecretDeliveryResponse.java:24: error: package java.     time does not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/ClientDetailResponse.java:21: error: package java.time      does not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/automation/v2/CreateSecretRequestV2.java:10: error:      cannot find symbol
 import java.util.Base64;
                 ^
   symbol:   class Base64
   location: package java.util
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/SecretDetailResponse.java:24: error: package java.time      does not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/AutomationSecretResponse.java:24: error: package java.     time does not exist
 import java.time.OffsetDateTime;
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/validation/ValidBase64Validator.java:18: error: cannot      find symbol
 import java.util.Base64;
                 ^
   symbol:   class Base64
   location: package java.util
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/automation/v2/SecretDetailResponseV2.java:10: error:      cannot find symbol
 import java.util.Base64;
                 ^
   symbol:   class Base64
   location: package java.util
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/ApiDate.java:40: error: cannot find symbol
     private static DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.     SSS'Z'");
                    ^
   symbol:   class DateTimeFormatter
   location: class ApiDateSerializer
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SecretContent.java:36: error: Did not generate      @AutoValue class for keywhiz.api.model.SecretContent because it references undefined types
 public abstract class SecretContent {
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SanitizedSecret.java:39: error: Did not generate      @AutoValue class for keywhiz.api.model.SanitizedSecret because it references undefined types
 public abstract class SanitizedSecret {
                 ^
 /home/vagrant/keywhiz/api/src/main/java/keywhiz/api/model/SecretSeries.java:38: error: Did not generate      @AutoValue class for keywhiz.api.model.SecretSeries because it references undefined types
 public abstract class SecretSeries {
                 ^
 30 errors
 [INFO] ------------------------------------------------------------------------
 [INFO] Reactor Summary:
 [INFO] 
 [INFO] Keywhiz (Parent) .................................. SUCCESS [  1.294 s]
 [INFO] Keywhiz Testing ................................... SUCCESS [  0.974 s]
 [INFO] Keywhiz API ....................................... FAILURE [  1.623 s]
 [INFO] Keywhiz Client .................................... SKIPPED
 [INFO] Keywhiz HKDF ...................................... SKIPPED
 [INFO] Keywhiz Model ..................................... SKIPPED
 [INFO] Keywhiz Server .................................... SKIPPED
 [INFO] ------------------------------------------------------------------------
 [INFO] BUILD FAILURE
 [INFO] ------------------------------------------------------------------------
 [INFO] Total time: 4.153 s
 [INFO] Finished at: 2015-11-05T14:17:53+00:00
 [INFO] Final Memory: 26M/264M
 [INFO] ------------------------------------------------------------------------
 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-     compile) on project keywhiz-api: Compilation failure -> [Help 1]
 org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.     plugins:maven-compiler-plugin:3.1:compile (default-compile) on project keywhiz-api: Compilation failure
         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.     java:108)
         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.     java:76)
         at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(     SingleThreadedBuilder.java:51)
         at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116)
         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361)
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
         at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
         at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213)
         at org.apache.maven.cli.MavenCli.main(MavenCli.java:157)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:606)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
         at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
         at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
 Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
         at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:862)
         at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:129)
         at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.     java:133)
         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
         ... 19 more
 [ERROR] 
 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
 [ERROR] 
 [ERROR] For more information about the errors and possible solutions, please read the following articles:
 [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
 [ERROR] 
 [ERROR] After correcting the problems, you can resume the build with the command
 [ERROR]   mvn <goals> -rf :keywhiz-api

website's links don't show up on mobile devices

The links on the right side (wiki, api, google group, stackoverflow) don't show up when the window is too narrow (yeah for responsive design!)

Add Kerberos GSSAPI for authentication

Would be nice to be able to use Kerberos and GSSAPI for authentication instead of SSL client certificates.

For the server and client there are no issues since it's Java. keywhiz-fs is a bit trickier/unknown but https://github.com/jmckaskill/gokerb could be used.

Since we (and I think you too, since you use a keytab in your example) has to deploy a keytab to the machine at provisioning anyway. Though, you probably already have a certificate from your CM already…

What do you think?

I am getting error at the time of creating client,group and secret at same.Facing errors while accessing secret

Hi @sul3n3t

import io.dropwizard.jackson.Jackson;

import java.io.IOException;
import java.io.InputStream;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;

import javax.ws.rs.core.MediaType;

import keywhiz.AuthHelper;
import keywhiz.KeywhizService;
import keywhiz.api.AutomationSecretResponse;
import keywhiz.api.ClientDetailResponse;
import keywhiz.api.GroupDetailResponse;
import keywhiz.api.SecretDetailResponse;
import keywhiz.client.KeywhizClient;
import keywhiz.testing.HttpClients;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.collect.ImmutableMap;
import com.google.common.io.Resources;
import com.squareup.okhttp.OkHttpClient;

public class CreateGroupAndClient {

    public static void main(String[] args) throws IOException {
        KeywhizClient keywhizClient = keywhizClient();
        keywhizClient.login("keywhizAdmin", "adminPass".toCharArray());
        ClientDetailResponse createClient = keywhizClient.createClient("TestClient");//creating client
        GroupDetailResponse createGroup = keywhizClient.createGroup("TestGroup", "Cyber money test");//creating group
        keywhizClient.enrollClientInGroupByIds((int) createClient.id,(int) createGroup.getId());
        ImmutableMap<String, String> myMap = ImmutableMap.<String, String> builder().put("client", "client").put("group", "Security").build();
        SecretDetailResponse createSecret = keywhizClient.createSecret("TestSecret", "","TestSecret", true, myMap);
        keywhizClient.grantSecretToGroupByIds(Math.toIntExact(createSecret.id),Math.toIntExact(createGroup.getId()));//grant Secret To GroupByIds

        KeywhizClient retriveClient = mutualSslClient();
        AutomationSecretResponse secretDetailsById = retriveClient.getSecretDetailsByName("TestSecret");//calls httpGet(String.format("/secret/%s", secretName))
        System.out.println(secretDetailsById.secret());

    }

    public static KeywhizClient keywhizClient() {
        String password = "ponies";
        KeyStore trustStore = keyStoreFromResource(
                "dev_and_test_truststore.p12", password);

        OkHttpClient httpClient = HttpClients
                .builder()
                .addRequestInterceptors(
                        new AuthHelper.XsrfRequestInterceptor("XSRF-TOKEN",
                                "X-XSRF-TOKEN"),
                        new AuthHelper.AcceptRequestInterceptor(
                                MediaType.APPLICATION_JSON))
                .build(trustStore, 4444);

        com.fasterxml.jackson.databind.ObjectMapper mapper = KeywhizService
                .customizeObjectMapper(io.dropwizard.jackson.Jackson
                        .newObjectMapper());
        return new KeywhizClient(mapper, httpClient);
    }

    private static KeyStore keyStoreFromResource(String path, String password) {
        KeyStore keyStore;
        try (InputStream stream = Resources.getResource(path).openStream()) {
            keyStore = KeyStore.getInstance("PKCS12");
            keyStore.load(stream, password.toCharArray());
        } catch (IOException | NoSuchAlgorithmException | CertificateException
                | KeyStoreException e) {
            throw new AssertionError(e);
        }
        return keyStore;
    }

    public static KeywhizClient mutualSslClient() {
        String password = "ponies";
        KeyStore keyStore = keyStoreFromResource("clients/client.p12", password);
        KeyStore trustStore = keyStoreFromResource(
                "dev_and_test_truststore.p12", password);

        OkHttpClient httpClient = HttpClients
                .builder()
                .withClientCert(keyStore, password)
                .addRequestInterceptors(
                        new AuthHelper.AcceptRequestInterceptor(
                                MediaType.APPLICATION_JSON))
                .build(trustStore, 4444);
        ObjectMapper mapper = KeywhizService.customizeObjectMapper(Jackson
                .newObjectMapper());
        return new KeywhizClient(mapper, httpClient);

    }

when i run above application the i am getting below error.

But i test with some test data like,

KeywhizClient retriveClient = mutualSslClient();
        AutomationSecretResponse secretDetailsById = retriveClient.getSecretDetailsByName("General_Password");//calls httpGet(String.format("/secret/%s", secretName))
        System.out.println(secretDetailsById.secret());

'General_Password' is test data came with keywhiz setup.

How to implement using java api

Sir,
I can use curl command to retrieve my secrets using below command,
curl --cert client.pem -k --trace trace.out -H "Content-Type:application/json" https://localhost:4444/automation/secrets/1

But i want to achieve this in java ,

import java.io.DataOutputStream;
import java.io.IOException;
import java.net.URL;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
        public java.security.cert.X509Certificate[] getAcceptedIssuers() {
            return null;
        }

        public void checkClientTrusted(X509Certificate[] certs,
                String authType) {
        }

        public void checkServerTrusted(X509Certificate[] certs,
                String authType) {
        }

    } };

    SSLContext sc = SSLContext.getInstance("SSL");
    sc.init(null, trustAllCerts, null);
    HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

    // Create all-trusting host name verifier
    HostnameVerifier allHostsValid = new HostnameVerifier() {
        public boolean verify(String hostname, SSLSession session) {
            return true;
        }
    };
    // Install the all-trusting host verifier
    HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);
//    Response r = given().contentType("application/json").when()
//          .post("https://localhost:4444/automation/secrets/1");
//  System.out.println(r.getBody().toString());


    URL url = new URL("https://localhost:4444/automation/secrets/1");
    HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
    con.setRequestMethod("GET");
  //  con.setRequestProperty("User-Agent", USER_AGENT);
  //  con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
  //  con.setRequestProperty("Authorization", "Basic " + authStringEnc);
    con.setRequestProperty("Content-Type", "application/json");

    String urlParameters = "";

    // Send post request
    con.setDoOutput(true);
    DataOutputStream wr = new DataOutputStream(con.getOutputStream());
    wr.writeBytes(urlParameters);
    wr.flush();
    wr.close();

    int responseCode = con.getResponseCode();
    System.out.println("\nSending 'POST' request to URL : " + responseCode);

how to retrieve secret using Web service call from java

Hi,
I want to consume a secret from java api.when i try to consume it from RestClient then i am getting reply as unauthorized.

Can you help how to do it.

I am unable to import as maven project into eclipse.

Hi ,
Error is:

Description Resource Path Location Type
Plugin execution not covered by lifecycle configuration: org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (execution: default-compile, phase: compile) pom.xml /keywhiz-testing line 5 Maven Project Build Lifecycle Mapping Problem

But from command prompt It is working.