cjstehno / ersatz

🤖 A simulated HTTP server for testing client code with configurable responses.

Ersatz Server

If you use this project and would like to help keep it running, please consider making a donation.

☕ Buy me a coffee?

Quick Links

• Site: https://cjstehno.github.io/ersatz
• Project: https://github.com/cjstehno/ersatz
• User Guide: https://cjstehno.github.io/ersatz/docs/user_guide.html
• Java API Docs: https://cjstehno.github.io/ersatz/javadoc/
• Groovy API Docs: https://cjstehno.github.io/ersatz-groovy/javadoc/

Introduction

The Ersatz Server is a HTTP client testing tool which allows for request/response expectations to be configured in a flexible manner. The expectations will respond in a configured manner to requests and allow testing with different responses and/or error conditions without having to write a lot of boiler-plate code.

Warning: v3.x is NOT directly backwards compatible with the 2.x releases. Some lesser-used features have been removed and the Groovy DSL has been extracted into its own extension library - see the What's New in 3.0 section of the User Guide for details and a migration guide.

Warning: v2.x has removed some unused features of the library and extracted the Groovy support into it's own extension library. See the What's New in 2.0 section of the User Guide for details and a migration guide.

Getting Started

Project artifacts are available via the Maven Central repository. Below are the dependency coordinates for Gradle and Maven, more details are provided in the Getting Started section of the User Guide.

Gradle

testImplementation 'io.github.cjstehno.ersatz:ersatz:4.0.1'

// or, for the Groovy DSL extensions
testImplementation 'io.github.cjstehno.ersatz:ersatz-groovy:4.0.1'

Maven

<dependency>
    <groupId>io.github.cjstehno.ersatz</groupId>
    <artifactId>ersatz</artifactId>
    <version>4.0.1</version>
    <scope>test</scope>
</dependency>

<!-- or, for the Groovy DSL extensions -->
<dependency>
    <groupId>io.github.cjstehno.ersatz</groupId>
    <artifactId>ersatz-groovy</artifactId>
    <version>4.0.1</version>
    <scope>test</scope>
</dependency>

Shadowed

There is a safe (shadowed) version of each library available, which is useful in cases where you already have a version of Undertow in use (to avoid version collisions). See the Shadow Jar section of the User Guide for more information.

Build Instructions

Ersatz is built using Gradle:

./gradlew clean build

Or, if the "shadow" artifact is desired:

./gradlew clean build shadowJar

If you are interested in building the website, with all documentation and reports, you can run the following:

./gradlew site

In order to build specific reports, run the appropriate one of the following:

./gradlew asciidoctor
./gradlew javadoc
./gradlew jacocoTestReport
./gradlew test

Publishing

To Local Maven Repo

You can publish the all the source, javadoc, "safe" and regular jars to your local maven repository (~/.m2/repository directory) using the following command:

./gradlew publishToMavenLocal -x signErsatzGroovyPublication signErsatzPublication

// or, if you have not built recently
./gradlew clean build publishToMavenLocal -x signErsatzGroovyPublication signErsatzPublication

The -x sign skips the signing step, which requires signing information. See the section on "signing" below if you need to have the locally published artifacts signed.

To Maven Central

Before you publish a release, be sure to generate a release build (see signing section for details):

./gradlew clean build shadowJar signErsatzGroovyPublication signErsatzPublication -Psigning.gnupg.keyName=<key-id> -Psigning.gnupg.passphrase=<key-pass>

Then, to publish the artifacts to the Maven Central Repository, run

./gradlew publish -PossrhUser=<jira-user> -PossrhPass=<jira-pass> -Psigning.gnupg.keyName=<key-id> -Psigning.gnupg.passphrase=<key-pass>

The additional "signing." properties are required to sign the artifacts, see the Signing section below for more details.

Once the artifacts have been published, sign-in to https://s01.oss.sonatype.org and navigate to the "Staging Repositories" and "Close" the published artifacts - this may take some time to appear. If there are errors, you can "Drop" it, fix them and publish again.

After you have successfully "closed" the staging repository, you can release it by pressing the "Release" button.

Signing

When publishing the artifacts to the Maven Central Repository, they need to be signed. In order to keep the signing information secret, the properties are added only when the publishing task is executed, on the command line.

When you want to sign the published artifacts, add the following parameters to the command line:

-Psigning.gnupg.keyName=<last-8-of-key> -Psigning.gnupg.passphrase=<key-password>

where <last-8-of-key> is the last 8 characters of the key, and <key-password> is the password for the key.

You can list the available keys using:

gpg -k

Documentation Site

Building

You can build the documentation website using the following:

./gradlew site

Which will build all Javadocs, User Guide, build reports and the website itself.

Publishing

Publishing the website is a bit of an odd process:

1. Create a separate clone of the ersatz project repo with a different name (e.g. ersatz-site).
2. In the ersatz-site repo checkout the gh-pages branch - you should see only website content in that repo now.
3. Publish the updated site content from the main project by running: rsync -r build/site/* ../ersatz-site/
4. In the ersatz-site project add, commit and push the changes into the gh-pages branch.

At this point the website will be published but it may take some time for GitHub to reload the changes.

License

Copyright (C) 2023 Christopher J. Stehno

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

ersatz's Issues

Publish to maven central

Configure project artifacts to be published in maven central repository as well as Jcenter.

Update groovy/spock version

Let's get the latest versions of spock, groovy and gradle.

Add JUnit Java example to Getting Started

The Java example is weak in the Getting Started section. Expand it to a full Java JUnit test example as the Groovy Spock example.

Also add simple Java example to the site landing page.

Finish unit testing

At this point there is just a quick cross-section of unit testing... finish this and then finish the integration testing in the HttpBuilder-NG project.

Gather requirements for 1.0

I think this project is nearing the point for a true 1.0 release - use this issue to gather a list of features required for 1.0.

Multiple value headers not matching

It seems that request with multiple value headers are not matched. Below is the expectation.

server.expectations {
    get('/api/hello') {
        called 1
        header 'Accept', 'application/json'
        header 'Accept', 'application/vnd.company+json'
        responder {
            code 200
                content  'msg': 'World', 'application/vnd.company+json'
            }
        }
}

However, this does not seem to match. Looking at the code it appears that only the first value is compared:

static RequestMatcher header(final String name, final Matcher<String> m) {
    new RequestMatcher(m, { ClientRequest cr -> cr.headers.getFirst(name) })
}

public String getFirst(String headerName) {
    HeaderValues headerValues = getEntry(headerName);
    if (headerValues == null) return null;
    return headerValues.getFirst();
}

Can this be fixed, or would you prefer I send a PR ?

Thanks

Unable to match requests with encoded chars in path

Related to http-builder-ng/http-builder-ng#162 found in HttpBuilder-NG the path matcher does not account for encoded parts of a path, such as:

/api/v4/projects/myteam%2Fmyrepo/repository/files/myfile.json

It will automatically decode it to:

/api/v4/projects/myteam/myrepo/repository/files/myfile.json

Figure out how to support these encoded paths.

Java 9 Support

Java 9 is out and it's not pretty. I upgraded, tried to build the project only to fail with odd errors... I then did a similar pattern on my other projects and got the same result. I then downgraded back to Java 8.

At this point, I don't have a Java 9 upgrade plan. My original understanding was that the modularization was an optional piece of functionality - it appears that it is not and that it also seems to break pretty much everything.

If you are comfortable with Java 9 and want to take a whack and getting Ersatz to compile and work, feel free to submit a Pull Request. If not for that, it might be a while unless I get a lot of outside pressure.

Truncate the logged body content

When posting multipart file content, the logged body is way too big... truncate the to-string on that down to like 1k or something manageable.

Add ContentType support to multipart DSL

The multipart content matcher configuration has content types as strings, provide overrides with the ContentType object.

Request recording

Consider/investigate request/response recording such that you can use Ersatz as a proxy in front of a live server, run a test or other requests against it and record the request/response captured as expectations.

• http://betamax.software/ - a Java library that does this sort of recording. I wonder if it could be used to do the heavy lifting or if it does things well enough that I don't need to integrate the functionality here.

support for clearing expectations

Currently expecations are additive. Add support for clearing them and investigate possibility of having a test share a server for all test methods without rebuilding it, but only clearing the expectations.

Add hamcrest expecation examples

The only mention of hamcrest is one line in the user guide - some examples would be nice.

Allow Closures as Matchers

It would be nice if a Closure<Boolean> could be used as an alternative to a Hamcrest matcher where they are allowed... I guess a Supplier<Boolean> could also be allowed.

Either adjust the interfaces to allow this or make a ClosureMatcher and a ConsumerMatcher implementation of the Matcher interface.

Auto-start with expectations

The pattern of:

ersatzServer.expectations {
    // expectation code..
}.start()

is common enough that it might be useful to provide a means of auto-starting after calling one of the expectation configuration methods (maybe a global config):

def ersatzServer = new ErsatzServer({
    autoStart true
})

Should still be false by default, and it should not cause errors if its enabled and start() is called - it should just be ignored in this case.

This will allow for cleaner code.

More complex header support

Currently, the request and response headers are simply Strings... HTTP headers can be more complicated. Consider adding support for more types of headers - is this even needed or will Strings work well enough?

Move authentication support into core

The BASIC and DIGEST authentication support should be converted to config properties rather than "features"

• the feature mechanism should be deprecated and removed by 1.0 with a migration strategy for existing code (probably just HttpBuilder-NG)
• it would be nice to be able to define BASIC, DIGEST and non-authenticated expectations on the same server but this would not be a requirement

Add something like:

def server = new ErsatzServer({
    basicAuthentication true
    digestAuthentication true
})

Request `body` matcher methods need to allow hamcrest matcher for content-type

The Request.body content matcher methods allow for Hamcrest matchers as values, but not for content types. It would be useful (especially in multipart matching) to allow for Hamcrest matchers in the content type fields. So that you could have something like:

expectations {
    post('/foo').body( any(), startsWith('text/') ).responds().code(201)
}

Cleanup unit testing

• Cleanup and simplify testing
• increase coverage a bit without being silly about it

Create standalone embedded proxy server

The proxy support added to HttpBuilder-NG has exposed the need for a more realistic yet still simple embedded proxy server. Undertow can be used as a proxy server so it would be nice to provide a simple interface for setting up a small embedded testing proxy server outside of the Ersatz server itself.

Replace the FileUpload API with JavaMail for multipart

The JavaMail API is a bit more generic in its Multipart implementation - let's use it instead.

Make sure it works for both request and response multipart content.
Document how to use it in both cases.

Explore using in Kotlin

Explore how Ersatz may be used from Kotlin.

• Are there any issues with using it? If so, what can be done to resolve them?
• Document how it may be used in the User Guide

More documentation

• Add more to the user guide
• Better groovy doc text
• More examples
• Clean up web site a bit

Ensure that gzip/deflate compression supported

Need to make sure that gzip, deflate compression are both supported.

Add Digest server feature

Add a ServerFeature to support DIGEST authentication. Undertow seems to support it already so this should be similar to the BASIC auth feature as far as implementation goes.

Add reference to kotlin usage example and blog post

Add reference to my recent blog post about REST development.
Also note the kotlin version of the project as an example of using with kotlin

Rework user guide

Rework the user guide to have better organization and more details and examples about each feature.

• Make sure the GroovyDocs have documentation links from override methods
• Add package info to GroovyDocs
• Try to find a way to get Markdown or AsciiDoc to work in GroovyDocs (maybe there is a way now)
• Re-align the sections of the User Guide (see comment below)
• More inline examples in GroovyDocs
• Add more content type examples for request/response transforming
• Examples of using Closures as Hamcrest matchers
• Information about using with Kotlin
• Any relevant information about using with Groovy 2.5
• add contributors section to user guide

Consider adding a shadow jar distribution

It would be interesting and helpful to provide a distribution version of the library as a shadow jar as an alternate dependency to the standard. This would keep the dependencies of this project from affecting the using project.

Raw request handling

It might be useful to add a means of writing a matcher directly against a request, something like:

server.expectations {
    match( matcher ).responds().body('ok', TEXT_PLAIN)
}

where matcher would be a hamcrest matcher given the ClientRequest object to match against. This would allow another level of flexibility in match configuration.

Note: that the match method would be generic to handle whatever the matcher matches, including the request method.

Https support

Add support for configuring HTTPS endpoints either via extension or other means.

• should be able to work out of the box or with provided keys/certs
• would be nice to be able to configure both HTTP and HTTPS on the same server - this may require refactoring of the extension mechanism (not a bad thing)

Ensure support for large upload/download files

Make sure that the uploading and downloading support specifications support very large files in some manner.

binary compatibility with Undertow 1.3.x

If not really necessary it would be nice if Ersatz server is compatible with Undertow 1.3.x (namely 1.3.15.Final) which is used by various spring boot related libraries (https://mvnrepository.com/artifact/io.undertow/undertow-core/1.3.25.Final/usages). This disqualifies from using Ersatz with Spring Boot or Grails 3. The construct offending the compatibility is currently server.listenerInfo:

            actualHttpPort = (server.listenerInfo[0].address as InetSocketAddress).port

            if (httpsEnabled) {
                actualHttpsPort = (server.listenerInfo[1].address as InetSocketAddress).port
            }

Add `any` method matcher

It would be useful to add an any request matcher such that it would match any request method - with the ability to refine the match with the rest of the configuration (query, headers, etc).

server.expectations {
    any('/foo').responds().body('ok', TEXT_PLAIN)
}

so that GET /foo and POST /foo would both be matched by this expectation.

Test with Groovy 2.5

Groovy 2.5 is coming out soon - it would be nice to know how things work before it's released.

Not sure we would want to upgrade right away but it would be good to have a test (shell) and some docs around using it - if there is any difference - I would think it would just work without any notes.

Will the new Java 8 lambda integration make the extra Groovy vs Java methods obsolete?

Come up with a support timeline for 2.5 - hopefully it will just work with both until I start using new features.

Better layout and structure of unit test

The tests are a bit of a hodge podge right now. There is good coverage but things could be cleaner and better organized.

Ability to delay or time-out response

It might be useful to have the ability to timeout or delay the response for testing how the client handles this.

expectations {
    get('/foo').responds {
        delay '1 min'
        code 200
    }
    get('/bar').responds {
        delay 3600 // ms
        code 200
    }
}

Use the time duration parsing from Vanilla for rich string times or just allow ms (maybe TimeCategory too).
This would slow down tests that use it, but I could see this being something a client would want to test.

Better logging of unmatched requests

It would be invaluable if the actual and expected requests were logged in more detail and if possible, identify where they didn't match. I think mock-server does this and prints out which part of the request did not match (whether it was the method, body, headers etc).

Also consider making the output Spock and/or JUnit friendly so that IDEs (esp IntelliJ) can show a diff.

Support for http 2

Undertow should already support this but I need to figure out how to enable and expose as an option.

Support for multi-part uploads

Add support for multi-part upload handling either in the core or as a feature.

Allow global config of request decoders

Currently the encoders/decoders are configured on a per-request/response basis. While a shared collection of them may be injected, it would be better to allow for global configuration of the desired defaults at the server or expectations configuration level.

Also ensure that there is a set of reusable common encoders/decoders for standard scenarios.

EncoderChain fails to resolve an encoder if object is a Map

If we try to encode a Map instance using the response encoder, the encoder chain logic fails to resolve it.

static final def JSON_ENCODER = { Object object ->
    toJson(object)
} as Function

def server = new ErsatzServer({
    encoder 'application/json', LinkedHashMap, JSON_ENCODER
})

I believe this is because in the ErsatzResponse class the getContent method tries to evaluate content.class but if content is a Map, Groovy thinks class is a key of the Map rather than a property accessor for getClass method.

@Override
String getContent() {
    if (content != null) {
        return encoderChain.resolve(contentType, content.class)?.apply(content) ?: (content as String)
    }
    return ''
}

The proposed solution is to use the method accessor rather than property shorthand in this instance.

return encoderChain.resolve(contentType, content.getClass())?.apply(content) ?: (content as String)

Add OAUTH support

Add support for some sort of OAUTH - could probably use pac4j or something similar.

https://github.com/pac4j/undertow-pac4j

Support for proxy

TBD

Use "simulated" instead of "mock"

Make the point that this is not really a mock server but a server simulator... might make things more clear.

Add Grails Guide link to docs

Add the grails guide link:

• http://guides.grails.org/grails-mock-http-server/guide/index.html

to the docs somewhere and attribute it to the author

Cookie Tesing Features

@cjstehno

1. Ability to specify more than key/value for cookies. Especially path and expiration would be very handy.
2. Ability to assert that cookies are not present in a request. This may be possible already with Hamcrest matchers. If so, I would like to add them to the http-builder-ng tests.

HTTPS support for proxy server

Currently the HTTPS support for the proxy server is not implemented. There is an issue in Undertow that needs to be resolved first:

https://issues.jboss.org/browse/UNDERTOW-1138

Support Options and Trace methods

Need to add support for OPTIONS and TRACE request matching.

• https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
• https://www.tutorialspoint.com/http/http_methods.htm
• http://ietf.org/rfc/rfc2616
• OPTIONS: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS

Documentation

Documentation needs to be written:

• site
• user guide
• source code docs

Web Sockets support

Should be able to support simulation of web socket request/response.

https://github.com/undertow-io/undertow/blob/master/examples/src/main/java/io/undertow/examples/websockets/WebSocketServer.java