This repository contains example code for all kinds of technologies and how to efficiently test them. All examples are based on our experience in different actual small and large scale projects.
-
General Practices
-
Technologies
This showcase demonstrates the basics of writing efficient automated tests for Spring Boot applications.
Basic principles:
-
Write isolated functionality tests for your own code.
-
Write small specific technology-integration tests for your usage of Spring Boot features and other used technologies like databases, message brokers, HTTP clients etc.
-
Write a few very high level application end-to-end smoke tests to verify that everything fits together.
The "Test Pyramid" is a simplified model describing how software testing should be done in different levels of granularity and how many tests should be on which level. A very good article about that topic was written by Ham Vocke in 2018 and published on Martin Fowler’s Blog titled The Practical Test Pyramid.
Based on that generalized model, a test automation focused pyramid for backend applications could look something like this:
Let’s dive deeper into this model with the help of a simplified application slice:
This application manages a library of books. It offers an HTTP API for consumers and data is stored in an SQL database.
These tests are written to make sure that the code you have written does exactly what it is supposed to do. Depending on what the code under test actually does, the scope of these tests can be divided into several groups:
-
Pure Functions: Tests will vary different input parameter combinations and check that the result is equal to what is expected.
-
Stateful Functions: For these functions, the result of their invocation is dependent on the state of their parent component (e.g. objects). Tests usually involve setting up a new instance for each test. Other than that the tests themselves are pretty similar to those of pure functions.
-
Orchestration Components: These components "orchestrate" the invocation and data transfer of multiple dependency components in order to achieve an overarching functionality. Their tests usually involve mocking the behaviour of the different dependencies and are focused on verifying that concerns like error handling, invocation order, correct data transfer etc. are handled as intended.
Because the tests only refer to your own code and everything else is mocked, they are extremely fast and can be run by the thousands in a very short time. This makes them the best tests to get quick feedback on the core components, e.g. the business logic, of your application.
In our example application, isolated functionality tests would be used mainly in the BookCollection component.
Important Methods and Technologies:
Technology integration tests are used to verify code that you have written to use a particular technology. Examples include, but are not limited to:
-
HTTP endpoints (
@Controller,@RestController) -
caching (
@Cachable) -
transactions (
@Transactional) -
asynchronous invocation (
@Async) -
asynchronous messaging (
@KafkaListener,@RabbitListener,KafkaTemplate,@AmqpTemplate) -
event handling (
@EventListener) -
method-level security (
@RolesAllowed,@PreAuthorize,@PostAuthorize,@Secured) -
web security configuration
-
database access (
JpaRepository,MongoRepository,JdbcTemplateetc.) -
HTTP client calls (
RestTemplate,HttpClient,WebClientetc.)
The goal is not to check if a given technology works. Instead, it is to check if you are using the technology correctly to achieve your goal.
As an example, let’s assume that you are connecting to a SQL database using JDBC and have written a SQL query to read some data. You don’t need to test that the JDBC driver or the database works. What you need to test is whether you have written valid SQL that will return the desired result when used with a particular database.
Since technology integration tests involve bootstrapping some kind of technology (external services, framework features etc.), they are a lot slower that isolated tests. At least the initial setup will usually take a couple of seconds, while each single test will most likely take only a couple of milliseconds.
In our example application, technology integration tests would be used to test the in the BookRestController and BookRepository components.
Important Methods and Technologies:
-
WireMock for simulating external HTTP services
-
Testcontainers for running and managing Docker containers in your tests (e.g. for databases)
-
Spring Boot Test Slices (
@WebMvcTest,@JpaTest,@SpringBooTest(classes=[MyCustomConfig::class])etc.)
End-to-end tests are written from the perspective of a user of our software. Particularly crucial here is which options the user has for interacting with the application under test. Frontend single-page applications are usually tested end-to-end using a browser and the backend is simulated. Meanwhile, backend applications, which are our focus here, are tested using their API. Any Dependencies, like other services or databases, are either simulated or replaced by test instances.
Questions that end-to-end tests can answer, and a combination of just isolated and technology integration tests can’t:
-
Does my application start given a default configuration? → Do all my components fit and are all required compontents part of the application context.
-
Does my global error handling work for all of my endpoints? → If there are global error handlers, testing them in each and every relevant technology integration test is error-prone (you might forget them) and redundant.
-
Do my global security rules work? → A lot of security aspects are defined globally. So the same logic as for global error handlers applies here.
In addition to questions like this, it is generally useful to include a couple of smoke tests. These kinds of tests execute one or two happy path scenarios per endpoint, just to see that the whole control flow from request to response works. Basically if "everything fits and works together".
In our example application, the end-to-end tests would use the BooksRestController’s HTTP endpoints and the BooksRepository’s database would be a test instance.
The scope of an end-to-end test starts with the available input channels of the application under test as they would be used in production and ends where the application’s responsibility ends.
The impact of an application’s architecture on its overall testability can be demonstrated using the following three examples. Let’s start with a rather abstracted and well-structured architecture and degrade that abstraction with each following example:
Architecture #1 is basically the classical 3 layer architecture:
-
The BooksRestController handles the translation of the HTTP protocol, and the public language (external model) into business logic, and the internal domain model.
-
The BooksCollection handles all core business logic and acts exclusively on the internal domain model.
-
The BooksRepository is responsible for the persistence of the state of the internal domain model in some kind of database.
Having a clear separation of concerns with each component focusing on a single job (e.g. translating business logic into SQL), it is very easy to also write tests that focus on that job and do not need to take too much else into consideration.
The BooksCollection can be 100% tested in isolation, since it does not rely on any outside technology. This component als contains all the important core behaviour for handling books. What one might call business logic.
The dependency to the BooksRepository is mocked and therefore completely under the tests' control. So in this architecture our feedback loop for the most important parts of our application is very fast.
Both the BooksRestController and BooksRepository are such small components, who’s only task is to translate business calls from and to a specific technology, that their isolated tests would cover the same scenarios that their technology integration will have to cover anyway. Therefore, isolated tests for these components are not necessary.
Both the BooksRestController and BooksRepository components handle integration with different technologies.
BooksRestController handles HTTP communication and translates our public language into our internal domain model.
Tests for this component should therefore involve HTTP and focus on whether requests are understood and responses are created correctly.
(@WebMvcTest, @WebFluxTest)
BooksRepository takes our SQL commands and uses a JDBC driver to talk to a database.
Tests for this component should involve a database in order to validate our commands are correctly written.
(@JdbcTest, @DataJdbcTest, @DataJpaTest, @DataMongoTest,etc)
Architecture example #2 removes the "business" layer, or more general the technology-independent components. Leaving the BooksRestController to interact directly with the BooksRepository.
This mix of responsibilities for the BooksRestController has an immediate impact on the lower levels of the test automation pyramide.
The two remaining components from example #1 contain technology specific code, which needs to be tested with technology integration tests. There are no real purely isolated testable components left. But because the business logic has to go somewhere, it is more than likely that all of that code would now be part of the BooksRestController.
This makes BooksRestController the one component that now does two things: Translating our public language from HTTP and executing business logic upon these requests. Therefore, it might be useful to write both isolated and technology integration tests for this component.
Writing those tests in a sustainable manner can be hard though. Instead of writing tests which represent business rules and are based on business inputs and outcomes (aka the value of your code), the tests now need to start and end with a technical perspective. Technical data (e.g. request headers, query parameters, request / response abstractions etc.) need to be simulated as input. That makes it hard to write tests that focus on those business value of your code.
Along with the new challenges for isolated tests, the technology integration tests are harder to write as well.
While the BooksRestController tests of example #1 could focus solely on testing the translation of HTTP requests into responses, they now need to know all the business rules as well. Just writing an example request and checking if the BookCollection mock is invoked with the correct parameter is not possible when the requests are directly translated into actions and side effects.
As with example #1, everything else is already tested either by isolated or technology integration tests, the only tests remaining are:
-
Global security rules.
-
Happy path smoke tests.
With those, our application is thoroughly - but also more challengingly - tested and ready to be deployed.
Example #3 removes all concepts of separation of concern / layers and puts the BooksRestController in charge of everything. From translating the public language to interacting directly with the database, all while also containing any business logic. Basically there is no architecture, but there is a big ball of mud.
Doing this, kills any hope for writing small and focused tests or having different kinds of tests at all. Purely technical white-box isolated tests for a single do-it-all component are basically unmaintainable. Each tests setup has to consider which database state to set up based on which logical path will be traversed based on a specific HTTP request. This makes the tests fragile, complex to write and hard to understand.
Without other components to mock, there is also no real advantage to writing technical integration tests. Bootstrapping the application only partially does not really save any startup time but does add a lot more complexity. Simply writing everything as end-to-end tests is usually the only option left.
With less design (e.g. fewer abstractions, bigger multi-use components etc.) in the production code, the ability to write efficient tests decreases. From example #1 to #2 the difference is not yet as serious as from #2 to #3, so there is a point at which not all aspects of the application are testable without excessive effort. The basic principle is: The better the production code is decomposed / structured, the more of it can be verified purely with isolated and individual technology integration tests.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent