A tongue-in-cheek demonstration of a handful of technologies working together to achieve something completely arbitrary.
An elaborate, over-engineered backend solution for doing a single generic "thing" using an HTTP endpoint.
This project utilises tools and architectures that I have become comfortable with during my time as an engineer. It can be used as a minimal-example for the packages and concepts used. It can be used as a template to build projects that actually do things.
Python 3.10 and Pipenv are required to build, run and test.
Build the environment using
pipenv install
or
pipenv install --dev
if running the tests.
Run the service using uvicorn:
pipenv run serve
(note that hitting the endpoint will result in errors because, well, it's all pretend).
Run the tests by running one of
pipenv run test-static
pipenv run test-unit
pipenv run test-behave
for running the static (ruff, black, mypy for styles and types), unit and behave tests respectively.
Enterprise FizzBuzz for its humour, but imagine less silliness and more learning.
Here, I'll explain the code in order to describe the concepts demonstrated. Each section will contain a header, a link to any file I'm referring to and an explanation including links to any external references (these links are available in a section at the bottom too). Often, there will be points of uncertainty or discussion as this design is not intended to be a series of dogmatic rules, but rather choices between many valid options.
This entry point of the code includes a pattern called an Application Factory (which is a concept I've borrowed from my use of it in Flask).
In the create_app function we could define a bunch of app-level settings, but this is mostly used for its synergy with the Dependency Injector pattern (inspiration from the Python Dependency Injector pattern itself). Simply, we define our app and our container (containing our dependencies). The initialisation of the container 'wires' itself through config inside the Container class (although, arguably this could be moved to here to keep the config in one place).
Finally, we make the app globally available which allows us to run the program using some ASGI service (uvicorn, etc).
This file is purely there to define our dependencies using Dependency Injector. They include default instantiations which are the ones which will be used in the live service. Everything defined in this file can be overridden in tests, as will be seen later.
The following sections focus on the conventional layers which I've used here to support appropriate separation of concerns and testibility.
In a more expansive project, this file (like most of the files here) could become it's own folder. I've intentionally chosen to not clog the project structure here with superfluous files/folders (keeping the YAGNI principle in mind while still demonstrating complexity).
Routes is intended to be a few things:
- Functionally, this is where our FastAPI routes are defined.
- Logically, this is Domain Driven Design "Service" (Services come in many shapes, some good articles on this here and here).
While I call this a Service, it's only because there is some conditionality controlling the flow of behaviour (we either compute the result or not based on the data in the database). You could split this logic out into a dedicated "Domain Service" and have this file exist simply to define routes.
We use dependency injection here to allow a dependency to be defined dynamically, and not inside the running code. Leveraging dependency injection allows us to write highly testable code (there's a good article on it here). By defining our dependencies in this way, they are easily overrideable (more on this later).
Ideally, this route function would do as little as possible, but there's a few things I always deem acceptable in moderation:
- Logging: logging here usually allows for clear and simple logging which is directly related to the flow of behaviour in the system. Digging through logs is one thing, but digging for where a log message is defined is another.
- Safe exception handling: here we handle a missing entry in a database. Having this handled here ensures that the flow of the route's logic is in 1 place, not spread between multiple places.
Whenever possible, I try to keep the route functions as simple as possible. Really, all we're doing here is:
- (Optionally) retrieve some data from a repository.
- (Optionally) perform some business logic by using domain level functions.
- (Optionally) save some data to a repository.
- Return data in an agreed format.
Note three of those are optional, they can be omitted but rarely should they be rearranged. Ideally they shouldn't be chained either (avoid doing this then that then the other - the function would be definitely doing too much (Clean Code) and become a maintenance nightmare).
This is where the business logic for our application lives. Ours counts characters present in a string - we define two classes, one to perform the behaviour and one as a return type. We use some nice Pydantic validation on CharCount to ensure that we always create valid objects.
Our repository class is designed as a way for our service to save the things we want to save. The only thing interesting we do here is to allow for the client dependency to be set.
The goal here is to define, in one place, all of the models which our are used to bring data in or send data out from the service. I commonly use the suffix DTO (Data Transfer Object) as a way to visually differentiate them from other models.
Having all of the models defined in one place also allows us to have all of our input validation in one place. Pydantic provides type validation upon object instantiation, and combined with FastAPI (like we do in routes.py here and here) gives us automatic 422 status code responses when our inputs aren't valid. As we did in domains.py, we can extend the validation on these DTO classes to be more complex as the need arises.
A final thing to note is that FastAPI can use these models and routes to generate an OpenAPI specification (example tutorial here) which can be used by a number of platforms to run, mock, describe etc our service. This could be done even if the DTO-style models weren't in one place, but it's nicer that they are.
It's my personal preference to 1, minimise the use of custom exceptions where Python built-ins could be used instead, and 2, keep all custom exceptions in one place - usually with a small note about their intended usage.
I use Pytest for unit testing and attempt to keep the hierarchy as flat as possible by only writing fully isolated function tests (no test classes) with highly descriptive names in the format
def test_function_under_test_input_description_expected_behaviour():
...
There are a bunch of variations on this pattern, but it mostly contains the same information
I avoid test classes to encourage better isolation (also I find that the terminal output is clearer to read).
The behave testing is incredibly powerful for testing system-wide behaviour and allowing you to make certain assertions about data-at-rest afterwards (through mocking the database). Behave allows us to write canned expressions (steps) to populate "Given, When, Then" Scenarios in Gherkin syntax.
Where behave excels in this example is its synergy with FastAPI's test client (direct access to the HTTP endpoints) and Dependency Injectors containers (overriding allows us direct access to the dependencies).
The environment file for Behave allows us to set behaviour that we want to happen when the tests run. In this we set mocks, and define that they are refreshed between every Scenario. We also define the app's test client. We attach all of these to Behave's context object which manages data between steps (this way, it's all accessible in the steps themselves).
Here, we define a minimal mock Client to ensure that we can pretend that we're saving data. Only the functions used by the application are covered, and we only do enough to ensure that we can verify that the right things have been saved. This could easily be replaced by mongomock if we're fancy, just a standard Mock/MagicMock if we're not.
We use the datastore property later to enable verification that our application submits things to the database correctly (Given we have a request, When we hit an endpoint, Then data is written to the database). Arguably it's enough to say a 2XX HTTP status code is returned (we do this too), but its nice to be sure.
TODO
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent