Schematic is a Clojure library which aids in assembling Component systems
from configuration data. That configuration data is often read from an edn file.
The structure of the configuration data was inspired by Integrant.
The goal of Schematic is to reduce the complexity of programmatic
construction of Components, while continuing to use existing Component workflows
and tools.
The primary features provided by Schematic are:
- specifying an initialization function for a Component
- declaring Component dependencies (for use in
component/using) - declaring merge definitions to enable configuration sharing among Components
- assembling a System using only a subset of the available Components
Schematic starts with a configuration map. From this map, Schematic will assemble a Component system map, ready to be started.
Each key/value pair in the configuration map will become a system key and component in the Component system map.
Schematic uses some special keys to identify
- The constructor for the component (
:sc/create-fn) - The Component dependencies for the components (
:sc/refs) - Additional rules to manipulate the component's configuration (
:sc/merge)
The component's configuration is passed to its constructor as part of building the Component system map.
It is ok to omit the :sc/create-fn constructor when a plain map is sufficient.
This may occur when the component doesn't need to implement any protocols,
or when the component exists as a source of configuration for other components.
The simplest example is:
(require '[com.walmartlabs.schematic :as sc]
'[com.stuartsierra.component :as component])
(def config {:app {:host "localhost"}})
(->> (sc/assemble-system config)
(component/start-system)
(into {}) ;; put it into a map just so it prints in the REPL
)
;; => {:app {:host "localhost"}}In this case, Schematic didn't really do any work, since there were no declared refs, merge-defs, or create-fns.
A more complete example is:
(require '[com.walmartlabs.schematic :as sc]
'[com.stuartsierra.component :as component])
(defrecord App [server api-keys]
component/Lifecycle
(start [this]
(assoc this :started true))
(stop [this]
(assoc this :started nil)))
(def config
{:app {:sc/create-fn `map->App
:sc/refs {:server :webserver}
:sc/merge [{:to [:api-keys] :from [:api :keys]}]}
:webserver {:sc/merge [{:from [:host-config]}]
:port 8080}
:host-config {:host "localhost"}
:api {:username "user"
:keys [1 2 3 4]}})
(->> (sc/assemble-system config)
(component/start-system)
(into {}) ;; put it into a map just so it prints in the REPL
)
;; =>
;; {:api {:username "user", :keys [1 2 3 4]},
;; :host-config {:host 8080},
;; :webserver {:host "localhost", :port 8080},
;; :app #user.App{:server {:host "localhost", :port 8080},
;; :api-keys [1 2 3 4],
;; :started true}}Notice, in the above, that the Schematic keys (:sc/create-fn, etc.) have
been removed, and the configuration for the :app component has been
extended via :sc/merge.
Of course, in a more realistic example, the configuration data would be read from an EDN file, rather than hard-coded into the application.
The following sections will cover aspects seen in the above example.
Example (edn):
{:app {:sc/create-fn user/map->App
:sc/refs {:server :webserver
:api-keys :api-keys}}
:webserver {:sc/refs [:host]
:port 8080}
:host 8080
:api-keys [1 2 3 4]}In this example, :app, :webserver, host and :api-keys are all top-level items,
and will be used to create components and/or be injected into components.
Top-level items can be any type of data, but only associative data structures will
receive dependency injection metadata for their references.
Component dependencies are declared via a key of :sc/refs with the value being
a map or vector having the same semantics as component/using.
Example:
:app {:sc/create-fn user/map->App
:sc/refs {:server :webserver
:api-keys :api-keys}}
:webserver {:sc/refs [:host]}In :app, the refs are a map of component-ids, where the key is the local id of the component,
and the value is the global id of the component.
In :webserver, a vector is provided, indicating that the local id and global id of the
component are the same.
To specify that a particular configuration item can participate in component/Lifecycle functions (start/stop),
the special key -- :sc/create-fn -- is used to declare the namespace-qualified name of the function which
which will be called to create the Component instance. This will usually be the default map->Record function which defrecord
creates for us. But it can also be any regular function which returns an object which implements
the Lifecycle interface, and accepts a single map as an argument.
Example (using a map->Record constructor function):
(ns com.business.system
(:require [com.walmartlabs.schematic :as sc]
[com.stuartsierra.component :as component]))
(defrecord App [server api-keys thread-pool-size]
component/Lifecycle
(start [this] this)
(stop [this] this))
(def config {:app {:sc/create-fn 'com.business.system/map->App
:sc/refs {:server :webserver
:api-keys :api-keys}
:thread-pool-size 20}
;; remaining config omitted
})Note :sc/create-fn 'com.business.system/map->App which will cause the App record to be created
when the system is assembled.
Example (using plain function):
(ns com.business.system
(:require [com.walmartlabs.schematic :as sc]
[com.stuartsierra.component :as component]))
(defrecord App [server api-keys thread-pool-size]
component/Lifecycle
(start [this] this)
(stop [this] this))
(defn make-app [{:keys [thread-pool-size] :or {thread-pool-size 10}}]
(map->App {:thread-pool-size thread-pool-size}))
(def config {:app {:sc/create-fn 'com.business.system/make-app
:sc/refs {:server :webserver
:api-keys :api-keys}
:thread-pool-size 20}
;; remaining config omitted
})In the 'plain function' example, you can see that it is possible to set a default value
for thread-pool-size in the make-app function if a value were not to be provided in the config.
That is just one possible use case for using a plain function constructor.
For each top-level, map? item in the config, Schematic will extract a list of the declared
:sc/refs and attach the needed Component metadata by calling (component/using component ref-map).
The resulting value will be a Component which can be started and stopped.
In complex applications, there may be configuration that is shared or duplicated across components,
the :sc/merge key is a set of rules for selecting, renaming, and injecting global configuration
into a component's specific configuration.
The merge happens before :sc/refs are processed.
:host-config {:host "localhost"
:port 8080}
:app {:sc/merge [{:from [:host-config]}]}In this example, :host-config map will be merged into the :app map.
:sc/mergeaccepts a vector of merge-defs.- a merge-def can provide
:to,:from, and:selectkeys :toindicates the key/key-path in the component config where the results will be merged. It defaults to[].:fromindicates the key/key-path from the global config where values will be copied from:selectis either a vector of key-names or a map of local-key-names to from-key-names (names of fields in:from). It defaults to the special keyword:all, which indicates to copy the entire:fromdata object.
The actual work of processing merge-defs, analyzing the :sc/refs, and adding the dependency metadata happens in the
com.walmartlabs.schematic/assemble-system function. That function takes a config map
and returns a System map which can be started and stopped.
Example:
(->> {:webserver {:sc/refs [:host :port]}
:host "localhost"
:port 8080}
(sc/assemble-system)
(component/start-system))
;; => #<SystemMap>assemble-system optionally takes a configuration map
and an optional list of component-ids to include in the final system.
In some cases it might be desirable to include only a subset of the available config when assembling a system, so that only the needed components are started and stopped. Two possible such scenarios are:
- The config is shared among multiple applications, which each need a portion of the components at runtime, but not all of them.
- At the REPL, it might be useful to get a particular component and start it, such as a database connection.
In such cases, the two argument version of assemble-system can be invoked, passing
the list of component ids as the second parameter.
There are the top-level components which must be included in the final system.
Only these top-level components and all of their transitive dependencies will be included in
the final system map.
Example (include single app):
(-> {:app-1 {:sc/refs {:conn :db-conn
:product-api :product-api}}
:app-2 {:sc/refs {:conn :db-conn
:customer-api :customer-api}}
:db-conn {:sc/refs {:host :db-host}
:username "user"
:password "secret"}
:product-api {}
:customer-api {}
:db-host "localhost"}
(sc/assemble-system [:app-1])
(component/start-system)
((partial into {})))
;; =>
;; {:product-api {},
;; :app-1 {:conn {:host "localhost", :username "user", :password "secret"}, :product-api {}},
;; :db-host "localhost",
;; :db-conn {:host "localhost", :username "user", :password "secret"}}Notice that :app-2 and :customer-api have not been included in the final system.
This filtering of components occurs before components are properly instantiated (via the :sc/create-fn function).
Example (development at the REPL):
(let [config {:app-1 {:sc/refs {:conn :db-conn
:product-api :product-api}}
:app-2 {:sc/refs {:conn :db-conn
:customer-api :customer-api}}
:db-conn {:sc/refs {:host :db-host}
:username "user"
:password "secret"}
:product-api {}
:customer-api {}
:db-host "localhost"}
system (-> (sc/assemble-system config [:db-conn :product-api])
(component/start-system))
{:keys [db-conn product-api]} system]
(println (into {} system))
;; do something here with db-conn or product-api
(component/stop system)
nil)
;; {:product-api {},
;; :db-host localhost,
;; :db-conn {:host localhost, :username user, :password secret}}
;; => nil In this example, we were able to use just the database component and product-api to do some testing in the REPL.
Copyright (c) 2017-present, Walmart Inc.
Distributed under the Apache Software License 2.0.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent