chanikag / config-mapper

config-mapper

This repository is to maintain the codes that used to project single toml file configurations into WSO2 products specific configurations.

Phases

Parsing

Read the config from the TOML based configuration file which will be the single source of truth for all the product configuration. No processing is done on this phase and whatever the configs read will be passed to the next phase

Mapping

Responsible for mapping the product configuration keys to the keys used by component level. This should be a 1:1 mapping. If there are 1:n mappings from TOML to product configs, need to use Inferring to get it done.

Mapping is done using the file named “key-mappings.json”. The format of that file is as below,

{

 "key_from_toml_file": "key_used_by_component"

}

Inferring

Responsible for inferring configuration values, based on other configuration that the user has already provided in the configuration source.

Eg.

• If the user specifies the database is ‘mysql’, we can infer the driver class, validation query and connection URL (assuming he has already provided host,port,db name etc)
• If the user specifies the userstore as ‘readwriteldap’, there are several LDAP related connection properties we can pre-populate with defaults.

Defaults

Responsible for providing the default values for the configurations. These values will be taken only if the user has not provided the configuration value or it has not been inferred in the previous phrase.

Resolve Placeholders

Responsible for replacing placeholders that are given in the configuration values. The placeholders can be used to resolve environment variables, system properties or any other configuration value from the pipeline. Please refer the following subsection on how to use placeholders.

Resolve Units

Responsible for resolving the units from human readable/understandable unit to component accepted unit. For example, at the component level “foo_timeout” may accept milliseconds. However, by default, we want to set it to 1 minute. That means we need to do some math and set it to 1601000, Which is 60000. The drawbacks are,

• User need to know what units are expected
• Need to do the math and give the value in expected unit
• May get to misconfigure due to inconsistent unit usages. (There are configs that use ms, seconds, minutes. Therefore the user may assume the same unit for two configs)

To overcome this problem, this phrase of the pipeline convert the units from the user given unit to component expected units. The user only needs to suffix the unit to the value when configuring. Eg. In the above case, the user may give either of,

“foo_timeout = 1m” or “foo_timeout = 60s”

Validation

Responsible for context-aware validation of the configuration values and show any errors in case there are validation failures. This can be used to mandate certain configuration and make sure the values match provided regular expression.

Dependent versions

• com.fasterxml.jackson.core:jackson-annotations:jar:2.7.0
• com.fasterxml.jackson.core:jackson-core:jar:2.7.9
• com.fasterxml.jackson.core:jackson-databind:jar:2.7.9.1
• com.google.guava:guava:jar:22.0
• com.google.re2j:re2j:jar:1.2
• com.hubspot.jinjava:jinjava:jar:2.4.14
• org.xmlunit:xmlunit-core:jar:2.6.2
• org.xmlunit:xmlunit-matchers:jar:2.6.2
• org.xmlunit:xmlunit-assertj:jar:2.6.2
• net.consensys.cava:cava-toml:jar:0.5.0