eclipse / configjsr Goto Github PK

In sensitive environments I would like to know the source of each configuration entry. Configuration can change the behaviour of an whole application, so the final source of an entry provided by the configuration system during runtime is a relevant topic. So my questions:

• How the information can be accessed from a configuration about the source (and optionally other metdata) of an entry? From the API side this could be soemthing as
  Map<String,String> getMetadata(String key);
• Do you see alternatives how this requirement can be covered?

Unfortunately the API is the more simple aspect. The question is how the configuration system get's to this information. I see the following options:

1. The ConfigSource is extended with the same API method to provide metadata for an entry. This comes with the disadvantage that the configuration has to reevaluate each access to the API method, since ConfigSources can change and any cached metadata gets invalid if a more significant ConfigSource provides a value for a key later.
2. The ConfigSource returns not only a String, but a more complex value, which also contains the corresponding metadata. This is how it is done in Tamaya with PropertyValue, PropertySource

Other use cases, where this metadata mechanism has shown to be useful is support for collections, where I can provide the expected collection type (set, list, map, subtype mappings) as metadata.

Currently the specification only references how to handle errors as a side-effect of the desired behavior with the following two behaviors identified:

1. If a required configuration parameter is missing, a DeploymentException should be thrown.

2. If an implicit converter cannot convert a value, an IllegalArgumentException should be thrown (really this applys to explicit converters too since the Converter interface includes the equivalent throws statement).

I'd like to suggest that this leaves a lot of latitude in how these Exceptions are thrown and specifically how the configuration's deficits are passed back to the developer (and ultimately to the user). Ultimately, the entire state of the configuration system should be evaluated and all the deficits enumerated in a way that makes it easy to iterate over the problems. I think the javax.validation system does a pretty good job as it uses a ConstraintViolationException to pass along a Set<ConstrainViolation<?>> that can be easily parsed. I'd suggest that each validation problem, whether it's a missing required parameter or an issue with a converter should contain a field that only contains the parameter's name as well as a message describing the problem.

As a concrete example, the Wildfly Swarm implementation of the microprofile-config API produces a DeploymentException with the following message:

org.jboss.weld.exceptions.DeploymentException: Error while validating Configuration
No Config Value exists for ldap.service.host
No Config Value exists for ldap.client.password
No Config Value exists for ldap.client.dn

This message actually has two problems as follows:

1. If this message is subsequently printed out to a logging system, many static code analysis tools will flag this as a security issue.

2. If the developer wants to catch this message and provide additional information and/or formatting to the message, parsing these lines out of the message requires extra code.

Finally, since JSR-382 is intended to encompass both Java SE and Java EE (yeah I know) projects, it's very likely that the developer will want to provide a customized help message for a CLI project or a detailed error dialog for a JavaFX (AWT, Swing, etc) project when a configuration is invalid in some way. It would be great to standardize this across all implementations!

In Tamaya a target type can have multiple converters registered. The order is defined by @Priority annotations and the fully qualified class name. If a converter cannot convert a value the next in order is called until a value could be converted or no more converters are available.

• Pro: You can add additional formats for a type easily, without having to support all other formats existing as well.
• Contra: More complexity

One additional feature that Tamaya provides is the possibility to intercept the evaluation of configuration. The main mechanism here is the PropertyFilter, which can apply changes to the raw String value, before applying any converter logic. This mechanism has shown to be very useful for security based filtering and implementing placeholder/resolver logic. The question is if that should be a vendor specific feature, or it should be part of the official API.

The original design of DeltaSpike was defering the Config to just String/String (key/value).
Any typesafe behaviour and other features were built on top of the TypedResolver (our ConfigValue).
This is actually a much more consistent approach.

So should we remove anything beyond String/String from Config and defer to ConfigValue?
This would make our API much cleaner.
The only real argument against this might be backward compatibility with MicroProfile-config.

It would be a very helpful feature -- not only for container / orchestration environments -- to trigger updates in ConfigSources that originate from properties files, when these files are updated.

Spec says:

A custom Converter can define a priority with the @javax.annotation.Priority annotation. If a Priority annotation isn’t applied, a default priority of 100 is assumed. The Config will use the Converter with the highest Priority for each target type.

We need to add a test that add 2 converters (with different @priority) for the same type and only the one with the highest priority is used.

It seems that Apache Tamaya has a feature we miss yet in MP-config and thus also in the config JSR.
eclipse/microprofile-config#248

It's basically about picking up implicit conversion behaviour based on some certain criteria.

• Does a constructor(String) exist for the target class?
• Does a valueOf(String) method exist?
• Does a parse(String) method exist?

I think it would be rather easy and very beneficial to add this to our spec. Wdyt?

Initiated by @tomas-langer we had a lot of discussions during the last EG meetings about consistent atomic access to configuration.

Consider having myapp.host + myapp.port which belong together.

now if you switch from myserver:8080 to otherserver:9090, then there is no guarantee to have consistent access.

String url = config.getValue("myapp.host");
Integer port = config.getValue("myapp.port);

Imagine that inbetween those calls the config gets updated :(

This might require to switch back ConfigSource from getPropertyNames() to go Map<String, String> getProperties() as we had it in DeltaSpike and my initial proposal.
Or find another solution.

We should add Array in our builtin converter. Archius has this support and it is universally useful.

Dash characters are also commonly used in Java property keys but are not allowed in OS environment variables. The mapping between the two that's defined in the ConfigSources specification (https://github.com/eclipse/ConfigJSR/blob/master/spec/src/main/asciidoc/configsources.asciidoc#default-configsources) should therefore consider how the mapping of dashes should occur.

Related to #24.

The ConfigProvider (EDIT on name... apologies) has a static initializer.

For static initialization I usually use this pattern See https://en.wikipedia.org/wiki/Initialization-on-demand_holder_idiom to avoid unnecessary loading.

Thus

private static final ConfigProviderResolver INSTANCE = ConfigProviderResolver.instance();

Should be:

private static class LazyHolder { private static final ConfigProviderResolver INSTANCE = ConfigProviderResolver.instance(); }

Also sort of off topic but I was actually trying to get a configuration standard 2 years ago: https://github.com/agentgt/configfacade

I didn't have the clout or time to devote to pushing it so I'm excited about the future of this library.

I hadn't separated out enough of the "helper" functionality from the base interfaces in my version as I was still experimenting with the ergonomics of the library as well as I was also trying to add reloadable behavior and listeners (similar to archaius).

I think reloadable/listening behavior is important and might file an issue to maybe add support if thats OK.

I'm integrating the ConfigAccessor to my config JSR implementation and the following concepts from ConfigAccessor spec are not explained or specified in https://github.com/eclipse/ConfigJSR/blob/master/spec/src/main/asciidoc/configaccessor.asciidoc#configaccessor:

• withCurrentProjectStage(boolean with) does not define what a project state is.
• strictly(boolean strictly) has a local link for property value resolution sequence that does not exist.
• parameterizedBy(String propertyName) has a local link for ConfigResolver.getPropertyAwarePropertyValue that does not exist

There is also disprecancy between the spec and the API.
The ConfigAccessor does not define some methods specified in the spec (e.g. withCurrentProjectStage, strictly) while some in the interface are not mentioned (e.g. withLookupChain).

The Javadoc of withLookupChain mentions a javaconfig.projectStage property that is not defined anywhere. Is the name of this property significant (I assume it is since it starts with javaconfig).

Currently the whole config approach assumes that all the information is available in the whole ClassLoader/application.

If someone want's to guard against such an access, then this should best be done via the SecurityManager. But how to integrate this best?

Should we introduce a ProjectStage in JSR-382?

At the moment modules other than the API use a package name org.eclipse.configjsr. That seems legitimate (it is unlikely another Eclipse project uses that name), but most other Eclipse projects used as RI for JSRs did pick a "friendly" code-name, e.g. EclipseLink, Yasson, etc.

Could we also try to find such a code-name?

as @Emily-Jiang noted in PR #62 we need to add test which cover the following features of ConfigAccessor:

• useConverter
• getKey
• getResolvedKey
• getValue(ConfigSnapShot)
• ConfigSource#value changed stuff

In Apache DeltaSpike you can define an interface which gets automatically provided as accessor for configured values

http://deltaspike.apache.org/documentation/configuration.html#_interface_based_configuration

It's basically the approach first seen in Owner.
You simply write an interface and the getters automatically get implemented to look up a configured value.

Are we interested in providing such a mechanism in JSR-382?

Splitting this out from #9 into a separate ticket as discussed in our last EG meeting.

When people make use of ConfigValue they often want to trigger some action when a new value get's picked up.
Remember that ConfigValue is only needed for configured values which might change during runtime.

Often a user wants to be aware when the underlying value got changed. Either to log it out, or to perform some other code.

This is essentially done by registering a lambda as callback which gets called whenver the underlying config gets accessed an a change gets detected.
https://github.com/struberg/ConfigJSR-1/blob/i9_configValue/api/src/main/java/javax/config/ConfigValue.java#L162

Here is a code examples to highlight why this is an important use case:

@ApplicationScoped
public class FtpService {
  private ConfigValue<String> urlCfg;

  @Inject 
  void setConfig(Config cfg) {
    cfg.access("myapp.ftp.url")
      .cacheFor(1, TimeUnits.DAY)
      .evaluateVariables(true)
      .withLookupChain("${myapp.tenant}")
      .onChange((propertyName, oldV, newV) 
          -> log.info("Switchint to a new FTP URL from {} to {}", oldV, newV));
  }

  public void store(String name, byte[] content) {
    FtpConnection = access(urlCfg.getValue());
  }
}

This is really important for our Ops team. Otherwise they don't really have a clue when the system eventually picked up new values!

The onChange callback might also be used to e.g. clean cached values, etc. It's really very versatile and an important use case!

Some operating systems allow only alphabetic characters or an underscore(_), in environment variables. Other characters such as ., /, etc may be disallowed. Therefore, we do an environment name mapping: replace all non-alpha-numeric with _/

In order to set a value for a config property that has a name containing such disallowed characters from an environment variable, the following rules are used.
This ConfigSource searches for potentially 3 environment variables with a given property name (e.g. {@code "com.ACME.size"}):

1. Exact match (i.e. {@code "com.ACME.size"})
2. Replace the character that is neither alphanumeric nor '_' with '_' (i.e. {@code "com_ACME_size"})
3. Replace the character that is neither alphanumeric nor '_' with '_' and convert to upper case (i.e. {@code "COM_ACME_SIZE"})

add readme doc

We should add support for Java 9 module for the Config API.

I propose we use the javax.config name for the module.

Should the variable evaluation be enabled by default when using ConfigAccessor?
And do we also like to enable variable evaluation for @Inject @ConfigProperty by default?

In case an application wants to provide an integrated feature (e. g. "edit" button in the GUI) to edit configuration values, it would be great to have support for writeable configuration values (possibly in a future version of this API).

Currently the load SPI method strictly is relying on the ServiceLoader:

private static ConfigProviderResolver loadSpi(ClassLoader cl) {
        ...
        ServiceLoader<ConfigProviderResolver> sl = ServiceLoader.load(
                            ConfigProviderResolver.class, cl);
        ...
}

Adding a more complex service discovery mechanism is not required, but what would be a nice addition that would help in special environments is allowing a system property to define the explicit class to be loaded, instead of throwing an exception if multiple classes are registrered.
Summarizing my proposal is to add a system property javax.config.spi.ConfigProviderResolver to pass explicitly the class to be loaded. If not set, the default service loader functionality should be used as of now, also throwing error, when multiple implementations are registered.

Based on #26 I had a look at the spec and saw that we currently define the default config properties file as META-INF/javaconfig.properties. Actually I'm not really happy with the name and the place. I think that the META-INF folder is a valid option but would prefer to add support for a properties file in the root folder (conflicts can be handled by ordinal). Next to this I ask myself if javaconfig.properties as a name really makes sense? Why not using something like application.properties (as spring does)?

Adding a Converter defined by a lamba expression fails:

ConfigBuilder b = ConfigProviderResolver.instance().getBuilder();
b.withConverters(val -> val);

Using an anonymous class works fine:

Converter<String> converter = new Converter<String>(){
            @Override
            public String convert(String value) {
                return value;
            }
        }; 
b.withConverters(converter);

SO we have probably to rework this part of the API. In Tamaya we therefore have things like:

ConfigBuilder withConverters(Class type, Converter... converters);

respectively

<T> ConfigBuilder withConverters(TypeLiteral<T> type, Converter<T>... converters);

Is this an issue, or how the target type can be evaluated?

Plus @tomas-langer had the idea to split it into 2 methods:

• addLookupSuffix(String)
• addLookupSuffix(ConfigAccessor<String>)

Often configuration values are stored and retrieved from an external remote resource, server or service in order to better facilitate a central cohesive management of your environment. The result of this is that configuration values for a particular application/service are often changed on the external service.

Right now if an application uses the Configuration API it will not know if or when a particular configuration value was changed or updated. The change would eventually get detected the next time a particular configuration value was queried through the API (if it's not cached). However for values that are typically only read at startup, they may never get updated.

On top of updating the values when queried, we should have a relatively lightweight API to enable ConfigSources to notify the application using the Configuration API of any (or selected) config value changes that occur within the ConfigSource. The notification could be a simple CDI event we define or we can enable applications to register a listener in non CDI environments.

How updates are detected would be left to the ConfigSource implementation (polling, watches on files, ...).

The same API or functionality could be used to notify the application of updates when a config file changes, as discussed in #8 .

In the spec, the list of method tested for implicit conversion (https://github.com/eclipse/ConfigJSR/blob/master/spec/src/main/asciidoc/converters.asciidoc#implicit-converters) does not mention any required visibility for the methods.

In the TCK however, we test that private methods and constructors are valid in https://github.com/eclipse/ConfigJSR/blob/master/tck/src/main/java/org/eclipse/configjsr/PrivateImplicitConverterTest.java

I think that we should not support private methods for implicit conversion.
If a Java library has restricted visibility of constructor or methods, I don't think there is a good reason to ignore the restriction and use the methods anyway.
Furthermore, if the Config JSR is used in a Java 9 modular environment, accessing private module requires that the Java module "opens up" to the javax.config module.

ConfigAccessor provides many new features.
Should these features also be part of @ConfigProperty?
And should we introduce additional parameters or additional (stacked) annotations?

JSR-377 (https://github.com/jsr377/jsr377-api), the Desktop/Embedded Application Framework JSR, has as goal to standardize how applications can be built that target desktop, mobile, and embedded environments.
As part of these standardization effort JSR-377 has identified the configuration space as one of its key areas. Looking at the current state of JSR-382 there are some concerns on the viability of consuming 382 as is from 377's POV. Allow me to explain,
382 exposes a javax.config.Config type that's the source of key/values pertaining to configuration properties. Instances of Config can be queried programmatically, given a key you may get a value, simple pull model (1). There's also a push model via annotations (ConfigProperty) that complement dependency injection (2). Finally, there are times where the value obtained is not of the appropriate type, thus type conversion (3) must occur.

From 377's POV, (1) provides the minimum definition for querying key/values. Additional behavior can be built on top of javax.config.Config such as javax.application.configuration.Configuration (see https://github.com/jsr377/jsr377-api/blob/master/jsr377-api/src/main/java/javax/application/configuration/Configuration.java). Thus point (1) is covered.

Point (2) defines javax.config.inject.ConfigProperty as a Qualifier for injection. This annotation defines enough behavior to be usable by 377 given certain defaults. 377 defines its own annotation, javax.application.configuration.Configured, (see https://github.com/jsr377/jsr377-api/blob/master/jsr377-api/src/main/java/javax/application/configuration/Configured.java) with additional properties that come in handy given the uses cases we've seen so far. Now, 377 could use both annotations assuming that if 382's is used then you get default, well-defined behavior for those "missing" properties that 377's annotation provides. Thus point (2) is also covered.

So far, not changes to 382 are required for 377 to work with it. However it's point (3) that brings concerns to us as for natural reasons to 382 the Converter API expects the source type to be a String, whereas in 377 we reuse the Converter API to be able to transform resource values (such as Images, Icons, Colors, Gradients, etc). The sources of these resources may be textual sources (properties, XML, JSON, YAML) where the source type is a String, but they could also be programmatic sources such as Java resource bundles, Groovy scripts, Kotlin scripts, where the source type may already by of the expected type, or it could be some other type (not a String) that can be transformed into a target. Take for example an Insets instance, the source could be Insets, a number, a list of 4 numbers, a Map with (top,bottom,left,right) coordinates, some other thing.

The risk for 377 for adopting 382 as it currently stands, is that 377 would have to define its own Converter API to make resources work. I'd like to start a discussion on the Converter API to figure out what can be done to have this two JSRs cooperating with one another.

Good afternoon!

We have 2 possible ways to define the implicit Converter rules (#13):

A.) Either we can pick up only the list of combinations (name + param Type) of the Types in the JDK. This is the wording used in microprofile-config:

• the target type {@code T} has a {@code public static T of(String)} method, or
• the target type {@code T} has a {@code public static T valueOf(String)} method, or
• the target type {@code T} has a public Constructor with a String parameter, or
• the target type {@code T} has a {@code public static T parse(CharSequence)} method

B.) Use any combination of String and CharSequence param Types with a list of method names.
This is the wording currently used in ConfigJSR. This list is longer but it is easier to remember for users as it is just a list of method names with String and :

• the target type {@code T} has a {@code static T of(String)} method, or
• the target type {@code T} has a {@code static T of(CharSequence)} method, or
• the target type {@code T} has a {@code static T valueOf(String)} method, or
• the target type {@code T} has a {@code static T valueOf(CharSequence)} method, or
• the target type {@code T} has a {@code static T parse(String)} method, or
• the target type {@code T} has a {@code static T parse(CharSequence)} method, or
• the target type {@code T} has a Constructor with a String parameter, or
• the target type {@code T} has a Constructor with a CharSequence parameter

Now we need more community feedback about which way we should go in ConfigJSR.

Create a ConfigJSR-tck github repo for the TCK.
Has to be done by Eclipse.

Currently we only support injecting a @ConfigProperty Provider<T> but this has some issues on Weld.
So we might think about additionally also supporting injection of Supplier <T>

From Tomas:

Find a way to resolve the following usecase: 
Yaml config like this:


sockets:
  

 - name:  name1    

   port: 80
  
 - name:
  name2    

   port: 9090
      

   address: “192.168.1.1"


API expectation: Config.asList(“sockets”, SocketConfig.class) [Tomas]

this allows custom implementations to do a proper cleanup.

TCK tests are in the org.eclipse.configjsr package but the doc and a test mentions the javax.config.tck package.

Either the doc and the test must reference org.eclipse.configjsr or the package must be renamed to javax.config.tck.

Personnally, I'd prefer to use org.eclipse.configjsr.tck as the package name (and not "pollute" javax namespace for the TCK classes).

The gitter link on the main page shows to
https://gitter.im/eclipse/microprofile-config, which IMO is not correct. Either remove them of create a corresponding JSR Gitter and update the link.

As the name of environment variable can only use alphabets and _ , it is different to override some config properties as environment variables.

e.g.
com.acme.me="blah"

This property cannot be specified as environment variable. How about to directly translate . to _ for the corresponding environment variables.

com.acme.me -> com_acme_me
In this way, this property can be defined or override in environment variable.

to match the ConfigAccessor API name

In Apache DeltaSpike we support configure some.key=http://${bla.server}/some/Endpoint
where bla.server is yet another config entry.

https://issues.apache.org/jira/browse/DELTASPIKE-1126

This is also covered in the original ConfigJSR proposal:
https://github.com/struberg/javaConfig/blob/configValue/api/src/main/java/javx/config/ConfigValue.java#L54

Do we want to support this kind of functionality in JSR-382?

Remove the EPL which is a left-over of the initial Eclipse setup and add the ALv2 license as required by the ConfigJSR-382 proposal.

The @ConfigProperty has a target like

@Target({METHOD, FIELD, PARAMETER, TYPE})

Do we currently support METHOD and TYPE?
Looks like in most cases, it's for fields.

For version, I would only keep FIELD and PARAMETER that we can easily support

Happy to push a PR if we are ok.

add an example
add more explanation

We should add a 4th provided ConfigSource for a properties files that resides in the file system --
additionally to the classpath. Examples are locations such as /etc/javaconfig/ or ~/.javaconfig/. In order to support both Windows and Unix systems the default path to the properties file (whatever may chosen) should be configured, e.g. using a environment variable.

Synch up with MicroProfile Config and bring the following two major changes from MicroProfile Config:
eclipse/microprofile-config#325
and replace no-numberic-alpha with _ in environment variable

The current documentation (as generated into pdf) is missing a nice ending.
There should be a section that wraps up, has links, references etc.

The following was briefly discussed at the JCP Ec meeting in Hursley, after Emily presented the current status of JSr-382 to the EC. Gil Tene from Azul Systems asked if it would be possible to design this JSR so that it could potentially be added to Java SE at a later date.

This can happen more easily if the API is split in two: the programmatic API without any external dependencies; and the annotation based API that depends on 330/365. The split would have to be managed by two JSRs.

Additional EC members expressed their support for this idea (myself included) thus I bring it here for your consideration.

In order to respect everyone's value, each PR must have an approval before it gets merged. I will investigate how to put this in github. In the meanwhile, please respect this rule. If you feel this should not be forced, please have your concerns aired out now.