puppetlabs / cpp-hocon

A C++ port of the Typesafe Config library.

C++ HOCON Parser

This is a port of the TypesafeConfig library to C++.

The library provides C++ support for the HOCON configuration file format.

          MMMMMMMMMMMMMMMMMMMM
         .====================.
         MMMMMMMMMMMMMMMMMMMMMM
        .MMMMMMMMMMMMMMMMMMMMMM.
    ===.7MMMIN7NMMMMMMMMM7M=MMMM,===
    MMM.7MM:     DMMMMM7    :MMM=MMM
    MMM.7MM,     DMMMMM?    ~MMM=MMM
    MMM.7MMM~++~?MMMMMMM~++~MMMM=MMM
        .MMMMMMMMMMMMMMMMMMMMMM.
         MMMMMMMMMMMMMMMMMMMMMM
         MMMMMMMMMMMMMMMMMMMMMM
         MMM      HOCON     MMM
         MMMMMMMMMMMMMMMMMMMMMM
          .MMMMMMMMMMMMMMMMMM.
          .MMMMMMMMMMMMMMMMMM.
       .MMMMMMMMMMMMMMMMMMMMMMMM
 .    MMMMMMMMMMMMM88MMMMMMMMMM8MM    .
7=MMMMMMMM++ A CONFIG FILE  ++M8MMMMMM7=
M=MMMMMMMM+ FORMAT DESIGNED  +M8MMMMMM7M
M=MMMMMMMM++  FOR HUMANS   ++M8MMMMMM7M
 =MMMMMMMMMMMMMMMMM88MMMMMMMMMM8MMMMMM7
  7MM.88MMMMMMMMMMM88MMMMMMMMMMO88 MM8
  7MM   MMMMMMMMMMM88MMMMMMMMMM8   MM8
  7MM   MMMMMMMMMMM88MMMMMMMMMM8   MM8
  7MM   MMMDMMMM?MM88MM?MMMMOMM8   MM8

To get started, install it, then to parse a file:

#include <hocon/parser/config_document_factory.hpp>
#include <fstream>

using hocon::config_document_factory::parse_file;

int main(int argc, char** argv) {
    auto doc = parse_file("file.conf");
    doc = doc->with_value_text("a", "42");

    std::ofstream out("file.conf");
    out << doc->render();
    return 0;
}

If you build cpp-hocon with -DBUILD_SHARED_LIBS=ON, then the example can be built with

c++ example.cc -o example -std=c++11 -lcpp-hocon

You can use hocon::config_document_factory::parse_string to parse a string. config_document is used to modify a file while preserving all formatting. Use config to read from the config or if you don't care about preserving formatting.

Note that file extensions matter. A .conf file will be parsed as HOCON, a .json file will be parsed as JSON, and other extensions will be ignored.

See the docs for more.

Caveats

This is a mostly complete implementation of the HOCON format. It currently has some known limitations

• Include requires the location specifier, i.e. include "foo" won't work but include file("foo") will. URL is not yet implemented, and classpath won't be supported as it makes less sense outside of the JVM.
• Unicode testing is absent so support is unknown. There are likely things that won't work.

Install

Build Requirements

• OSX or Linux
• GCC >= 4.8 or Clang >= 3.4 (with libc++)
• CMake >= 3.2.2
• Boost Libraries >= 1.54
• Leatherman

Pre-Build

Prepare the cmake release environment:

$ mkdir release
$ cd release
$ cmake ..

Optionally, also prepare the debug environment:

$ mkdir debug
$ cd debug
$ cmake -DCMAKE_BUILD_TYPE=Debug ..

Building

1. Enter your build environment of choice, i.e. cd release or cd debug
2. make
3. (optional) install with make install

Testing

Run tests with make test.

cpp-hocon's Issues

Resolving inheritence/overloading

Continuing with highlighting the differences with typesafe-config.

Given file test-2.conf:

Common {
    common = "common"
}

Foo = ${Common}                                                                                                                                                                                
Foo = {
    home = "/tmp"
    home = ${?HOME}
}

it fails to resolve with "resolve_substitutions() did not give us a resolved object" (it works with typesafe-config).

For the record, this variant does not work in neither of them, though I was expecting it to work:

Common {
    common = "common"
}

Foo = ${Common} {
    home = "/tmp"
    home = ${?HOME}
}

It fails with:

# simple_config_object::replace_child did not find {
    # file: test-2.conf
    # unresolved merge of 2 values follows (
    #     unmerged value 0 from file: test-2.conf
    "home" : "/tmp",
    #     unmerged value 1 from file: test-2.conf
    "home" : ${?HOME}
    # ) end of unresolved merge

}

and

com.typesafe.config.ConfigException$BugOrBroken: SimpleConfigObject.replaceChild did not find SimpleConfigObject({"home":"/tmp","home":${?HOME}}) in SimpleConfigObject({"Common":{"common":"common"},"Foo":${Common}{"home":"/tmp","home":${?HOME}}})

And this variant works in both of them but is more verbose:

Common {
    common = "common"
}

Foo-content {
    home = "/tmp"
    home = ${?HOME}
}
Foo = ${Common} ${Foo-content}

Fails to build with Boost 1.72

Describe the Bug

Regressed by boostorg/filesystem@68ec5b1fb66b. From error log:

[100%] Linking CXX executable ../../bin/libcpp-hocon_test
ld: error: undefined symbol: typeinfo for boost::filesystem::filesystem_error
>>> referenced by parseable.cc
>>>               ../CMakeFiles/libprojectsrc.dir/src/parseable.cc.o:(.data+0x0)
c++: error: linker command failed with exit code 1 (use -v to see invocation)

Expected Behavior

Builds fine

Steps to Reproduce

1. cmake .
2. make

Environment

• Boost 1.72 snapshot (downstream)
• FreeBSD 12.1 amd64

Additional Context

Can be worked around by removing the following line

cpp-hocon/lib/CMakeLists.txt

Line 94 in cdfa711

add_subdirectory(tests)

"include" with files crashes

This test program:

    string
        fn1 = "/tmp/prod-msgw.xx"s,
        fn2 = "/tmp/services.conf"s;
    ofstream included(fn1.c_str());
    included << "{ \"b\":\"Hi!\" }}" << endl;
    included.close();

    ofstream includer(fn2.c_str());
    includer << "{ include file(\"" << fn1 << "\" )}" << endl;
    includer.close();

    auto cfg = hocon::config::parse_file_any_syntax(fn2)->resolve();

...leaves this stack:

#0  0x000000000089e47a in hocon::config_parse_options::set_syntax(config_syntax) const ()
#1  0x00000000008ac0c8 in hocon::simple_includer::clear_for_include(hocon::config_parse_options const&) ()
#2  0x00000000008aa7fc in hocon::simple_include_context::parse_options() const ()
#3  0x00000000008aafe9 in hocon::simple_includer::include_file_without_fallback(std::shared_ptr<hocon::config_include_context const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) ()
#4  0x00000000008ab85f in hocon::simple_includer::include_file(std::shared_ptr<hocon::config_include_context const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) const ()
#5  0x00000000008ab48f in hocon::simple_includer::proxy::include_file(std::shared_ptr<hocon::config_include_context const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) const ()
#6  0x00000000008f167a in hocon::config_parser::parse_context::parse_include(std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<hocon::config_value const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<hocon::config_value const> > > >&, std::shared_ptr<hocon::config_node_include const>) ()
#7  0x00000000008f42f3 in hocon::config_parser::parse_context::parse_object(std::shared_ptr<hocon::config_node_object const>) ()
#8  0x00000000008f6494 in hocon::config_parser::parse_context::parse_value(std::shared_ptr<hocon::abstract_config_node_value const>, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) ()
#9  0x00000000008f8989 in hocon::config_parser::parse_context::parse() ()
#10 0x00000000008f8d76 in hocon::config_parser::parse(std::shared_ptr<hocon::config_node_root const>, std::shared_ptr<hocon::config_origin const>, hocon::config_parse_options, std::shared_ptr<hocon::config_include_context const>) ()
#11 0x00000000008a17c5 in hocon::parseable::raw_parse_value(std::unique_ptr<std::istream, std::default_delete<std::istream> >, std::shared_ptr<hocon::config_origin const>, hocon::config_parse_options const&) const ()
#12 0x00000000008a1f43 in hocon::parseable::raw_parse_value(std::shared_ptr<hocon::config_origin const>, hocon::config_parse_options const&) const ()
#13 0x00000000008a5ce8 in hocon::parseable::parse_value(std::shared_ptr<hocon::config_origin const>, hocon::config_parse_options const&) const ()
#14 0x00000000008a67cd in hocon::parseable::parse_value(hocon::config_parse_options const&) const ()
#15 0x00000000008a6c19 in hocon::parseable::parse(hocon::config_parse_options const&) const ()
#16 0x00000000008ad31f in hocon::simple_includer::from_basename(std::shared_ptr<hocon::name_source>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, hocon::config_parse_options) ()
#17 0x0000000000831bd3 in hocon::config::parse_file_any_syntax(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, hocon::config_parse_options) ()
#18 0x0000000000831cf8 in hocon::config::parse_file_any_syntax(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) ()

Example doesn't compile, and segfaults when you fix the compilation issues

The example in https://github.com/puppetlabs/cpp-hocon/blob/6ba46c077facd9339115f075d1521d20584826fd/README.md doesn't compile:

• write_to_file isn't found in leatherman.file_util
• should be doc->, not doc.
• result of with_value_text is a new document that's ignored
• unclear how to link against cpp-hocon

Once you fix those things and compile the example, it segfaults in with_value_text.

Valgrind errors (no observable impact)

Just wanted to point out that cpp-hocon is getting some errors in valgrind. I didn't try to track it down any further, as it would require a debug build and is easy to reproduce.

This code:

 auto conf = config::parse_string(R"(
                                         foo : 10
                                         bar : The quick brown fox
        )");

, when run under valgrind produces:

==103534== Conditional jump or move depends on uninitialised value(s)
==103534==    at 0x99913B: hocon::config_document_parser::parse_context::parse_value(std::shared_ptr<hocon::token const>) (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x999679: hocon::config_document_parser::parse_context::consolidate_values(std::vector<std::shared_ptr<hocon::abstract_config_node const>, std::allocator<std::shared_ptr<hocon::abstract_config_node const> > >&) (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x9973AC: hocon::config_document_parser::parse_context::parse_object(bool) (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x99B99A: hocon::config_document_parser::parse_context::parse() (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x99C6A7: hocon::config_document_parser::parse(hocon::token_iterator, std::shared_ptr<hocon::config_origin const>, hocon::config_parse_options) (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x95223C: hocon::parseable::raw_parse_value(std::unique_ptr<std::istream, std::default_delete<std::istream> >, std::shared_ptr<hocon::config_origin const>, hocon::config_parse_options const&) const (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x95292C: hocon::parseable::raw_parse_value(std::shared_ptr<hocon::config_origin const>, hocon::config_parse_options const&) const (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x9567C7: hocon::parseable::parse_value(std::shared_ptr<hocon::config_origin const>, hocon::config_parse_options const&) const (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x9572A8: hocon::parseable::parse_value(hocon::config_parse_options const&) const (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x957F0A: hocon::parseable::parse() const (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x8E0631: hocon::config::parse_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, hocon::config_parse_options) (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)
==103534==    by 0x8E0777: hocon::config::parse_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /home/jharris/.CLion2016.2/system/cmake/generated/cpp-2a0ba01b/2a0ba01b/Debug/out/mc/bin/maintests)

Self-Referential Substitutions don't work

I'm using the example from documentation: https://github.com/lightbend/config/blob/master/HOCON.md#examples-of-self-referential-substitutions

A self-reference resolves to the value "below" even if it's part of a path expression. So for example:

foo : { a : { c : 1 } }
foo : ${foo.a}
foo : { a : 2 }

Here, ${foo.a} would refer to { c : 1 } rather than 2 and so the final merge would be { a : 2, c : 1 }.
Recall that for a field to be self-referential, it must have a substitution or value concatenation as its value. If a field has an object or array value, for example, then it is not self-referential even if there is a reference to the field itself inside that object or array.

And I'm getting crash:
terminate called after throwing an instance of 'hocon::bug_or_broken_exception' what(): resolve_substitutions() did not give us a resolved object
Also I'm getting crash in cases below:

a = {aa = abba}
a.aa = mama
b = ${a} { bb = mia }
# Everything below is failing
b.aa = mea
b.bb = culpa
b = { aa = mea, bb = culpa }
c = ${b.aa}
c = ${b.bb}

Looks like this library implementation doesn't follow the HOCON documentation...

include quoted string

Reporting a few issues with including the configuration files. The behavior is compared with the reference implementation (typesafe-config)

First, here is the example config files:

test-1.conf

Foo {
    foo = "foo"
}

Bar {
    include "bar"
}

bar.conf:

bar = "bar"

Using this command (scala) I get the same result for all the cases described below:

scala> println(ConfigFactory.parseResourcesAnySyntax("test-1.conf").resolve().root().render() )
{
    # test-1.conf @ file: test-1.conf: 5
    "Bar" : {
        # bar.conf @ file: bar.conf: 1
        "bar" : "bar"
    },
    # test-1.conf @ file: test-1.conf: 1
    "Foo" : {
        # test-1.conf @ file: test-1.conf: 2
        "foo" : "foo"
    }
}

From C++ side I am using:

        try {
            auto cfg = hocon::config::parse_file_any_syntax(filename)->resolve();
            std::cout << "ok - validate " << filename << std::endl;
            std::cout << cfg->root()->render() << std::endl;
        } catch (const hocon::config_exception &e) {
            std::cout << "not ok - validate " << filename << std::endl;
            std::cout << "# " << e.what() << std::endl;
             failed--;
        }                                                                                                                           

with the same file from the same directory, the result is:

ok - validate test-1.conf
{
    # file: test-1.conf
    "Bar" : {},
    # file: test-1.conf
    "Foo" : {
        # file: test-1.conf
        "foo" : "foo"
    }
}

(the content of the "Bar" section is missing).

using instead:

Bar {
    include "bar.conf"                                                                                                                                                                         
}

I get an exception: "reader() should not be called on resources".

Using include file("bar") or include file("bar.conf") works as expected

How can I merge two configs?

I'm able to create parse one existing config using hocon::config::parse_string, but when I use with_fallback so I can merge it with other config, it returns config_mergeable which doesn't contain any public methods to be rendered to std::string. I also could not find any example of how to do that here. Maybe it's not supported by this library?

Appreciate all the help!

Array at root?

If the JSON I want is an array at the top level, how do I parse that from a HOCON string?

I get an exception when parsing "[ { op: replace } ]"
string: has type list rather than object at file root

Automatically regenerate gh-pages branch on release

Use https://github.com/marketplace/actions/doxygen-action to automatically generate regenerate docs in the gh-pages branch on release.

We probably need to stop templating Doxyfile and namespaces.dox for this to work well.

included file root

Continuation of the #94, it's different problem but the setup is the same (so skipping the Scala/C++ code).

if we have the config files:
test-1.conf:

Foo {
    foo = "foo"
}

Bar {
    include file("bar")                                                                                                                                                                        
}

bar.conf:

bar = "bar"
doubleBar = ${bar}

it fails the substitution with: "Could not resolve subtitution to a value: ${bar}".

If we use instead:

doubleBar = ${Bar.bar}

it works and the substitution is performed.

In the reference implementation (typesafe-config) both cases succeed.

Some basic examples needed

Thanks for the work on this library. I really need a few examples of the API in use. The unit tests aren't much help, and the Doxygen doesn't show any usage examples.

For instance, what's unwrapped mean? And am I supposed to just downcast config_value to the proper type in a big switch ?

Why is include name hocon/xxx but library is libcpp-hocon?

Normally the library onject and header names are consistent -- is this an accident? Are you likely to change this?

I can happily make a pull request but if this would require other changes to your process so that you would ignore it, I won't bother. (I understand the priority of cpp-hocon to puppetlabs is not high -- understandable)

Is there a way to write configuration parameters to files ?

I appreciate the work you have done to provide a C++ support for the HOCON configuration file format. I wonder if there is API available to write configuration parameters to files at runtime or any schedules about this.

the include command causes an error

Hi folks,

If I try to parse a .conf file with an include directive I get the error Error: reader() should not be called on resources

I am calling hocon::program_options::parse_file(filename, reasoner_options, true);

I'm running on MacOS.

Also I'm not sure how to specify the search directories for "include" though at the moment we have a single flat directory so this doesn't matter.

Tests fail to compile with pre-release glibc and leatherman-vendored Catch

Describe the Bug

In the next version of glibc, 2.34, SIGSTKSZ will be changed from a preprocessor constant to an expression involving a runtime function call. This has broken a lot of software that used it as a stack or static buffer size.

In leatherman, a copy of the Catch unit test framework (version 1.x rather than the current 2.x) is bundled in the “vendor” area. It is part of the public API, and cpp-hocon uses it for its unit tests. Since the SIGSTKSZ change, the cpp-hocon build fails with:

In file included from /usr/include/signal.h:315,
                 from /usr/include/leatherman/vendor/catch.hpp:6448,
                 from /builddir/build/BUILD/cpp-hocon-0.3.0/lib/tests/main.cc:2:
/usr/include/leatherman/vendor/catch.hpp:6471:33: error: size of array 'altStackMem' is not an integral constant-expression
 6471 |         static char altStackMem[SIGSTKSZ];
      |                                 ^~~~~~~~
/usr/include/leatherman/vendor/catch.hpp:6522:45: error: size of array 'altStackMem' is not an integral constant-expression
 6522 |     char FatalConditionHandler::altStackMem[SIGSTKSZ] = {};
      |                                             ^~~~~~~~

Expected Behavior

The unit tests build and work

Steps to Reproduce

Steps to reproduce the behavior:

1. Find a platform with a pre-release version of glibc 2.34 or later, such as Fedora Rawhide
2. Build cpp-hocon

Environment

• Version 0.3.0
• Platform Fedora Rawhide/35 and 34

Additional Context

From the glibc 2.34 release notes:

* Add _SC_MINSIGSTKSZ and _SC_SIGSTKSZ.  When _SC_SIGSTKSZ_SOURCE or
  _GNU_SOURCE are defined, MINSIGSTKSZ and SIGSTKSZ are no longer
  constant on Linux.  MINSIGSTKSZ is redefined to sysconf(_SC_MINSIGSTKSZ)
  and SIGSTKSZ is redefined to sysconf (_SC_SIGSTKSZ).

Here is the upstream Catch issue for both 1.x and 2.x: catchorg/Catch2#2178

I am the maintainer of the cpp-hocon package in Fedora Linux. I have worked around this by patching the CMakeLists files to use the system copy of catch (catch1-devel in Fedora), which contains a workaround.

It would be great if there were a CMake Boolean I could set to use an external or system copy of Catch without patching the build system. I would prefer to do that even after leatherman fixes its copy of Catch.

releases aren't buildable

The release tarballs only have the lib and include directories, but not top level files such as (critically) CMakeLists.txt.

$ curl -sL https://github.com/puppetlabs/cpp-hocon/releases/download/0.1.4/cpp-hocon.tar.gz | tar tz | grep CMakeLists.txt
$

Good reference identified as "part of a cycle of substitutions"

Describe the Bug

The following configuration snippet

base {
}

app = ${base} {
  a {
    b = 1
  }
  c = [
    ${app.a.b}
  ]
}

was complained as "${app.a.b} was part of a cycle of substitutions."

Expected Behavior

The above configuration is OK.

Steps to Reproduce

Steps to reproduce the behavior:

	auto config = hocon::config::parse_string(R"(
base {
}

app = ${base} {
  a {
    b = 1
  }
  c = [
    ${app.a.b}
  ]
}
)"
	)->resolve();

Environment

• Version [cpp-hocon 0.3.0]
• Platform [Ubuntu 18.04, g++ 10.3]