Recursive includes do not work with nested directories about pyyaml-include

[sam-s]

Here is what works for me:

import yaml
class RecursiveLoader(yaml.FullLoader):  # pylint: disable=too-many-ancestors
    """YAML Loader with recursive `!include` constructor.
    https://gist.github.com/joshbode/569627ced3076931b02f
    https://stackoverflow.com/q/528281/850781"""
    def __init__(self, stream):
        try:
            self._root = os.path.dirname(stream.name)
        except AttributeError:
            self._root = os.path.curdir
        super().__init__(stream)
    def include(self, node):
        """Include the file referenced at the node."""
        filename = os.path.join(self._root, self.construct_scalar(node))
        extension = os.path.splitext(filename)[1]
        with open(filename, 'r') as fd:
            if extension in ('.yaml', '.yml'):
                return yaml.load(fd, Loader=self.__class__)
            if extension in ('.json', ):
                return json.load(fd)
            return fd.readlines()
yaml.add_constructor('!include', RecursiveLoader.include, RecursiveLoader)

Thanks!

[tanbro]

Recursive includes are complex ...

But if the problem is that the Tag can not get the pathname correctly, maybe absolute absolute pathname or use a base_dir for the Tag could solve:

The constructor searchs including files with iglob, from CWD.

And it's construct function has an argument base_dir, default to None.

• When it's set, the tag will load other YAML files with base_dir as prefix dir, eg:

  new a construcor:

  import yaml
  from yamlinclude import YamlIncludeConstructor
  
  constructor = YamlIncludeConstructor(base_dir='/your/conf/dir')
  yaml.FullLoader.add_constructor('!inc', constructor)

  sub-config files are in dir /your/conf/dir

  The entry YAML file's path is /path/of/app/config.yml , content is:

  bar: !inc bar.yml

  The constructor does things like this:

  base_dir = "/your/conf/dir"
  pathname = "bar.yml"
  pathname = os.path.join(base_dir, pathname)
  iterable = iglob(pathname, recursive=recursive)
  for path in filter(os.path.isfile, iterable):
        # load and parse bar.yml
        # ... ...

• When it's None, the tag will load other YAML files from CWD,

can this help?

Thanks!

[sam-s]

Thank you for your reply.
I think every include should be taken relative to the including file, not the top-level file.
Abs path in YamlIncludeConstructor(base_dir=..) will not help.

[tanbro]

Emm, each including file is relative to the file includes them ... it's more like XML Pointer or $ref in OpenAPI spec.

Also i met a similar situation recently - seperated YAML config files on S3.

i think https://tools.ietf.org/html/rfc6901 describes things for that.