Comments (8)
@ba1dr Thanks for your advice!
Yes, it's nice and natural to parse the relative file from where the current YAML is.
But i can not to get information about current file, as you wrote.
For the case of that "config files in different directories.", i think current API can't do that beautifully, but to use absolute path is workable.
from pyyaml-include.
jinjyaml is a Jinja2 template engine integration for PyYAML.
We can include files by Jina2's include
instruction:
Consider we have below YAML:
parent: !j2 |
{% include "child-1.yml" %}
{% include "child-2.yml" %}
then execute:
import jinja2
import jinjyaml
j2_env = jinja2.Environment(
loader = jinja2.FileSystemLoader(searchpath=your_base_dir)
)
j2_ctor = jinjyaml.Constructor()
yaml.add_constructor('!j2', j2_ctor)
doc = yaml.full_load(yaml_string)
data = jinjyaml.extract(doc, env=j2_env)
Jinja2's FileSystemLoader
would load child-1.yml and child-2.yml, relative to it's search path.
And we can even write a custom Jinja2 file loader, for particular purpose.
from pyyaml-include.
Hmm, no, I think Jinja2 would be an overkill. If using template engines - I'd better use config file on Python generated with Jinja2 rather than yaml.
Even with this include feature I am not sure if it is a good idea to use it as it breaks compatibility with other languages or scripts that do not support this tag..
from pyyaml-include.
Perhaps a YMAL's Json Pointer (if there be one) could be more fit for the case.
from pyyaml-include.
In case anyone's interested, my current workaround is this:
import contextlib
import os
import pathlib
import yaml
from yamlinclude import YamlIncludeConstructor
YamlIncludeConstructor.add_to_loader_class(loader_class=yaml.SafeLoader)
@contextlib.contextmanager
def working_directory(path: pathlib.Path):
prev_cwd = pathlib.Path.cwd()
os.chdir(path)
try:
yield
finally:
os.chdir(prev_cwd)
def load_config_file(file_path: pathlib.Path):
with working_directory(file_path.parent):
with file_path.open("r") as config_file:
return yaml.safe_load(config_file)
But obviously, the limitation is that any 2nd+ level include is relative to the first file, not any intermediate files, but luckily that's good enough for us right now 🙂
from pyyaml-include.
But i can not to get information about current file, as you wrote.
Actually @tanbro, you can! :)
One just had to change the base_dir
as they travel along, and extract the name of the file from the stream, then patch yaml.load
specifically.
EDIT: Updated the snippet, this is what we now use internally in an __init__.py
.
import yaml
from yamlinclude import YamlIncludeConstructor
YamlIncludeConstructor.add_to_loader_class(loader_class=yaml.FullLoader)
YamlIncludeConstructor.add_to_loader_class(loader_class=yaml.SafeLoader)
YamlIncludeConstructor.add_to_loader_class(loader_class=yaml.Loader)
YamlIncludeConstructor.add_to_loader_class(loader_class=yaml.BaseLoader)
include_tag = YamlIncludeConstructor.DEFAULT_TAG_NAME
yaml_load = yaml.load # Save original load function
def load_yaml(stream, Loader):
from pathlib import Path
path = Path(stream.name)
if include_tag not in Loader.yaml_constructors:
return yaml_load(stream, Loader=Loader)
previous_base = Loader.yaml_constructors[include_tag].base_dir
Loader.yaml_constructors[include_tag].base_dir = path.parent.as_posix()
res = yaml_load(stream, Loader=Loader)
Loader.yaml_constructors[include_tag].base_dir = previous_base
return res
yaml.load = load_yaml # Use new one
del YamlIncludeConstructor
del yaml
from pyyaml-include.
The above would fail on strings (if used with e.g. yaml.load(f.read())
, or some local definitions).
One can add an isinstance(stream, io.TextIOWrapper)
for validation as needed.
EDIT: Like so:
yaml_load = yaml.load # Save original load function
def load_yaml(stream, Loader):
from pathlib import Path
from yamlinclude import YamlIncludeConstructor
from io import TextIOWrapper
tag = YamlIncludeConstructor.DEFAULT_TAG_NAME
if tag not in Loader.yaml_constructors or not isinstance(stream, TextIOWrapper):
# If tag is included in the stream but we can't get the file location, we can't assume
# anything about the relative file location
return yaml_load(stream, Loader=Loader)
path = Path(stream.name)
previous_base = Loader.yaml_constructors[tag].base_dir
Loader.yaml_constructors[tag].base_dir = path.parent.as_posix()
res = yaml_load(stream, Loader=Loader)
Loader.yaml_constructors[tag].base_dir = previous_base
return res
yaml.load = load_yaml # Use new one
@ba1dr @1ace You might be interested ^
from pyyaml-include.
#26 provides a way to include files relatively
from pyyaml-include.
Related Issues (20)
- Python 3.6 unsupported after releasing `1.2.post1` HOT 2
- Publish sdist to PyPI HOT 2
- two files in one dict? HOT 8
- Include and achors HOT 3
- include specific components from other yamls? HOT 2
- Does not support top level include with other keys HOT 2
- Could you update the PyPI to new version to add PyYAML 6.0 support? HOT 2
- How to concatenate two yaml files? HOT 1
- Ability for default value HOT 1
- Wildcard with Hash/Dict/Map HOT 1
- Idea: include from remote storages (notably git) HOT 3
- Yaml anchor support for include yaml files HOT 1
- Use tag with anchors across a single yaml config file HOT 2
- It seems that I can't rewrite the inherited argments from base `yaml` file. HOT 5
- Python 3.10 unsopported? HOT 2
- yaml writer? HOT 1
- How to make the package work with setuptools lower than 61.0.0 HOT 4
- The library looks to by typed, but the `py.typed` file is missing HOT 2
- Can we use alias with include? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyyaml-include.