Comments (8)
I think this must come from the underlying YAML parsers.
Is it just the E notation floats, though? I think the immediate solution is to just switch to the decimal notation unfortunately.
from hyperpyyaml.
yeah it's just the E notation. Oh well, okay.
from hyperpyyaml.
Oh wait @Gastron it also happens with 0.00001
and 0.000001
, but not 0.0001
from hyperpyyaml.
Whoa. I was also able to reproduce with 0.00001
, and that it works fine with 0.0001
.
I did some digging and the cause is a little convoluted. This happens because the YAML gets loaded twice: once with ruamel.yaml, then the overloads get added, and then the whole thing gets dumped back as YAML, and loaded again with "bells and whistles" with PyYAML. The floats, when they are dumped out as YAML, probably get their basic Python string representation, which turns to scientific (E) notation at 0.00001.
And it is apparently a known issue that PyYAML does not parse the scientific notation right, see this issue: yaml/pyyaml#173
There's probably something we could do to the floats to dump them in the right format, or we could maybe patch the PyYAML float matching, see this nice StackOverflow answer
from hyperpyyaml.
This should be possible to fix in hyperpyyaml by adding a custom representer for floats. The default representer already has the needed lines of code
if getattr(self.serializer, 'use_version', None) == (1, 1):
if '.' not in value and 'e' in value:
# Note that in some cases `repr(data)` represents a float number
# without the decimal parts. For instance:
# >>> repr(1e17)
# '1e17'
# Unfortunately, this is not a valid float representation according
# to the definition of the `!!float` tag in YAML 1.1. We fix
# this by adding '.0' before the 'e' symbol.
value = value.replace('e', '.0e', 1)
but this part is not executed since getattr(self.serializer, 'use_version', None) == (1, 1)
does not return True.
It can therefore also be fixed by setting the version of the ruamel_yaml object which dumps after resolving references and changing overrides in resolve_references() by adding ruamel_yaml.version = (1, 1)
after the ruamel_yaml object is initialised. I tested this on my code, and it works, but I'm not sure if changing the version has other negative effects.
from hyperpyyaml.
Thanks @larsrgit, that's an interesting way to fix it! I wonder if this leaves out some cases where overrides get injected; they're not parsed by ruamel, and I am not sure if they get represented by this code or not. I'd imagine that patching the PyYAM-side parsing might be more robust.
from hyperpyyaml.
In my understanding of the hyperpyyaml code this should handle all cases where the e notation is introduced by the intermediate dumping.
So it would handle all cases where the user does not specify his variables (in the yaml or in overrides) in e notation without the dot. This way the user would at least have the full control.
But you are right, patching the pyyaml parsing during loading would be better in the way that it would also work if the user does not know that pyyaml requires the e-notation to be with a dot.
from hyperpyyaml.
This stack overflow answer should provide a template for solving this if anyone is interested https://stackoverflow.com/a/30462009
from hyperpyyaml.
Related Issues (16)
- Include statement documentation HOT 1
- Unexpected behaviour when using references with the results of !apply HOT 6
- hyperpyyaml sometimes loads scientific e notation as string HOT 2
- Overriding any value get parsed as a string HOT 3
- `!applyref` two functions at a time HOT 1
- tuple with `!ref` tag HOT 2
- Cannot include documents with object `!ref`s HOT 4
- Support new version of ruamel.yaml
- Saving config files with !new tags HOT 2
- `!include` statement with certain files not working HOT 1
- `hyperpyyaml/core.py:718: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead`
- Bug with anchors and `!apply` HOT 8
- pypi version has not been updated to git version HOT 1
- Could not `!ref` a `!apply` value HOT 1
- overrides not working properly for load_hyperpyyaml with !include
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hyperpyyaml.