Comments (13)
It's coming! It'll probably look something like this: #23 (comment).
from parsimonious.
Suppress and Select are two pieces of sugar that would be nice. Perhaps a recursive flattener like in PyPy would help as well. I'll do a survey of Parsimonious grammars in the wild and see what pops out.
Spelling strawman:
foo = "," <something_important> ","
bar = >something_boring< something_important >another_boring_thing< also_important third_important
multiple = <multiple selected? terms> "boring"
from parsimonious.
Transforms should also (do I even need to mention it?) be heritable, so we can suppress whitespace once and have it stick elsewhere:
_ = >" "+<
things = thing _ thing _ thing # would return just thing, thing, thing
from parsimonious.
<<thing>>
or >>thing<<
are possibilities for recursive flattener thing spelling. Haven't thought them out at all yet, but it took me months to think of them, apparently. They're nice because then all the transforms are just >s and <s.
However, I probably won't implement flattening until I see some compelling use cases. RPython's page gives a right-recursive grammar as an example, and their example, at least, is much simpler represented using a quantifier.
from parsimonious.
In summary, I can think of 4 transforms sufficiently useful to include:
- Igoring
- Selecting
- Flattening
- Lifting
from parsimonious.
Writing and testing grammars in Parsimonious has been gloriously simple, but transforming the parse tree has so far been difficult. For example, I'm trying to write a general JSON-izer for Parsimonious parse trees that just spits out node info for the named rules (not anonymous literal matches, for example), and I'm having trouble doing it by implementing NodeVisitor.generic_visit
.
It would be a huge help to have more explicit support for transforms like lifting and ignoring (#29, #36), or some documentation/examples about how to implement them myself (#99 , #48). Are there plans to visit (no pun intended) these issues in the near-future? Alternately, could you link to any snippets of Parsimonious code that do things like the ignoring or lifting?
from parsimonious.
Thanks for asking! First, I'm going to be talking about Parsimonious on Podcast.__init__ in the next month or two, so I'm going to try to get some real docs in place before then.
Second, I haven't had much time to hack on Parsimonious lately, but tree transforms are the next thing on the roadmap after switching the precedence of /
and (space)—that is, ors and sequences—to match what every other PEG parser does, including the original paper.
Finally, there's lot of ignoring going on in my grammar in DXR: see https://github.com/mozilla/dxr/blob/master/dxr/query.py#L309, which takes a term like "+foo:bar". It looks at plus
and filter
, but it completely ignores colon
, and the colon propagates no further up the visitation chain.
Does that help?
from parsimonious.
Sweet! I enjoyed listening to you on Talk Python to Me, and I'm looking forward to your next interview.
I think I understand the ignoring happening at your link; thank you. What do you do if you look at your node or its children, and decide you wanted to ignore everything? Simply return
ing will return None
, but that's still something. For example, in the JSON-izer I'm playing with, generic_visit(node, children)
looks at node.expr_name
, and returns nothing if that is empty. None
becomes null
in json.dumps
, though, and I'm trying to pretend it was never visited (or possibly lift its children, if found). I think this is a kind of tree reduction; I'm just not sure how to implement it.
from parsimonious.
What do you do if you look at your node or its children, and decide you wanted to ignore everything? Simply returning will return None, but that's still something.
You want the node to not exist in the tree? Arrange to lop it off in the visitor for the parent of that node. For instance, when it sees None
come back from visit_some_child_thing(), have it omit that item from its return value.
from parsimonious.
(I didn't mean to hijack this issue. It seemed like the right place for this.) I think I could JSON-ize any parse tree, regardless of rule structure, so I'm only using generic_visit
. There is no visit_some_child_thing
. This feels do-able, with some better understanding of how to prune generically.
Here's a gist illustrating the visitor using your "(( bold stuff ))" example. I think the "expr_name": "text"
node should be an only child, as "((" and "))" are anonymous siblings, but instead it has the two null
s. This seems like an ignoring operation.
For instance, when it sees None come back from visit_some_child_thing(), have it omit that item from its return value.
Ah, I should "de-None
" the children. Got it, thank you!
If this visitor encounters an anonymous parent with named children, it returns the children (a list of dictionaries), when I really just intend to lift the children to the level of the parent, which sounds like a simple lifting operation. Still not sure how to implement this one...
Thanks for your responsiveness.
from parsimonious.
I think you're almost there. It looks like you now have "uninteresting" nodes returning None. Now if you simply filter out the Nones from out
before you return it, you might be in business.
from parsimonious.
That was it. And, as far as lifting children, that was as simple as flattening a children
list that may contain lists. Thank you for your help!
from parsimonious.
from parsimonious.
Related Issues (20)
- Use mypyc when building wheels HOT 1
- Repetition doesn't seem to work HOT 1
- Newest releases missing in conda-forge HOT 5
- What is the official file name extension for grammars in `parsinomous` DSL? HOT 5
- Raise an exception from the grammar? HOT 3
- How to stop consuming after find reserved word
- Python 3.11 : cannot import name 'getargspec' from 'inspect' HOT 3
- Or operator issue HOT 3
- Feature Request: Next Valid Token
- Struggling with what I thought should be a simple grammar HOT 1
- requires regex package HOT 9
- Bug: Unexpected ParserError HOT 1
- How to handle nodes with identical names at different levels of the tree? HOT 2
- INI parsing example does not work for files starting with empty lines HOT 1
- How to search for substring with grammar?
- IncompleteParseError
- How to get better parse errors?
- want to skip escaped newlines in input
- [request] support repeated elements with separators (or intercalated matches)
- [request] support lazy and possessive quantifiers
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from parsimonious.