Comments (5)
Right now there is no mechanism for terms precedence.
The simplest one which is just whichever is first passed in terms(...) call in parser definition should have the biggest precedence shouldn't be hard to implement.
I will consider adding the feature in near future.
from ctpg.
After a brief analyze of the code I can tell that the term that is first on the terms(...) call in parser definition is actually matched first.
So I can't see the problem here, can you please provide some code?
Is "u" actually separate term? Why? It seems it should be part of the number term.
from ctpg.
I'm trying to write a parser for Java's MANIFEST.MF format (spec) and this behavior would be useful for me:
B ) If there is no suitable rule for the matched term, fallback and see if there is a suitable term (case: Unexpected Identifier)?
A MANIFEST.MF file looks like:
Manifest-Version: 1.0
Implementation-Title: foobar
Implementation-Version: 1.2.3
The language specification requires the value for Manifest-Version
to be in a specific format:
version-info: Manifest-Version : version-number
version-number: digit+{.digit+}*
So I use the following regex_term
:
// version-number: digit+{.digit+}*
// digit: {0-9}
constexpr char version_number_pattern[] = "[0-9](\\.[0-9])*";
constexpr regex_term<version_number_pattern> version_number("version-number");
And the rule:
version_info("Manifest-Version", ':', ' ', version_number)
For other key-value pairs, I use a more general regex_term
:
// otherchars: any UTF-8 character except NUL, CR and LF
// I match these one at a time, unfortunately adding a * to make [^\r\n]* segfaults gcc. But that's another issue.
constexpr char otherchar_pattern[] = "[^\\r\\n]";
constexpr regex_term<otherchar_pattern> otherchar("otherchar");
My problem is that the version-number
term is a special case of the otherchar
term. If I list version-number
first in the terms(...)
call, then MyKey: 1
would not parse correctly since the parser parses the 1
as a version-number
, not as otherchar
, and errors Syntax error: Unexpected 'version-number'
.
If I instead list otherchar
first in the terms
list, then I get the opposite error whenever my grammar expects a version-number
: Syntax error: Unexpected 'otherchar'
.
I don't have much experience with parsing, so I may be simply misunderstanding something.
from ctpg.
First of all, if your language doesn't do any specific whitespace processing (like indentation sensitive languages) you should not include them in the grammar at all, ctpg generated parsers skip whitespace by default.
I would do it like you do, with version-number term defined first so it has precedence, and simply include the version-number term in a syntax rule for 'other key value pairs'. Can you show me the grammar as well?
from ctpg.
First of all, if your language doesn't do any specific whitespace processing (like indentation sensitive languages) you should not include them in the grammar at all, ctpg generated parsers skip whitespace by default.
Thanks for your reply. Yes, I might be able to get by with only parsing newlines, but the grammar does not have an equivalent of semicolons, lines are terminated only with newlines.
I would do it like you do, with version-number term defined first so it has precedence, and simply include the version-number term in a syntax rule for 'other key value pairs'.
I think I see what you're saying. In general though, this could be a lot of manual work. Correct me if I'm wrong, but you'd need to:
- For each terminal
a
in the terms list:- Create a nonterminal
A
with typestring
and the ruleA(a) >= create<string>()
- In the existing rules, replace occurrences of
a
withA
- For each terminal
b
that precedesa
:- If there exists an instance of
a
that could be recognized asb
(e.g. in my case,a = otherchar
,b = version-string
): - Add the rule
A(b) >= create<string>()
- If there exists an instance of
- Create a nonterminal
Can you show me the grammar as well?
Sure, here is the parser I've got so far: https://gist.github.com/evan-goode/a5d97348f96328c956792c0660e414b0. And the grammar specification is here: https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Manifest_Specification.
from ctpg.
Related Issues (20)
- Make tests and examples executables conditional HOT 6
- Constexpr parsing: non-constexpr containers HOT 5
- Parsing exponent notation in json-parser HOT 3
- Ignore all whitespaces but newlines HOT 3
- string_view buffer HOT 2
- [question] Cant Parameterize and "Seperate out" Parser HOT 7
- [question] Trouble Making Argument Lists HOT 1
- [Question] Is it applicable to parsing markdown? HOT 2
- [Question] Did I exceed the maximum number of rules? HOT 1
- [Feture Request] Integration with CTRE HOT 3
- how long to next release ? HOT 8
- Maybe it is possible that removing lexer scanner? HOT 2
- Big Parsers? HOT 1
- Constexpr parsing of an array of ints HOT 6
- [Question] how to make the lexer NOT match a particular string HOT 2
- [question solved] constexpr variable 'p' must be initialized by a constant expression HOT 1
- Simple expression example and AST HOT 3
- [MSVC] Compilation error, unable to resolve operator overload of `ctpg::stdex::cbitset<N>::operator==` HOT 6
- Documentation wrong? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ctpg.