GithubHelp home page GithubHelp logo

sirthias / parboiled Goto Github PK

View Code? Open in Web Editor NEW
1.3K 52.0 157.0 63.1 MB

Elegant parsing in Java and Scala - lightweight, easy-to-use, powerful.

Home Page: http://parboiled.org

License: Apache License 2.0

Java 80.12% Scala 19.88%

parboiled's Introduction

parboiled's People

Contributors

based2 avatar bgalek avatar brjeter avatar butlermh avatar cleishm avatar cwensel avatar deining avatar dependabot[bot] avatar domsj avatar github-actions[bot] avatar gkossakowski avatar iinegve avatar jboynes avatar jrudolph avatar kenwenzel avatar leachbj avatar scala-steward-bot avatar sethtisue avatar simonwibberley avatar simy4 avatar sirthias avatar slide avatar tomaswolf avatar vpeurala avatar xuwei-k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

parboiled's Issues

ASM 4.0

Hi,

In the setup/download instructions it says that ASM 3.3.1 is required. Is ASM 4.0 supported? We've moved on to ASM 4.0 and would like to include parboiled.

Thanks.

Problems with abc example

It is quite easy to get parboiled to throw an IllegalStateException with the a^n b^n c^n example, e.g. with input abbbbbcc or aaabbbbbbbbcc:

Enter an a^n b^n c^n expression (single RETURN to exit)!
aaabbbbbbbbcc
Exception in thread "main" java.lang.IllegalStateException
at org.parboiled.common.Preconditions.checkState(Preconditions.java:119)
at org.parboiled.parserunners.RecoveringParseRunner.run(RecoveringParseRunner.java:149)
at org.parboiled.parserunners.AbstractParseRunner.run(AbstractParseRunner.java:81)
at org.parboiled.parserunners.AbstractParseRunner.run(AbstractParseRunner.java:76)
at flow.test.ParserTest.main(ParserTest.java:46)

The exception is thrown here:

public ParsingResult<V> run(InputBuffer inputBuffer) {
    // [...]
    if (!lastParsingResult.matched) {
        // [...]
        if (!getRootMatcher().isNodeSuppressed()) {
            performFinalRun();
            checkState(lastParsingResult.matched);   // <-- here
        }
    }
    return lastParsingResult;
}

I have just started using parboiled, but from what I can tell, an internal exception of this kind should not be thrown here. Is this correct?

Furthermore, it is possible to get the parser to seemingly accept invalid input as valid, e.g. aabbbcc gets read as aabbcc:

Enter an a^n b^n c^n expression (single RETURN to exit)!
aabbbcc
aabbbcc = null

[S]E 'aabbcc'
  [OneOrMore] 'aa'
    ['a'] 'a'
    ['a'] 'a'
  [B]E 'bbcc'
    ['b'] 'b'
    [Optional]E 'bc'
      [B]E 'bc'
        ['b'] 'b'
        [Optional]
        ['c'] 'c'
    ['c'] 'c'

Support for ACTION in Groovy

Currently the use of action in parsers written in Groovy classes throws an exception.

I think that's because Parboiled.createParser() can't do anything about BaseParser.ACTION() with Groovy classes/objects. I mean this class extension stuff.

Without push("fin") it works just fine (see ExampleGroovyParser.groovy below). For comparision I also wrote an ExampleJavaParser.java.

Exception in thread "main" org.parboiled.errors.GrammarException: Illegal rule definition: Unwrapped action expression!
    at org.parboiled.support.Checks.ensure(Checks.java:49)
    at org.parboiled.BaseActions.check(BaseActions.java:435)
    at org.parboiled.BaseActions.push(BaseActions.java:172)
    at org.parboiled.BaseActions$push.callCurrent(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:44)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:149)
    at ExampleGroovyParser.start(ExampleGroovyParser.groovy:12)
    at ExampleGroovyParser$$parboiled.start(Unknown Source)
    at ExampleGroovyParser$$parboiled$start.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
    at ExampleGroovyParser.main(ExampleGroovyParser.groovy:16)

Groovy class (ExampleGroovyParser.groovy)

import org.parboiled.BaseParser
import org.parboiled.Parboiled
import org.parboiled.Rule
import org.parboiled.annotations.BuildParseTree
import org.parboiled.parserunners.ReportingParseRunner
import org.parboiled.support.ParseTreeUtils

@BuildParseTree
class ExampleGroovyParser extends BaseParser {
    // start = "a"+ "b"+
    Rule start() { Sequence(OneOrMore(String("a")), 
                   OneOrMore(String("b")), push("fin")) }

    static void main(String[] args) {
        ExampleGroovyParser parser = Parboiled.createParser(ExampleGroovyParser)
        def parseRunner = new ReportingParseRunner(parser.start())

        def input = "aabbb"
        def result = parseRunner.run(input)
        println ParseTreeUtils.printNodeTree(result)
    }
}

Java class (ExampleJavaParser.java)

import org.parboiled.BaseParser;
import org.parboiled.Parboiled;
import org.parboiled.Rule;
import org.parboiled.annotations.BuildParseTree;
import org.parboiled.parserunners.ReportingParseRunner;
import org.parboiled.support.ParseTreeUtils;
import org.parboiled.support.ParsingResult;

@BuildParseTree
public class ExampleJavaParser extends BaseParser<Object> {
    // start = "a"+ "b"+
    Rule start() { return Sequence(OneOrMore(String("a")), 
                   OneOrMore(String("b")), push("fin")); }

    @SuppressWarnings({ "rawtypes", "unchecked" })
    public static void main(String[] args) {
        ExampleJavaParser parser = Parboiled.createParser(ExampleJavaParser.class);
        ReportingParseRunner parseRunner = new ReportingParseRunner(parser.start());

        String input = "aabbb";
        ParsingResult result = parseRunner.run(input);
        System.out.println(ParseTreeUtils.printNodeTree(result));
    }
}

Another utility function: adapt `normal* (special normal*)*` to Parboiled

Yet another musing... I use regexes a lot, and find myself to use this pattern very often. There are a lot of situations where this pattern is used. For instance, to recognize double quoted strings, one would define:

  • normal as [^"],
  • special as \\.,

and the full regex is ^"[^"]*(\\.[^"]*)*"$.

This pattern can be effectively used in Parboiled as well (for matching multiline comments, or even quoted strings as above), so I coded a utility function for that, with the options of altering the quantifiers so that they can become + on each element:

    /**
     * Adaptation of the {@code normal* (special normal*)*} regex pattern
     *
     * <p>The three booleans allow for the first {@code normal},
     * the inside {@code normal} or the group to be empty,
     * respectively. If disallowed, the {@code *} quantifier (ie,
     * {@link BaseParser#ZeroOrMore(Object)} is turned into a {@code +}
     * quantifier (ie, {@link BaseParser#OneOrMore(Object)}.</p>
     *
     * @param normal the normal rule
     * @param special the special rule
     * @param allowFirstEmpty allow first normal to be empty
     * @param allowInsideEmpty allow insider normal to be empty
     * @param allowGroupEmpty allow group to be empty
     * @return the matching rule
     */
    Rule NSN(final Rule normal, final Rule special,
        final boolean allowFirstEmpty, final boolean allowInsideEmpty,
        final boolean allowGroupEmpty)
    {
        final Rule firstNormal = allowFirstEmpty
            ? ZeroOrMore(normal) : OneOrMore(normal);
        final Rule insiderNormal = allowInsideEmpty
            ? ZeroOrMore(normal) : OneOrMore(normal);
        final Rule group = allowGroupEmpty
            ? ZeroOrMore(Sequence(special, insiderNormal))
            : OneOrMore(Sequence(special, insiderNormal));

        return Sequence(firstNormal, group);
    }

Are there chances that it makes it into parboiled?

Broken code example links in wiki, no develop branch

The links to code examples in the wiki all point to http://github.com/sirthias/parboiled/tree/develop/*. There is no develop branch though. I fixed the links in two instances before noticing that all (?) links were broken. Before hunting down every link manually I think it would be better that someone with write access to the wiki git repository does some search and replace all.

The PegDownProcessor doesn't seem to be threadsafe

When I run my tests concurrently I get this kind of stacktrace:

 Error while parsing rule 'Doc/ZeroOrMore/Sequence/Block/FirstOf/Verbatim' at input position (line 1, pos 1):

_hello_
^

java.util.NoSuchElementException (MatcherContext.java:328)
org.parboiled.MatcherContext.runMatcher(MatcherContext.java:328)
org.parboiled.matchers.FirstOfMatcher.match(FirstOfMatcher.java:40)
org.parboiled.parserunners.BasicParseRunner$Handler.match(BasicParseRunner.java:119)
org.parboiled.MatcherContext.runMatcher(MatcherContext.java:315)
org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)
org.parboiled.parserunners.BasicParseRunner$Handler.match(BasicParseRunner.java:119)
org.parboiled.MatcherContext.runMatcher(MatcherContext.java:315)
org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)
org.parboiled.parserunners.BasicParseRunner$Handler.match(BasicParseRunner.java:119)
org.parboiled.MatcherContext.runMatcher(MatcherContext.java:315)
org.parboiled.matchers.ZeroOrMoreMatcher.match(ZeroOrMoreMatcher.java:39)
org.parboiled.parserunners.BasicParseRunner$Handler.match(BasicParseRunner.java:119)
org.parboiled.MatcherContext.runMatcher(MatcherContext.java:315)
org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)
org.parboiled.parserunners.BasicParseRunner$Handler.match(BasicParseRunner.java:119)
org.parboiled.MatcherContext.runMatcher(MatcherContext.java:315)
org.parboiled.parserunners.BasicParseRunner$Handler.matchRoot(BasicParseRunner.java:115)
org.parboiled.parserunners.BasicParseRunner.runRootContext(BasicParseRunner.java:105)
org.parboiled.parserunners.BasicParseRunner.runRootContext(BasicParseRunner.java:98)
org.parboiled.parserunners.ReportingParseRunner.runRootContext(ReportingParseRunner.java:73)
org.parboiled.parserunners.BasicParseRunner.run(BasicParseRunner.java:92)
org.parboiled.parserunners.BasicParseRunner.run(BasicParseRunner.java:87)
org.parboiled.parserunners.BasicParseRunner.run(BasicParseRunner.java:83)
org.pegdown.Parser.parseRawBlock(Parser.java:995)
org.pegdown.PegDownProcessor.markdownToHtml(PegDownProcessor.java:83)

Thanks for telling me if there's any workaround for this.

Introduce cut marker optimization

Essentially this optimization will allow for "early exits" in FirstOf rules.

Consider this example

FirstOf(
    Sequence(A(), B(), C()),
    Sequence(A(), D())
)

Currently parboiled will try the second alternative in all cases where the first alternative failed.
However, if the first alternative already matched A and B and only failed at C the second alternative would not actually need to be tried since the already matched A B is not a prefix of the second alternative.
This analysis can be done statically after rule tree generation and so called "cut markers" can be introduced at all points, where FirstOf alternatives have matched far enough to become "non-prefixes" of all following alternatives.

Possible bug in prevText() - certain chars missing

Hello!

Today is my first day with PB, and i've come across a very unexpected result in one of my rules:

public String endDate = null;

public Rule UtcDateTime() {
return Sequence( Digit(), Digit(), Digit(), Digit(), // YYYY
         Digit(), Digit(), // mm
         Digit(), Digit(), // dd
         'T',
         Digit(), Digit(), // HH
         Digit(), Digit(), // MM
         Digit(), Digit(), // SS
         'Z' );
}

public Rule EndDate() {
return Optional( Sequence( UtcDateTime(), actionEndDate ) );
}

private Action actionEndDate = new Action() {
    public boolean run(Context context) {
    endDate = prevText();
    return true;
    }};

That parses fine, but when i look at the value of endDate after the parse, the value is missing the 'T' and 'Z' characters from the UtcDate() rule. My assumption is that rules created by automatic conversion from char are somehow treated differently?

e.g. when i parse this value: 20101121T112100Z, my endDate value = 20101121112100 (that is, the 'T' and 'Z' are gone), which is surprising to me, considering that the parser treats it as a String instead of a Date.

Is this a bug or a misunderstanding on my part? If it's the latter, how can i ensure that the 'T' and 'Z' characters being parsed become part of my collected output?

The complete test class (i'm parsing vCal recurrence rule) is pasted below (based on the Time example code), but i'm fairly certain that the wiki is going to mangle it pretty badly...

package org.parboiled.examples.vcal;
import org.parboiled.;
import org.parboiled.support.
;
import org.parboiled.matchers.;
import org.parboiled.common.
;

/**

  • Parser for very relaxed time literals. Demonstrates parse tree node selection via labels.
    */
    public class VCalRRuleParser extends BaseParser {

    public String ruleType = null;
    public String duration = null;
    public String interval = null;
    public String endDate = null;

    public Rule Head() {
    resetRuleState();
    return Sequence( this.FirstOf( Daily(),
    Weekly() ),
    EndDate(),
    Eoi()
    );
    }

    // WTF: If this is private then parboiled causes an IllegalStateException!?!?!?!?
    void resetRuleState() {
    ruleType
    = duration
    = interval
    = endDate
    = null;
    }

    private Action actionInterval = new Action() {
    public boolean run(Context context) {
    interval =
    //""+nodeByLabel("interval").getValue()
    prevText()
    ;
    return true;
    }};
    private Action actionDuration = new Action() {
    public boolean run(Context context) {
    duration =
    //""+nodeByLabel("duration").getValue()
    prevText()
    ;
    return true;
    }};
    private Action actionEndDate = new Action() {
    public boolean run(Context context) {
    endDate =
    //""+nodeByLabel("endDate").getValue()
    prevText()
    ;
    return true;
    }};

    private class ActionRuleType implements Action {
    private String t = null;
    ActionRuleType(String t) {
    this.t = t;
    }
    public boolean run(Context context) {
    ruleType = this.t;
    return true;
    }
    };

    private static final Rule WS = new CharSetMatcher( Characters.of(' ','\t') );

    public Rule Daily() {
    return Sequence( 'D', Interval(),
    Optional( Sequence( WS, Duration() ) ),
    new ActionRuleType("D")
    );
    }

    public Rule Weekly() {
    return Sequence( 'W', Interval(),
    // TODO: Weekday List
    Optional( Duration() ),
    new ActionRuleType("W")
    );
    }

    public Rule MonthlyP() {
    return Sequence( 'M', 'P', Interval(),
    // TODO: Month List
    Optional( Duration() ),
    new ActionRuleType("MP")
    );
    }

    public Rule MonthlyD() {
    return Sequence( 'M', 'D', Interval(),
    // TODO: Day List
    Optional( Duration() ),
    new ActionRuleType("MD")
    );
    }

    public Rule YearlyP() {
    return Sequence( 'Y', 'P', Interval(),
    // TODO: Month List
    Optional( Duration() ),
    new ActionRuleType("MP")
    );
    }

    public Rule YearlyD() {
    return Sequence( 'Y', 'D', Interval(),
    // TODO: Day List
    Optional( Duration() ),
    new ActionRuleType("MD")
    );
    }

    public Rule UtcDateTime() {
    return Sequence( Digit(), Digit(), Digit(), Digit(), // YYYY
    Digit(), Digit(), // mm
    Digit(), Digit(), // dd
    'T',
    Digit(), Digit(), // HH
    Digit(), Digit(), // MM
    Digit(), Digit(), // SS
    'Z' );
    }

    public Rule EndDate() {
    return Optional( Sequence( UtcDateTime(), actionEndDate ) );
    }

    public Rule Interval() {
    return Sequence( IntegerVal(), actionInterval );
    }

    public Rule Duration() {
    return Sequence( '#', IntegerVal(), actionDuration );
    }

    public Rule Digit() {
    return CharRange('0', '9');
    }

    public Rule IntegerVal() {
    return OneOrMore(Digit());
    }

    }

ReportingParseRunner assumes only one parse error

The ReportingParseRunner currently assumes that it creates the only parse error during its location match:

result = runLocatingMatch(inputBuffer);
Preconditions.checkState(!result.matched); // we failed before so we should really be failing again
Preconditions.checkState(result.parseErrors.size() == 1);

However, a parser might throw an ActionException during the parse run, which would cause one or more errors to already be present prior to the locating match, violating this assumption.

Support for Scala 2.9.1

Since the current release of Scala's play! framework still uses Scala 2.9.1, is there any change that parboiled supports this Scala version, too?

nTimes(1, "a") will throw ClassCastException

It happened in "parboiled-scala"

class MyParser extends Parser {
    def xxx = rule {
        nTimes(1, "a")
    }    
}

object MyParser {
    val parser = new MyParser { override val buildParseTree = true }
    val input = "any"
    val result = TracingParseRunner(parser.xxx).run(input)        
}

It will throw:

Exception in thread "main" java.lang.ClassCastException: org.parboiled.matchers.CharMatcher cannot be cast to org.parboiled.matchers.SequenceMatcher
at org.parboiled.scala.Parser$$anonfun$nTimes$1.apply(Parser.scala:268)
at org.parboiled.scala.Parser$$anonfun$nTimes$1.apply(Parser.scala:268)
at org.parboiled.scala.package$.make(package.scala:121)
at org.parboiled.scala.Parser$class.nTimes(Parser.scala:267)

System ClassLoader used in AsmUtils.

The current AsmUtils seems to use a system ClassLoader, which is a no-go under any container (ant, j2ee, etc).

In practice, the following Scala code:
Thread.currentThread.setContextClassLoader (classOf[SiteParser].getClassLoader)
val parser = org.parboiled.Parboiled.createParserSiteParser, String
val result = org.parboiled.ReportingParseRunner.run[String](parser.Url, url);
throws:
java.lang.RuntimeException: Error creating extended parser class: Error at instruction 17: Error loading class 'fastcms.nmy7sje5.SiteParser' for rule method analysis
at org.parboiled.Parboiled.createParser(Parboiled.java:58)
org.parboiled.asm.tree.analysis.AnalyzerException: Error at instruction 17: Error loading class 'fastcms.nmy7sje5.SiteParser' for rule method analysis
at org.parboiled.asm.tree.analysis.Analyzer.analyze(Unknown Source)
at org.parboiled.transform.InstructionGraphCreator.process(InstructionGraphCreator.java:40)
at org.parboiled.transform.ParserTransformer.runMethodTransformers(ParserTransformer.java:64)
at org.parboiled.transform.ParserTransformer.extendParserClass(ParserTransformer.java:48)
at org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:40)
at org.parboiled.Parboiled.createParser(Parboiled.java:54)
java.lang.RuntimeException: Error loading class 'fastcms.nmy7sje5.SiteParser' for rule method analysis
at org.parboiled.transform.AsmUtils.getClassForInternalName(AsmUtils.java:67)
at org.parboiled.transform.AsmUtils.isAssignableTo(AsmUtils.java:250)
at org.parboiled.transform.AsmUtils.isCallOnContextAware(AsmUtils.java:301)
at org.parboiled.transform.InstructionGraphNode.(InstructionGraphNode.java:58)
at org.parboiled.transform.RuleMethod.setGraphNode(RuleMethod.java:192)
at org.parboiled.transform.RuleMethodInterpreter.createNode(RuleMethodInterpreter.java:102)
at org.parboiled.transform.RuleMethodInterpreter.naryOperation(RuleMethodInterpreter.java:89)
at org.parboiled.asm.tree.analysis.Frame.execute(Unknown Source)
at org.parboiled.asm.tree.analysis.Analyzer.analyze(Unknown Source)
at org.parboiled.transform.InstructionGraphCreator.process(InstructionGraphCreator.java:40)
at org.parboiled.transform.ParserTransformer.runMethodTransformers(ParserTransformer.java:64)
at org.parboiled.transform.ParserTransformer.extendParserClass(ParserTransformer.java:48)
at org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:40)
at org.parboiled.Parboiled.createParser(Parboiled.java:54)
java.lang.ClassNotFoundException: fastcms.nmy7sje5.SiteParser
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.parboiled.transform.AsmUtils.getClassForInternalName(AsmUtils.java:65)
at org.parboiled.transform.AsmUtils.isAssignableTo(AsmUtils.java:250)
at org.parboiled.transform.AsmUtils.isCallOnContextAware(AsmUtils.java:301)
at org.parboiled.transform.InstructionGraphNode.(InstructionGraphNode.java:58)
at org.parboiled.transform.RuleMethod.setGraphNode(RuleMethod.java:192)
at org.parboiled.transform.RuleMethodInterpreter.createNode(RuleMethodInterpreter.java:102)
at org.parboiled.transform.RuleMethodInterpreter.naryOperation(RuleMethodInterpreter.java:89)
at org.parboiled.asm.tree.analysis.Frame.execute(Unknown Source)
at org.parboiled.asm.tree.analysis.Analyzer.analyze(Unknown Source)
at org.parboiled.transform.InstructionGraphCreator.process(InstructionGraphCreator.java:40)
at org.parboiled.transform.ParserTransformer.runMethodTransformers(ParserTransformer.java:64)
at org.parboiled.transform.ParserTransformer.extendParserClass(ParserTransformer.java:48)
at org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:40)
at org.parboiled.Parboiled.createParser(Parboiled.java:54)

Check for "FirstOf alternative prefix of later one" problem

The mistake of having a FirstOf alternative be a prefix of a subsequent one is common and sometimes hard to see.
It'd be great if parboiled could automatically detect these grammar problems and report them.
Actually this would likely be a byproduct of implementing #19

Node.getValue() not working.

Hi Sirthias,

Thanks a lot for the wonderful framework. I am trying it out for one of my upcoming project.

I am writing the following code in a Action class.

public boolean run(Context cntxt) {
System.out.println("okfsokfokfsdkf");
System.out.println("Get first match="+cntxt.getFirstMatchChar());
System.out.println("Get match="+cntxt.getMatch());
//System.out.println("Get sub node 0="cntxt.getSubNodes());
List<Node> nodeList = cntxt.getSubNodes();
for(Node node : nodeList){
// System.out.println("Value:"+node.getChildren().get(0));
for(int i=0;i< node.getChildren().size();i++){
Node cnode = (Node)node.getChildren().get(i);
System.out.println("Cnode Value:"+cnode.getValue());
}

    }
    return true;      
}

when I try to print the value of node using cnode.getValue() it prints null. cnode.getLabel() is working fine. Here is my parse tree:

okfsokfokfsdk
Get first match=i
Get match=i="3"
Cnode Value:null
Cnode Value:null
[VariableDeclarator] 'i="3"'
[Sequence] 'i="3"'
[Identifier] 'i'
[StringLiteral] '"3"'
[ZeroOrMore] '3'

Thanks & Regards,
Abhishek

Create Repeat() and Join() rules

While exercising with parboiled, mainly to write a simple parser for CSS color expressions, I have found something quite painful to do: I had three or six hex digits to match, and I had three items to look for separated by commas.

So, I figured out that instead of having to write:

// repeat 3 times
Sequence(r, r, r)
// join with a comma
Sequence(r, comma, r, comma, r)

I'd create some utilities to handle this:

https://gist.github.com/1525449

Now I just have to write:

// repeat 3 times
Repeat(3, r)
// join 3 items by a comma
Join(3, r, comma)

The gist link above contains the result of my musings. It'd be nice to see this in parboiled!

Side note: the Digit() and HexDigit() rules are used so often that it would be nice to see them integrated as well

Parboiled can use the default class loader only

Parboiled can use the default class loader only that can cause problem in OSGi environment when AsmUtil uses loadClass. There is a solution, the context class loader (it can be injected in OSGi activator) Here is the patch for it.

index 9b48d30..fa8f7f1 100644
--- a/parboiled-java/src/main/java/org/parboiled/transform/AsmUtils.java
+++ b/parboiled-java/src/main/java/org/parboiled/transform/AsmUtils.java
@@ -50,6 +50,9 @@ class AsmUtils {
         checkArgNotNull(clazz, "clazz");
         String classFilename = clazz.getName().replace('.', '/') + ".class";
         InputStream inputStream = clazz.getClassLoader().getResourceAsStream(classFilename);
+        if (inputStream == null) {
+            inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(classFilename);
+        }
         return new ClassReader(inputStream);
     }

@@ -72,7 +75,12 @@ class AsmUtils {
                 try {
                     clazz = AsmUtils.class.getClassLoader().loadClass(className);
                 } catch (ClassNotFoundException e) {
-                    throw new RuntimeException("Error loading class '" + className + "' for rule method analysis", e);
+                    // If class not found trying the context classLoader
+                    try {
+                        Thread.currentThread().getContextClassLoader().loadClass(className);
+                    } catch (ClassNotFoundException e2) {
+                        throw new RuntimeException("Error loading class '" + className + "' for rule method analysis", e2);
+                    }
                 }
             }
             classForDesc.put(classDesc, clazz);

PARBOILED MISSING FILE?

Dear,

I download 2 jar file to my application is : pegdown-1.4.1.jar and parboiled-java-1.5.1.jar to run the parse markdown syntax to html code.

But unfortunately, I can not run the PegDownProcessor because the system show me the error :
Caused by: java.lang.ClassNotFoundException: org.parboiled.errors.GrammarException

I check on your source but I can't find class GrammarException.
So could you help me about is this the file was deleted from your jar file or it is missing?

Many thanks!

Using with Google Guice servlets.

I'm trying to write Simple example from wiki(http://wiki.github.com/sirthias/parboiled/simple-example) in Tomcat with Guice framework:

protected void service(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
String input = "1+2";
CalculatorParser parser = Parboiled.createParser(CalculatorParser.class);
ParsingResult<?> result = ReportingParseRunner.run(parser.expression(), input);
String parseTreePrintOut = ParseTreeUtils.printNodeTree(result);
resp.getWriter().println(parseTreePrintOut);
}

And there is an error message at starting up:
java.lang.RuntimeException: Error creating extended parser class: Class not found
org.parboiled.Parboiled.createParser(Parboiled.java:51)
test.config.EmulationServlet.service(EmulationServlet.java:33)
javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:216)
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:141)
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93)
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:63)
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:122)
com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)

java.io.IOException: Class not found
org.parboiled.asm.ClassReader.a(Unknown Source)
org.parboiled.asm.ClassReader.(Unknown Source)
org.parboiled.asm.ClassReader.(Unknown Source)
org.parboiled.transform.ClassNodeInitializer.transform(ClassNodeInitializer.java:55)
org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:37)
org.parboiled.Parboiled.createParser(Parboiled.java:47)
test.config.EmulationServlet.service(EmulationServlet.java:29)
javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:216)
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:141)
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93)
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:63)
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:122)
com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)

When I'm to run this code as native Java application(without Tomcat and Guice) it works good.

Is there a way to solve this issue? Perhaps this is due to the conflict between Guice bytecode generator(http://code.google.com/p/google-guice/wiki/ClassLoading) and bytecode generator in Parboiled? If so, how could I switching off one of them for this case?

Thank you.

Classloaders issues

I wanted to use Pegdown in an IntelliJ IDEA plugin, but the use of Thread.currentThread().getContextClassLoader() in AsmUtils makes it impossible (I guess there are other places too). The library also probably does not work in OSGi environments too because of this.

IncompatibleClassChangeError

I have tried to upgrade from 1.0.2 (parbolied-java + core) to 1.1.3 today and I have encountered the following issue:

java.lang.IncompatibleClassChangeError: org/parboiled/transform/ClassNodeInitializer :     org/parboiled/transform/ClassNodeInitializer
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at org.parboiled.transform.ParserTransformer.extendParserClass(ParserTransformer.java:44)

I have tried using asm 3.3.1, 4.0 and 4.1, but I still get the same result. Is it only me or is it a common problem?

Thanks.

How to implement "get all chacaters before and after '|' "?

At first, I must say it's really a good job you have done. Parboiled is so powerful and easier to parse some custom DSLs. Thank you!

And I want to ask a question here:

Suppose the input is:

abcdef | 111222
including Non-English characters like 中文 | Some other characters 怎么办

I want to write a rule, to match the string before and after the character '|'. But I don't know how to write it, because there may be some non-english characters there, that I can't specify exactly what are these characters, but I know what characters I don't want.

So, the rule should be:

Rule Line() {
return Sequence(OneOrMore(AnyCharactersNot("|")), Ch('|'), OneOrMore(AnyCharactersNot("|\r\n")), AnyOf("\r\n"));
}

I don't know how to implement the method:

Rule AnyCharactersNot(String chars)

It seem Test() and TestNot() can't be used here neither.

Issue in CalculatorParser

When using the Java calculator examples provided (CalculatorParser3 or CalculatorParser4), some expressions are not evaluated correctly.
For instance when entering:
3 * (1 + (5 / ((4 - 7) * (1 + 2) + 7) - (3 * 9)))
the result returned is 114.0 but it should really be -85.5.

The issue boils down to using Action variables in some of the rules. The same action variable is used for multiple operators and the variable can be 'set' multiple times before 'get' is called.
Using action variables in recursive rules seems like it could lead to errors.
In the Calculator example, a simple operator stack (i.e. java.util.Stack) can be used instead of the Action variables.

Duplicate nodes when skipping Sequence() nodes within FirstOf() rules

Edit: simplified example code.
Edit: added version info

I think the duplicated [A] node created by the demo parser (see code below) shouldn't be there :-)

The bug is reproducible using the following maven artifact:

        <dependency>
            <groupId>org.parboiled</groupId>
            <artifactId>parboiled-java</artifactId>
            <version>1.0.1</version>
        </dependency>
import org.parboiled.BaseParser;
import org.parboiled.Parboiled;
import org.parboiled.Rule;
import org.parboiled.annotations.BuildParseTree;
import org.parboiled.errors.ErrorUtils;
import org.parboiled.parserunners.ReportingParseRunner;
import org.parboiled.support.ParseTreeUtils;
import org.parboiled.support.ParsingResult;

public class DuplicatesTest {

    @BuildParseTree
    public static class TestGrammarParser extends BaseParser<Object> {

        Rule Duplicates() {
            return FirstOf(Sequence("A", "C").skipNode(), "A");
        }

    }

    public static void main(String[] args) {

        TestGrammarParser parser = Parboiled.createParser(TestGrammarParser.class);

        ReportingParseRunner<Object> parseRunner = new ReportingParseRunner<Object>(parser.Duplicates());

        String toParse = "ABC";

        ParsingResult<Object> parsingResult = parseRunner.run(toParse);

        if (parsingResult.matched) {
            System.out.println(ParseTreeUtils.printNodeTree(parsingResult));
            if (parsingResult.parseTreeRoot.getChildren().size() > 1) {
                System.out.println("Looks like Sequence didn't undo the creation of its 'A' subnode?");
            } else {
                System.out.println("OK");
            }
        } else {
            System.err.println(ErrorUtils.printParseErrors(parsingResult));
        }
    }

}

AsmUtils tries to make ClassLoader.findLoadedClass accessible

This line in AsmUtils:173 is a no-go with Google AppEngine:

        Class<?> classLoaderBaseClass = Class.forName("java.lang.ClassLoader");
        Method findLoadedClassMethod = classLoaderBaseClass.getDeclaredMethod("findLoadedClass", String.class);

        // protected method invocation --> FAIL!
        findLoadedClassMethod.setAccessible(true);

Under GAE, this line fails with the following exception:

Caused by: java.lang.SecurityException: java.lang.IllegalAccessException: Reflection is not allowed on protected final java.lang.Class java.lang.ClassLoader.findLoadedClass(java.lang.String)
at com.google.appengine.runtime.Request.process-674657286aa9df1b(Request.java)
at java.lang.reflect.Method.setAccessible(Method.java:134)
at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:173)

Upgrade to ASM 4

We're looking at Java 8 support in Play Framework. This is going to eventually require an upgrade to ASM 5 (currently there's an alpha release out there with initial Java 8 support), since there are many new features in Java 8 bytecode that ASM chokes on, and apparently support for these features can't be implemented in a backwards compatible way. But before we get there we'll probably need an upgrade to ASM 4.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.