indentation parsing doesn't work about eo

[alexpanov]

@yegor256 do you think there needs to be a vote about that too? ;)

[g4s8]

I've adapted python indentation parser (from @jachinte comment) for EO grammar.
Does anyone want to review it? I'm not familiar with ANTLR, so I can make a mistake. Just want to try EO at the earliest moment.

[martinstraus]

Identation is required? That's nasty.

[Tertioptus]

why white-space parsing anyway?

[yegor256]

@Tertioptus it's compact and effective

[g4s8]

it's compact and effective

Furthermore, it's a necessary feature.
Without indentation parsing we can't understand method (or ctor) to pass argument in some cases.

E.g. if we have two objects:

object one as String:
    String @s1
    String @s2
    ctor(String s)
        one(s, "")
    ctor(String s1, String s2)
    String string():
        concat s1, s2

object two as String:
    String @s1
    String @s2
    ctor(String s)
        one(s, "")
    ctor(String s1, String s2)
    String string():
        concat s2, s1

this code

print:
    one:
        two:
            "A",
            "B"

can be interpreted in a two ways: one(two("A", "B")) and one(two("A"), "B")
and return "AB" or "BA".

[stain]

I think https://github.com/yshavit/antlr-denter can do it for the reference implementation using antlr.

[yegor256]

@stain many thanks for the antlr-denter link. Let's use it! Can you submit a pull request?

[stain]

I can have a look at using antlr-denter, but not until the weekend I'm afraid :-(

It is much harder to write a regular grammar without block end characters - I tried to make some lookehad rules but couldn't get it quite to work - perhaps too late at night..

(To summarize: My idea was to make a lookahead rule to consume space on either end of a line:

    NEWLINE = '\r\n' | '\n'
    SPACE = ' '
    indent = <SPACE> <SPACE>
    morestatements = <indent> #"[^\n]*" <NEWLINE> statements)

and then use this as a positive and negative lookahead in the actual recursive rule statements:

    statement = object | number | string | etc
    statements = ((&(morestatements) <indent> statement <NEWLINE> <indent> statements)) | 
                 (!(morestatements) <indent> statement <NEWLINE>)

The theory would be that statements would require recursion if there's equivalent space on the next line, or require non-recursion (no more statements) if there isn't equivalent space.

But to do this in a forward-looking grammar without too much trickery we would have to:

• Insist on a fixed indentation pattern (e.g. two spaces)
• Not allow non-indented comments within a block
• Always/never use newline in various patterns (e.g. after :)

Doing this I see why many languages like Java use { and } together with ; - it makes it easier to write a simple grammar. But I don't think having a simple grammar is a good requirement for "eo" - as long as we don't end up with weird workarounds in the syntax (e.g. a series indented newlines as substitutes for }!)

[stain]

My naive whitespace-agnostic grammar found 7 alternative interpretation of @g4s8's example with print: one: two: "A", "B" -- "B" can be an argument for print, one or two -- and in addition both one and two could be consider top-level objects or an argument. If we had a more traditional () or {} block for constructor it shouldn't be a problem.

Let's not forget the "no syntactic sugar" mantra -- lack of block characters is in a way also syntactic sugar..

[jachinte]

@stain I think it would be syntactic sugar if removing those characters was part of an alternative syntax, that is, the eo language supports { and } to define block scopes, but a new syntax allows indentation-based scopes.

As you are using Antlr, my first suggestion is to look at the python grammar in the Antlr repository. This StackOverflow question can also give you some insights.

[pchmielowski]

@g4s8 I think, you should create a pull request and we'll review and discus it

[yegor256]

fixed, thanks all

[0pdd]

@yegor256 all 6 puzzles are solved here: #104, #105, #106, #129, #131, #95.