Code organisation for larger parser about combine

[Marwes]

You can look at the json parser and gluon parser. Though if you do not need to share state between the parsers you can omit the structure passed as &self.

fn parse_ident(&self, input: I) -> ParseResult<Id, I> {
    self.parser(ParserEnv::<I, F>::parse_ident2)
        .map(|x| x.0)
        .parse_stream(input)
}

Basically just wrap the parser in a function with the signature for parser and within that function just call parse_stream. With some extra work do it like integer in the json parser. By calling parse_lazy you tell the parser to delay emitting error messages so the integer wraps the parser(integer_) (which does not emit an error) into parser(integer_).expected("integer") to ensure that the error message actual displays what was expected. I can try to explain parse_lazy in a better way if you want but most of the time you don't need to worry about it. It should just be relevant in hot loops.

[Marwes]

This is something impl Trait will make much nicer eventually (and with maximum performance as well). It might be possible to add some macros to make this nicer as well.

[Yamakaky]

I see. Will there be performance difference with using local variables like in the example?

    let integer = spaces()
        //parse a string of digits into an i32
        .with(many1(digit()).map(|string: String| string.parse::<i32>().unwrap()));

    //Parse integers separated by commas, skipping whitespace
    let mut integer_list = sep_by(integer, spaces().skip(char(',')));

I didn't need shared state for the nom parser, so free functions should be fine.

[Marwes]

Nope, there should be no difference at all.

[Yamakaky]

Cool!

[Yamakaky]

I started the implementation: https://github.com/Yamakaky/dcpu/blob/combine/src/assembler/parser/combine_parser.rs
It doesn't compile and PN and SN are dummy type declarations. Could you quickly review it?

[Marwes]

Since you take a &str as input you may want to consider using range (zero-copy) parsers https://docs.rs/combine/2.0.0/combine/range/index.html (which I now realize I have forgotten to add doc comments on, they have examples at least) . Also, since you use boxed parsers you may want to be careful so you don't recreate the parser to often (harming performance). But of course always profile before optimizing. Otherwise it looks as I would expect! 👍

[Yamakaky]

I tried boxed parser since I didn't really know how to return one otherwise...

[Marwes]

Ah, sorry. The way to "return" a parser without boxing right now is to write roughly

fn pos_number(input: &str) -> ParseResult<&str, int> {
    raw_number().and_then(|(digits, base)| u16::from_str_radix(digits,
                                                                    base)).parse_stream(input)
}

// Use pos_number
parser(pos_number).parse("abc");

That way the type is hidden by the function.

[Yamakaky]

Does your pos_number work with Or?

[Marwes]

Yep, parser is just a way to make a Parser from a function.

[Marwes]

Alternatively, with a bit more boilerplate you could write this.

fn pos_number() -> Expected<FnParser<fn (&str) -> ParseResult<&str, int>>> {
    parser(pos_number_).expected("positive number")
}
fn pos_number_(input: &str) -> ParseResult<&str, int> {
    raw_number().and_then(|(digits, base)| u16::from_str_radix(digits,
                                                                    base)).parse_lazy(input)
}

pos_number().parse("abc");

By calling parse_lazy over parse_stream it becomes slightly more efficient if pos_number are used in or. This is because the parser holds off on generating error information for as long as possible until an entire chain of or parsers has failed.

Making a note to properly explain how this works somewhere in the documentation

[Yamakaky]

Make an other note: generate pos_number using a macro ^^

[Marwes]

Perhaps. That would work in this case but it would not be possible to define a parser that was generic if it were a macro. Most of my own parsers are generic which is why I haven't added it.

fn pos_number<I>() -> Expected<FnParser<fn (I) -> ParseResult<I, int>>> where I : Stream<Item = char> {
    parser(pos_number_).expected("positive number")
}
fn pos_number_<I>(input: I) -> ParseResult<I, int> where I : Stream<Item = char> {
    raw_number().and_then(|(digits, base)| u16::from_str_radix(digits,
                                                                    base)).parse_lazy(input)
}

[Marwes]

Had a thought, as long as the parsers are not incredibly complex or mutually recursive an easy way to get re-usability could be to use macros which produce parsers.

macro_rules! pos_number {
    () => { raw_number().and_then(|(digits, base)| u16::from_str_radix(digits, base)) }
}

pos_number!().parse_*(input)

[Yamakaky]

why not ^^