timiskhakov / exploringspansandpipelines Goto Github PK

This could result in a stackoverflow (if the line is too long). You want to stack alloc only if the length is reasonably small (say 1-256 bytes), anything above that you can use the memory pool, or use the SequenceReader to parse the ReadOnlySequence.

On Windows FileGenerator creates a file using a line terminator different from \n that one of the file parsers expects. As a result line parsing breaks while trying to parse a Guid value.

Quick and dirty workaround would be changing the generator to use \n:

public static async Task Generate(string file, int numberOfLines)
{
    var sb = new StringBuilder();
    foreach (var line in CreateContent(numberOfLines))
    {
        sb.Append($"{line}\n");
    }

    await File.WriteAllTextAsync(file, sb.ToString());
}

But a proper solution would be changing the parser to take the Windows version of the line terminator into account.

Thanks to Mark Lynn for pointing out to this issue.

Hi, I read your article about how to use span to read files with zero allocation and I find it very releated to my project record parser which is basically a "line parser" which build using the line/fine configurations. It generates using expression tree feature the code you would handwriting yourself using span and all the boilerplate.

It offers builders for read/write files of type delimited (csv, tsv, etc) and fixed-length (cobol, bank stuffs, etc).

anyway, just sharing because I see synergy and may find it interesting in your projects

example:

[Fact]
public void Given_value_using_standard_format_should_parse_without_extra_configuration()
{
    var reader = new VariableLengthReaderBuilder<(string Name, DateTime Birthday, decimal Money, Color Color)>()
        .Map(x => x.Name, indexColumn: 0)
        .Map(x => x.Birthday, 1)
        .Map(x => x.Money, 2)
        .Map(x => x.Color, 3)
        .Build(";");
  
    var result = reader.Parse("foo bar baz ; 2020.05.23 ; 0123.45; LightBlue");
  
    result.Should().BeEquivalentTo((Name: "foo bar baz",
                                    Birthday: new DateTime(2020, 05, 23),
                                    Money: 123.45M,
                                    Color: Color.LightBlue));
}