reegeek / structlinq

Implementation in C# of LINQ concept with struct

StructLinq

Implementation in C# of LINQ concept with struct to reduce drastically memory allocation and improve performance. Introduce IRefStructEnumerable to improve performance when element are fat struct.

---

• Installation
• Usage
• Performances
• Features
  • BCL bindings
  • Transformers
  • Converters
  • LINQ Extensions
  • Other Extensions
• IRefStructEnumerable

---

Installation

This library is distributed via NuGet. To install StructLinq :

PM> Install-Package StructLinq

Usage

StructLinq use massively generic concept and struct "specialization".

using StructLinq;
 
int[] array = new [] {1, 2, 3, 4, 5};

int result = array
                .ToStructEnumerable()
                .Where(x => (x & 1) == 0, x=>x)
                .Select(x => x *2, x => x)
                .Sum();

x=>x is used to avoid boxing (and allocation) and to help generic type parameters inference. You can also improve performance by using struct for Where predicate and select function.

Performances

All benchmark results are in here. For example following linq sequence:

   list
     .Where(x => (x & 1) == 0)
     .Select(x => x * 2)
     .Sum();

can be replace by:

  list
    .ToStructEnumerable()
    .Where(x => (x & 1) == 0)
    .Select(x => x * 2)
    .Sum();

or if you want zero allocation by:

 list
   .ToStructEnumerable()
   .Where(x => (x & 1) == 0, x=>x)
   .Select(x => x * 2, x=>x)
   .Sum(x=>x);

or if you want zero allocation and better performance by:

  var where = new WherePredicate();
  var select = new SelectFunction();
  list
    .ToStructEnumerable()
    .Where(ref @where, x => x)
    .Select(ref @select, x => x, x => x)
    .Sum(x => x);

Benchmark results are:

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19042
Intel Core i7-8750H CPU 2.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.101
  [Host]     : .NET Core 5.0.1 (CoreCLR 5.0.120.57516, CoreFX 5.0.120.57516), X64 RyuJIT
  DefaultJob : .NET Core 5.0.1 (CoreCLR 5.0.120.57516, CoreFX 5.0.120.57516), X64 RyuJIT

Method	Mean	Error	StdDev	Ratio	Gen 0	Gen 1	Gen 2	Allocated
LINQ	65.116 μs	0.6153 μs	0.5756 μs	1.00	-	-	-	152 B
StructLinqWithDelegate	26.146 μs	0.2402 μs	0.2247 μs	0.40	-	-	-	96 B
StructLinqWithDelegateZeroAlloc	27.854 μs	0.0938 μs	0.0783 μs	0.43	-	-	-	-
StructLinqZeroAlloc	6.872 μs	0.0155 μs	0.0137 μs	0.11	-	-	-	-

StructLinq is significatively faster than default LINQ implementation.

Features

Duck typing with foreach is available with zero allocation for IStructEnumerable.

BCL

Following class have a StructLinq extension method for IStructEnumerable:

• IEnumerable<T>
• T[]
• List<T> (in Struct.Linq.BCL)
• Dictionary<TKey, TValue> (in Struct.Linq.BCL)
• Hashset<T> (in Struct.Linq.BCL)
• ImmutableArray<T> (in Struct.Linq.BCL)

Converters

Following converters are available for :

• ToArray
• ToList (in Struct.Linq.BCL)
• ToEnumerable

LINQ Extensions

Following extensions are available for :

• Aggregate
• All
• Any
• Concat
• Contains
• Count
• Distinct(zero allocation)
• ElementAt
• ElementAtOrDefault
• Empty
• Except(zero allocation)
• First
• FirstOrDefault
• Intersect(zero allocation)
• Last
• LastOrDefault
• Max
• Min
• OrderBy(zero allocation)
• OrderByDescending
• Range
• Repeat
• Reverse(zero allocation)
• Select
• SelectMany
• Skip
• SkipWhile
• Sum
• Take
• TakeWhile
• Union(zero allocation)
• Where

Other Extensions

• LongCount
• UIntCount
• Order
• TryFirst

IRefStructEnumerable

    public interface IRefStructEnumerable<out T, out TEnumerator>
        where TEnumerator : struct, IRefStructEnumerator<T>
    {
        TEnumerator GetEnumerator();
    }

    public interface IRefStructEnumerator<T>
    {
        bool MoveNext();

        void Reset();

        ref T Current { get; }
    }

ref Current allows to avoid copy. I should be very useful when T is a fat struct.

Duck typing with foreach with ref is available with zero allocation for IRefStructEnumerable.

BCL

Following class have a StructLinq extension method for IRefStructEnumerable:

• T[]
• List<T> (in Struct.Linq.BCL)

Converters

Following converters are available for :

• ToArray
• ToList (in Struct.Linq.BCL)
• ToEnumerable

LINQ Extensions

Following extensions are available for :

• All
• Any
• Concat
• Contains
• Count
• Distinct
• ElementAt
• ElementAtOrDefault
• Except
• First
• FirstOrDefault
• Intersect
• Last
• Select
• Skip
• SkipWhile
• Sum
• Take
• TakeWhile
• Union
• Where

Other Extensions

• LongCount
• UIntCount
• TryFirst

structlinq's Issues

StructEnumerable.Join

StructEnumerable.Empty

StructEnumerable.Contains

StructEnumerable.FirstOrDefault

StructEnumerable.TakeWhile

StructEnumerable.ThenBy / StructEnumerable.ThenByDescending

Add dictionnary wrappers

StructEnumerable.Skip

StructEnumerable.OrderBy / StructEnumerable.OrderByDescending

Challenging default OrderBy algo.

StructEnumerable.GroupJoin

StructEnumerable.TakeLast

StructEnumerable.All

StructEnumerable.Intersect

StructEnumerable.First / StructEnumerable.FirstOrDefault

StructEnumerable.Append

StructEnumerable.AggregateBy / StructEnumerable.SumBy

Apply an aggregator during a GroupBy.

StructEnumerable.DefaultIfEmpty

StructEnumerable.Single / StructEnumerable.SingleOrDefault

StructEnumerable.Repeat

StructEnumerable.Cast

StructEnumerable.SkipLast

StructEnumerable.GroupBy

StructEnumerable.SequenceEqual

StructEnumerable.Take

StructEnumerable.Prepend

IRefStructCollection

Add IRefStructCollection like IStructCollection

StructEnumerable.ElementAtOrDefault

StructEnumerable.Any

StructEnumerable.SkipWhile

ToArray()

StructEnumerable.OrderByDescending

StructEnumerable.SelectMany

StructEnumerable.OrderByDescending

StructEnumerable.Concat

StructEnumerable.Reverse

StructEnumerable.OfType

Optimize ArrayEnumerator

Use IL code for Current

StructEnumerable.Safe

ArrayEnumerator is not safe.
MoveNext always increment index.

target C# 9 with func pointers (it can still compile into old CLR)

I think you can use inline static methods with function pointers and it will avoid boilerplate of creating classes as C# does not have inline classes.

StructEnumerable.ElementAt / StructEnumerable.ElementAtOrDefault

ToList()

For loops in benchmarks

May be you could add for loop doing same as benchmarks on page?

StructEnumerable.Average

Compute also other moment.
https://en.wikipedia.org/wiki/Moment_(mathematics)

StructEnumerable.Union

StructEnumerable.Last / StructEnumerable.LastOrDefault

Improve ToArray()

StructEnumerable.Zip

StructEnumerable.Except

Implement StructEqualityComparer for int, long,....

Optimize Count for Array, List...