ledjon-behluli / orleanspaces Goto Github PK

While getting all data collected from the director grain is the more efficient way, since it involves n + 1 calls, where n is the number of
partitions (i.e. store grains), and 1 is a call from the agent to the director, and all calls to the store grains are done in parallel via Task.WhenAll. We need to keep in-mind the potential size of the whole tuple space, so when the number of partitions grows a lot, this call to the director might result in contention for ThreadPool threads and may lead to thread starvation, therefor degrading performance.

An alternative is for the director to expose an IAsyncEnumerable way to load the data, where each call to it will result in a single batch of loading the data. This "batch" basically means the director calls the store grains one-by-one and streams back the result which the agent call append to its in-memory dataset. This does result in a slower loading of the whole tuple space, as there are 2*n calls, each time a call to the director + 1 call to the partition of a store, but ultimately should not lead to potential thread starvation.

We should have a configurable behavior for this in the SpaceClientOptions.

In SpaceTuple.cs, I modified to avoid crash:

public override string ToString() => fields == null ? "!! fields is null !!" : $"({string.Join(", ", fields)})";

Run Concurrent:

...
READER 0: (exchange-key, 0)
READER 4: !! fields is null !!
READER 6: (exchange-key, 6)
...

Usually 1 or two Readers have null fields.

Agents must rehydrate their state upon startup, this is missing currently and needs to be resolved

Enhance 'AddOrleanSpace()' on the silo builder by means of accepting 2 optional options configs.

  public static ISiloBuilder AddOrleanSpaces(
      this ISiloBuilder builder,
      Action<SpaceServerOptions>? configureServerOptions = null,
      Action<SpaceClientOptions>? configureClientOptions = null)
{
...
}

If user provides an non-null configureClientOptions and specifies SpaceKind.XXX then that space (therefor also the agent) is available in the silo too.

• Create new analyzer that detects usages of generic tuples & templates that should be specialized ones.
• Create associated fixer that converts i.e:
  SpaceTuple(1, 1, 1) -> IntTuple(1, 1, 1)
SpaceTuple('a', 'b', 'c') -> CharTuple('a', 'b', 'c')
  etc...

Makes it more feasible to work with the generic tuples as a whole array can be passed as opposed to array elements.
This is useful when clients build an array of elements in a loop and want to pass it down, or if they already get an array from a source that they can not control

• ValueTask CountAsync() -> int Count {get;}
• ValueTask : PeekAsync(template) -> tuple : Peek(template)
• ValueTask<IEnumerable> : ScanAsync(template) -> IEnumerable : Enumerate(template)
• IAsyncEnumerablePeekAsync() -> IAsyncEnumerableEnumerateAsync()

In certain cases like where the agent is used to perform only writes, or the application needs fast startup times, it is useful to allow a configurable way to load (or not) the space contents. Best way is to have this on the SpaceOptions.

The functionality needs to be complimented via a new method on the ISpaceAgent (and generic versions) called ReloadAsync which acts as loading the space for agents that have been configured to NOT load space upon startup, or in general a reload of the space (for whatever reasons)

Hi,

Is this project production ready?

Adjust OSA003 to NOT report on direct array passing because of #6

• Enumerate(template = default)
• EnumerateAsync(template = default);

Both methods should enumerate only tuples that match the given template, or in cases when template = default, than it should enumerate all tuples in the respective space(s).

Partitioning the space for each space kind allows us to avoid performance impacts due to frequent updates to a large list of tuples, because each update involves serializing and persisting the entire dataset.