GenericTensor 2.0: plan

Alright, time for improvement

1 action - 1 interface

Currently we have a large interface which forces the user to implement all its methods. It's not as good as it was thought to be:

1. You may want not to implement a method, but you can't guess whether some GT's function uses it or not
2. The interface is not extendable - adding a new method immediately breaks backward comaptibility

Instead, we will have one method in each interface (there will be multiple interfaces - like IAddAction<A, B, C> etc.).

Now we will be able to constrain each function separately, and the tensor itself won't be constrained (the type will have only one type argument - T). It makes all things much more convenient, and it also allows for more advanced API - for example, adding tensors would look like:

Tensor<C> AddElementwise<A, B, C, TAdd>(Tensor<A> a, Tensor<B> b) where TAdd : IAdd<A, B, C>

Non-generic entry point class API

Currently we need to type the typename with its arguments. Instead, there will be a non-generic static class to handle those things: thanks to type inference, we will type less.

No sparse transformation

Currently, there are some transformations, like transposition and slicing, which makes the tensor "sparse" - that is, instead of rewriting elements themselves, they rewrite the meanings of indices. That means, that going over a just created 2D tensor by width and height would be much faster than if we transpose that tensor beforehand.

In new API such a feature will remain in some different more explicit form, for example, TensorView<T>. It behaves similarly to BCL's Span<T>, which is basically a view on some piece of memory. Likewise, TensorView just allows to read/write to a tensor, but is not a tensor itself. Might be useful for some operations, where the user doesn't want to think about the order of axes or size of a tensor. Though not all API will be available for TensorView<T>.

But tensors itself will be strictly linear/dense. Slicing and transposition will produce a new dense tensor.

Hardware acceleration

Now, with dense tensors we can use three types of acceleration:

1. Multithreading
2. SIMD (HonkPerf.NET.GenericSIMD)
3. GPU

Though the third one is questioned, the first two are definitely doable and will require new methods with type constrained to unmanaged. As simple as that.

New project name - GenericTensor.NET. New type name - Tensor.

Alright, this one is questionable. Should we rename to a more "modern" name? It would mean a new "slot" in nuget packages, and hence, not visible by current users (if there are any aside from AngouriMath, hehe).

New operations

We could implement convoluted map for it. Example of convoluted map:
Imagine matrix A

1 2 3
4 5 6
7 8 9

and then code:

B = A.ConvolutedMap(width: 2, height: 2, step: 1, view: TensorView -> view[0, 0] + view[0, 1] + view[1, 0] + view[1, 1]);

then B is

12 16
24 28

This way we can basically process, reshape, "bend", collapse or add axes anyhow we need. Upscale for instance:

A =
    1 2 3
    4 5 6
    7 8 9

B = A.ConvolutedMap(
    inputWidth: 2,
    inputHeight: 2,
    step: 2,
    outputWidth: 3,
    outputHeight: 3,
    (source: TensorView, destination: TensorView) ->
        destination[0, 0] = source[0, 0]
        destination[0, 2] = source[0, 1]
        destination[2, 0] = source[1, 0]
        destination[2, 2] = source[1, 1]
        destination[0, 1] = (destination[0, 0] + destination[0, 2]) / 2
        destination[1, 0] = (destination[0, 0] + destination[2, 1]) / 2
        etc...
)

Note, that we don't limit to 2d convolution here, though not sure where we could ever need 4D+ convolutions.

Unresolved questions

Should there be single type Tensor or by dimension - Tensor1D, Tensor2D, etc.? If the latter, should there be TensorND or just cover a fixed number of dimensions?

Like +, -, /, *. If so, what should "/" and "*" do?

So the code in GT is not working in a natively compiled app. Needs to be fixed

I am using the latest release 1.0.4 for my .NET project

When inverting a matrix twice with <GenTensor>.InvertMatrix(), i am expecting it to compute the inverse back to the original matrix, but the program is stuck and will not continue. I tried waiting for a few minutes but nothing has happened.

The matrix i used was one of type GenTensor<Entity, EntityWrapper> containing only diagonal elements ("-1", "1", "r^2", "r^2 * sin(theta)^2")

Thank for sharing this intresting packge.
Is there any fft or SVD function in this package?

Thanks for sharing this interesting package.

Will this package support MKL or OpenBlas as backend to accelerate matrix inverse computing speed?

There are 168 warnings generated on build currently, not cool

Name of the type

For now we will assume it's Tensor. However it's opionated, because it might be useful to avoid calling it the same name as other libs.

Operations

Namespace: GenericTensor.Core.

Interfaces with method Invoke: () -> A

IAdditiveIdentity<A>
IMultiplicativeIdentity<A>

Interfaces with method Invoke: A -> B

INegate<A, B>

Interfaces with method Invoke: A x B -> C

IAdd<A, B, C>
ISubtract<A, B, C>
IMultiply<A, B, C>
IDivide<A, B, C>

Interfaces with method Invoke: A -> bool

IEqualsAdditiveIdentity<A>

Interfaces with method Invoke: A x B -> bool

IAreEqual<A, B>

Interfaces with method Invoke: A -> string

IToString<A>

Interfaces with method Invoke: A -> byte[]

ISerialize<A>

Interfaces with method Invoke: byte[] -> A

IDeserialize<A>

Tensor

The main type where all these methods will be is Tensor (but not Tensor<T>!).

Composition

static Tensor<T> Concat<T>(params Tensor<T>[] ts, int axis = -1);
byte[] Serialize<T, TSerializer>() where TSerializer : ISerialize<T>;
Tensor<T> Slice<T>(params Either<int, Index, Range>[] dims);
static Tensor<T> Stack<T>(params Tensor<T>[] ts);
Tensor<T> Transpose<T>(Tensor<T> a);
Tensor<T> Transpose<T>(Tensor<T> a, int axis1, int axis2);
void TransposeInplace(Tensor<T> a);
void TransposeInplace(Tensor<T> a, int axis1, int axis2);

Math operations

...

Elementwise:

void AddPointwise<A, B, C, TAdder>(Tensor<A> a, Tensor<B> b, Tensor<C> destination) where TAdder : struct, IAdder<A, B, C>;
Tensor<C> AddPointwise<A, B, C, TAdder>(Tensor<A> a, Tensor<B> b) where TAdder : struct, IAdder<A, B, C>;

Should be added so that projects using GT could catch them all and wrap into their own

When is it not sufficient to assign references directly?

As proposed by mednik, to gaussian-eliminate A|I_n matrix (first part) instead of finding the adjoint.

• Add a module supporting SIMD
  - [ ] Replace using Linoffset with using Span<T> instead of T[]

Instead, it should be required to override ConstantsAndFunctions's statis method ToString.

https://github.com/WhiteBlackGoose/GenericTensor/blob/master/GenericTensor/Functions/ToString.cs#L43

#6 (comment)

It might be a good idea instead to return an error code.

var m2 = GenTensor<double, DoubleWrapper>.CreateMatrix(new double[,]
    {
        { 1, 0, 70 },
        { 0, 1,  0 },
        { 0, 0,  1 }
    }
);

m2.InvertMatrix();
Console.WriteLine(m2);

Returns

Matrix[3 x 3]
1    -0   70
-0   1    -0
0    -0   1

It currently returns IEnumerable, which has a virtual call. We can make it much faster by writing a value type state machine (foreach does not force IEnumerable).