stellar-group / blaze_tensor Goto Github PK
View Code? Open in Web Editor NEW3D Tensors for Blaze (https://bitbucket.org/blaze-lib/blaze)
License: Other
3D Tensors for Blaze (https://bitbucket.org/blaze-lib/blaze)
License: Other
In the following code:
blaze::DynamicArray<4UL, int> arr(2,2,2,4);
for (std::size_t c = 0; c != 4; ++c)
{
auto arr_tensor = blaze::quatslice(blaze::trans(arr, { 3,0,1,2 }), c);
for (std::size_t l = 0; l != 2; ++l)
{
auto arr_slice = blaze::columnslice(arr_tensor, l);
for (std::size_t i = 0; i != 2; ++i)
{
for (std::size_t j = 0; j != 2; ++j)
{
arr_slice(i, j) = 1;
}
}
}
}
the compiler sees arr_slice
as a blaze::ColumnSlice_<blaze::QuatSlice<const blaze::DQuatTransExpr<blaze::DynamicArray<4Ui64,int>>>>
which is const. Therefore, we get:
c:\repos\blaze_app_1\prog.cpp(2161): error C3892: 'arr_slice': you cannot assign to a variable that is const
I am calculating the outer product of a 1-D and a 2-D array:
blaze::DynamicMatrix<int> m{{42, 1, 1}, {4, 5, 6}};
blaze::DynamicVector<int> v{1, 2};
blaze::DynamicTensor<int> result(v.size(), m.rows(), m.columns(),-10);
for (std::size_t i=0; i!=m.rows();++i)
{
auto slice = blaze::rowslice(result, i);
slice = blaze::trans(blaze::outer(v, blaze::row(m, i)));
}
while the code in the above block works, I like to change the order of arrays in blaze::outer
so I would get rid of the extra trans
:
for (std::size_t i=0; i!=m.rows();++i)
{
auto slice = blaze::rowslice(result, i);
slice = blaze::outer(blaze::row(m, i), v);;
}
binary '=': no operator found which takes a right-hand operand of type 'ReturnType' (or there is no acceptable conversion)
Here is an incomplete list of things that have to be implemented:
3D Tensor operations
Blaze data structures
blaze::StaticTensor<T, ...>
blaze::UniformTensor<T>
blaze::HybridTensor<T>
Blaze views
blaze::PageSlices
blaze::RowSlices
blaze::ColumnSlices
blaze::ravel
for matrices and tensors (flatten into a 1D sequence of values) (#9, #10)blaze::dilatedsubvector
(#22)blaze::dilatedsubmatrix
(#25)blaze::dilatedsubtensor
(#30)blaze::flip
as a view to flip the array over all its axesUsing Blaze, the result of a matrix * vector is the same as their dot product:
blaze::DynamicVector<int> a{1, 1, 0};
blaze::DynamicMatrix<int> b{{1, 2, 3}, {-1, -2, -3}};
auto ans = b * a;
would results in
( 3 )
( -3 )
Having a matrix * matrix is also results in their dot product:
blaze::DynamicMatrix<int> a{{1, 0}, {1, -1}, {2, -2}};
blaze::DynamicMatrix<int> b{{1, 2, 3}, {-1, -2, -3}};
auto ans = b * a;
( 9 -8 )
( -9 8 )
That would be really useful to have the same overloads for a tensor. I think we need to add the equivalents of dot3d1d, dot3d2d and dot2d3d.
For example, for calculating tensor * vector (like in dot3d1d), the for
loop in the following:
blaze::DynamicVector<int> a{1, 1, 0};
blaze::DynamicTensor<int> b{{{1, 2, 3}, {-1, -2, -3}},
{{4, 5, 6}, {-4, -5, -6}}};
blaze::DynamicMatrix<int> ans(b.pages(), b.rows());
for (std::size_t i = 0; i != b.pages(); ++i)
blaze::row(blaze::submatrix(ans, i, 0, 1, b.rows()), 0) =
blaze::trans(blaze::pageslice(b, i) * a);
would results in the desired ans = b*a
.
Although blaze has covered several fundamental operations, it does not have an operation to calculate the average. I believe having mean
calculations can be really helpful.
As an example, I need to implement some pooling operation. In the simplest case, I have:
blaze::DynamicTensor<int> t{
{{1, 2, 3, 4}, {5, 6, 7, 8}, {9, 10, 11, 12}},
{{-1, -2, -3, -4}, {-5, -6, -7, -8}, {-9, -10, -11, -12}}};
std::size_t filter_depth = 2;
std::size_t filter_height = 2;
std::size_t filter_width = 2;
blaze::DynamicTensor<double> result(t.pages() - filter_depth + 1,
t.rows() - filter_height + 1, t.columns() - filter_width + 1);
for (std::size_t p = 0; p != result.pages(); ++p)
for (std::size_t r = 0; r != result.rows(); ++r)
for (std::size_t c = 0; c != result.columns(); ++c)
result(p, r, c) = blaze::sum(blaze::subtensor(
t, p, r, c, filter_depth, filter_height, filter_width));
result /= (filter_depth * filter_height * filter_width);
I need to use blaze::sum
and a division. In more complex cases, the subtensor has a different size for each group of elements of my result tensor and I have to calculate divisions for each case separately.
See:
https://bitbucket.org/blaze-lib/blaze/commits/5191c499b68d40fa0b08f7a36b4ae5720a97b7b7
(the Phylanx build is now broken)
Blaze supports the softmax operation and its rowwise and columnwise overloads (ref).
I like to request for the blaze_tensor to support this operation for tensors. As an example:
#include <iostream>
#include <blaze/Math.h>
int main()
{
blaze::StaticTensor<double, 2UL, 3UL, 3UL> A{ { { 1.0, 2.0, 3.0 }
, { 4.0, 1.0, 2.0 }
, { 3.0, 4.0, 1.0 } },
{ { 3.0, 6.0, 2.0}
, { -2.0, 2.0, 0.0 }
, { 1.0, 1.0, 3.0 } } };
blaze::StaticTensor<double, 2UL, 3UL, 3UL> B;
B = blaze::softmax<blaze::columnwise>(A);
std::cout << B << "\n";
return 0;
}
results in
[[[ 0.09003057, 0.24472847, 0.66524096],
[ 0.84379473, 0.04201007, 0.1141952 ],
[ 0.25949646, 0.70538451, 0.03511903]],
[[ 0.04661262, 0.93623955, 0.01714783],
[ 0.01587624, 0.86681333, 0.11731043],
[ 0.10650698, 0.10650698, 0.78698604]]]
An example of possible python implementation is:
import numpy as np
def softmax(x, axis=-1):
y = np.exp(x - np.max(x, axis, keepdims=True))
return y / np.sum(y, axis, keepdims=True)
however, the blaze implementation would be mappped on the above function if we use softmax<rowwise>
as axis=1
and softmax<columnwise>
as axis=0
.
I like to ask blaze_tensor to add support for the l2Norm
since it is one of the most useful norms in machine learning. A numpy equivalent implementation is:
def l2_norm(x):
y = np.max(np.sum(x ** 2))
return np.sqrt(y)
The following code:
struct isSame
{
bool operator()(double lhs, double rhs) const
{
return lhs == rhs;
}
};
blaze::DynamicArray<4UL, int> q{{{{{-5, -3, -1, -2, 0}}}}};
blaze::DynamicTensor<int> t{{{-5, -3, -1, -2, 0}}};
std::cout<<blaze::reduce(blaze::map(q, q, isSame{}), std::logical_and<bool>{})<<"\n";
std::cout<<blaze::reduce(blaze::map(t, t, isSame{}), std::logical_and<bool>{})<<"\n";
would print:
0
1
The following code generates compile time error:
blaze::DynamicArray<4UL, int> q{{{{{1, 2}, {3, 4}},
{{5, 6}, {7, 8}},
{{9, 10}, {11, 12}}},
{{{13, 14}, {15, 16}},
{{17, 18}, {19, 20}},
{{21, 22}, {23, 24}}}}};
blaze::DynamicTensor<int> t{{{-5, -3}, {-4, 1}, {0, 2}},
{{-5, -3}, {-4, 1}, {0, 2}}};
blaze::quatslice(blaze::trans(q, {3, 0, 1, 2}), 0) % t;
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(171): error C2794: 'OppositeType': is not a member of any direct or indirect base class of 'blaze::INVALID_TYPE'
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(171): note: see reference to alias template instantiation 'blaze::OppositeType_t<blaze::INVALID_TYPE>' being compiled
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(1205): note: see reference to class template instantiation 'blaze::DTensDTensSchurExpr<blaze::QuatSlice<AT>,blaze::DynamicTensor<ET>>' being compiled
1> with
1> [
1> AT=blaze::DQuatTransExpr<blaze::DynamicArray<4,int>>,
1> ET=int
1> ]
1>c:\repos\blaze_app_1\prog.cpp(2060): note: see reference to function template instantiation 'decltype(auto) blaze::operator %<blaze::QuatSlice<AT>,blaze::DynamicTensor<ET>>(const blaze::DenseTensor<blaze::QuatSlice<AT>> &,const blaze::DenseTensor<blaze::DynamicTensor<ET>> &)' being compiled
1> with
1> [
1> AT=blaze::DQuatTransExpr<blaze::DynamicArray<4,int>>,
1> ET=int
1> ]
1>c:\repos\hpx\libs\parallel_executors\include\hpx\parallel\execution_policy.hpp(648): note: see reference to class template instantiation 'hpx::parallel::execution::parallel_policy_executor<hpx::launch>' being compiled
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(171): error C2938: 'blaze::OppositeType_t<blaze::INVALID_TYPE>' : Failed to specialize alias template
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(172): error C2794: 'TransposeType': is not a member of any direct or indirect base class of 'blaze::INVALID_TYPE'
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(172): note: see reference to alias template instantiation 'blaze::TransposeType_t<blaze::INVALID_TYPE>' being compiled
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(172): error C2938: 'blaze::TransposeType_t<blaze::INVALID_TYPE>' : Failed to specialize alias template
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(173): error C2794: 'ElementType': is not a member of any direct or indirect base class of 'blaze::INVALID_TYPE'
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(173): note: see reference to alias template instantiation 'blaze::ElementType_t<blaze::INVALID_TYPE>' being compiled
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(173): error C2938: 'blaze::ElementType_t<blaze::INVALID_TYPE>' : Failed to specialize alias template
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(176): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(176): error C2144: syntax error: 'unknown-type' should be preceded by ';'
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(176): error C2208: 'unknown-type': no members defined using this type
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(449): error C2955: 'blaze::SIMDTrait': use of class template requires template argument list
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\simd\simdtrait.h(296): note: see declaration of 'blaze::SIMDTrait'
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(449): error C2039: 'size': is not a member of 'blaze::SIMDTrait<T>'
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(449): error C2440: 'initializing': cannot convert from 'overloaded-function' to 'const size_t'
1>c:\repos\blaze_tensor\blaze_tensor\math\expressions\dtensdtensschurexpr.h(449): note: Context does not allow for disambiguation of overloaded function
1>Done building project "app1.vcxproj" -- FAILED.
If we didn't need to transpose q
this would have successfully compiled.
Note that blaze::quatslice(blaze::trans(q, {3, 0, 1, 2}), 0)
is equivalent to this:
(( 1 3 ) ( 5 7 ) ( 9 11 ) )
(( 13 15 ) ( 17 19 ) ( 21 23 ) )
This might be interesting to look at in order to generate more complicated tensor expressions:
Having the following code:
blaze::DynamicTensor<std::int64_t> t{{{1, 2, 3}}, {{4, 5, 6}}};
std::cout<<blaze::trans(t, {1, 0, 2}) << std::endl;
results in
(( 1 2 3 ) ( 4 5 6 ) )
which is the correct response.
If we don't use an initializer list to pass the indices, the result is different. In this example:
std::array<std::int64_t, 3> v = {1, 0, 2};
blaze::DynamicTensor<std::int64_t> t{{{1, 2, 3}}, {{4, 5, 6}}};
std::cout<<blaze::trans(t, v.data(), v.size()) << std::endl;
results in:
(( 1 4 ) )
(( 2 5 ) )
(( 3 6 ) )
Note that the above result is the same as the default transposition std::cout << blaze::trans(t) << std::endl;
, therefore the given indices are not taken into consideration in at least one of the DTensTransExpr constructors.
Having the following code
blaze::DynamicTensor<int> t1{{{1, 2, 33},{2, 13, 4}}};
blaze::DynamicTensor<int> t2{{{0, 2, 10, -2},{42, 4, 1, 6}}};
blaze::DynamicTensor<int> result(t1.pages(), t1.columns(), t2.columns());
blaze::pageslice(result, 0) = blaze::trans(blaze::pageslice(t1, 0))*blaze::pageslice(t2, 0);
I get:
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\expressions\matrix.h(1041): error C2672: 'blaze::PageSlice<MT>::assign': no matching overloaded function found
1> with
1> [
1> MT=blaze::DynamicTensor<int>
1> ]
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\expressions\matrix.h(1101): note: see reference to function template instantiation 'void blaze::assign_backend<MT,false,blaze::DynamicMatrix<Type,true>>(blaze::Matrix<MT,false> &,const blaze::Matrix<blaze::DynamicMatrix<Type,true>,true> &)' being compiled
1> with
1> [
1> MT=blaze::PageSlice<blaze::DynamicTensor<int>>,
1> Type=int
1> ]
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\smp\hpx\densematrix.h(304): note: see reference to function template instantiation 'void blaze::assign<MT,false,blaze::DynamicMatrix<Type,true>,true>(blaze::Matrix<MT,false> &,const blaze::Matrix<blaze::DynamicMatrix<Type,true>,true> &)' being compiled
1> with
1> [
1> MT=blaze::PageSlice<blaze::DynamicTensor<int>>,
1> Type=int
1> ]
1>c:\repos\blaze_tensor\blaze_tensor\math\views\pageslice\dense.h(880): note: see reference to function template instantiation 'void blaze::smpAssign<MT,false,blaze::DynamicMatrix<Type,true>,true>(blaze::Matrix<MT,false> &,const blaze::Matrix<blaze::DynamicMatrix<Type,true>,true> &)' being compiled
1> with
1> [
1> MT=blaze::PageSlice<blaze::DynamicTensor<int>>,
1> Type=int
1> ]
1>c:\repos\blaze_app_1\prog.cpp(1535): note: see reference to function template instantiation 'blaze::PageSlice<MT> &blaze::PageSlice<MT>::operator =<blaze::TDMatDMatMultExpr<MT1,MT2,false,false,false,false>,true>(const blaze::Matrix<blaze::TDMatDMatMultExpr<MT1,MT2,false,false,false,false>,true> &)' being compiled
1> with
1> [
1> MT=blaze::DynamicTensor<int>,
1> MT1=blaze::DMatTransExpr<blaze::PageSlice<blaze::DynamicTensor<int>>,true>,
1> MT2=blaze::PageSlice<blaze::DynamicTensor<int>>
1> ]
1>c:\repos\blaze_app_1\prog.cpp(1535): note: see reference to function template instantiation 'blaze::PageSlice<MT> &blaze::PageSlice<MT>::operator =<blaze::TDMatDMatMultExpr<MT1,MT2,false,false,false,false>,true>(const blaze::Matrix<blaze::TDMatDMatMultExpr<MT1,MT2,false,false,false,false>,true> &)' being compiled
1> with
1> [
1> MT=blaze::DynamicTensor<int>,
1> MT1=blaze::DMatTransExpr<blaze::PageSlice<blaze::DynamicTensor<int>>,true>,
1> MT2=blaze::PageSlice<blaze::DynamicTensor<int>>
1> ]
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\views\check.h(138): note: see reference to class template instantiation 'blaze::Check<false>' being compiled
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\views\check.h(121): note: see reference to class template instantiation 'blaze::Check<true>' being compiled
1>c:\repos\hpx\hpx\parallel\execution_policy.hpp(648): note: see reference to class template instantiation 'hpx::parallel::execution::parallel_policy_executor<hpx::launch>' being compiled
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\expressions\matrix.h(1041): error C2784: 'EnableIf<Condition,T>::Type blaze::PageSlice<MT>::assign(const blaze::DenseMatrix<MT2,false> &)': could not deduce template argument for 'const blaze::DenseMatrix<MT2,false> &' from 'const blaze::DynamicMatrix<Type,true>'
1> with
1> [
1> T=void,
1> MT=blaze::DynamicTensor<int>
1> ]
1> and
1> [
1> Type=int
1> ]
1>c:\repos\blaze_tensor\blaze_tensor\math\views\pageslice\dense.h(327): note: see declaration of 'blaze::PageSlice<MT>::assign'
1> with
1> [
1> MT=blaze::DynamicTensor<int>
1> ]
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\expressions\matrix.h(1041): error C2784: 'DisableIf<Condition,T>::Type blaze::PageSlice<MT>::assign(const blaze::DenseMatrix<MT2,false> &)': could not deduce template argument for 'const blaze::DenseMatrix<MT2,false> &' from 'const blaze::DynamicMatrix<Type,true>'
1> with
1> [
1> T=void,
1> MT=blaze::DynamicTensor<int>
1> ]
1> and
1> [
1> Type=int
1> ]
1>c:\repos\blaze_tensor\blaze_tensor\math\views\pageslice\dense.h(324): note: see declaration of 'blaze::PageSlice<MT>::assign'
1> with
1> [
1> MT=blaze::DynamicTensor<int>
1> ]
But I think I should be able to do the assignment since the slices have the same shape.
The following code does not generate the desired output:
blaze::DynamicTensor<int> t{{{1, 2, 3}, {4, 5, 6}},
{{7, 8, 9}, {10, 11, 12}}};
auto b = blaze::trans(t);
std::cout << blaze::pageslice(b, 0);
If instead of auto
we use blaze::DynamicTensor<int>
the code would generate the correct results.
Having more than one page, sum
does not calculate correctly.
blaze::DynamicTensor<int> t{{{1,0,0}, {0,0,0}},
{{1,0,0}, {0,0,0}}};
std::cout << blaze::sum(t) << std::endl;
3
The correct response is 2.
In order to implement reshape
and flatten
a ravel
expression is useful.
An implementation of bias_add
uses extracting the quatern value and then add it to the original array, so it needs add
.
I like to ask for a 1-D view of 2-D and 3-D arrays in blaze. I believe NumPy ravel in its default order (C) does the same thing and it is really useful in implementing products.
NumPy outer uses the flattened arrays where they are >1d. In other cases it is the same as tensordot(.,.,axes=0)
Also, in case of numpy.tensordot(a,b,axes=2)
where a
and b
are 2-D arrays we can calculate numpy.dot(a.ravel(), b.ravel())
In the following code, I defined a tensor and tried to access a vector of it. When I print out the size I get 1
but still, I seem to be able to assign a value to its second element.
blaze::DynamicTensor<double> t(1, 2, 9, 0);
std::cout<<(blaze::row(
blaze::columnslice(blaze::subtensor(t, 0, 0, 0, 1, 1, 3), 0),
0).size())<<std::endl;
blaze::row(
blaze::columnslice(blaze::subtensor(t, 0, 0, 0, 1, 1, 3), 0),
0)[1] = 100;
std::cout<<t<<std::endl;
1
(( 0 0 0 0 0 0 0 0 0 ) ( 100 0 0 0 0 0 0 0 0 ) )
The Blaze header IntegerSequence.h moved from blaze/math to blaze/util. See https://bitbucket.org/blaze-lib/blaze/commits/4cec1773fa38372e24981081dde4da0a2b6bcf25
int main()
{
blaze::DynamicVector<int> v{ 1, 2, 3, 4, 5, 6};
blaze::dilatedsubvector(blaze::subvector(v,1,5), 1, 2, 3);
}
( 3 )
( 6 )
1>c:\repos\blaze_tensor\blaze_tensor\math\views\dilatedsubvector\dense.h(582): error C2338: Subvector type detected
1>c:\repos\blaze_tensor\blaze_tensor\math\views\dilatedsubvector.h(405): note: see reference to class template instantiation 'blaze::DilatedSubvector<VT,false,true>' being compiled
1> with
1> [
1> VT=blaze::Subvector<blaze::DynamicVector<int,false>,blaze::unaligned,false,true>
1> ]
1>c:\repos\blaze_app_1\prog.cpp(1978): note: see reference to function template instantiation 'decltype(auto) blaze::dilatedsubvector<VT,false,>(blaze::Vector<VT,false> &&,size_t,size_t,size_t)' being compiled
1> with
1> [
1> VT=blaze::Subvector<blaze::DynamicVector<int,false>,blaze::unaligned,false,true>
1> ]
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\views\check.h(138): note: see reference to class template instantiation 'blaze::Check<false>' being compiled
1>c:\repos\vcpkg\installed\x64-windows\include\blaze\math\views\check.h(121): note: see reference to class template instantiation 'blaze::Check<true>' being compiled
1>c:\repos\hpx\hpx\parallel\execution_policy.hpp(648): note: see reference to class template instantiation 'hpx::parallel::execution::parallel_policy_executor<hpx::launch>' being compiled
1>Done building project "app1.vcxproj" -- FAILED.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.