xtensor-stack / xtensor-python Goto Github PK
View Code? Open in Web Editor NEWPython bindings for xtensor
License: BSD 3-Clause "New" or "Revised" License
Python bindings for xtensor
License: BSD 3-Clause "New" or "Revised" License
Looks like when I try to build the test project standalone with the xtensor-python-dev
package (which pulls xtensor-dev
and pybind11-dev
), the necessary include path for the Python headers is missing:
[ 55%] Building CXX object CMakeFiles/test_xtensor_python.dir/main.cpp.o
/usr/bin/c++ -I/home/gvaillan/debian/packages/xtensor-python/build/googletest-src/googletest/include -march=native -Wunused-parameter -Wextra -Wreorder -Wconversion -std=c++14 -O3 -DNDEBUG -o CMakeFiles/test_xtensor_python.dir/main.cpp.o -c /home/gvaillan/debian/packages/xtensor-python/test/main.cpp
/home/gvaillan/debian/packages/xtensor-python/test/main.cpp:9:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
I wonder which part in the root CMakeLists.txt
is adding it. Perhaps the Numpy CMake detection which is missing from test/CMakeLists.txt
?
running build_ext building 'pyaabbtree' extension C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\Eelco\Miniconda3\envs\pyaabbtree\Include -IC:\Users\Eelco\AppData\ Roaming\Python\Python36\Include -IC:\Users\Eelco\Miniconda3\envs\pyaabbtree\lib\site-packages\numpy\core\include -IC:\Users\Eelco\Dropbox\Git\xtensor\include -IC:\Users\Eelco\Miniconda3\env s\pyaabbtree\include -IC:\Users\Eelco\Miniconda3\envs\pyaabbtree\include -IC:\Users\Eelco\Miniconda3\envs\pyaabbtree\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUD E" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETF XSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include \\winrt" /EHsc /Tpsrc/main.cpp /Fobuild\temp.win-amd64-3.6\Release\src/main.obj /EHsc /DVERSION_INFO=\"0.0.1\" main.cpp c:\users\eelco\dropbox\git\xtensor\include\xtensor\xfunction.hpp(132): error C2039: 'layout_type': is not a member of 'xt::pyarray<double>' src/main.cpp(18): note: see declaration of 'xt::pyarray<double>' src/main.cpp(25): note: see reference to class template instantiation 'xt::xfunction<std::plus<double>,double,const xt::pyarray<double> &,xt::xscalar<const int>>' being compiled c:\users\eelco\dropbox\git\xtensor\include\xtensor\xfunction.hpp(132): error C2065: 'layout_type': undeclared identifier c:\users\eelco\dropbox\git\xtensor\include\xtensor\xfunction.hpp(133): error C2039: 'contiguous_layout': is not a member of 'xt::pyarray<double>' src/main.cpp(18): note: see declaration of 'xt::pyarray<double>' c:\users\eelco\dropbox\git\xtensor\include\xtensor\xfunction.hpp(133): error C2065: 'contiguous_layout': undeclared identifier c:\users\eelco\dropbox\git\xtensor\include\xtensor\xfunction.hpp(133): error C2975: 'B': invalid template argument for 'xt::and_c', expected compile-time constant expression c:\users\eelco\dropbox\git\xtensor\include\xtensor\xutils.hpp(210): note: see declaration of 'B' error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
Am I using incompatible versions of the various packages? Sadly I cant see right away if this is some form of trivial error without familiarizing myself with the codebase first.
The shape_type
for pyarray should be a friend class to pyarray that takes a reference to that pyarray, being a proxy on the numpy array shape buffer and dimensionality (by reference).
This should make it easier to track undetected pybind11 bugs in the master branch.
As in xarray/xtensor
See the Debian CI test logs for version 0.10.9 and version 0.10.10.
[==========] Running 32 tests from 3 test cases.
[----------] Global test environment set-up.
[----------] 15 tests from pyarray
[ RUN ] pyarray.initializer_constructor
[ OK ] pyarray.initializer_constructor (0 ms)
[ RUN ] pyarray.shaped_constructor
[ OK ] pyarray.shaped_constructor (0 ms)
[ RUN ] pyarray.strided_constructor
[ OK ] pyarray.strided_constructor (0 ms)
[ RUN ] pyarray.valued_constructor
[ OK ] pyarray.valued_constructor (0 ms)
[ RUN ] pyarray.strided_valued_constructor
[ OK ] pyarray.strided_valued_constructor (0 ms)
[ RUN ] pyarray.copy_semantic
[ OK ] pyarray.copy_semantic (0 ms)
[ RUN ] pyarray.move_semantic
[ OK ] pyarray.move_semantic (0 ms)
[ RUN ] pyarray.extended_constructor
[ OK ] pyarray.extended_constructor (0 ms)
[ RUN ] pyarray.reshape
[ OK ] pyarray.reshape (0 ms)
[ RUN ] pyarray.access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:321: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec(2, 1)
Which is: 9
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:316: row_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:332: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec(2, 1)
Which is: 2
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:327: column_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:343: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec(2, 1)
Which is: 2
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:338: central_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:354: Failure
Expected: vec(2, 0, 0)
Which is: 16
To be equal to: vec(2, 0)
Which is: -1
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:349: unit_shape access
[ FAILED ] pyarray.access (0 ms)
[ RUN ] pyarray.indexed_access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:390: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec[index1]
Which is: 9
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:385: row_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:400: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec[index1]
Which is: 2
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:395: column_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:410: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec[index1]
Which is: 2
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:405: central_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:422: Failure
Expected: vec(2, 0, 0)
Which is: 16
To be equal to: vec[id1]
Which is: -1
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:415: unit_shape access
[ FAILED ] pyarray.indexed_access (0 ms)
[ RUN ] pyarray.broadcast_shape
[ OK ] pyarray.broadcast_shape (0 ms)
[ RUN ] pyarray.iterator
[ OK ] pyarray.iterator (0 ms)
[ RUN ] pyarray.initializer_list
[ OK ] pyarray.initializer_list (0 ms)
[ RUN ] pyarray.zerod
[ OK ] pyarray.zerod (0 ms)
[----------] 15 tests from pyarray (0 ms total)
[----------] 14 tests from pytensor
[ RUN ] pytensor.initializer_constructor
[ OK ] pytensor.initializer_constructor (0 ms)
[ RUN ] pytensor.shaped_constructor
[ OK ] pytensor.shaped_constructor (0 ms)
[ RUN ] pytensor.strided_constructor
[ OK ] pytensor.strided_constructor (0 ms)
[ RUN ] pytensor.valued_constructor
[ OK ] pytensor.valued_constructor (0 ms)
[ RUN ] pytensor.strided_valued_constructor
[ OK ] pytensor.strided_valued_constructor (0 ms)
[ RUN ] pytensor.copy_semantic
[ OK ] pytensor.copy_semantic (0 ms)
[ RUN ] pytensor.move_semantic
[ OK ] pytensor.move_semantic (0 ms)
[ RUN ] pytensor.extended_constructor
[ OK ] pytensor.extended_constructor (0 ms)
[ RUN ] pytensor.reshape
[ OK ] pytensor.reshape (0 ms)
[ RUN ] pytensor.access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:321: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec(2, 1)
Which is: 9
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:316: row_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:332: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec(2, 1)
Which is: 2
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:327: column_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:343: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec(2, 1)
Which is: 2
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:338: central_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:354: Failure
Expected: vec(2, 0, 0)
Which is: 16
To be equal to: vec(2, 0)
Which is: -1
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:349: unit_shape access
[ FAILED ] pytensor.access (0 ms)
[ RUN ] pytensor.indexed_access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:390: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec[index1]
Which is: 9
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:385: row_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:400: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec[index1]
Which is: 2
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:395: column_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:410: Failure
Expected: vec(2, 1, 0)
Which is: 20
To be equal to: vec[index1]
Which is: 2
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:405: central_major access
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:422: Failure
Expected: vec(2, 0, 0)
Which is: 16
To be equal to: vec[id1]
Which is: -1
Google Test trace:
/tmp/autopkgtest-virt-lxc.shared.v25qf1sr/downtmp/autopkgtest_tmp/test_common.hpp:415: unit_shape access
[ FAILED ] pytensor.indexed_access (1 ms)
[ RUN ] pytensor.broadcast_shape
[ OK ] pytensor.broadcast_shape (0 ms)
[ RUN ] pytensor.iterator
[ OK ] pytensor.iterator (0 ms)
[ RUN ] pytensor.zerod
[ OK ] pytensor.zerod (0 ms)
[----------] 14 tests from pytensor (1 ms total)
[----------] 3 tests from pyvectorize
[ RUN ] pyvectorize.function
[ OK ] pyvectorize.function (0 ms)
[ RUN ] pyvectorize.lambda
[ OK ] pyvectorize.lambda (0 ms)
[ RUN ] pyvectorize.complex
[ OK ] pyvectorize.complex (0 ms)
[----------] 3 tests from pyvectorize (0 ms total)
[----------] Global test environment tear-down
[==========] 32 tests from 3 test cases ran. (1 ms total)
[ PASSED ] 28 tests.
[ FAILED ] 4 tests, listed below:
[ FAILED ] pyarray.access
[ FAILED ] pyarray.indexed_access
[ FAILED ] pytensor.access
[ FAILED ] pytensor.indexed_access
4 FAILED TESTS
pybind::array
's accessors to not require creating a temporary array, by using the same method as in xtensor.I tried this based on example 1, but it seems that there no type casters like pybind11's stl.h
which will auto-cast std::vector
and etc. to python types.
double _sum_of_sines(xt::xarray<double> &m)
{
auto sines = xt::sin(m);
return std::accumulate(sines.cbegin(), sines.cend(), 0.0);
}
double sum_of_sines(xt::pyarray<double> &m)
{
return _sum_of_sines(m.cast<xt::xarray<double>>());
}
It seems noconvert is not supported? I tried adding noconvert to my (now classic) summer.cc:
#include <numpy/arrayobject.h>
#include "pybind11/numpy.h" // complex
#include "pybind11/pybind11.h"
#include "pybind11/operators.h"
#include "numpy/arrayobject.h"
//#include "ndarray/pybind11.h"
#include "xtensor/xtensor.hpp"
#include "xtensor/xcontainer.hpp"
#include "xtensor-python/pyarray.hpp"
#include <complex>
namespace py = pybind11;
PYBIND11_PLUGIN (summer) {
if (_import_array() < 0) {
PyErr_SetString(PyExc_ImportError, "numpy.core.multiarray failed to import");
return nullptr;
}
py::module m("summer", "pybind11 example plugin");
// m.def("sum", [](xt::pyarray<double> const& x) {
// xt::xtensor<double, 1> tmp(x);
// double sum = 0;
// for (auto e : tmp)
// sum += e;
// return sum;
// });
m.def("sum", [](xt::pyarray<std::complex<double>> const& x) {
std::complex<double> sum = 0;
for (auto e : x)
sum += e;
return sum;
},
py::arg("inp").noconvert()
);
m.def("sum", [](xt::pyarray<double> const& x) {
double sum = 0;
for (auto e : x)
sum += e;
return sum;
},
py::arg("inp").noconvert()
);
return m.ptr();
}
It seems to have no effect. If I reverse the order of the defs (double is 1st), then when passing a complex array I get the numpy warning of discarding imag part. If I use the order above, the time for processing float array is much more than for complex array, suggesting there is conversion.
Just as xtensor
provides both xarray<T>
and xtensor<T, n>
, it would be nice if xtensor-python
provided a fixed-dimensionality container (that would raise at call time if an array(-like) with invalid dimensionality was passed in from python).
Hi guys, I find that xtensor-python will be installed under the envs/include
directory rather than envs/include/python/
when using conda, and user will need to specify the include path if they do not manually install it.
pybind11
is installed in envs/include/python/
by default.
I have run into a few issues with a similar allocator to the one added in #23 and wanted to warn you about them so you can decide to either remove it or fix it.
PyMem_(Malloc|Free)
requires the GIL.PyMem_Malloc
to be freed.The issue with 1 is that it makes it hard to release the gil because any internal allocations will require the gil and it is not obvious to a consumer what can allocate.
The issue with 2 is that if you store an object at global scope whose destructor invokes PyMem_Free
you will get a double free or other failure because the memory was already released.
For my application I have just switched to PyMem_Raw(Malloc|Calloc|Free)
but those are wrappers around malloc|calloc|free with slightly different NULL and 0 handling. I feel bad for adding this without sufficient testing and I hope you haven't had too many issues because of this.
If I'm understanding the docs for xarray::size()
correctly, then in the 1D case, size()
and shape()[0]
should always be identical. Is that correct?
In the following pyarray
test case, size()
is not consistent with its shape()
. Notice that xtensor thinks that col0
has a size is 10
even though its shape is only (5,)
I discovered this using the latest releases of xtensor
and xtensor-python
, but I can reproduce the behavior using the latest master branches.
Test case:
import numpy as np
from example import print_xt_info
rows = np.zeros((5,2), dtype=np.uint32)
print(f"rows:\t size: {rows.size} shape: {rows.shape} strides: {rows.strides}")
print_xt_info(rows)
col0 = rows[:,0]
print(f"col0:\t size: {col0.size} shape: {col0.shape} strides: {col0.strides}")
print_xt_info(col0)
Output:
rows: size: 10 shape: (5, 2) strides: (8, 4)
xt_a: size: 10 shape: [5,2,] strides: [2,1,]
col0: size: 5 shape: (5,) strides: (8,)
xt_a: size: 10 shape: [5,] strides: [2,]
#include <iostream>
#include <cstdint>
#include "pybind11/pybind11.h"
#define FORCE_IMPORT_ARRAY
#include "xtensor-python/pyarray.hpp"
namespace py = pybind11;
using std::uint32_t;
template <class T>
std::string seq_to_string(T v)
{
std::ostringstream ss;
ss << "[";
for ( auto x : v )
ss << x << ",";
ss << "]";
return ss.str();
}
inline void print_xt_info(xt::pyarray<uint32_t> const & xa)
{
std::cout << "xt_a:\t"
<< " size: " << xa.size()
<< " shape: " << seq_to_string(xa.shape())
<< " strides: " << seq_to_string(xa.strides())
<< std::endl << std::endl;
}
PYBIND11_PLUGIN(example)
{
xt::import_numpy();
py::module m("example", "");
m.def("print_xt_info", print_xt_info);
return m.ptr();
}
This should be done via the xoptional assembly proposed in xtensor-stack/xtensor#238.
@wolfv added a transpose method to the two main containers of xtensor
. We should probably add the same to pyarray
.
How can I use xtensor-python with numpy arrays? I'd want to call functions implemented with xtensor c++ from python passing numpy arrays, and also return numpy arrays to python.
Is there a discussion or mail list for these questions?
Thanks
As described in #30, binding arrays of complex works fine with pybind11
2.0.1 but crashes with the master branch. The bug is triggered after many calls, like in the benchmarks.
The benchmarks should be added to the CI scripts in order to trigger the bug.
I want to construct a pytensor of all zeros. Should the following compile (it doesn't):
void F(){
xt::pytensor<int,2> x (xt::zeros<int> ({1,1}));
}
This would avoid the need to call pybind11 API in the type_caster.
Is there a way to go from a xt::pytensor to xt::xtensor and back again without copying the underlying data ?
If so, it would be nice with some documentation on how to do it. If not, then I have a feature request :) .
Trying out broadcast. The test is trying to broadcast the 3rd arg (int) to an array matching the shape of the 1st arg, x. x and y are 1d arrays.
The error seems to be coming from the py::print
.def("__call__", [](hist_t& h, xt::pyarray<double> const& x, xt::pyarray<double> const& y, int w) {
if (x.shape() != y.shape())
throw std::runtime_error ("histogram2d::shape mismatch");
auto ws = xt::broadcast (w, {x.shape()[0]});
py::print (ws.shape());
for (size_t s = 0; s < x.size(); ++s)
h (x(s), y(s), ws(s));
},
py::arg("x"), py::arg("y"), py::arg("w")=1)
python3 ./test_histogram2d.py
Traceback (most recent call last):
File "./test_histogram2d.py", line 5, in <module>
h (np.zeros (10), np.zeros(10))
RuntimeError: make_tuple(): unable to convert arguments of types 'std::tuple<std::array<unsigned long, 1ul> const&>' to Python object
Putting an xt::pytensor
inside an std::tuple
seems to cause a memory leak, e.g.:
void leak() {
xt::pytensor<float, 1> ones = xt::ones<float>({10});
std::tuple<xt::pytensor<float, 1>> tup(ones);
}
Using pybind11's py::tuple
doesn't cause the same problem.
Am I doing something wrong here or is this a bug?
See https://github.com/daniel-ziegler/tuple_leak/blob/master/tuple_fun.cpp for a full demonstration of the problem; you can install the package and run https://github.com/daniel-ziegler/tuple_leak/blob/master/demo.py. Intriguingly, it looks like the size of the array doesn't affect how much memory gets leaked.
I'm on pybind11 version 2.2.3 and xtensor-python version 0.19.0.
xtensor-0.16.3
xsimd-4.1.6
xtensor-blas-0.11.0
This code:
https://gist.github.com/nbecker/251112eb10e8effbdeb0545eabef6b31
compiled with
g++ -o mag_sqr.cpython-36m-x86_64-linux-gnu.os -c -g -std=c++1z -O3 -march=native -fvisibility=hidden -flto -DNDEBUG -ffast-math -fPIC -DHAVE_CBLAS -DXTENSOR_USE_XSIMD -I/home/nbecker/.local/include/xtensor-blas/flens -I/home/nbecker/.local/include -I/usr/include/eigen3 -I/home/nbecker/.local/lib/python3.6/site-packages/numpy/core/include -I/usr/include/python3.6m mag_sqr.cc
results in errors (rather long, not copied here)
Pyarray and pytensor rely on pybind11 make_caster<T>::name()
to determine the string representation of its value_type
(which is then used for the auto-generated docstring). However, make_caster
assigns the name "int" to every integral type and "float" to every floating type (except for "char"->"str" and "bool"->"bool").
While this makes perfect sens for vanilla Python, this becomes confusing with Numpy where the precise numeric type is important. For example, defining the two overloaded test
functions in the module m
:
m.def("test", [](xt::pyarray<int>&a){});
m.def("test", [](xt::pyarray<long>&a){});
leads to the following docstring:
test(*args, **kwargs)
Overloaded function.
1. test(arg0: numpy.ndarray[int]) -> None
2. test(arg0: numpy.ndarray[int]) -> None
where the two overloads seem to have an identical signature.
Would it be possible to bypass pybind11 make_caster
for types that corresponds to numpy dtypes such that we would get a correct docstring ? (something like: )
1. test(arg0: numpy.ndarray[int32]) -> None
2. test(arg0: numpy.ndarray[int64]) -> None
First of all, it was really easy to get started with xtensor-python
using your cookiecutter template. Thanks!
I tried modifying one of the examples in a seemingly trivial way: I replaced double
with uint8
. And now it crashes! Any idea what's going on?
Here's the condensed example, adapted from example2()
:
#include "pybind11/pybind11.h"
#include "xtensor/xarray.hpp"
#define FORCE_IMPORT_ARRAY
#include "xtensor-python/pyarray.hpp"
#include "xtensor-python/pyvectorize.hpp"
namespace py = pybind11;
using std::uint8_t;
// I'm using uint8_t here instead of double
inline xt::pyarray<uint8_t> example2(xt::pyarray<uint8_t> &m)
{
return m + 2;
}
PYBIND11_PLUGIN(example_module)
{
using namespace pybind11::literals;
xt::import_numpy();
py::module m("example_module", "");
m.def("example2", example2);
return m.ptr();
}
I tried to run this test program:
import numpy as np
from example_module import example2
a = np.zeros((10,), np.uint8)
b = example2(a)
And it crashes with the following error:
Traceback (most recent call last):
File "test2.py", line 5, in <module>
b = example2(a)
RuntimeError: Precondition violation!
Internal error: trivial_assigner called with unequal types.
/miniforge/envs/flyem-forge/include/xtensor/xassign.hpp(377)
I tried other dtypes, too:
Crashes: uint8_t
, uint16_t
, int8_t
, int16_t
No crash: uint32_t
, uint64_t
, int32_t
, int64_t
, float
, double
Setup details: I'm using OSX and clang
with libc++
, with the following conda packages:
numpy 1.13.1 py36_0
python 3.6.2 0
xtensor 0.12.1 0 conda-forge
xtensor-python 0.14.0 0 conda-forge
pybind11 2.1.1 py36_0 conda-forge
(very first issue on github, be indulgent please :) )
When running
(on Sierra 10.12.6, in a conda env in which I previously installed xtensor 0.10.9 and xtensor-python 0.12.1):
pip install ./my_package_created_thanks_to_the_cool_cookiecutter_github_repository (it is named "dyna"),
the following error is returned:
"
Processing ./dyna
Requirement already satisfied: pybind11>=2.0.1 in ./env/lib/python3.6/site-packages (from dyna==0.0.1)
Requirement already satisfied: numpy in ./env/lib/python3.6/site-packages (from dyna==0.0.1)
Installing collected packages: dyna
Running setup.py install for dyna ... error
Complete output from command /Users/jcalais/Prog/workspace/dyna/env/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/x7/h2yj87t94x9gb6nkm0qg1ck40000gn/T/pip-geq3z0bp-build/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/x7/h2yj87t94x9gb6nkm0qg1ck40000gn/T/pip-fon09ep0-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
creating var
creating var/folders
creating var/folders/x7
creating var/folders/x7/h2yj87t94x9gb6nkm0qg1ck40000gn
creating var/folders/x7/h2yj87t94x9gb6nkm0qg1ck40000gn/T
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/jcalais/Prog/workspace/dyna/env/include -arch x86_64 -I/Users/jcalais/Prog/workspace/dyna/env/include -arch x86_64 -I/Users/jcalais/Prog/workspace/dyna/env/include/python3.6m -c /var/folders/x7/h2yj87t94x9gb6nkm0qg1ck40000gn/T/tmpxc3s13qj.cpp -o var/folders/x7/h2yj87t94x9gb6nkm0qg1ck40000gn/T/tmpxc3s13qj.o -std=c++14
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/jcalais/Prog/workspace/dyna/env/include -arch x86_64 -I/Users/jcalais/Prog/workspace/dyna/env/include -arch x86_64 -I/Users/jcalais/Prog/workspace/dyna/env/include/python3.6m -c /var/folders/x7/h2yj87t94x9gb6nkm0qg1ck40000gn/T/tmp3qr7nhob.cpp -o var/folders/x7/h2yj87t94x9gb6nkm0qg1ck40000gn/T/tmp3qr7nhob.o -fvisibility=hidden
building 'dyna' extension
creating build
creating build/temp.macosx-10.7-x86_64-3.6
creating build/temp.macosx-10.7-x86_64-3.6/src
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/jcalais/Prog/workspace/dyna/env/include -arch x86_64 -I/Users/jcalais/Prog/workspace/dyna/env/include -arch x86_64 -I/Users/jcalais/Prog/workspace/dyna/env/include/python3.6m -I/Users/jcalais/.local/include/python3.6m -I/Users/jcalais/Prog/workspace/dyna/env/lib/python3.6/site-packages/numpy/core/include -I/Users/jcalais/Prog/workspace/dyna/env/include -I/Users/jcalais/Prog/workspace/dyna/env/Library/include -I/Users/jcalais/Prog/workspace/dyna/env/include/python3.6m -c src/main.cpp -o build/temp.macosx-10.7-x86_64-3.6/src/main.o -stdlib=libc++ -mmacosx-version-min=10.7 -DVERSION_INFO="0.0.1" -std=c++14 -fvisibility=hidden
In file included from src/main.cpp:3:
In file included from /Users/jcalais/Prog/workspace/dyna/env/include/xtensor/xmath.hpp:20:
In file included from /Users/jcalais/Prog/workspace/dyna/env/include/xtensor/xoperation.hpp:18:
/Users/jcalais/Prog/workspace/dyna/env/include/xtensor/xstrides.hpp:88:26: error: no viable overloaded operator[] for type 'const const xt::pystrides_iterator<8>'
return arg * strides[dim] + raw_data_offset<size_type, S, dim + 1>(strides, args...);
^~~~~~~ ~~~
/Users/jcalais/Prog/workspace/dyna/env/include/xtensor/xstrides.hpp:116:28: note: in instantiation of function template specialization 'xt::detail::raw_data_offset<unsigned long, const xt::pystrides_iterator<8>, 0, unsigned long>' requested here
return detail::raw_data_offset<size_type, const typename S::const_iterator, dim, Arg, Args...>(view, arg, args...);
^
/Users/jcalais/Prog/workspace/dyna/env/include/xtensor/xcontainer.hpp:377:27: note: in instantiation of function template specialization 'xt::data_offset<unsigned long, xt::pystrides_adaptor<8>, 0, unsigned long>' requested here
size_type index = data_offset<size_type>(strides(), static_cast<size_type>(args)...);
^
src/main.cpp:20:13: note: in instantiation of function template specialization 'xt::xcontainer<xt::pyarray >::operator()' requested here
return m(0);
^_
inline reference operator[] (difference_type n) { return *(p_current + n) / N; }
^
1 error generated.
error: command 'gcc' failed with exit status 1**
"
Adding test suites for pyarray
and pytensor
used just like xarray
and xtensor
would probably be the best way to check basic funcitonnality.
Hi,
we should improve the way we are currently looking for xtensor-python dependencies:
find_package(xtl REQUIRED)
message(STATUS "Found xtl: ${xtl_INCLUDE_DIRS}/xtl")
find_package(xtensor REQUIRED)
message(STATUS "Found xtensor: ${xtensor_INCLUDE_DIRS}/xtensor")
find_package(pybind11 REQUIRED)
message(STATUS "Found pybind11: ${pybind11_INCLUDE_DIRS}/pybind11")
find_package(NumPy REQUIRED)
message(STATUS "Found numpy: ${NUMPY_INCLUDE_DIRS}")
I think we could add the following:
find_package
callnumpy
is a direct dependency, it's not documented; which version range is known to work?xtl
is not only an indirect dependency of xtensor
(which would mean we can remove the find_package
and take automatically added PUBLIC
dependency of the xtensor
dependency) but also a direct dependency used in xtensor-python
code: expose it in the readme with its version range as wellmaybe it's impossible...
Thank you for developing xtensor and xtensor-python, which are very practical libraries.
In some cases, I want to write the following functions and export to Python with pybind11:
void foo(const xt::pytensor<double, 1> &a)
{
xt::xtensor<double, 1> b = 0.5 * a;
}
This function can be compiled with g++ (5.4.0@ubuntu 16.04).
However, the following Python code stops with Segmentation Fault (core dumped):
foo(np.array([1]))
although this code works correctly:
foo(np.array([1,2,3]))
If you have some ideas about this bug and don't have enough time for debugging,
I would like to try fixing according to your comments.
This section should describe the best practices when functions need to be exposed to both C++ (operating on xtensor / xarray
) and Python (operating on pytensor / pyarray
)
I am having build issues wit the latest release of xtensor-python (0.20.0)
and xtensor (0.17.4) / xtl (0.4.16).
The lines
#define FORCE_IMPORT_ARRAY
#include "xtensor-python/pyarray.hpp"
are causing:
from /home/cpape/Work/software/src/z5/src/python/lib/z5py.cxx:7:
/home/cpape/Work/software/conda/miniconda3/envs/z5-bld/include/xtensor/xiterator.hpp: In function ‘constexpr auto xt::detail::trivial_begin(C&)’:
/home/cpape/Work/software/conda/miniconda3/envs/z5-bld/include/xtensor/xiterator.hpp:406:25: error: ‘xtl::mpl’ has not been declared
return xtl::mpl::static_if<has_storage_iterator<C>::value>([&](auto self)
^~~
/home/cpape/Work/software/conda/miniconda3/envs/z5-bld/include/xtensor/xiterator.hpp: In function ‘constexpr auto xt::detail::trivial_end(C&)’:
/home/cpape/Work/software/conda/miniconda3/envs/z5-bld/include/xtensor/xiterator.hpp:418:25: error: ‘xtl::mpl’ has not been declared
return xtl::mpl::static_if<has_storage_iterator<C>::value>([&](auto self)
^~~
/home/cpape/Work/software/conda/miniconda3/envs/z5-bld/include/xtensor/xiterator.hpp: In function ‘constexpr auto xt::detail::trivial_begin(const C&)’:
/home/cpape/Work/software/conda/miniconda3/envs/z5-bld/include/xtensor/xiterator.hpp:430:25: error: ‘xtl::mpl’ has not been declared
return xtl::mpl::static_if<has_storage_iterator<C>::value>([&](auto self)
^~~
/home/cpape/Work/software/conda/miniconda3/envs/z5-bld/include/xtensor/xiterator.hpp: In function ‘constexpr auto xt::detail::trivial_end(const C&)’:
/home/cpape/Work/software/conda/miniconda3/envs/z5-bld/include/xtensor/xiterator.hpp:442:25: error: ‘xtl::mpl’ has not been declared
return xtl::mpl::static_if<has_storage_iterator<C>::value>([&](auto self)
^~~
I am using Ubuntu 18.04 and gcc 7.3.
Note that this error does not occur with xtensor-python
(0.19.1), xtensor
(0.16.4) and xtl
(0.4.16)
Please see the below commands to learn about this issue.
root@docker-host:/home/vagrant# uname -a
Linux docker-host 3.13.0-151-generic #201-Ubuntu SMP Wed May 30 14:22:13 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
=======================================
root@docker-host:/opt/anaconda3/pkgs# conda install -c conda-forge xtensor-python
Solving environment: done
# All requested packages already installed.
root@docker-host:/opt/anaconda3/pkgs#
================================
root@docker-host:/home/vagrant# cat > test.py
import numpy as np
import xtensor_python_test as xt
v = np.arange(15).reshape(3, 5)
s = xt.sum_of_sines(v)
print(s)
=================================
root@docker-host:/home/vagrant# python test.py
Traceback (most recent call last):
File "test.py", line 2, in <module>
import xtensor_python_test as xt
ModuleNotFoundError: No module named 'xtensor_python_test'
When we have a function taking a pyarray but pass in a numpy array of integers, the numpy array is implicitly converted to a pyarray of doubles (but this creates a copy of the buffer).
We should think about emitting a warning in this case, or make the warning user modifiable.
Or at least clearly document it.
This would be really helpful with the testing of xtensor-python.
Hi,
currently xtensor python brings seamless integration of xarray
and xtensor
types with numpy. However xtensor_fixed
type is not supported. Hence the following definitions work (with m
an instance of pybind11::module
):
m.def("test_xarray", [](){return xt::xarray<int>{1,2};});
m.def("test_xtensor", [](){return xt::xtensor<int, 1>{1,2};});
while this one fails
m.def("test_xtensor_fixed", [](){return xt::xtensor_fixed <int, xt::xshape<2>>{1,2};});
at runtime. When trying to call the method from python, the following error is reported:
TypeError: Unable to convert function return value to a Python type! The signature was
() -> xt::xfixed_container<int, xt::fixed_shape<2ul>, (xt::layout_type)1, xt::xtensor_expression_tag>
Would it be possible to add seamless integration with xtensor_fixed
?
inline auto add(xt::pyarray<double> &a, xt::pyarray<double> &b)
{
auto res = xt::pyarray<double>();
res.reshape(a.shape());
xt::noalias(res) = (a + b);
return res;
}
fails to compile with:
src/main.cpp: In function ‘auto add(xt::pyarray<double>&, xt::pyarray<double>&)’:
src/main.cpp:26:26: error: no matching function for call to ‘xt::pyarray<double>::reshape(const inner_shape_type&)’
res.reshape(a.shape());
^
In file included from /home/wolfv/Programs/xtensor-python/include/xtensor-python/pyarray.hpp:20:0,
from src/main.cpp:8:
/home/wolfv/Programs/xtensor-python/include/xtensor-python/pycontainer.hpp:222:17: note: candidate: void xt::pycontainer<D>::reshape(const shape_type&) [with D = xt::pyarray<double>; xt::pycontainer<D>::shape_type = std::vector<long unsigned int>]
inline void pycontainer<D>::reshape(const shape_type& shape)
^~~~~~~~~~~~~~
/home/wolfv/Programs/xtensor-python/include/xtensor-python/pycontainer.hpp:222:17: note: no known conversion for argument 1 from ‘const inner_shape_type {aka const xt::xbuffer_adaptor<long unsigned int>}’ to ‘const shape_type& {aka const std::vector<long unsigned int>&}’
/home/wolfv/Programs/xtensor-python/include/xtensor-python/pycontainer.hpp:236:17: note: candidate: void xt::pycontainer<D>::reshape(const shape_type&, xt::layout_type) [with D = xt::pyarray<double>; xt::pycontainer<D>::shape_type = std::vector<long unsigned int>]
inline void pycontainer<D>::reshape(const shape_type& shape, layout_type l)
^~~~~~~~~~~~~~
/home/wolfv/Programs/xtensor-python/include/xtensor-python/pycontainer.hpp:236:17: note: candidate expects 2 arguments, 1 provided
/home/wolfv/Programs/xtensor-python/include/xtensor-python/pycontainer.hpp:249:17: note: candidate: void xt::pycontainer<D>::reshape(const shape_type&, const strides_type&) [with D = xt::pyarray<double>; xt::pycontainer<D>::shape_type = std::vector<long unsigned int>; xt::pycontainer<D>::strides_type = std::vector<long unsigned int>]
inline void pycontainer<D>::reshape(const shape_type& shape, const strides_type& strides)
^~~~~~~~~~~~~~
/home/wolfv/Programs/xtensor-python/include/xtensor-python/pycontainer.hpp:249:17: note: candidate expects 2 arguments, 1 provided
How would one go about taking in (and returning) a numpy array of strings using xtensor-python (assuming ASCII)?
The use case is I have a numpy array containing a bunch of Base64 encoded JPEG images. I want to decode this batch using an OpenMP loop in C++. Ideally I should also be able to return a numpy array of strings.
I know I can work around this by creating a 2D numpy array of bytes (each row contains the ASCII string's bytes) but the problem is that it requires two passes since we have to find the max string length. Not to mention string conversions in python.
See https://xtensor-python.readthedocs.io/en/latest/pyvectorize.html
doxygenfunction: Unable to resolve multiple matches for function “xt::pyvectorize” with arguments () in doxygen xml output for project “xtensor-python” from directory: ../xml. Potential matches:
<elided>
I'm attempting to bind a xt::pyarray<xt::pyarray<double>>
to a Numpy array of arrays, but I am getting compilation errors instantiating the pyarray
. The error is a bit long to include here, the relevant bit is
pybind11/include/pybind11/numpy.h:260:81: error: no member named 'dtype' in 'pybind11::detail::npy_format_descriptor<xt::pyarray<double, 16>, void>'
return detail::npy_format_descriptor<typename std::remove_cv::type>::dtype();
coming from the call to pybind11::dtype::of<T>
in the return ofpyarray::raw_array_t
. Unfortunately, I don't know enough about the inner workings of pybind11
to determine if the problem lies there, or whether this is a xtensor-python
issue.
The problem I am trying to solve is to provide Python bindings to a code that uses xtensor
xarray
. Using xt::xarray<xt::xarray<double>>
in a C++-only situation works as expected. Actually, my case is limited in that I always have three inner arrays, so in C++ I would prefer to use std::array<xt::xarray<double>, 3>
(or perhaps xt::xtensor<xt::xarray<double>, 3>
) but I do not know how to transparently (without copies) expose those as an ndarray of three ndarrays to Python.
Example code that produces the compilation error:
#include <numeric>
#include "pybind11/pybind11.h"
#include "xtensor/xarray.hpp"
#include "xtensor/xmath.hpp"
#include "xtensor-python/pyarray.hpp"
double sum_of_dim(xt::pyarray<xt::pyarray<double>> & m, const int dim)
{
auto d = m(dim);
return std::accumulate(d.begin(), d.end(), 0.0);
}
PYBIND11_PLUGIN(example1_m)
{
pybind11::module m("example1_m", "Array of array test");
m.def("sum_of_dim", sum_of_dim, "Array of array function test");
return m.ptr();
}
I was trying to get an array view from cpp and call a python function back with it.
After some help, I managed to do it by converting the view to a xt::pytensor
before proceeding with the Python call (See code below).
void cpp_f(xt::pytensor<double, 2> my_arr, py::object py_f) {
auto v = xt::pytensor<double, 1>(xt::view(my_arr, 0, xt::all()));
py_f(v);
}
def test_view():
def py_f(v):
print(v)
cpp_f(np.array([ [1,2,3], [4,5,6], ]), py_f)
It would be nice to have some way to pass the view directly to the python function, instead of having to convert them. It seems that it is not possible to make bindings for xt::view
directly, though. I'll leave the original discussion here for reference:
Tarcísio Fischer @tarcisiofischer 14:49
Do you know if there'll be bindings for the xt::view anytime soon?
Johan Mabille @JohanMabille 14:50
the thing is numpy view and xtensor views are not exactly the same so we cannot bind view directly
Sylvain Corlay @SylvainCorlay 14:50
Not as a numpy array, but there is some hope for strided views on pyarrays / pytensor.
Johan Mabille @JohanMabille 14:50
we have xstrided_views that works like numpy views, we can make bindings for them
The attached code benchmarks about 10x slower than my earlier code (using ndarray), I don't know why
#include <numpy/arrayobject.h>
#include "pybind11/pybind11.h"
#include "pybind11/stl.h"
#include "xtensor/xarray.hpp"
#include "xtensor/xtensor.hpp"
#include "xtensor/xcontainer.hpp"
#include "xtensor/xbroadcast.hpp"
#include "xtensor/xbuilder.hpp"
#include "xtensor-python/pyarray.hpp"
#include "xtensor-python/pytensor.hpp"
#include <algorithm> // ?
namespace py = pybind11;
template<typename flt_t>
inline int nint (flt_t x) {
return x >= 0.0 ? int (x + 0.5) : int (x - 0.5);
}
//xt::pytensor<int,1> x (xt::zeros<int> ({1}));
void F(){
xt::pytensor<int,2> x (xt::zeros<int> ({1,1}));
}
template<typename flt_t>
struct histogram_2d {
histogram_2d (flt_t _minx, flt_t _maxx, flt_t _miny, flt_t _maxy,
flt_t _deltax, flt_t _deltay, bool _clip=true) :
minx (_minx), maxx (_maxx), miny (_miny), maxy (_maxy),
deltax (_deltax), deltay (_deltay), clip (_clip),
buckets (xt::zeros<int> ({int ((_maxx - _minx)/_deltax + 1), int ((_maxy - _miny)/_deltay + 1)}))
{}
histogram_2d (xt::pytensor<int,2> const& _buckets, flt_t _minx, flt_t _maxx,
flt_t _miny, flt_t _maxy, flt_t _deltax, flt_t _deltay, bool _clip=true) :
minx (_minx), maxx (_maxx), miny (_miny), maxy (_maxy),
deltax (_deltax), deltay (_deltay), clip (_clip), buckets (_buckets)
{}
flt_t apply (flt_t x, flt_t min, flt_t max) {
if (x > max)
if (clip)
return max;
else
throw std::runtime_error ("histogram: out of range");
else if (x < min)
if (clip)
return min;
else
throw std::runtime_error ("histogram: out of range");
else
return x;
}
auto operator() (flt_t x, flt_t y, int w=1) {
int x_index = nint ((apply (x, minx, maxx) - minx) / deltax);
int y_index = nint ((apply (y, miny, maxy) - miny) / deltay);
buckets(x_index, y_index) += w;
return *this;
}
// auto operator() (xt::xtensor<flt_t,1> x, xt::xtensor<flt_t,1> y,
flt_t minx, maxx, miny, maxy, deltax, deltay;
bool clip;
xt::pytensor<int,2> buckets;
};
PYBIND11_PLUGIN (histogram2d_c) {
if (_import_array() < 0) {
PyErr_SetString(PyExc_ImportError, "numpy.core.multiarray failed to import");
return nullptr;
}
py::module m("histogram2d_c", "pybind11 example plugin");
typedef histogram_2d<double> hist_t;
py::class_<hist_t> (m, "histogram2d")
.def(py::init<double,double,double,double,double,double,bool>(),
py::arg("minx"), py::arg("maxx"), py::arg("miny"), py::arg("maxy"),
py::arg("dx"), py::arg("dy"), py::arg("clip")=true)
.def("__call__", [](hist_t& h, double x, double y, int w) {
return h (x, y, w); },
py::arg("x"), py::arg("y"), py::arg("w")=1)
.def("__call__", [](hist_t& h, xt::pytensor<double,1> const& x,
xt::pytensor<double,1> const& y,
int w) {
if (x.shape() != y.shape())
throw std::runtime_error ("histogram2d::shape mismatch");
//auto ws = xt::broadcast (w, {x.shape()[0]});
auto xpt = x.xbegin();
auto ypt = y.xbegin();
auto xend = x.xend(); // even moving this out of loop doesn't help
for (;
xpt != xend; ++xpt, ++ypt) {
h (*xpt, *ypt, w);
}
},
py::arg("x"), py::arg("y"), py::arg("w")=1)
.def("__call__", [](hist_t& h, xt::pytensor<double,1> const& x,
xt::pytensor<double,1> const& y,
xt::pytensor<int,1> const& w) {
if (x.shape() != y.shape() or x.shape() != w.shape())
throw std::runtime_error ("histogram2d::shape mismatch");
auto xpt = x.xbegin();
auto ypt = y.xbegin();
auto wpt = w.xbegin();
for (;
xpt != x.xend(); ++xpt, ++ypt, ++wpt) {
h (*xpt, *ypt, *wpt);
}
})
.def_readonly ("buckets", &hist_t::buckets)
.def ("__getstate__", [](hist_t const& h) {
return py::make_tuple (h.buckets, h.minx, h.maxx, h.miny, h.maxy, h.deltax, h.deltay, h.clip);
})
.def(py::init<xt::pytensor<int,2>const&,double,double,double,double,double,double,bool>())
.def ("__setstate__", [](hist_t & h, py::tuple t) {
if (t.size() != 8)
throw std::runtime_error("Invalid state!");
new (&h) hist_t (t[0].cast<xt::pytensor<int,2>>(),
t[1].cast<double>(),
t[2].cast<double>(),
t[3].cast<double>(),
t[4].cast<double>(),
t[5].cast<double>(),
t[6].cast<double>(),
t[7].cast<bool>()
);
})
.def_readonly ("minx", &hist_t::minx)
.def_readonly ("maxx", &hist_t::maxx)
.def_readonly ("miny", &hist_t::miny)
.def_readonly ("maxy", &hist_t::maxy)
.def_readonly ("deltax", &hist_t::deltax)
.def_readonly ("deltay", &hist_t::deltay)
.def_readonly ("clip", &hist_t::clip)
.def_property_readonly ("axes", [](hist_t& h) {
xt::pytensor<double,1> x (xt::arange (h.minx, h.maxx+h.deltax, h.deltax));
xt::pytensor<double,1> y (xt::arange (h.miny, h.maxy+h.deltay, h.deltay));
return py::make_tuple (x, y);
})
;
return m.ptr();
}
profile shows:
pprof --text /usr/bin/python3 ./test_histogram2d.prof
Using local file /usr/bin/python3.
Using local file ./test_histogram2d.prof.
Total: 953 samples
193 20.3% 20.3% 894 93.8% pybind11::cpp_function::initialize::{lambda#3}::_FUN [clone .lto_priv.119]
119 12.5% 32.7% 344 36.1% PyArray_NewFromDescr_int
101 10.6% 43.3% 101 10.6% __memmove_avx_unaligned_erms
47 4.9% 48.3% 47 4.9% PyArray_MultiplyList
45 4.7% 53.0% 45 4.7% PyMem_RawRealloc
When attempting to use xt::pyarray
in a class (where substituting xt::xarray
works perfectly) I receive the following error during compilation:
.../third_party/xtensor/include/xtensor/xcontainer.hpp:34:78: error: no type named 'storage_type' in 'struct xt::xcontainer_inner_types<xt::pyarray<double> >'
using storage_type = typename xcontainer_inner_types<D>::storage_type;
I have a class that's more or less as follows:
class SomeClass {
xt::pyarray<double> mean_;
xt::pyarray<double> variance_;
};
I'm using the latest xtensor, xtensor-simd, and xtensor-python as of may 4.
Using GCC6 (via homebrew) on OSX.
Hi guys, I tried the example 2. But I got this ImportError. It seems that xtensor_python_ARRAY_API is not defined. I tried grep
this in both xtensor
and xtensor-python
but I cannot find its definition.
It is used here in pycontainer.hpp
#define PY_ARRAY_UNIQUE_SYMBOL xtensor_python_ARRAY_API
Attempting to compile the following code results in an error (using xtensor 0.15.9 but the bug was still present when trying against 0.15.2):
mean_ = xt::sum(xt::cast<double>(x), xt::evaluation_strategy::immediate()) / x.size();
.../xtensor/include/xtensor/xreducer.hpp:84:28: error: 'class xt::xfunction<xt::detail::cast<double>::functor<float>, double, const xt::pyarray<float>&>' has no member named 'data'; did you mean 'at'?
auto begin = e.data().begin();
Looks like #120 still isn't fixed as of latest xtensor-python master (w/ xtensor 0.15.9).
Taking the result of xt::pyarray<float>.shape()
and calling reshape(<that value>)
on another array produces the following error
In file included from ?
warning: narrowing conversion of '((const xt::xcontainer<xt::pyarray<float> >*)(& x))->xt::xcontainer<xt::pyarray<float> >::size()' from 'xt::xcontainer<xt::pyarray<float> >::size_type {aka long unsigned int}' to 'double' inside { } [-Wnarrowing]
auto out = xt::pyarray<double>({x.size()});
^~~~~
warning: narrowing conversion of '((const xt::xcontainer<xt::pyarray<float> >*)(& x))->xt::xcontainer<xt::pyarray<float> >::size()' from 'xt::xcontainer<xt::pyarray<float> >::size_type {aka long unsigned int}' to 'double' inside { } [-Wnarrowing]
#define BH_TT xt::pyarray
auto out = xt::pyarray<double>({x.size()});
^~~~~
...
.../xtensor-python/include/xtensor-python/pycontainer.hpp:345:30: error: reinterpret_cast from type 'xt::detail::xbuffer_storage<long unsigned int*, std::allocator<long unsigned int> >::const_pointer {aka const long unsigned int*}' to type 'npy_intp* {aka long int*}' casts away qualifiers
PyArray_Dims dims = {reinterpret_cast<npy_intp*>(shape.data()), static_cast<int>(shape.size())};
I'm using GCC7.
Similarly, instantiating a new pyarray using the shape of another array fails with the following error:
error: no matching function for call to 'xt::pyarray<double>::pyarray(const inner_shape_type&)'
We need to be able to iterate over an axis of an expression, with iterators that resolve to proxy references on xviews along that axis.
The most general case might require a dynamic xview.
Test files are here:
https://gist.github.com/nbecker/13c6ca2869d8b5266b6a1e5ae64e9806
This is not much different than previous benchmark.
The actual benchmark is at the end of test_logsumexp.py
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.