wjakob / nanobind Goto Github PK
View Code? Open in Web Editor NEWnanobind: tiny and efficient C++/Python bindings
License: BSD 3-Clause "New" or "Revised" License
nanobind: tiny and efficient C++/Python bindings
License: BSD 3-Clause "New" or "Revised" License
I would like to build a C++/nanobind
library that contains Python stubs, to be able to easy to use it, like the great native Python libraries (example: pandas, numpy, os, etc)
The main idea is to get the auto-complete drop down
inside code editors like VS Code
.
Example:
import myLib
obj = myLib.MyClass()
obj. # here will show the drop down menu showing the MyClass public methods
Another thing is to have the functions signatures to be able to see in VS Code
how many input arguments each C++
function expect.
Example:
import myLib
obj = myLib.myClass()
obj.foo( # here the editor shows the function signature
Is there a way to get these 2 features above using nanobind
?
Thank you,
No response
I wrote a conan recipe for nano bind as we would like to use this in our project. But I am getting an error there because nano bind does not offer an install target. This is ideally needed to avoid having to embed nano bind and keep it a separate project.
Would it be possible add an install target to install your files? Thanks.
pybind11 also has an install target.
No response
It would be great to add support for numpy float16 and Eigen::half.
In old time of pybind11, this can be done by adding follow code:
namespace pybind11 {
namespace detail {
constexpr int NPY_FLOAT16 = 23;
template <> struct npy_format_descriptor<Eigen::half> {
static constexpr auto name = _("float16");
static pybind11::dtype dtype() {
handle ptr = npy_api::get().PyArray_DescrFromType_(NPY_FLOAT16);
return reinterpret_borrow<pybind11::dtype>(ptr);
}
};
} // namespace detail
} // namespace pybind11
#include <Eigen/Core>
#include <nanobind/eigen/dense.h>
#include <nanobind/nanobind.h>
using float_type = Eigen::half;
typedef Eigen::Matrix<float_type, Eigen::Dynamic, Eigen::Dynamic,
Eigen::RowMajor>
Matrix;
Matrix matmul(const Matrix &a, const Matrix &b) { return a * b; }
NB_MODULE(lib, m) { m.def("matmul", &matmul); }
I'd like to move a project from pybind to nanobind since it requires very low overheads in the bindings, and there isn't that much code in the bindings overall.
However, I'm not sure about the following things in nanobind:
object_api.contains()
to check whether a Python object like dict/set contains another value. Is it correct that we just check the return value for nullptr
/ so basically just check whether the returned object is valid ?d[py::int_(0)]
. It looks like nanobind has moved this to a direct numeric accessor: d[0]
should return a accessor<num_item>
. Is that correct ?-
when accessing numpy array with ndarray<>
, nanobind fails when binding on a read-only object.
seems that
Line 165 in a965f5b
PyBUF_RECORDS
requires write permission. errors ValueError: buffer source array is read-only
#include <nanobind/nanobind.h>
#include <nanobind/ndarray.h>
namespace nb = nanobind;
NB_MODULE(myext, m) {
m.def("foo", [](nb::object o) {
auto a = nb::cast<nb::ndarray<>>(o);
printf("%ld\n", a.ndim());
});
}
>>> import myext
>>> import numpy as np
>>> a = np.broadcast_to(1, (2, 3))
>>> b = np.arange(16)
>>> myext.foo(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RuntimeError: nanobind::cast_error
>>> myext.foo(b)
1
nanobind failed to compile on Linux with pp310.
If lower version of pp or use cpython, it compiles fine.
Not sure if works on Windows.
I am using nanobind as a submodule in my project.
Linux:
/project/nanobind/src/nb_type.cpp:426:12: error: ‘Py_am_send’ was not declared in this scope
426 | Ei<i1, Py_##p2##_##name, \
| ^~~
/project/nanobind/src/nb_type.cpp:517:5: note: in expansion of macro ‘E’
517 | E(81, as_async, am, send),
| ^
/project/nanobind/src/nb_type.cpp:428:52: error: no matching function for call to ‘Ei<81, <expression error>, 208, 220>()’
428 | offsetof(PyHeapTypeObject, p1.p2##_##name)>()
| ^
/project/nanobind/src/nb_type.cpp:517:5: note: in expansion of macro ‘E’
517 | E(81, as_async, am, send),
| ^
/project/nanobind/src/nb_type.cpp:398:83: note: candidate: ‘template<unsigned int I1, unsigned int I2, unsigned int Offset1, unsigned int Offset2> constexpr nanobind::detail::nb_slot nanobind::detail::Ei()’
398 | template <size_t I1, size_t I2, size_t Offset1, size_t Offset2> nb_slot constexpr Ei() {
| ^~
/project/nanobind/src/nb_type.cpp:398:83: note: template argument deduction/substitution failed:
/project/nanobind/src/nb_type.cpp:428:52: error: template argument 2 is invalid
428 | offsetof(PyHeapTypeObject, p1.p2##_##name)>()
| ^
/project/nanobind/src/nb_type.cpp:517:5: note: in expansion of macro ‘E’
517 | E(81, as_async, am, send),
| ^
gmake[2]: *** [CMakeFiles/nanobind-static.dir/nanobind/src/nb_type.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/nanobind-static.dir/all] Error 2
gmake: *** [all] Error 2
/Users/runner/work/apngasm-python/apngasm-python/nanobind/src/nb_type.cpp:517:5: error: use of undeclared identifier 'Py_am_send'
E(81, as_async, am, send),
^
/Users/runner/work/apngasm-python/apngasm-python/nanobind/src/nb_type.cpp:426:12: note: expanded from macro 'E'
Ei<i1, Py_##p2##_##name, \
^
<scratch space>:200:1: note: expanded from here
Py_am_send
^
/Users/runner/work/apngasm-python/apngasm-python/nanobind/src/nb_type.cpp:601:57: error: invalid application of 'sizeof' to an incomplete type 'nanobind::detail::nb_slot const[]'
} else if (slot * sizeof(nb_slot) < (int) sizeof(type_slots)) {
^~~~~~~~~~~~
2 errors generated.
make[2]: *** [CMakeFiles/nanobind-static.dir/nanobind/src/nb_type.cpp.o] Error 1
Full log from my GitHub runner:
Failed, Linux x86: https://github.com/laggykiller/apngasm-python/actions/runs/5847075688/job/15852936378
Failed, LInux x64: https://github.com/laggykiller/apngasm-python/actions/runs/5847259600/job/15853343119
Failed, MacOS: https://github.com/laggykiller/apngasm-python/actions/runs/5848670338/job/15856185771
Success, MacOS: https://github.com/laggykiller/apngasm-python/actions/runs/5847662364/job/15854124322
No response
There is a missing return value in this function in tensor.h
at line 271:
nanobind/include/nanobind/tensor.h
Line 271 in 8555cf8
tensor &operator=(const tensor &t) {
detail::tensor_inc_ref(t.m_handle);
detail::tensor_dec_ref(m_handle);
m_handle = t.m_handle;
m_tensor = t.m_tensor;
}
No response
Since the primary way nanobind offers itself to be consumed is by having it be built in the consuming project's build system, it should take great care in not littering the consuming project's build output when no-brainer warnings are enabled. Here is an example output from MSVC:
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
[6/16] Building CXX object python\CMakeFiles\nanobind.dir\__\venv\Lib\site-packages\nanobind\src\nb_func.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(117,1): warning C4365: 'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch
return put(buf + i, digits - i);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(131): warning C4365: 'return': conversion from '__int64' to 'size_t', signed/unsigned mismatch
size_t size() const { return m_cur - m_start; }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(132): warning C4365: 'return': conversion from '__int64' to 'size_t', signed/unsigned mismatch
size_t remain() const { return m_end - m_cur; }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(136,1): warning C4365: 'initializing': conversion from '__int64' to 'size_t', signed/unsigned mismatch
size_t old_alloc_size = m_end - m_start,
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(138,1): warning C4365: 'initializing': conversion from '__int64' to 'size_t', signed/unsigned mismatch
used_size = m_cur - m_start,
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(581,80): warning C4365: '=': conversion from 'int' to 'uint8_t', signed/unsigned mismatch
args_flags[i] = arg_convert ? (uint8_t) cast_flags::convert : 0;
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(807,47): warning C4365: 'initializing': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t nargs = NB_VECTORCALL_NARGS(nargsf);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(297) : warning C6323: Use of arithmetic operator on Boolean type(s).
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(276) : warning C6387: 'fc->descr' could be '0'.: Lines: 147, 148, 150, 151, 152, 153, 154, 155, 156, 157, 158, 160, 161, 162, 165, 210, 211, 213, 217, 218, 220, 239, 241, 245, 246, 249, 250, 253, 255, 258, 259, 261, 273, 274, 275, 276
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(285) : warning C6387: 'fc->descr_types' could be '0'.: Lines: 147, 148, 150, 151, 152, 153, 154, 155, 156, 157, 158, 160, 161, 162, 165, 210, 211, 213, 217, 218, 220, 239, 241, 245, 246, 249, 250, 253, 255, 258, 259, 261, 273, 274, 275, 276, 281, 282, 283, 285
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(483) : warning C6255: _alloca indicates failure by raising a stack overflow exception. Consider using _malloca instead.
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(484) : warning C6255: _alloca indicates failure by raising a stack overflow exception. Consider using _malloca instead.
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(485) : warning C6255: _alloca indicates failure by raising a stack overflow exception. Consider using _malloca instead.
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(580) : warning C6386: Buffer overrun while writing to 'args'.: Lines: 425, 426, 427, 429, 431, 432, 434, 436, 438, 466, 475, 479, 482, 483, 484, 485, 514, 515, 516, 518, 519, 520, 523, 525, 528, 532, 535, 536, 537, 538, 539, 541, 542, 544, 577, 580, 581, 536, 537, 538, 539, 541, 542, 544, 577, 580
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_func.cpp(581) : warning C6386: Buffer overrun while writing to 'args_flags'.: Lines: 425, 426, 427, 429, 431, 432, 434, 436, 438, 466, 475, 479, 482, 483, 484, 485, 514, 515, 516, 518, 519, 520, 523, 525, 528, 532, 535, 536, 537, 538, 539, 541, 542, 544, 577, 580, 581, 536, 537, 538, 539, 541, 542, 544, 577, 580, 581
[7/16] Building CXX object python\CMakeFiles\nanobind.dir\__\venv\Lib\site-packages\nanobind\src\nb_type.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_type.cpp(666,92): warning C4365: 'argument': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
std::memcpy(tp->tp_members, temp_tp->tp_members, tp->tp_itemsize * Py_SIZE(temp_tp));
^
[8/16] Building CXX object python\CMakeFiles\nanobind.dir\__\venv\Lib\site-packages\nanobind\src\nb_enum.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
[9/16] Building CXX object python\CMakeFiles\nanobind.dir\__\venv\Lib\site-packages\nanobind\src\common.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\common.cpp(27,57): warning C4365: 'initializing': conversion from 'int' to 'size_t', signed/unsigned mismatch
size_t size = vsnprintf(buf, sizeof(buf), fmt, args);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\common.cpp(258,81): warning C4365: 'initializing': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_VECTORCALL_NARGS(nargsf) + (kwnames ? NB_TUPLE_GET_SIZE(kwnames) : 0);
^
[10/16] Building CXX object python\CMakeFiles\nanobind.dir\__\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(99,47): warning C4365: 'argument': conversion from 'int32_t' to 'size_t', signed/unsigned mismatch
scoped_pymalloc<Py_ssize_t> strides(t.ndim), shape(t.ndim);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(99,62): warning C4365: 'argument': conversion from 'int32_t' to 'size_t', signed/unsigned mismatch
scoped_pymalloc<Py_ssize_t> strides(t.ndim), shape(t.ndim);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(102,18): warning C4365: 'argument': conversion from 'int32_t' to 'size_t', signed/unsigned mismatch
strides[i] = (Py_ssize_t) t.strides[i] * view->itemsize;
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(103,16): warning C4365: 'argument': conversion from 'int32_t' to 'size_t', signed/unsigned mismatch
shape[i] = (Py_ssize_t) t.shape[i];
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(214,48): warning C4365: 'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch
scoped_pymalloc<int64_t> strides(view->ndim);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(215,46): warning C4365: 'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch
scoped_pymalloc<int64_t> shape(view->ndim);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(316,44): warning C4365: 'argument': conversion from 'int32_t' to 'size_t', signed/unsigned mismatch
scoped_pymalloc<int64_t> strides(t.ndim);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(322,35): warning C4365: '=': conversion from 'size_t' to 'T', signed/unsigned mismatch
with
[
T=Py_ssize_t
]
strides[i] = accum;
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\nb_ndarray.cpp(330,35): warning C4365: '=': conversion from 'size_t' to 'T', signed/unsigned mismatch
with
[
T=Py_ssize_t
]
strides[i] = accum;
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_call.h(111) : warning C6255: _alloca indicates failure by raising a stack overflow exception. Consider using _malloca instead.
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_call.h(111) : warning C6255: _alloca indicates failure by raising a stack overflow exception. Consider using _malloca instead.
[11/16] Building CXX object python\CMakeFiles\nanobind.dir\__\venv\Lib\site-packages\nanobind\src\error.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(117,1): warning C4365: 'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch
return put(buf + i, digits - i);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(131): warning C4365: 'return': conversion from '__int64' to 'size_t', signed/unsigned mismatch
size_t size() const { return m_cur - m_start; }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(132): warning C4365: 'return': conversion from '__int64' to 'size_t', signed/unsigned mismatch
size_t remain() const { return m_end - m_cur; }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(136,1): warning C4365: 'initializing': conversion from '__int64' to 'size_t', signed/unsigned mismatch
size_t old_alloc_size = m_end - m_start,
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\buffer.h(138,1): warning C4365: 'initializing': conversion from '__int64' to 'size_t', signed/unsigned mismatch
used_size = m_cur - m_start,
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\error.cpp(123,56): warning C4365: 'argument': conversion from 'int' to 'uint32_t', signed/unsigned mismatch
buf.put_uint32(PyFrame_GetLineNumber(frame));
^
[12/16] Building CXX object python\CMakeFiles\nanobind.dir\__\venv\Lib\site-packages\nanobind\src\trampoline.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
[13/16] Building CXX object python\CMakeFiles\mhef-pp-bindings.dir\source\main.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
[14/16] Building CXX object python\CMakeFiles\nanobind.dir\__\venv\Lib\site-packages\nanobind\src\implicit.cpp.obj
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(393): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return PyBytes_Size(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(398): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_TUPLE_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(415): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_LIST_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(431): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
size_t size() const { return NB_DICT_GET_SIZE(m_ptr); }
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(518,73): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const tuple &t) { return NB_TUPLE_GET_SIZE(t.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(519,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const list &l) { return NB_LIST_GET_SIZE(l.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(520,71): warning C4365: 'return': conversion from 'Py_ssize_t' to 'size_t', signed/unsigned mismatch
NB_INLINE size_t len(const dict &d) { return NB_DICT_GET_SIZE(d.ptr()); }
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\include\nanobind\nb_types.h(556,1): warning C4365: 'argument': conversion from 'size_t' to 'Py_ssize_t', signed/unsigned mismatch
detail::slice_compute(m_ptr, size, start, stop, step, slice_length);
^
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\implicit.cpp(39) : warning C6387: 'data' could be '0'.: Lines: 18, 20, 21, 25, 26, 28, 29, 30, 29, 37, 39
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\implicit.cpp(40) : warning C6011: Dereferencing NULL pointer 'data'. : Lines: 18, 20, 21, 25, 26, 28, 32, 33, 34, 37, 39, 40
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\implicit.cpp(69) : warning C6387: 'data' could be '0'.: Lines: 49, 51, 52, 56, 57, 59, 60, 61, 60, 68, 69
C:\dev\mhef-pp\venv\Lib\site-packages\nanobind\src\implicit.cpp(70) : warning C6011: Dereferencing NULL pointer 'data'. : Lines: 49, 51, 52, 56, 57, 59, 63, 64, 65, 68, 69, 70
There are also instances of alloca
and unchecked malloc
(malloc
in a C++ project?) which should be remedied ASAP.
No response
Thanks for this new endeavour.
I'm trying to play around, and the example from the docs does not compile:
#include <nanobind/nanobind.h>
#include <nanobind/ndarray.h>
namespace nb = nanobind;
const float data[] = { 1, 2, 3, 4, 5, 6, 7, 8 };
NB_MODULE(my_ext, m)
{
m.def("ret_numpy", []() {
size_t shape[2] = { 2, 4 };
return nb::ndarray<nb::numpy, float, nb::shape<2, nb::any>>(
data, /* ndim = */ 2, shape);
});
}
/Users/tdegeus/data/prog/tests/test_nanobind/my_ext.cpp:43:12: error: no matching constructor for initialization of 'nb::ndarray<nb::numpy, float, nb::shape<2, nb::any>>'
return nb::ndarray<nb::numpy, float, nb::shape<2, nb::any>>(
^
/Users/tdegeus/miniforge3/envs/test/lib/python3.11/site-packages/nanobind/include/nanobind/ndarray.h:251:5: note: candidate constructor not viable: no known conversion from 'const float[8]' to 'void *' for 1st argument
ndarray(void *value,
^
/Users/tdegeus/miniforge3/envs/test/lib/python3.11/site-packages/nanobind/include/nanobind/ndarray.h:246:14: note: candidate constructor not viable: requires single argument 'handle', but 3 arguments were provided
explicit ndarray(detail::ndarray_handle *handle) : m_handle(handle) {
^
/Users/tdegeus/miniforge3/envs/test/lib/python3.11/site-packages/nanobind/include/nanobind/ndarray.h:269:5: note: candidate constructor not viable: requires single argument 't', but 3 arguments were provided
ndarray(const ndarray &t) : m_handle(t.m_handle), m_dltensor(t.m_dltensor) {
^
/Users/tdegeus/miniforge3/envs/test/lib/python3.11/site-packages/nanobind/include/nanobind/ndarray.h:273:5: note: candidate constructor not viable: requires single argument 't', but 3 arguments were provided
ndarray(ndarray &&t) noexcept : m_handle(t.m_handle), m_dltensor(t.m_dltensor) {
^
/Users/tdegeus/miniforge3/envs/test/lib/python3.11/site-packages/nanobind/include/nanobind/ndarray.h:244:5: note: candidate constructor not viable: requires 0 arguments, but 3 were provided
ndarray() = default;
^
It does compile without the const-ness of data
The Type Caster for STL std::vector<bool>
is not working due to the following range based for loop:
See https://stackoverflow.com/a/34085592/9759769.
An example demonstrating the error would be:
#include <nanobind/nanobind.h>
#include <nanobind/stl/vector.h>
#include <vector>
namespace nb = nanobind;
using namespace nb::literals;
// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables, misc-use-anonymous-namespace, readability-identifier-length)
NB_MODULE(nanobind_example_ext, m)
{
m.def("vector_bool", []() { return std::vector<bool>{false, true, false, true}; });
}
which yields on GCC 12.3.0:
[build] /nanobind_example/.venv/lib/python3.11/site-packages/nanobind/include/nanobind/stl/detail/nb_list.h:62:13: error: cannot bind non-const lvalue reference of type ‘std::_Bit_reference&’ to an rvalue of type ‘std::_Bit_iterator::reference’
[build] 62 | for (auto &value : src) {
[build] | ^~~
[build] ninja: build stopped: subcommand failed.
Hi,
I have use case that defines a base class in our codebase, e.g.
// the header file, named as `test.h`
#pragma once
struct Scoreboard {
Scoreboard() = default;
};
// our nanobind cpp file that is compiled to `base.cpython-39-darwin.so`
#include "test.h"
#include "nanobind/nanobind.h"
namespace nb = nanobind;
NB_MODULE(base, m) {
nb::class_<Scoreboard>(m, "Scoreboard")
.def(nanobind::init<>{});
}
In user side, they can derived the base class.
// user's nanobind cpp file that is compiled to `derived.cpython-39-darwin.so`
#include "test.h"
#include "nanobind/nanobind.h"
namespace nb = nanobind;
struct MyScoreboard: public Scoreboard {
MyScoreboard() = default;
};
NB_MODULE(derived, m) {
nanobind::module_::import_("base");
nb::class_<MyScoreboard, Scoreboard>(m, "MyScoreboard")
.def(nanobind::init<>{});
}
When trying to import the derived class MyScoreboard
:
from derived import MyScoreboard
It gives below error (in debug mode): Critical nanobind error: nanobind::detail::nb_type_new("MyScoreboard"): base type "Scoreboard" not known to nanobind!
.
The root cause is that the std::type_info
of Scoreboard
in two shared libraries are difference and it causes the nb_type_lookup
function fails. (my environment is M1 (Arm64) with clang, so it is linked with libstdc++ and it seems this is some behavior observed with libstdc++).
After some more search, I found this is actually a issue mentioned in pybind11: pybind/pybind11#912 and the solution is to have custom hash function for std::type_info
: https://github.com/pybind/pybind11/pull/915/files. The only drawback seems to me is that it could slow things down as we need hashing on string.
So any plan to port that fix from pybind11 or any other idea to fix this issue?
See the code in problem description
The title is by @Yhg1s, I'm just the messenger here.
We happened to stumble over this in connection with pybind11, then I looked in nanobind and see that's also adding capsules to builtins
:
Line 379 in 61393ad
Thomas wrote:
No response
std::exception
isn't exported but the subclasses in the library are, causing this compilation error:
'dllexport' implies default visibility, but 'class nanobind::index_error' has already been declared with a different visibility
This was fixed in PyBind11 with this commit: https://github.com/pybind/pybind11/pull/3132/files
No response
The documentation lists
Removed features include:
[...]
● Support for evaluating Python code strings was removed.
[...]
I'd like to cast my vote for adding something like
nb:exec(code_as_string, nb::globals())
No response
When porting some old pybind11 code, I found that nb::tuple{}
creates a tuple
class with a nullptr handle, which promptly segfaults if doing anything with it. I feel like in order to be consistent with list
and dict
, this should have a default constructor like:
tuple() : object(PyTuple_New(0), detail::steal_t()) { }
It would be great to have
size_t ndarray::size
and to be able to use e.g. std::copy
on shape
and stride
, i.e. to make their pointers somehow public.
Would this be considered?
No response
When I was working on #119 and #121, I started to see test build breaks in our local environment, even on unmodified nanobind code. I've uploaded a gist with the entire set of compiler errors that I saw. I think the issue is that ::NumDimensions
was not added until version 3.4, if I read the upstream issue correctly.
Debian buster (what I am using at the moment) only has 3.3.7. Even Debian bullseye (the current "stable") only has 3.3.9. The Ubuntu image GitHub is using for ubuntu-latest
(jammy) has 3.4.0, as does Debian bookwarm (the current "testing" release, next stable).
Would you like a PR to bump the minimum version to 3.4.0?
No response
Hi,
I need to create something like this: https://github.com/pybind/pybind11_json
for nanobind and for rapidjson.
I fail to understand how to construct a "bool" binding object with nanobind.
How to create a bool object currently?
Also as I saw "::float_" was also dropped in favor of "::int_", am I getting it right?
Thanks,
The configuration step succeeds, but the subsequent build step fails. It looks like the tests step on each other.
cmake -GNinja -DCMAKE_BUILD_TYPE=RelWithDebInfo -Bbuild/x64-Release .
cmake --build ./build/x64-Release
ninja: error: build.ninja:520: multiple rules generate tests/libnanobind.dylib [-w dupbuild=err]
I am on a MacBook Pro 2019, Monterey 12.4 if that is any helpful.
If we have a function which takes a const reference, it should be possible to do something like this:
m.def("get_dim", [](const nb::tensor<const double>& x){ return x.ndim(); });
and send in an unwriteable numpy array:
w = np.array([[2.0,2.0]])
w.setflags(write=False)
assert (module.get_dim(w) == 2)
But this doesn't seem to work at the moment:
E TypeError: get_dim(): incompatible function arguments. The following argument types are supported:
E 1. get_dim(arg: tensor[dtype=float64], /) -> int
E
E Invoked with types: ndarray
Removing the write=False
, it works fine, but sometimes we have an unwriteable array from somewhere else...
No response
The conversion between list
and std::vector<T>
fails when the list is empty.
Sometimes we need to pass an empty list, and it should give std::vector<T>()
- an empty vector.
# Add this to test_stl.py
def test33_list():
print(t.identity_list([]))
pytest test_stl.py
E TypeError: identity_list(): incompatible function arguments. The following argument types are supported:
E 1. identity_list(arg: Sequence[int], /) -> Sequence[int]
E
E Invoked with types: list
I used https://github.com/wjakob/nanobind_example. I got "unsupported operand type(s) for +: 'getset_descriptor' and 'str'" in ipython interpreter.
Is this an issue at ipython side?
Run the followings in ipython (in python interpreter is okay)
import nanobind_example
nanobind_example.add
v1.2.0 was released with internals version 7:
Line 20 in 33d80d9
But the documentation says it should be version 8: https://nanobind.readthedocs.io/en/latest/changelog.html#version-1-2-0-april-24-2023
No response
Hi,
For codes like below:
#include <nanobind/nanobind.h>
namespace nb = nanobind;
using namespace nanobind::literals;
class Struct {
public:
Struct() = default;
int a () { return a_; }
private:
int a_{100};
};
class MyStruct: public Struct {
public:
MyStruct() = default;
};
NB_MODULE(test_member_func_ptr_ext, m) {
nb::class_<MyStruct>(m, "MyStruct")
.def(nb::init<>())
.def_prop_ro("a", &MyStruct::a);
}
from test_member_func_ptr_ext import MyStruct
def test_member_func_ptr():
my_struct = MyStruct()
assert my_struct.a == 100
I'm getting the error msg:
def test_member_func_ptr():
my_struct = MyStruct()
> assert my_struct.a == 100
E TypeError: (): incompatible function arguments. The following argument types are supported:
E 1. (self) -> int
E
E Invoked with types: test_member_func_ptr_ext.MyStruct
It is a bit confusing just seen from the message, but the issue is that &MyStruct::a
is actually &Struct::a
, so it cause the cpp_function
func to create a lambda with input to be Struct
instead of MyStruct
. Since Struct
is unknown to nanobind, it complains.
I think a fix is something similar to what is already done in def_ro
func: do a static assertion and create the lambda with the type T
:
template <typename C, typename D, typename... Extra>
NB_INLINE class_ &def_ro(const char *name, D C::*p,
const Extra &...extra) {
static_assert(std::is_base_of_v<C, T>,
"def_ro() requires a (base) class member!");
def_prop_ro(name,
[p](const T &c) -> const D & { return c.*p; }, extra...);
return *this;
}
See the codes in problem description
The README goes into some detail on various API changes, but the (large) apparent change from buffer -> tensor (not just in name but also in usage) appears to be undocumented. Some detail on this would be greatly appreciated.
No response
enums which are wrapped by nanobind don't have the value
and name
attribute, but they do have __name__
I can work around this with e.g.
nb::enum_<cpp_enum>(m, "cpp_enum")
.value("first_enum", cpp_enum::first_enum)
.def_prop_ro("name", [](nb::object obj){return nb::getattr(obj, "__name__");});
but maybe it should be builtin?
Python native enums have name
and value
attributes.
No response
I'm maintaining a library that used pybind11/eigen.h. It appears this header is not in nanobind, but there is the eigen
directory. How do we port binding code that relied on eigen.h
to nanobind?
In particular, this library used const Eigen::Matrix4f&
I noticed that when a class member function has a default argument value that is of a enum class type, nanobind will warn that the enum type is leaked. For example, here is the output from the attached code:
% python -c "import nanobind_example"
nanobind: leaked 3 instances!
nanobind: leaked 1 types!
- leaked type "Color"
nanobind: this is likely caused by a reference counting issue in the binding code.
In contrast, default value for function arguments does not seem to cause this warning.
#include <nanobind/nanobind.h>
#include <iostream>
namespace nb = nanobind;
using namespace nb::literals;
enum class Color { Red, Green, Blue };
class ColorMap {
public:
ColorMap() = default;
};
NB_MODULE(nanobind_example_ext, m) {
nb::enum_<Color>(m, "Color")
.value("Red", Color::Red)
.value("Green", Color::Green)
.value("Blue", Color::Blue);
nb::class_<ColorMap>(m, "ColorMap")
.def(nb::init<>())
.def(
"print_color",
[](ColorMap& self, Color c) {
switch (c) {
case Color::Red:
std::cout << "red" << std::endl;
break;
case Color::Green:
std::cout << "green" << std::endl;
break;
case Color::Blue:
std::cout << "blue" << std::endl;
break;
}
},
"color"_a = Color::Red); // <-- This causes leak.
}
Currently, there is no type caster for char16_t
and char32_t
. They could be mapped to Python int as ord
in Python returns an int. I did this in my own code (I only needed char32_t
), and it works. I think removing && !is_std_char_v<T>>
in nb_cast.h
could work too.
namespace nanobind::detail {
template <>
struct type_caster<char32_t> {
using Value = char32_t;
template <typename T> using Cast = Value;
static constexpr auto Name = const_name("char32_t");
static constexpr bool IsClass = false;
Value value = '\0';
bool from_python(handle src, uint8_t flags, cleanup_list *cleanup) noexcept {
if (!isinstance<int_>(src)) {
return false;
}
value = static_cast<std::uint_least32_t>(int_(src));
return true;
}
static handle from_cpp(char32_t value, rv_policy policy, cleanup_list *cleanup) {
return int_(value);
}
explicit operator Value() {
return value;
}
};
}
When returning an enum
with def_property_readonly
, the wrong value is sometimes returned.
I have this problem in a larger wrapper code, but tried to reproduce here as a minimal example in a test.
In the code below, both get_enum()
and read_enum
should return Enum.A
, however read_enum
returns Enum.B
apparently.
What is also strange is that e2
below compares as equal to t.Enum.A
although it prints as t.Enum.B
.
// C++ code
class EnumHolder {
public:
EnumHolder() {}
Enum get_enum() { return Enum::A; }
};
nb::class_<EnumHolder>(m, "EnumHolder")
.def(nb::init<>())
.def("get_enum", &EnumHolder::get_enum)
.def_property_readonly("read_enum", &EnumHolder::get_enum);
# Python code
def test05_enum_property():
w = t.EnumHolder()
e1 = w.get_enum()
e2 = w.read_enum
print(e1, e2)
assert e1 == t.Enum.A
assert e2 == t.Enum.B
Testing configuration:
Mac OS X Ventura (13.0.1, ARM)
Python 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:38:29) [Clang 13.0.1 ]
Conda 22.9.0
cmake version 3.24.2
I was hoping to gain context on the following code in nanobind_add_module
:
if (APPLE)
add_library(${name} MODULE ${ARG_UNPARSED_ARGUMENTS})
target_include_directories(${name} PUBLIC ${Python_INCLUDE_DIRS})
target_link_options(${name} PRIVATE -undefined suppress -flat_namespace)
else()
Python_add_library(${name} MODULE ${ARG_UNPARSED_ARGUMENTS})
endif()
I admit I am not an expert in the CMake+Python combination, but the if ()
branch does not appear to be required.
The breakage occurs because Python_INCLUDE_DIRS
does not actually get defined by find_package(Python ...)
.
What I tried (after removing the APPLE
specific cmake logic) was the following:
find_package(Python 3.8 COMPONENTS Interpreter Development.Module REQUIRED)
nanobind_add_module(test_wrapper ...)
This appears to work fine, provided that Python_add_library
is used.
If it seems correct and suitable to you, I can submit a PR that makes this change. Let me know if I have misunderstood something. Cheers.
No response
Build a python3.11 interpreter via:
Python-3.11.4 $ ./configure --with-address-sanitizer --with-undefined-behavior-sanitizer --with-openssl=$(brew --prefix openssl) --prefix=/some/dir
Python-3.11.4 $ make -j `nproc`
Python-3.11.4 $ make install
Create a venv and make sure that cmake
finds this interpreter, then run the nanobind
unit tests. Then run:
n/tests ❯❯❯ python3 -m pytest test_functions.py::test29_traceback -s -v
====================================================== test session starts =======================================================
platform darwin -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0 ---
cachedir: .pytest_cache
configfile: pyproject.toml
collected 1 item
test_functions.py::test29_traceback =================================================================
==2380==ERROR: AddressSanitizer: heap-use-after-free on address 0x00010c1f3e70 at pc 0x00010cfaf498 bp 0x00016f02e9e0 sp 0x00016f02e9d8
READ of size 8 at 0x00010c1f3e70 thread T0
#0 0x10cfaf494 in Py_DECREF(_object*) object.h:537
#1 0x10cfa8854 in Py_XDECREF(_object*) object.h:602
#2 0x10cfa8600 in nanobind::python_error::~python_error() error.cpp:82
#3 0x10cfa8944 in nanobind::python_error::~python_error() error.cpp:74
#4 0x1813e2e44 in __cxa_decrement_exception_refcount+0x38 (libc++abi.dylib:arm64e+0x17e44) (BuildId: 52aa13e2567c36c294947b892fdbf24532000000200000000100000000040d00)
#5 0x10cef9f10 in nanobind_init_test_functions_ext(nanobind::module_&)::$_53::operator()(nanobind::callable) const test_functions.cpp:189
#6 0x10cef8b04 in _object* nanobind::detail::func_create<false, true, nanobind_init_test_functions_ext(nanobind::module_&)::$_53, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, nanobind::callable, 0ul, nanobind::scope, nanobind::name>(nanobind_init_test_functions_ext(nanobind::module_&)::$_53&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> (*)(nanobind::callable), std::__1::integer_sequence<unsigned long, 0ul>, nanobind::scope const&, nanobind::name const&)::'lambda'(void*, _object**, unsigned char*, nanobind::rv_policy, nanobind::detail::cleanup_list*)::__invoke(void*, _object**, unsigned char*, nanobind::rv_policy, nanobind::detail::cleanup_list*) nb_func.h:105
#7 0x10cf29bf4 in nanobind::detail::nb_func_vectorcall_simple(_object*, _object* const*, unsigned long, _object*) nb_func.cpp:736
#8 0x100e97b38 in PyObject_Vectorcall call.c:299
0x00010c1f3e70 is located 16 bytes inside of 56-byte region [0x00010c1f3e60,0x00010c1f3e98)
freed by thread T0 here:
#0 0x102046fa4 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x42fa4) (BuildId: f0a7ac5c49bc3abc851181b6f92b308a32000000200000000100000000000b00)
#1 0x1011b4920 in tb_dealloc traceback.c:175
#2 0x100ec2eb8 in BaseException_clear exceptions.c:88
#3 0x100ec2a40 in BaseException_dealloc exceptions.c:102
#4 0x10cfaf524 in Py_DECREF(_object*) object.h:538
#5 0x10cfa8854 in Py_XDECREF(_object*) object.h:602
#6 0x10cfa84b0 in nanobind::python_error::~python_error() error.cpp:80
#7 0x10cfa8944 in nanobind::python_error::~python_error() error.cpp:74
#8 0x1813e2e44 in __cxa_decrement_exception_refcount+0x38 (libc++abi.dylib:arm64e+0x17e44) (BuildId: 52aa13e2567c36c294947b892fdbf24532000000200000000100000000040d00)
#9 0x10cef9f10 in nanobind_init_test_functions_ext(nanobind::module_&)::$_53::operator()(nanobind::callable) const test_functions.cpp:189
#10 0x10cef8b04 in _object* nanobind::detail::func_create<false, true, nanobind_init_test_functions_ext(nanobind::module_&)::$_53, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, nanobind::callable, 0ul, nanobind::scope, nanobind::name>(nanobind_init_test_functions_ext(nanobind::module_&)::$_53&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> (*)(nanobind::callable), std::__1::integer_sequence<unsigned long, 0ul>, nanobind::scope const&, nanobind::name const&)::'lambda'(void*, _object**, unsigned char*, nanobind::rv_policy, nanobind::detail::cleanup_list*)::__invoke(void*, _object**, unsigned char*, nanobind::rv_policy, nanobind::detail::cleanup_list*) nb_func.h:105
#11 0x10cf29bf4 in nanobind::detail::nb_func_vectorcall_simple(_object*, _object* const*, unsigned long, _object*) nb_func.cpp:736
#12 0x100e97b38 in PyObject_Vectorcall call.c:299
previously allocated by thread T0 here:
#0 0x102046e68 in wrap_malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x42e68) (BuildId: f0a7ac5c49bc3abc851181b6f92b308a32000000200000000100000000000b00)
#1 0x1011db468 in _PyObject_GC_New gcmodule.c:2298
#2 0x1011b5304 in PyTraceBack_Here traceback.c:253
#3 0x1010b0560 in _PyEval_EvalFrameDefault ceval.c:5757
#4 0x1010afa8c in _PyEval_Vector ceval.c:6439
#5 0x100e97b38 in PyObject_Vectorcall call.c:299
#6 0x10cf94c78 in nanobind::detail::obj_vectorcall(_object*, _object* const*, unsigned long, _object*, bool) common.cpp:306
#7 0x10cefbe10 in nanobind::object nanobind::detail::api<nanobind::handle>::operator()<(nanobind::rv_policy)1>() const nb_call.h:136
#8 0x10cef9d18 in nanobind_init_test_functions_ext(nanobind::module_&)::$_53::operator()(nanobind::callable) const test_functions.cpp:186
#9 0x10cef8b04 in _object* nanobind::detail::func_create<false, true, nanobind_init_test_functions_ext(nanobind::module_&)::$_53, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, nanobind::callable, 0ul, nanobind::scope, nanobind::name>(nanobind_init_test_functions_ext(nanobind::module_&)::$_53&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> (*)(nanobind::callable), std::__1::integer_sequence<unsigned long, 0ul>, nanobind::scope const&, nanobind::name const&)::'lambda'(void*, _object**, unsigned char*, nanobind::rv_policy, nanobind::detail::cleanup_list*)::__invoke(void*, _object**, unsigned char*, nanobind::rv_policy, nanobind::detail::cleanup_list*) nb_func.h:105
#10 0x10cf29bf4 in nanobind::detail::nb_func_vectorcall_simple(_object*, _object* const*, unsigned long, _object*) nb_func.cpp:736
#11 0x100e97b38 in PyObject_Vectorcall call.c:299
SUMMARY: AddressSanitizer: heap-use-after-free object.h:537 in Py_DECREF(_object*)
Shadow bytes around the buggy address:
0x00702185e770: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x00702185e780: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x00702185e790: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x00702185e7a0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x00702185e7b0: fa fa fa fa 00 00 00 00 00 00 00 01 fa fa fa fa
=>0x00702185e7c0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd[fd]fd
0x00702185e7d0: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fd
0x00702185e7e0: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
0x00702185e7f0: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
0x00702185e800: fd fd fd fa fa fa fa fa 00 00 00 00 00 00 00 00
0x00702185e810: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Code is in the repo-only build config is necessary to reproduce.
When binding a function processing tensors, one can specify contiguity flags, like nb::c_contig
. Nanobind is expected to ensure the input tensors are contiguous in memory before calling the function.
However, when passing a non-contiguous PyTorch tensor to such a function will raise an error, instead of making it contiguous under the hood.
Modifying the Nanobind example project with the following function definition allows to reproduce this issue.
m.def("add", [](nb::ndarray<float, nb::c_contig> a, nb::ndarray<float, nb::c_contig> b) { return 0; }, "a"_a, "b"_a);
from nanobind_example import add
import drjit as dr
import torch
a = torch.ones((3,3))
b = dr.ones(dr.llvm.TensorXf, (3,3))
# DrJit tensors, works as expected
print(b,b)
print(b,b[::-1])
# PyTorch tensors, fails
print(add(a,a)) # Contiguous tensors, returns 0
print(add(a,a.T)) # a.T is non-contiguous, this fails
This raises the following issue
Traceback (most recent call last):
File "<stdin>", line 11, in <module>
TypeError: add(): incompatible function arguments. The following argument types are supported:
1. add(a: ndarray[dtype=float32, order='C'], b: ndarray[dtype=float32, order='C']) -> int
Invoked with types: torch.Tensor, torch.Tensor
In GitHub runner, cibuildwheel setups python 3.7, which is not supported by nanobind (Attempting to install nanobind would cause TypeError: copytree() got an unexpected keyword argument 'dirs_exist_ok'
)
The cause is https://github.com/wjakob/nanobind/blob/master/setup.py#L49 due to dirs_exist_ok
option not existing before python 3.8 (https://docs.python.org/3/library/shutil.html#shutil.copytree)
See: https://github.com/laggykiller/apngasm-python/actions/runs/5854803439/job/15871376469
However, if I install it in the venv setup by cibuildwheel, then:
CMake Error at cmake/QueryPythonForNanobind.cmake:37 (find_package):
Could not find a package configuration file provided by "nanobind" with any
of the following names:
nanobindConfig.cmake
nanobind-config.cmake
Add the installation prefix of "nanobind" to CMAKE_PREFIX_PATH or set
"nanobind_DIR" to a directory containing one of the above files. If
"nanobind" provides a separate development package or SDK, be sure it has
been installed.
See: https://github.com/laggykiller/apngasm-python/actions/runs/5855058420/job/15872111888
Of course I can just add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ext/nanobind)
, but I think we should also make the pip method possible.
No response
Hi,
How to embed Python 3 to use this library without need have python installed locally?
Thanks.
If 'name' is a macro, and it is passed to NB_MODULE(name, module) it is not expanded, because NB_MODULE is a macro itself.
The solution, if you decide to implement it (and the workaround) is easy, rename the macro NB_MODULE to, say, NB_MODULE_INTERNAL(), introduce a function called NB_MODULE(), and call NB_MODULE_INTERNAL() from NB_MODULE().
I assume that this is the case for other macros, but this one affected me...
In case you are curious, I'm generating bindings for great parts of CGAL (see https://www.cgal.org/). It is a generic library that extensively uses templates. There are many combinations of types that can be instantiated. We support (via CMake setting) the generation of more than one library that consist of bindings, which can co-exist and be imported by the same Python component. The names of these libraries are automatically generated. These names are passed to the code, as macros, to serve as the corresponding module names.
No response
I think the title says it all.
While this could be considered as a feature, it would be a bit hard to port binding code from pybind11 (or boost.python) and keep the python code that uses the bindings running.
No response
Thank you for your great job. I'm enjoying your tools as combining the OpenCV framework like below repository.
It seems a very kind and sophisticated tool at least for me.
https://github.com/yuki-inaho/nanobind_opencv_exercise
I noticed below code lines (especially the 83-th line) on the test code are a little suspectable for me.
To be exact, size_t x = 0; x < tensor.shape(1); ++x
will be correct.
Please sorry if something is wrong, thank you.
nanobind/tests/test_tensor.cpp
Lines 79 to 87 in 43a96c5
Hi. Thanks for opening this wonderful binding tool. I have a question regarding binding Pytorch C++ codes to Python. As written in the README that Nanobind works with only "a subset of C++", can it be a drop-in for the current Pybind 11 that Pytorch is using? Have you had any experience before? If you had, could you please share? Thank you very much.
Python 3.12 refactored the layout of the long object. The following patch is needed to compile modules against 3.12 (3.12b1 is the latest version as of this report). Unfortunately I won't have time to provide a properly tested PR that does the right thing with both the old and the new layout, but I'm creating this issue so this can at least be tracked.
--- common.cpp.orig 2023-05-30 04:18:52.879564557 +0000
+++ common.cpp 2023-05-30 04:18:57.819527549 +0000
@@ -824,7 +824,7 @@
Py_ssize_t size = Py_SIZE(o);
if (size == 0 || size == 1) {
- digit value_d = lo->ob_digit[0];
+ digit value_d = lo->long_value.ob_digit[0];
T value = (T) value_d;
*out = value;
return sizeof(T) >= sizeof(digit) || value_d == (digit) value;
@@ -835,7 +835,7 @@
return false;
} else {
if (size == -1) {
- sdigit value_d = - (sdigit) lo->ob_digit[0];
+ sdigit value_d = - (sdigit) lo->long_value.ob_digit[0];
T value = (T) value_d;
*out = value;
return sizeof(T) >= sizeof(sdigit) || value_d == (sdigit) value;
No response
Having bound a class Holder
with a member of type Eigen::Matrix2d as property, the created getter does not keep its Holder
alive. Hence, the last line of the Python example code accesses invalid memory.
type_caster<Eigen::Matrix<...>>::from_cpp simply ignores rv_policy::reference_internal, while I believe it should pass cleanup->self() as owner to the constructor of ndarray if called with rv_policy::reference_internal.
Based on commit 9139eda
C++ code:
#include <nanobind/nanobind.h>
#include <nanobind/eigen/dense.h>
namespace nb = nanobind;
struct Holder
{
Eigen::Matrix2d member = Eigen::Matrix2d::Ones();
};
NB_MODULE(nanobind_eigen_property, m)
{
nb::class_<Holder>(m, "Holder")
.def(nb::init())
.def_rw("member", &Holder::member);
}
Python code:
import nanobind_eigen_property
import numpy as np
ones = np.ones((2, 2))
holder = nanobind_eigen_property.Holder()
member = holder.member
# succeeds:
np.testing.assert_array_equal(member, ones)
del holder
# fails or crashes, since member now points to invalid memory:
np.testing.assert_array_equal(member, ones)
Hi everyone !
Thanks @wjakob for this new and shiny version of pybind ! :)
I am however experiencing some difficulties at runtime: when loading the module that nanobind
created, it comes short of finding the right binary for nanobind
library. I am on macOS Monterey v12.6.2 so the exact error message that comes up is (see below for the project hierarchy):
ImportError: dlopen(/path/to/the/project/level1/level2/level21/_my_binary.cpython-310-darwin.so, 0x0002):
Library not loaded: '@rpath/libnanobind.dylib'
Here is my setup:
v3.10.9
of python, propelled by CLang.Python 3.10.9 (main, Dec 15 2022, 10:44:50) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
v0.1.0
of nanobind
using pip
.Name: nanobind
Version: 0.1.0
Summary: Seamless operability between C++17 and Python
Home-page: https://github.com/wjakob/nanobind
Author: Wenzel Jakob
Author-email: [email protected]
License: BSD
Location: /opt/homebrew/lib/python3.10/site-packages
Requires:
Required-by: project
I am trying to refactor some parts of a pre-existing project to use nanobind
so I cannot really show the entire tree, but the part where I am currently trying to add the first nanobind
piece looks like the following.
project
|- ...
|- project-main
|-- setup.py
|-- CMakeLists.txt
|-- project
|--- level1
|---- __init__.py
|---- level2
|----- __init__.py
|----- level21
|------ __init__.py
|------ _my_binary.cpp
|------ my_binary.py
|------ ...
|----- level22
|------ __init__.py
|------ ...
|---- ...
|- ...
|- README.md
|- ...
This is _my_binary.cpp
, simply copy-pasted from the example repo:
#include <nanobind/nanobind.h>
namespace nb = nanobind;
using namespace nb::literals;
NB_MODULE(_my_binary, m) {
m.def("add", [](int a, int b) { return a + b; }, "a"_a, "b"_a);
}
This is what my CMakeLists.txt
looks like:
project(FAST)
cmake_minimum_required(VERSION 3.18...3.22 FATAL_ERROR)
# Create CMake targets for all Python components needed by nanobind
if (CMAKE_VERSION VERSION_GREATER_EQUAL 3.26)
find_package(Python 3.10 COMPONENTS Interpreter Development.Module Development.SABIModule REQUIRED)
else()
find_package(Python 3.10 COMPONENTS Interpreter Development.Module REQUIRED)
endif()
# Run `nanobind.cmake_dir()` from Python to detect where nanobind is installed
execute_process(
COMMAND "${Python_EXECUTABLE}" -c "import nanobind; print(nanobind.cmake_dir())"
OUTPUT_STRIP_TRAILING_WHITESPACE OUTPUT_VARIABLE NB_DIR)
list(APPEND CMAKE_PREFIX_PATH "${NB_DIR}")
# Now, import nanobind through CMake's find_package mechanism
find_package(nanobind CONFIG REQUIRED)
# We are now ready to compile the actual extension modules
nanobind_add_module(
_my_binary
STABLE_ABI
NB_SHARED # <-- I also tried with NB_STATIC here, same result.
project/level1/level2/level21/_my_binary.cpp
)
# Install directive for scikit-build
install(TARGETS _my_binary LIBRARY DESTINATION project/level1/level2/level21)
And this is the setup.py
found in the project-main
folder (from the root folder I simply run pip install project-main
):
import nanobind # noqa: F401
import os
from skbuild import setup
CURRENT_DIR = os.path.dirname(os.path.realpath(__file__))
PARENT_DIR = os.path.join(CURRENT_DIR, "..")
with open(os.path.join(PARENT_DIR, "VERSION")) as fid:
version = fid.readline().strip()
with open(os.path.join(PARENT_DIR, "requirements.txt")) as fid:
requirements = [
line.split("--")[0] for line in fid.read().splitlines() if "#" not in line
]
requirements_to_prune = [
"grpcio", # for apple M1 silicon chips, need a custom install wheel (see Makefile)
]
requirements = [
req
for req in requirements
if not any([prune_req in req for prune_req in requirements_to_prune])
]
package_name = "project-main"
setup(
name=package_name,
version=version,
description="",
author="",
python_requires=">=3.8.13",
packages=["project"],
install_requires=requirements,
)
Note that I pulled the latest version of skbuild
as well, v0.16.6
, and my CMake
version is v3.25.2
.
Can you guys see anything I might be doing wrong here ? 😬
Thank you for all the help you could provide !!
No response
This library is incredible. The overhead on very often-called, but fast functions (runtime of sub-microsecond) on a derived type is something like an order of magnitude faster. I'm moving from pybind11 to nanobind as we speak.
One thing I was unclear of is how STL containers, namely std::vector
, work in the interface. If I have a function like so that I want to wrap:
auto indexer (int i, const std::vector<double> &e){
return e[i];
}
does nanobind
still make a temporary? What if I make a numpy array and pass that to the function? It seems like the copies are avoided, but I am not 100% sure.
Related: it would be nice to add std::valarray
, as that allows for concise math stuff without the weight of Eigen.
In pybind11 we had the ability to specify .noconvert()
on an argument to avoid copies, and that compiles with nanobind
, but I wanted to confirm that the same thing is working in nanobind
without copying.
Alternatively, how would one access a numpy array buffer (1d of double) in nanobind
? The tensor machinery seems overkill.
Hi,
I was under the impression that, since the tensor class can have either host or GPU memory, it should be working fine with both C++ and CUDA, but I am having problems compiling a simple code with NVCC. The error message says:
nvcc fatal : 's': expected a number
I have checked the compiler args, it seems a problematic compiler option is causing the issue:
/usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -Dtest_dlpack_EXPORTS --options-file CMakeFiles/test_dlpack.dir/includes_CUDA.rsp -O3 -DNDEBUG --generate-code=arch=compute_52,code=[compute_52,sm_52] -Xcompiler=-fPIC -fno-stack-protector -Os -std=c++17 -MD -MT CMakeFiles/dir/src/bindings.cu.o -MF CMakeFiles/src/bindings.cu.o.d -x cu -rdc=true -c ~/cuda_test/src/bindings.cu -o CMakeFiles/src/bindings.cu.o
Could anyone help me solve this issue? A toy example is given below.
`bindings.cu:`
#include <nanobind/nanobind.h>
namespace nb = nanobind;
using namespace nb::literals;
NB_MODULE(test, m) {
m.def("add", [](int a, int b) { return a + b; }, "a"_a, "b"_a);
}
`CMakeLists.txt:`
cmake_minimum_required(VERSION 3.18...3.22)
project(test_dlpack CUDA CXX)
find_package(Python 3.8 COMPONENTS Interpreter Development.Module REQUIRED)
add_subdirectory(nanobind)
nanobind_add_module(test_dlpack
src/bindings.cu
)
It looks like the include path is not correctly set up for the tsl dependency used.
lpapp@Laszlos-MBP nanobind % cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -Bbuild/x64-Release .
-- The C compiler identification is AppleClang 13.1.6.13160021
-- The CXX compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python: /Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 (found suitable version "3.10.2", minimum required is "3.8") found components: Interpreter Development Development.Module Development.Embed
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/lpapp/Projects/nanobind/build/x64-Release
lpapp@Laszlos-MBP nanobind % cmake --build ./build/x64-Release
[ 3%] Building CXX object tests/CMakeFiles/nanobind.dir/__/src/nb_internals.cpp.o
In file included from /Users/lpapp/Projects/nanobind/src/nb_internals.cpp:12:
/Users/lpapp/Projects/nanobind/src/nb_internals.h:9:10: fatal error: 'tsl/robin_map.h' file not found
#include <tsl/robin_map.h>
^~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [tests/CMakeFiles/nanobind.dir/__/src/nb_internals.cpp.o] Error 1
make[1]: *** [tests/CMakeFiles/nanobind.dir/all] Error 2
make: *** [all] Error 2
lpapp@Laszlos-MBP nanobind %
Hi, I'm hit a segmentation fault or bus error raised from Python and below is a minimal code snapshot to reproduce it:
#include <unordered_map>
#include <string>
#include "nanobind/nanobind.h"
#include "nanobind/stl/shared_ptr.h"
#include "nanobind/stl/string.h"
namespace nb = nanobind;
using namespace nb::literals;
class Component {
public:
Component() = default;
virtual ~Component() = default;
};
class Param: public Component {
public:
Param() = default;
};
class Model: public Component {
public:
Model() = default;
void add_param(const std::string& name, std::shared_ptr<Param> p) {
params_[name] = std::move(p);
}
std::shared_ptr<Param> get_param(const std::string& name) {
if (params_.contains(name)) {
return params_[name];
}
return nullptr;
}
private:
std::unordered_map<std::string, std::shared_ptr<Param>> params_;
};
class ModelA: public Model {
public:
ModelA() {
add_param("a", std::make_shared<Param>());
add_param("b", std::make_shared<Param>());
}
};
NB_MODULE(test_nanobind_dynamic_attr, m) {
nanobind::set_leak_warnings(false);
nb::class_<Component>(m, "Component");
nb::class_<Param, Component>(m, "ParamBase");
nb::class_<Model, Component>(m, "Model", nb::dynamic_attr()).def(nb::init<>{})
.def("_get_param", &Model::get_param, "name"_a)
.def("_add_param", &Model::add_param, "name"_a, "p"_a);
nb::class_<ModelA, Model>(m, "ModelA").def(nb::init<>{});
}
from test_nanobind_dynamic_attr import Model, ModelA
def _get_parameter(self: Model, key: str):
p = self._get_param(key)
if p is not None: # cache it for fast access later
setattr(self, key, p)
return p
raise AttributeError(f"'key' not found in {self}")
# when an attribute doesn't exist, check if it is parameter in Model, if so, get it
Model.__getattr__ = _get_parameter
# Get segmentation fault or bus error. If I comment out below lines, I got :
# SystemError: Objects/dictobject.c:1464: bad argument to internal function
#
# def _print_model(self):
# return f"{self.__class__.__qualname__}()"
#
# Model.__str__ = _print_model
class Top(Model):
def __init__(self):
super().__init__()
self.model_a = ModelA()
top = Top()
print(top.model_a)
print(top.model_a.a) # "a" is a param added in C++, so we should get it through the `_get_parameter` function
Here is some output from ASAN/UBSAN build of Python and Nanobind:
python3 benchmarks.py
ModelA()
Include/object.h:502:9: runtime error: member access within misaligned address 0xcdcdcdcdcdcdcdcd for type 'PyObject' (aka 'struct _object'), which requires 8 byte alignment
0xcdcdcdcdcdcdcdcd: note: pointer points here
<memory cannot be printed>
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Include/object.h:502:9 in
AddressSanitizer:DEADLYSIGNAL
=================================================================
==2517==ERROR: AddressSanitizer: SEGV on unknown address 0xffffba29b9bbb9b9 (pc 0x0001011bb94c bp 0x00016f149610 sp 0x00016f149560 T0)
==2517==The signal is caused by a READ memory access.
#0 0x1011bb94c in _PyObject_GenericGetAttrWithDict object.c:1317
#1 0x1011bb36c in PyObject_GenericGetAttr object.c:1368
#2 0x101271478 in slot_tp_getattr_hook typeobject.c:7706
#3 0x1011b9c48 in PyObject_GetAttr object.c:916
#4 0x101482e30 in _PyEval_EvalFrameDefault ceval.c:3466
#5 0x101466ce0 in _PyEval_EvalFrame pycore_ceval.h:73
#6 0x1014665dc in _PyEval_Vector ceval.c:6439
#7 0x10146606c in PyEval_EvalCode ceval.c:1154
#8 0x10163af30 in run_eval_code_obj pythonrun.c:1714
#9 0x101636aa0 in run_mod pythonrun.c:1735
#10 0x1016321e4 in _PyRun_SimpleFileObject pythonrun.c:440
#11 0x10163199c in _PyRun_AnyFileObject pythonrun.c:79
#12 0x1016c1700 in Py_RunMain main.c:680
#13 0x1016c2be4 in pymain_main main.c:710
#14 0x1016c2f8c in Py_BytesMain main.c:734
#15 0x100cb7684 in main python.c:15
#16 0x18b807f24 (<unknown module>)
==2517==Register values:
x[0] = 0x000000016f149478 x[1] = 0x0000000000000000 x[2] = 0x0000000000000000 x[3] = 0x0000000107d007a0
x[4] = 0x0000000063000000 x[5] = 0x0000000000000000 x[6] = 0x0000000000000000 x[7] = 0x0000000000000000
x[8] = 0x00000001037c2000 x[9] = 0x000000000000d810 x[10] = 0x00000000000d4230 x[11] = 0x000000000000007d
x[12] = 0xcdcdcdcdcdcdcdcd x[13] = 0xffffff0000000000 x[14] = 0x0000000000000000 x[15] = 0x0000000000000000
x[16] = 0x00000003077fa0b4 x[17] = 0x0000000102d980a0 x[18] = 0x0000000000000000 x[19] = 0x00000001021ef6c0
x[20] = 0x0000000105b3ce80 x[21] = 0x19b9b9b9b9b9b9b9 x[22] = 0x0000000000000001 x[23] = 0x0000000000000001
x[24] = 0x0000000105e320a0 x[25] = 0x0000000000000000 x[26] = 0x0000000000000000 x[27] = 0x0000000000000000
x[28] = 0x0000007000020000 fp = 0x000000016f149610 lr = 0x00000001011bbf58 sp = 0x000000016f149560
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV object.c:1317 in _PyObject_GenericGetAttrWithDict
==2517==ABORTING
fish: Job 1, 'python3 benchmarks.py' terminated by signal SIGABRT (Abort)
In my example, I put the nb::dynamic_attr()
on the base class Model
, as this is our usage (user inherits it to created derived ModelA
). If I move the nb::dynamic_attr()
from the base class Model
to the derived class ModelA
, the code seems work fine. So I guess it must be something related with nb::dynamic_attr()
and inheritance.
My environment is: M1 (Arm64) with macOS Ventura, Python 3.9.17, with nanobind 1.5.1.
See the codes in the problem description.
MacOS 12.5, XCode CLI tools installed but not full XCode.
This block in nanobind-config.cmake
fails:
execute_process(
COMMAND xcodebuild -version
OUTPUT_VARIABLE NB_XCODE_VERSION_STR
OUTPUT_STRIP_TRAILING_WHITESPACE
ERROR_FILE /dev/null)
string(REGEX MATCH "Xcode ([0-9][0-9]?([.][0-9])+)" NB_XCODE_VERSION_MATCH ${NB_XCODE_VERSION_STR})
with this error message:
CMake Error at vendor/nanobind/cmake/nanobind-config.cmake:55 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
Call Stack (most recent call first):
vendor/nanobind/CMakeLists.txt:73 (find_package)
apparently because NB_XCODE_VERSION_STR
does not get set to a valid value. Here's what that returns when run directly:
❯ xcodebuild -version
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance
To resolve this, I replaced
COMMAND xcodebuild -version
with
COMMAND pkgutil --pkg-info=com.apple.pkg.CLTools_Executables | awk '/version:/ {print $2}'
which then runs successfully.
No response
I was following this doc (https://github.com/wjakob/nanobind/blob/master/docs/cmake.md) by setting set(NB_SHARED OFF CACHE INTERNAL "")
to see if the output will statically link with nanobind
. However, at 526e2c7 this would still produce both libnanobind.so
and my own Python ext .so
, with the latter one depending on the former.
After checking the cmake config:
nanobind/cmake/nanobind-config.cmake
Line 222 in 526e2c7
NB_STATIC
to nanobind_add_module()
. Something like below worked on my end now:
nanobind_add_module(my_py_ext ${EXT_SOURCE_FILES} NB_STATIC)
Seems like a stale doc issue?
No response
Originally posted by tbrekalo March 31, 2022
How do I interpret the following?
nanobind: leaked 1 types!
- leaked type "Pile"
nanobind: leaked 5 functions!
- leaked function "<anonymous>"
- leaked function "<anonymous>"
- leaked function "<anonymous>"
- leaked function "<anonymous>"
- leaked function "__init__"
nanobind: this is likely caused by a reference counting issue in the binding code.
Is it a bug on my part for getting something wrong in binding code or is it an issue with nanobind?
Side note: code causing this issue deals with shared pointers for another type that is not brought up by the error message.
Type 'Pile' is regular value type, I have a std::vector of them. So I am not sure how reference counting is being mentioned here.
Thanks for your help in advance.
Issue with false report of type leaks influenced by python type hints and imort of non-standard modules (in this case; seaborn 0.11)
Code that worked with version 1.3.2 no longer works with version 1.5.0:
Array managed on the C++ side can be read, but not modified, i.e. if modified on the C++ side, the updates can be read from Python but trying to update from Python e,g, with numpy.copyto() has no effect.
Sorry in advance, if my code was exploiting undefined behaviour.
///////////// C++ /////////////
#include <nanobind/nanobind.h>
#include <nanobind/ndarray.h>
namespace nb = nanobind;
class Foo {
public:
using array_type = nb::ndarray<nb::numpy, float, nb::shape<1, nb::any>>;
array_type numpy() {
size_t shape = 8;
return array_type(static_cast<void*>(data), 1, &shape);
}
Foo() : data(new float[8]) {}
~Foo() { delete[] data; }
float* data;
};
NB_MODULE(bughunt, m) {
nb::class_<Foo>(m, "Foo")
.def(nb::init<>())
.def("numpy", &Foo::numpy);
}
############### Python #################
import bughunt
import numpy as np
foo = bughunt.Foo()
print(foo.numpy())
np.copyto(foo.numpy(), np.arange(8, dtype=np.float32))
print(foo.numpy())
# both prints are the same, but expected values from arange
# works if compiled with version 1.3.2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.