GithubHelp home page GithubHelp logo

pybind / pybind11_abseil Goto Github PK

View Code? Open in Web Editor NEW
24.0 24.0 13.0 208 KB

Pybind11 bindings for the Abseil C++ Common Libraries

License: Other

Starlark 8.38% C++ 56.84% Python 26.21% Shell 1.60% C 0.15% CMake 6.81%

pybind11_abseil's People

Contributors

charlesbeattie avatar ejania avatar hawkinsp avatar jblespiau avatar junyer avatar kenoslund-google avatar laramiel avatar lwolfsonkin avatar mchinen avatar petebu avatar rickeylev avatar rwgk avatar tkoeppe avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

chuckx mchinen jblespiau global-localhost global19 global19-atlassian-net rainwoodman david-crouse rwgk 13693360699 mizux jiawen

pybind11_abseil's Issues

ImportError: generic_type: cannot initialize type "StatusCode": an object with that name is already defined

Hi,

I am trying to figure out some issues with using the pybind11_abseil.status usage. TensorFlow as well as some other Google projects are using the pybind11_abseil modules. I found that as of TensorFlow 2.14.0 (2.13.1 is fine), it looks like that using TensorFlow in conjunction with packages that use pybind11_abseil will fail with the error

ImportError: generic_type: cannot initialize type "StatusCode": an object with that name is already defined

This happens for example in google-deepmind/launchpad#44 and tensorflow/tflite-support#954. The error happens when a pybind11 extension that uses
py::google::ImportStatusModule() is called.

For example

import tensorflow
import launchpad # Fails when importing an extension that uses ImportStatusModule.

Reversing the order we will see the same issues this time coming from TensorFlow trying to import a module that uses
ImportStatusModule.

I noticed that there are some changes to the pybind11_abseil version used in TensorFlow 2.14.0, which incoporates some changes around the import_status_module, but I don't have a really good clue about what that change entails, it would be quite useful if you can provide some help.

Thanks!

Nanobind port

Nanobind looks fairly stable at this point. I recently got nanobind to work with my various Bazel projects and considering doing a port.

Are there any plans to release a nanobind_abseil? If not, I'm happy to take a stab at it in my spare time.

Support for absl::InlinedVector<T, N>?

I have a function that returns absl::InlinedVector<float, 4>, as well as a struct that contains one as a member.

I'm happy to implement this but would like to entertain suggestions for API design, especially for generic types T.

status target conflict with abseil-cpp ones

If a super build project use set(ABSL_ENABLE_INSTALL ON) or modify the pybind11_abseil CMakelists.txt as fallow

diff --git a/CMakeLists.txt b/CMakeLists.txt
index ceb65a8..4f4ed9c 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -19,8 +19,11 @@ endif()
 FetchContent_Declare(
   abseil-cpp
   URL https://github.com/abseil/abseil-cpp/archive/refs/tags/20230802.0.tar.gz
   URL_HASH
     SHA256=59d2976af9d6ecf001a81a35749a6e551a335b949d34918cfade07737b9d93c5)
+  #GIT_REPOSITORY "https://github.com/abseil/abseil-cpp.git"
+  #GIT_TAG "20230802.1"
+)
+set(ABSL_ENABLE_INSTALL ON)
 
 FetchContent_Declare(
   pybind11

note: same behaviour with the last abseil-cpp release

we got:

$ cmake -S. -Bbuild
-- The CXX compiler identification is GNU 13.2.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX17
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX17 - Success
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- pybind11 v2.12.0 dev1
-- Found Python: /usr/bin/python3.11 (found suitable version "3.11.6", minimum required is "3.6") found components: Interpreter Development.Module Development.Embed 
CMake Error at pybind11_abseil/CMakeLists.txt:153 (add_library):
add_library cannot create target "status" because another target with the
same name already exists.  The existing target is a static library created
in source directory
"/usr/local/google/home/corentinl/work/pybind11_abseil/build/_deps/abseil-cpp-src/absl/status".
See documentation for policy CMP0002 for more details.


CMake Error at pybind11_abseil/CMakeLists.txt:160 (target_link_libraries):
Attempt to add link library "status_pyinit_google3" to target "status"
which is not built in this directory.

This is allowed only when policy CMP0079 is set to NEW.


-- Configuring incomplete, errors occurred!

The code:

pybind11_abseil/pybind11_abseil/CMakeLists.txt

Lines 151 to 170 in 52f2739

# status ====================================================================
add_library(status SHARED status_py_extension_stub.cc)
add_library(pybind11_abseil::status ALIAS status)
target_include_directories(status INTERFACE $<BUILD_INTERFACE:${TOP_LEVEL_DIR}>)
set_target_properties(status PROPERTIES PREFIX "")
target_link_libraries(status PUBLIC status_pyinit_google3 absl::status)
# import_status_module =========================================================
add_library(import_status_module STATIC import_status_module.cc)
add_library(pybind11_abseil::import_status_module ALIAS import_status_module)
target_include_directories(import_status_module
INTERFACE $<BUILD_INTERFACE:${TOP_LEVEL_DIR}>)
target_link_libraries(import_status_module PUBLIC status)

The problem is that cmake TARGET names are not scoped thus the name clash, on my way to provide a patch in google/or-tools then a PR here...
would go for

# change logical target name
add_library(status_py_extension_stub SHARED status_py_extension_stub.cc)
# but keep the previous base name, "status" for the output file...
set_target_properties(status_py_extension_stub  PROPERTIES LIBRARY_OUTPUT_NAME "status")
# we can keep the ALIAS unchanged to pybind11_abseil::status
add_library(pybind11_abseil::status ALIAS status_py_extension_stub)

`status.so` does not build on MacOS with proper cc toolchain registration

Modern apple_support sets up a proper Apple CC toolchain.

One consequence of this is that the linker flag -undefined dynamic_lookup is no longer on by default (this was a bug. ld's default is -undefined error).

When switching to the modern toolchain, it causes status.so to fail because it does not depend on @pybind11//:pybind11. For these .sos, we should pass linkopts = ["-undefined", "dynamic_lookup"], as pybind_extension does here.

I'm happy to author a PR.

Error when converting absl::Span<const bool>

When trying to specify a py::arg with C++ type absl::Span (or more specifically in my case absl::optional<absl::Span>), I get an compile error:

./third_party/pybind11_abseil/absl_casters.h:378:12: error: no matching function for call to 'MakeSpan'
return absl::MakeSpan(static_cast<std::vector<value_type>&>(caster));
...

I guess this might be related to std::vector being potentially specialized as a bitset instead of an array of byte sized bools (abseil/abseil-cpp#644). It would be nice to be able to define an py::arg as absl::Span though if it's possible to side-step this issue.

Deprecated Abseil types ie.(`Breakdown`) in use

It appears that pybind11_abseil uses deprecated types. ie. Breakdown, From_Datetime. Will/when will this be updated?

These issues seem to appear even when using the 20230802.0 tag of abseil-cpp as specified in the WORKSPACE file.

For a more complete example of the deprecated features see a failed build:

external/pybind11_abseil/pybind11_abseil/absl_casters.h:132:33: error: 'Breakdown' is deprecated: Use `absl::TimeZone::CivilInfo`. [-Werror=deprecated-declarations]
  132 | constexpr absl::Time::Breakdown kDatetimeInfiniteFuture = {
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~
In file included from external/com_google_absl/absl/time/clock.h:26,
                 from external/com_google_absl/absl/synchronization/internal/kernel_timeout.h:30,
                 from external/com_google_absl/absl/synchronization/mutex.h:74,
                 from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
                 from external/com_google_absl/absl/strings/cord.h:91,
                 from external/com_google_absl/absl/status/internal/status_internal.h:23,
                 from external/com_google_absl/absl/status/status.h:59,
                 from external/pybind11_abseil/pybind11_abseil/status_caster.h:11,
                 from external/pybind11_abseil/pybind11_abseil/status_casters.h:20,
                 from pybind_example.cc:16:
external/com_google_absl/absl/time/time.h:805:62: note: declared here
  805 |   struct ABSL_DEPRECATED("Use `absl::TimeZone::CivilInfo`.") Breakdown {
      |                                                              ^~~~~~~~~
In file included from pybind_example.cc:21:
external/pybind11_abseil/pybind11_abseil/absl_casters.h:133:57: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::weekday' [-Werror=missing-field-initializers]
  133 |     9999, 12, 31, 23, 59, 59, absl::Microseconds(999999)};
      |                                                         ^
external/pybind11_abseil/pybind11_abseil/absl_casters.h:133:57: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::yearday' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:133:57: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::offset' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:133:57: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::is_dst' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:133:57: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::zone_abbr' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:134:33: error: 'Breakdown' is deprecated: Use `absl::TimeZone::CivilInfo`. [-Werror=deprecated-declarations]
  134 | constexpr absl::Time::Breakdown kDatetimeInfinitePast = {
      |                                 ^~~~~~~~~~~~~~~~~~~~~
In file included from external/com_google_absl/absl/time/clock.h:26,
                 from external/com_google_absl/absl/synchronization/internal/kernel_timeout.h:30,
                 from external/com_google_absl/absl/synchronization/mutex.h:74,
                 from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
                 from external/com_google_absl/absl/strings/cord.h:91,
                 from external/com_google_absl/absl/status/internal/status_internal.h:23,
                 from external/com_google_absl/absl/status/status.h:59,
                 from external/pybind11_abseil/pybind11_abseil/status_caster.h:11,
                 from external/pybind11_abseil/pybind11_abseil/status_casters.h:20,
                 from pybind_example.cc:16:
external/com_google_absl/absl/time/time.h:805:62: note: declared here
  805 |   struct ABSL_DEPRECATED("Use `absl::TimeZone::CivilInfo`.") Breakdown {
      |                                                              ^~~~~~~~~
In file included from pybind_example.cc:21:
external/pybind11_abseil/pybind11_abseil/absl_casters.h:135:43: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::weekday' [-Werror=missing-field-initializers]
  135 |     1, 1, 1, 0, 0, 0, absl::ZeroDuration()};
      |                                           ^
external/pybind11_abseil/pybind11_abseil/absl_casters.h:135:43: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::yearday' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:135:43: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::offset' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:135:43: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::is_dst' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:135:43: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::zone_abbr' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:147:72: error: 'Breakdown' is deprecated [-Werror=deprecated-declarations]
  147 |                                 const absl::Time::Breakdown& bd_special) {
      |                                                                        ^
external/pybind11_abseil/pybind11_abseil/absl_casters.h:147:72: error: 'Breakdown' is deprecated [-Werror=deprecated-declarations]
external/pybind11_abseil/pybind11_abseil/absl_casters.h: In member function 'bool pybind11::detail::type_caster<absl::lts_20230802::Time>::load(pybind11::handle, bool)':
external/pybind11_abseil/pybind11_abseil/absl_casters.h:233:29: error: 'Breakdown' is deprecated: Use `absl::TimeZone::CivilInfo`. [-Werror=deprecated-declarations]
  233 |       absl::Time::Breakdown bd_py = {
      |                             ^~~~~
In file included from external/com_google_absl/absl/time/clock.h:26,
                 from external/com_google_absl/absl/synchronization/internal/kernel_timeout.h:30,
                 from external/com_google_absl/absl/synchronization/mutex.h:74,
                 from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
                 from external/com_google_absl/absl/strings/cord.h:91,
                 from external/com_google_absl/absl/status/internal/status_internal.h:23,
                 from external/com_google_absl/absl/status/status.h:59,
                 from external/pybind11_abseil/pybind11_abseil/status_caster.h:11,
                 from external/pybind11_abseil/pybind11_abseil/status_casters.h:20,
                 from pybind_example.cc:16:
external/com_google_absl/absl/time/time.h:805:62: note: declared here
  805 |   struct ABSL_DEPRECATED("Use `absl::TimeZone::CivilInfo`.") Breakdown {
      |                                                              ^~~~~~~~~
In file included from pybind_example.cc:21:
external/pybind11_abseil/pybind11_abseil/absl_casters.h:240:73: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::weekday' [-Werror=missing-field-initializers]
  240 |           absl::Microseconds(PyDateTime_DATE_GET_MICROSECOND(src.ptr()))};
      |                                                                         ^
external/pybind11_abseil/pybind11_abseil/absl_casters.h:240:73: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::yearday' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:240:73: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::offset' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:240:73: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::is_dst' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:240:73: error: missing initializer for member 'absl::lts_20230802::Time::Breakdown::zone_abbr' [-Werror=missing-field-initializers]
external/pybind11_abseil/pybind11_abseil/absl_casters.h:271:21: error: 'absl::lts_20230802::Time absl::lts_20230802::FromDateTime(int64_t, int, int, int, int, int, absl::lts_20230802::TimeZone)' is deprecated: Use `absl::FromCivil(CivilSecond, TimeZone)`. [-Werror=deprecated-declarations]
  271 |       value = absl::FromDateTime(
      |                     ^~~~~~~~~~~~
In file included from external/com_google_absl/absl/time/clock.h:26,
                 from external/com_google_absl/absl/synchronization/internal/kernel_timeout.h:30,
                 from external/com_google_absl/absl/synchronization/mutex.h:74,
                 from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
                 from external/com_google_absl/absl/strings/cord.h:91,
                 from external/com_google_absl/absl/status/internal/status_internal.h:23,
                 from external/com_google_absl/absl/status/status.h:59,
                 from external/pybind11_abseil/pybind11_abseil/status_caster.h:11,
                 from external/pybind11_abseil/pybind11_abseil/status_casters.h:20,
                 from pybind_example.cc:16:
external/com_google_absl/absl/time/time.h:1384:13: note: declared here
 1384 | inline Time FromDateTime(int64_t year, int mon, int day, int hour, int min,
      |             ^~~~~~~~~~~~
In file included from pybind_example.cc:21:
external/pybind11_abseil/pybind11_abseil/absl_casters.h:273:72: error: 'absl::lts_20230802::Time absl::lts_20230802::FromDateTime(int64_t, int, int, int, int, int, absl::lts_20230802::TimeZone)' is deprecated: Use `absl::FromCivil(CivilSecond, TimeZone)`. [-Werror=deprecated-declarations]
  273 |           PyDateTime_GET_DAY(src.ptr()), 0, 0, 0, absl::LocalTimeZone());
      |                                                                        ^
In file included from external/com_google_absl/absl/time/clock.h:26,
                 from external/com_google_absl/absl/synchronization/internal/kernel_timeout.h:30,
                 from external/com_google_absl/absl/synchronization/mutex.h:74,
                 from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
                 from external/com_google_absl/absl/strings/cord.h:91,
                 from external/com_google_absl/absl/status/internal/status_internal.h:23,
                 from external/com_google_absl/absl/status/status.h:59,
                 from external/pybind11_abseil/pybind11_abseil/status_caster.h:11,
                 from external/pybind11_abseil/pybind11_abseil/status_casters.h:20,
                 from pybind_example.cc:16:
external/com_google_absl/absl/time/time.h:1384:13: note: declared here
 1384 | inline Time FromDateTime(int64_t year, int mon, int day, int hour, int min,
      |             ^~~~~~~~~~~~
In file included from pybind_example.cc:21:
external/pybind11_abseil/pybind11_abseil/absl_casters.h:273:72: error: 'absl::lts_20230802::Time absl::lts_20230802::FromDateTime(int64_t, int, int, int, int, int, absl::lts_20230802::TimeZone)' is deprecated: Use `absl::FromCivil(CivilSecond, TimeZone)`. [-Werror=deprecated-declarations]
  273 |           PyDateTime_GET_DAY(src.ptr()), 0, 0, 0, absl::LocalTimeZone());
      |                                                                        ^
In file included from external/com_google_absl/absl/time/clock.h:26,
                 from external/com_google_absl/absl/synchronization/internal/kernel_timeout.h:30,
                 from external/com_google_absl/absl/synchronization/mutex.h:74,
                 from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
                 from external/com_google_absl/absl/strings/cord.h:91,
                 from external/com_google_absl/absl/status/internal/status_internal.h:23,
                 from external/com_google_absl/absl/status/status.h:59,
                 from external/pybind11_abseil/pybind11_abseil/status_caster.h:11,
                 from external/pybind11_abseil/pybind11_abseil/status_casters.h:20,
                 from pybind_example.cc:16:
external/com_google_absl/absl/time/time.h:1384:13: note: declared here
 1384 | inline Time FromDateTime(int64_t year, int mon, int day, int hour, int min,
      |             ^~~~~~~~~~~~
In file included from pybind_example.cc:21:
external/pybind11_abseil/pybind11_abseil/absl_casters.h: In static member function 'static pybind11::handle pybind11::detail::type_caster<absl::lts_20230802::Time>::cast(const absl::lts_20230802::Time&, pybind11::return_value_policy, pybind11::handle)':
external/pybind11_abseil/pybind11_abseil/absl_casters.h:295:27: error: 'Breakdown' is deprecated: Use `absl::TimeZone::CivilInfo`. [-Werror=deprecated-declarations]
  295 |     absl::Time::Breakdown t = src.In(absl::UTCTimeZone());
      |                           ^
In file included from external/com_google_absl/absl/time/clock.h:26,
                 from external/com_google_absl/absl/synchronization/internal/kernel_timeout.h:30,
                 from external/com_google_absl/absl/synchronization/mutex.h:74,
                 from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
                 from external/com_google_absl/absl/strings/cord.h:91,
                 from external/com_google_absl/absl/status/internal/status_internal.h:23,
                 from external/com_google_absl/absl/status/status.h:59,
                 from external/pybind11_abseil/pybind11_abseil/status_caster.h:11,
                 from external/pybind11_abseil/pybind11_abseil/status_casters.h:20,
                 from pybind_example.cc:16:
external/com_google_absl/absl/time/time.h:805:62: note: declared here
  805 |   struct ABSL_DEPRECATED("Use `absl::TimeZone::CivilInfo`.") Breakdown {
      |                                                              ^~~~~~~~~
In file included from pybind_example.cc:21:
external/pybind11_abseil/pybind11_abseil/absl_casters.h:295:57: error: 'absl::lts_20230802::Time::Breakdown absl::lts_20230802::Time::In(absl::lts_20230802::TimeZone) const' is deprecated: Use `absl::TimeZone::At(Time)`. [-Werror=deprecated-declarations]
  295 |     absl::Time::Breakdown t = src.In(absl::UTCTimeZone());
      |                                                         ^
In file included from external/com_google_absl/absl/time/clock.h:26,
                 from external/com_google_absl/absl/synchronization/internal/kernel_timeout.h:30,
                 from external/com_google_absl/absl/synchronization/mutex.h:74,
                 from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
                 from external/com_google_absl/absl/strings/cord.h:91,
                 from external/com_google_absl/absl/status/internal/status_internal.h:23,
                 from external/com_google_absl/absl/status/status.h:59,
                 from external/pybind11_abseil/pybind11_abseil/status_caster.h:11,
                 from external/pybind11_abseil/pybind11_abseil/status_casters.h:20,
                 pybind_example.cc:16:
external/com_google_absl/absl/time/time.h:833:13: note: declared here
  833 |   Breakdown In(TimeZone tz) const;
      |             ^~

Provide a fine grained mechanism for specifying `PYBIND11_ABSEIL_STATUS_MODULE_PATH`

I am using pybind11_abseil with pybind11_bazel.

I recently updated pybind11_abseil to HEAD and was surprised when my tests failed trying to import pybind11_absil.status. It took some time to track down the offending line to pybind11::google::ImportStatusModule(); on the C++ side and then to track down how the path is discovered.

With Bazel and assuming the default external repository name of pybind11_abseil, the corresponding Python import is from pybind11_abseil.pybind11_abseil import status. It took a while to find the relevant #define in README.md.

As the README accurately states, one must pass -D PYBIND11_ABSEIL_STATUS_MODULE_PATH=<path> directly on the command line (via bazel build --cxxopt="-D ...").

I was hoping for a more fine grained customization point via a macro that wraps @pybind11_abseil//pybind11_abseil:status_casters. At the moment, it is a pure pybind_library with no way to pass a defines or local_defines without setting --cxxopt globally.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.