Support for new ESP32 Arduino core (3.x) and IDF (5.1) about nimble-arduino

[h2zero]

Oh, using this with IDF is not recommended since you have the option there to enable nimble. You will want to switch from this repo to esp-nimble-cpp.

[shlomozippel]

Thanks, confirming esp-nimble-cpp worked 🎉

I also tested NimBLE-Arduino with arduino framework (without espidf) and still ran into the same errors with the new esp32 core v3.x, so leaving this issue open for now

[h2zero]

I also tested NimBLE-Arduino with arduino framework (without espidf) and still ran into the same errors with the new esp32 core v3.x, so leaving this issue open for now

This is due to using platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5; in that fork he has enabled NimBLE in the config, the official release will not.

[h2zero]

What is it that is no longer compatible? This library doesn't really use any Arduino core functions so there shouldn't be any changes required from there. This does contain a slightly older NimBLE version but that should not matter.

[shlomozippel]

When I first build it I get:

                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/porting/nimble/include/os/os.h:42,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/nimble/include/nimble/ble.h:26,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEAddress.h:23,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEAddress.cpp:19:
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/porting/npl/freertos/include/nimble/nimble_npl_os.h:457:4: error: 'esp_timer_handle_t' does not name a type; did you mean 'TimerHandle_t'?
  457 |    esp_timer_handle_t handle;
      |    ^~~~~~~~~~~~~~~~~~```

So I tried adding `CONFIG_BT_NIMBLE_USE_ESP_TIMER=n` to sdkconfig, which then resulted in a whole bunch of other errors:

```.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:90:62: error: 'CONFIG_BTDM_SCAN_DUPL_TYPE' was not declared in this scope; did you mean 'CONFIG_BTDM_SCAN_DUPL_TYPE_DATA'?
   90 | uint8_t                     NimBLEDevice::m_scanFilterMode = CONFIG_BTDM_SCAN_DUPL_TYPE;
      |                                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                                              CONFIG_BTDM_SCAN_DUPL_TYPE_DATA
In file included from .pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/porting/nimble/include/syscfg/syscfg.h:9,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/nimble/include/nimble/ble.h:25,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEAddress.h:23,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEAdvertisedDevice.h:20,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEScan.h:20,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.h:22,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:18:
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp: In static member function 'static void NimBLEDevice::init(const std::string&)':
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_EVT_COUNT' was not declared in this scope; did you mean 'MYNEWT_VAL_BLE_HCI_ACL_OUT_COUNT'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:41:48: note: in expansion of macro 'MYNEWT_VAL'
   41 |     #define DEFAULT_BT_LE_HCI_EVT_HI_BUF_COUNT MYNEWT_VAL(BLE_TRANSPORT_EVT_COUNT)
      |                                                ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:235:33: note: in expansion of macro 'DEFAULT_BT_LE_HCI_EVT_HI_BUF_COUNT'
  235 |     .ble_hci_evt_hi_buf_count = DEFAULT_BT_LE_HCI_EVT_HI_BUF_COUNT,                     \
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_EVT_DISCARDABLE_COUNT' was not declared in this scope; did you mean 'CONFIG_BT_NIMBLE_TRANSPORT_EVT_DISCARD_COUNT'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:42:48: note: in expansion of macro 'MYNEWT_VAL'
   42 |     #define DEFAULT_BT_LE_HCI_EVT_LO_BUF_COUNT MYNEWT_VAL(BLE_TRANSPORT_EVT_DISCARDABLE_COUNT)
      |                                                ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:236:33: note: in expansion of macro 'DEFAULT_BT_LE_HCI_EVT_LO_BUF_COUNT'
  236 |     .ble_hci_evt_lo_buf_count = DEFAULT_BT_LE_HCI_EVT_LO_BUF_COUNT,                     \
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_MAX_PERIODIC_ADVERTISER_LIST' was not declared in this scope; did you mean 'DEFAULT_BT_LE_MAX_PERIODIC_ADVERTISER_LIST'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:32:56: note: in expansion of macro 'MYNEWT_VAL'
   32 |     #define DEFAULT_BT_LE_MAX_PERIODIC_ADVERTISER_LIST MYNEWT_VAL(BLE_MAX_PERIODIC_ADVERTISER_LIST)
      |                                                        ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:237:29: note: in expansion of macro 'DEFAULT_BT_LE_MAX_PERIODIC_ADVERTISER_LIST'
  237 |     .ble_ll_sync_list_cnt = DEFAULT_BT_LE_MAX_PERIODIC_ADVERTISER_LIST,                 \
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_ACL_SIZE' was not declared in this scope; did you mean 'CONFIG_BT_NIMBLE_TRANSPORT_ACL_SIZE'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:35:40: note: in expansion of macro 'MYNEWT_VAL'
   35 |     #define DEFAULT_BT_LE_ACL_BUF_SIZE MYNEWT_VAL(BLE_TRANSPORT_ACL_SIZE)
      |                                        ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:255:25: note: in expansion of macro 'DEFAULT_BT_LE_ACL_BUF_SIZE'
  255 |     .ble_acl_buf_size = DEFAULT_BT_LE_ACL_BUF_SIZE,                                     \
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_ACL_FROM_LL_COUNT' was not declared in this scope; did you mean 'CONFIG_BT_NIMBLE_TRANSPORT_ACL_FROM_LL_COUNT'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:36:41: note: in expansion of macro 'MYNEWT_VAL'
   36 |     #define DEFAULT_BT_LE_ACL_BUF_COUNT MYNEWT_VAL(BLE_TRANSPORT_ACL_FROM_LL_COUNT)
      |                                         ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:256:26: note: in expansion of macro 'DEFAULT_BT_LE_ACL_BUF_COUNT'
  256 |     .ble_acl_buf_count = DEFAULT_BT_LE_ACL_BUF_COUNT,                                   \
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_EVT_SIZE' was not declared in this scope; did you mean 'MYNEWT_VAL_BLE_HCI_TRANSPORT_EMSPI'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:37:44: note: in expansion of macro 'MYNEWT_VAL'
   37 |     #define DEFAULT_BT_LE_HCI_EVT_BUF_SIZE MYNEWT_VAL(BLE_TRANSPORT_EVT_SIZE)
      |                                            ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:257:29: note: in expansion of macro 'DEFAULT_BT_LE_HCI_EVT_BUF_SIZE'
  257 |     .ble_hci_evt_buf_size = DEFAULT_BT_LE_HCI_EVT_BUF_SIZE,                             \
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:873:16: error: 'struct esp_bt_controller_config_t' has no member named 'mode'
  873 |         bt_cfg.mode = ESP_BT_MODE_BLE;
      |                ^~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:874:16: error: 'struct esp_bt_controller_config_t' has no member named 'ble_max_conn'
  874 |         bt_cfg.ble_max_conn = CONFIG_BT_NIMBLE_MAX_CONNECTIONS;
      |                ^~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:876:16: error: 'struct esp_bt_controller_config_t' has no member named 'normal_adv_size'
  876 |         bt_cfg.normal_adv_size = m_scanDuplicateSize;
      |                ^~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:877:16: error: 'struct esp_bt_controller_config_t' has no member named 'scan_duplicate_type'
  877 |         bt_cfg.scan_duplicate_type = m_scanFilterMode;
      |                ^~~~~~~~~~~~~~~~~~~
*** [.pio/build/usba-c2/lib41b/NimBLE-Arduino/NimBLEDevice.cpp.o] Error 1
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEScan.cpp: In member function 'void NimBLEScan::clearDuplicateCache()':
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEScan.cpp:456:5: error: 'esp_ble_scan_dupilcate_list_flush' was not declared in this scope
  456 |     esp_ble_scan_dupilcate_list_flush();
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*** [.pio/build/usba-c2/lib41b/NimBLE-Arduino/NimBLEScan.cpp.o] Error 1```

Also, important to note that these errors only happen when my board is ESP32-C2. I get different errors when I build for ESP32-S3:

```In file included from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/porting/nimble/include/modlog/modlog.h:30,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/include/host/ble_hs_log.h:23,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/include/host/ble_hs.h:38,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/include/host/ble_gap.h:31,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/src/ble_gap_priv.h:26,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/src/ble_hs_priv.h:27,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/src/ble_gap.c:26:
.pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/src/ble_gap.c: In function 'ble_gap_log_duration':
/Users/shlomo/.platformio/packages/framework-espidf/components/log/include/esp_log.h:265:27: error: format '%d' expects argument of type 'int', but argument 6 has type 'int32_t' {aka 'long int'} [-Werror=format=]```

and more similar warnings that are treated as errors.

Feels like a combination of upgrading from idf 4.x to 5.x where some of the ble / nimble headers changed, and probably some build flags changed too

[shlomozippel]

Platformio.ini to repro:

[env]
framework = arduino, espidf
platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
lib_deps = 
  h2zero/NimBLE-Arduino @ ^1.4.1

[env:usba-c2]
board = esp32-c2-devkitm-1

[h2zero]

Closing this as it is compiling and working fine with the latest Arduino core master branch. The issue here is using a third party core which provides the NimBLE core sources via IDF config.