[ServerDev] Some feedback about nimble-arduino

[h2zero]

Thanks for testing this out, I’ll make the change to sdkconfig right away.

Descriptors work differently in NimBLE and I need to work on their implementation still. That said NimBLE automatically creates the 2902 descriptor if you enable notifications/ indications and will work but not callbacks for that yet.

Yes there is something you missed because I forgot to mention it.

Before you make the first call to start advertising you have to start the gatts server.

In your code try adding:
pServer->start();
Right before advertising start.

I’ll implement this directly in the advertising start function soon.

[beegee-tokyo]

Sounds great. Will try tomorrow again.

The reason why I use the descriptor callback is that after a client connects I send a data package. If I send it before notifications are enabled the client looses that data package. Hope you can implement this callback.

Will report back tomorrow.

[h2zero]

Great!
I just updated it as well to include the sdkconfig changes as well as handling the server start automatically when starting advertising so your code should work now as is.

[beegee-tokyo]

Found the problem.
My advertising data is too long (more than 31 bytes). If I shorten my device name to 4 characters advertising of one 128bit UUID + the name works. If I go 1 character longer the startAdvertise calls abort.

In Arduino-ESP32 BLE library some data is put into the scan response.
in void BLEAdvertising::start() of ESP32 BLE library:

	if (!m_customScanResponseData && m_scanResp) {
		m_advData.set_scan_rsp = true;
		m_advData.include_name = m_scanResp;
		m_advData.include_txpower = m_scanResp;
		errRc = ::esp_ble_gap_config_adv_data(&m_advData);
		if (errRc != ESP_OK) {
			log_e("<< esp_ble_gap_config_adv_data (Scan response): rc=%d %s", errRc, GeneralUtils::errorToString(errRc));
			return;
		}

Maybe something to do in NimBLE as well!

[h2zero]

Yes, I was looking at that as well. I think the start function should calculate the advertising packet size and make adjustments based on that.
It’s on the todo list now.

[h2zero]

No longer on the todo list, had some free time at work and it's now done.

Advertising will now limit data to 31 bytes and move the name to scan response. tx power is now also in the scan response. Now there is room in the advertisement packet for a 128bit uuid and a 10 char name. If larger than that there is room for 29 char name in scan response.

Descriptors are about 80% implemented as well, should be ready soon.

[beegee-tokyo]

Looking better and better. Advertising works and my Android app can connect.

Some statistics about Flash, RAM and heap usage. Heap is checked after all other functions are initialized (display, LoRa, BLE). Beside of BLE the app is the same, just one time using NimBLE and other time using ESP32 BLE

NimBLE

RAM: [= ] 9.4% (used 30920 bytes from 327680 bytes)
Flash: [===== ] 46.7% (used 611495 bytes from 1310720 bytes)

Internal Total heap 322872, internal Free Heap 257848

ESP32 BLE

RAM: [= ] 12.7% (used 41608 bytes from 327680 bytes)
Flash: [======== ] 78.0% (used 1022499 bytes from 1310720 bytes)

Internal Total heap 291572, internal Free Heap 166676

[h2zero]

Glad to hear it's working for you and those stats are a significant improvement!

I just pushed an update that enables the 2902 descriptor and the onwrite callback is working, onread is todo still. NimBLE handles that one automatically so I need to find the api for the callbacks onread to work.

Also since NimBLE takes care of that descriptor internally I did the same, the 2902 is automatically created in the library as well so there is no need to call addDescriptor in the setup code unless you want to set the callbacks or something else.

[beegee-tokyo]

Descriptor callback works fine.
onWrite callback works fine.
Thanks for the update.

Update:

ESP32 is now connected to an Android phone for 7 hours. Sending messages back and forth every 30 seconds using NimBLE.
Not a single reset or other error.

[h2zero]

Good stuff, thanks for the update.

Hopefully I can get the descriptor code finished up soon, then start on security and examples.

[beegee-tokyo]

Just as reference, here is how I get the descriptor and add the descriptor callback:

/** Descriptor for the BLE-UART TX characteristic */
NimBLEDescriptor *txDescriptor;

// Add callback for notifcation and indication changes
txDescriptor = pCharacteristicUartTX->getDescriptorByUUID("2902");
if (txDescriptor != NULL)
{
	myLog_d("Found 2902 descriptor!");
	txDescriptor->setCallbacks(new DescriptorCallbacks());
}

And the callback looks like

/**
 * Callbacks for BLE client descriptor changes
 * on BLE UART characteristic
 */
class DescriptorCallbacks : public NimBLEDescriptorCallbacks
{
	void onWrite(NimBLEDescriptor *pDescriptor)
	{
		uint8_t *descrValue;
		descrValue = pDescriptor->getValue();
		if (descrValue[0] & (1 << 0))
		{
			bleUARTnotifyEnabled = true;
		}
		else
		{
			bleUARTnotifyEnabled = false;
		}
	};
};

[h2zero]

Thanks!

Just pushed an update that allows for any type of descriptor to be added to a characteristic as well.
Haven't had time to do much testing but the quick test I did seems to be working.

Not sure yet how to implement the onRead callback for 2902 but I don't think it's important enough to be concerned with at this point. Not much use for it I don't think.

[beegee-tokyo]

Latest version doesn't compile under PlatformIO anymore.
I went down the commits and the problem starts after commit #f43533d. That is when you changed the sdkconfig.h file.
Starting from commit #0281973 compilation fails with missing FreeRTOS defines (in PlatformIO):

In file included from C:/users/beegee/.platformio/packages/framework-arduinoespressif32/cores/esp32/Arduino.h:32:0,
                 from src/main.h:2,
                 from src\BLE\ble_esp32.cpp:3:
C:/users/beegee/.platformio/packages/framework-arduinoespressif32/tools/sdk/include/freertos/freertos/FreeRTOS.h:244:3: error: #error configMAX_TASK_NAME_LEN must be set to a minimum of 1 in FreeRTOSConfig.h
  #error configMAX_TASK_NAME_LEN must be set to a minimum of 1 in FreeRTOSConfig.h
   ^

In ArduinoIDE I get redefinition errors and then compilation stops with a "expected type-specifier before NimBLE2902" error

In file included from B:\Projects\Arduino\libraries\NimBLE-Arduino\src/nimble/nimble_npl.h:26:0,

                 from B:\Projects\Arduino\libraries\NimBLE-Arduino\src/os/os.h:38,

                 from B:\Projects\Arduino\libraries\NimBLE-Arduino\src/nimble/ble.h:26,

                 from B:\Projects\Arduino\libraries\NimBLE-Arduino\src/NimBLEAddress.h:20,

                 from B:\Projects\Arduino\libraries\NimBLE-Arduino\src/NimBLEAdvertisedDevice.h:20,

                 from B:\Projects\Arduino\libraries\NimBLE-Arduino\src/NimbleScan.h:19,

                 from B:\Projects\Arduino\libraries\NimBLE-Arduino\src/NimBLEDevice.h:20,

                 from B:\Projects\Arduino\NimBLE-UART\NimBLE-UART.ino:22:

B:\Projects\Arduino\libraries\NimBLE-Arduino\src/nimble/sdkconfig.h:12:0: warning: "CONFIG_LOG_BOOTLOADER_LEVEL" redefined

 #define CONFIG_LOG_BOOTLOADER_LEVEL 3

 ^

In file included from C:\Users\beegee\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4/tools/sdk/include/log/esp_log.h:20:0,
                 from C:\Users\beegee\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\cores\esp32/esp32-hal-log.h:128,
                 from C:\Users\beegee\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\cores\esp32/esp32-hal.h:50,
                 from C:\Users\beegee\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\cores\esp32/Arduino.h:35,
                 from sketch\NimBLE-UART.ino.cpp:1:
C:\Users\beegee\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4/tools/sdk/include/config/sdkconfig.h:248:0: note: this is the location of the previous definition
 #define CONFIG_LOG_BOOTLOADER_LEVEL 0

...

In file included from B:\Projects\Arduino\NimBLE-UART\NimBLE-UART.ino:22:0:
B:\Projects\Arduino\NimBLE-UART\NimBLE-UART.ino: In function 'void setup()':
B:\Projects\Arduino\libraries\NimBLE-Arduino\src/NimBLEDevice.h:55:23: error: expected type-specifier before 'NimBLE2902'
 #define BLE2902       NimBLE2902
                       ^
B:\Projects\Arduino\NimBLE-UART\NimBLE-UART.ino:87:40: note: in expansion of macro 'BLE2902'
   pTxCharacteristic->addDescriptor(new BLE2902());

[h2zero]

Thanks for pointing this out, I haven't tested using PlatformIO. I've been doing some work with the descriptors and trying to minimize the sdkconfig to work in as many arduino versions as possible.

If you could try the latest update I did, about 5 mins ago, and change your code from:
pTxCharacteristic->addDescriptor(new BLE2902());
to:
pTxCharacteristic->createDescriptor("2902");

and let me know if you have any problems it would be appreciated.

I've been trying to maintain compatibility with the old library but some things just didn't make sense to keep, that is one of them.

Also where you create your characteristics you should change any:
BLECharacteristic::PROPERTY_XXXXX
to:
PROPERTY_XXXXX

since I have changed those to a defined value instead as it's more compatible with NimBLE.

Edit:
Had a look at the sdkconfig.h in the nimble folder, I need to play around with this, not sure why that is being used by Platform IO, but you could try adding:
#define CONFIG_FREERTOS_MAX_TASK_NAME_LEN 16
to the sdkconfig in the main folder.

Seems I might need to make changes to the NimBLE library files so there is only 1 sdkconfig :).

[beegee-tokyo]

Still no success on PlatformIO. Adding
#define CONFIG_FREERTOS_MAX_TASK_NAME_LEN 16
just leads to the next missing config.

On ArduinoIDE it compiles with latest version (many redefine warnings). But then simple BLE-UART example code crashes. Maybe I missed something?
Code:

/*
    Video: https://www.youtube.com/watch?v=oCMOYS71NIU
    Based on Neil Kolban example for IDF: https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLE%20Tests/SampleNotify.cpp
    Ported to Arduino ESP32 by Evandro Copercini

   Create a BLE server that, once we receive a connection, will send periodic notifications.
   The service advertises itself as: 6E400001-B5A3-F393-E0A9-E50E24DCCA9E
   Has a characteristic of: 6E400002-B5A3-F393-E0A9-E50E24DCCA9E - used for receiving data with "WRITE"
   Has a characteristic of: 6E400003-B5A3-F393-E0A9-E50E24DCCA9E - used to send data with  "NOTIFY"

   The design of creating the BLE server is:
   1. Create a BLE Server
   2. Create a BLE Service
   3. Create a BLE Characteristic on the Service
   4. Create a BLE Descriptor on the characteristic
   5. Start the service.
   6. Start advertising.

   In this example rxValue is the data received (only accessible inside that function).
   And txValue is the data to be sent, in this example just a byte incremented every second.
*/
#include <NimBLEDevice.h>
#include <NimBLEServer.h>
#include <NimBLEUtils.h>
//#include <BLE2902.h>

BLEServer *pServer = NULL;
BLECharacteristic * pTxCharacteristic;
bool deviceConnected = false;
bool oldDeviceConnected = false;
uint8_t txValue = 0;

// See the following for generating UUIDs:
// https://www.uuidgenerator.net/

#define SERVICE_UUID           "6E400001-B5A3-F393-E0A9-E50E24DCCA9E" // UART service UUID
#define CHARACTERISTIC_UUID_RX "6E400002-B5A3-F393-E0A9-E50E24DCCA9E"
#define CHARACTERISTIC_UUID_TX "6E400003-B5A3-F393-E0A9-E50E24DCCA9E"


class MyServerCallbacks: public BLEServerCallbacks {
    void onConnect(BLEServer* pServer) {
      deviceConnected = true;
    };

    void onDisconnect(BLEServer* pServer) {
      deviceConnected = false;
    }
};

class MyCallbacks: public BLECharacteristicCallbacks {
    void onWrite(BLECharacteristic *pCharacteristic) {
      std::string rxValue = pCharacteristic->getValue();

      if (rxValue.length() > 0) {
        Serial.println("*********");
        Serial.print("Received Value: ");
        for (int i = 0; i < rxValue.length(); i++)
          Serial.print(rxValue[i]);

        Serial.println();
        Serial.println("*********");
      }
    }
};


void setup() {
  Serial.begin(115200);

  // Create the BLE Device
  BLEDevice::init("UART Service");

  // Create the BLE Server
  pServer = BLEDevice::createServer();
  pServer->setCallbacks(new MyServerCallbacks());

  // Create the BLE Service
  BLEService *pService = pServer->createService(SERVICE_UUID);

  // Create a BLE Characteristic
  pTxCharacteristic = pService->createCharacteristic(
                        CHARACTERISTIC_UUID_TX,
                        PROPERTY_NOTIFY
                      );

  //  pTxCharacteristic->addDescriptor(new BLE2902());
  pTxCharacteristic->createDescriptor("2902");
  
  BLECharacteristic * pRxCharacteristic = pService->createCharacteristic(
      CHARACTERISTIC_UUID_RX,
      PROPERTY_WRITE
                                          );

  pRxCharacteristic->setCallbacks(new MyCallbacks());

  // Start the service
  pService->start();

  // Start the server
  pServer->start();

  // Start advertising
  pServer->getAdvertising()->start();
  Serial.println("Waiting a client connection to notify...");
}

void loop() {

  if (deviceConnected) {
    pTxCharacteristic->setValue(&txValue, 1);
    pTxCharacteristic->notify();
    txValue++;
    delay(10); // bluetooth stack will go into congestion, if too many packets are sent
  }

  // disconnecting
  if (!deviceConnected && oldDeviceConnected) {
    delay(500); // give the bluetooth stack the chance to get things ready
    pServer->startAdvertising(); // restart advertising
    Serial.println("start advertising");
    oldDeviceConnected = deviceConnected;
  }
  // connecting
  if (deviceConnected && !oldDeviceConnected) {
    // do stuff here on connecting
    oldDeviceConnected = deviceConnected;
  }
}

Error log from terminal:

GAP procedure initiated: stop advertising.
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC      : 0x400dbce8  PS      : 0x00060130  A0      : 0x800d3fc1  A1      : 0x3ffc5ce0  
A2      : 0x3ffc8c70  A3      : 0x3ffc87c8  A4      : 0x3ffc87c4  A5      : 0x3ffc5d28  
A6      : 0x3ffc5d6e  A7      : 0x3ffc2601  A8      : 0x00000001  A9      : 0xfefefefe  
A10     : 0x00000000  A11     : 0x00000001  A12     : 0x00000000  A13     : 0x00000030  
A14     : 0x00000000  A15     : 0x3ffc8b4c  SAR     : 0x00000018  EXCCAUSE: 0x0000001c  
EXCVADDR: 0xfefefefe  LBEG    : 0x4000c349  LEND    : 0x4000c36b  LCOUNT  : 0x00000000  

Backtrace: 0x400dbce8:0x3ffc5ce0 0x400d3fbe:0x3ffc5d00 0x400d1be5:0x3ffc5d90 0x400e4863:0x3ffc5de0 0x4008d4d5:0x3ffc5e00

Rebooting...
ets Jun  8 2016 00:22:57

Decoded backtrace from exception says:

PC: 0x400dbce8: ble_gatts_count_cfg at B:\Projects\Arduino\libraries\NimBLE-Arduino\src\nimble\host\src\ble_gatts.c line 2058
EXCVADDR: 0xfefefefe

Decoding stack results
0x400dbce8: ble_gatts_count_cfg at B:\Projects\Arduino\libraries\NimBLE-Arduino\src\nimble\host\src\ble_gatts.c line 2058
0x400d3fbe: NimBLEService::start() at B:\Projects\Arduino\libraries\NimBLE-Arduino\src\NimBLEService.cpp line 162
0x400d1be5: setup() at B:\Projects\Arduino\NimBLE-UART/NimBLE-UART.ino line 98
0x400e4863: loopTask(void*) at C:\Users\beegee\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\cores\esp32\main.cpp line 14
0x4008d4d5: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c line 143

[beegee-tokyo]

Solution for sdkconfig.h problem:

1. in NimBLE-Arduino/src rename sckconfig.h to NimConfig.h
2. delete sdkconfig.h in NimBLE-Arduino/src/nimble
3. change all includes from #include "sdkconfig.h" to #include "NimConfig.h"

Compiles without problems under both ArduinoIDE and PlatformIO.

Content of NimConfig.h (two sdkconfig.h defines are required to get all working!)

#pragma once
#define CONFIG_BT_NIMBLE_MEM_ALLOC_MODE_INTERNAL 1
#define CONFIG_BT_NIMBLE_MAX_CONNECTIONS 3
#define CONFIG_BT_NIMBLE_MAX_BONDS 3
#define CONFIG_BT_NIMBLE_MAX_CCCDS 8
#define CONFIG_BT_NIMBLE_L2CAP_COC_MAX_NUM 0
#define CONFIG_BT_NIMBLE_PINNED_TO_CORE_0 1
#define CONFIG_BT_NIMBLE_PINNED_TO_CORE 0
#define CONFIG_BT_NIMBLE_TASK_STACK_SIZE 4096
#define CONFIG_BT_NIMBLE_ROLE_CENTRAL 1
#define CONFIG_BT_NIMBLE_ROLE_PERIPHERAL 1
#define CONFIG_BT_NIMBLE_ROLE_BROADCASTER 1
#define CONFIG_BT_NIMBLE_ROLE_OBSERVER 1
#define CONFIG_BT_NIMBLE_NVS_PERSIST 1
#define CONFIG_BT_NIMBLE_SM_LEGACY 1
#define CONFIG_BT_NIMBLE_SM_SC 1
#define CONFIG_BT_NIMBLE_SVC_GAP_DEVICE_NAME "nimble"
#define CONFIG_BT_NIMBLE_GAP_DEVICE_NAME_MAX_LEN 31
#define CONFIG_BT_NIMBLE_ATT_PREFERRED_MTU 256
#define CONFIG_BT_NIMBLE_SVC_GAP_APPEARANCE 0x0
#define CONFIG_BT_NIMBLE_ACL_BUF_COUNT 12
#define CONFIG_BT_NIMBLE_ACL_BUF_SIZE 255
#define CONFIG_BT_NIMBLE_HCI_EVT_BUF_SIZE 70
#define CONFIG_BT_NIMBLE_HCI_EVT_HI_BUF_COUNT 30
#define CONFIG_BT_NIMBLE_HCI_EVT_LO_BUF_COUNT 8

#define CONFIG_BT_ENABLED 1
#define CONFIG_BTDM_CONTROLLER_MODE_BLE_ONLY 1

[beegee-tokyo]

And last one today.

After making above changes the NimBLE-UART.no example compiles and works under ArduinoIDE without problems using your latest commit.

Same for my own application, working now with the latest commit.

Don't know how the test the Eddystone stuff.

[h2zero]

Awesome, thanks for the help, seems I'll need to do some more work with the config files.

Unfortunately NimBLE has a lot of relative path includes and some of them involve the sdkconfig which is why there are 2 of them.

I didn't want to touch the NimBLE core files if I didn't have to so updating it would be a simple matter of replacing files, but it seems the best solution to minimize problems for others would be to make some changes in there.

I haven't tested the eddystone stuff either, was looking for examples from the original library but couldn't find any. I just copied the files in so they would be there for anyone that has code that uses it. I'll have a play with that later on.

[beegee-tokyo]

The changes I did are only the includes in the NimBLE files so that should be manageable. And PlatformIO is way better than ArduinoIDE. Compilation is 3 times faster, you can jump directly to variables and functions declarations/definitions, make global defines, easy way to support different boards,.... You should give it a try. . I think it is important to keep compatibility to both IDEs.
I will look as well for some Eddystone examples tomorrow.

[h2zero]

Thanks, yes I will try out PlatformIO. I mainly use the idf and arduino as a component in eclipse.

I did find the bug causing the crash in BLEuart. Only happens when using arduino. Just pushed an update to fix that.

Now I’ll play around with sdkconfig :).

[beegee-tokyo]

Eclipse is as good as PlatformIO!
I just feel ArduinoIDE is like a toy for kids.

You did a great job with the NimBLE library. I will try tomorrow to use it with a custom characteristic instead of the BLE UART.

[h2zero]

Yeah Arduino ide and compiler are a too limited but great for a quick and dirty project. If you use the IDF you can just clone the espressif nimble library and add the bits from the v4 idf branch to v3.2 and stick it in your project/components folder. So much better and only 1 sdkconfig lol.

I plan to put that up as a repo as well when the cpp library is ready.

[h2zero]

So it seems arduino initialization was releasing the memory for BLE because of the config setup so I implemented your suggested changes and so far so good.

Thanks!

[beegee-tokyo]

I tested the iBeacon example from the BLE library with NimBLE and it works just fine.

I looked for Eddystone example code, but there is nothing that uses the BLEEddystoneURL or BLEEddystoneTLM stuff.

In fact I think that these two classes are incomplete. At least for BLEEddystoneURL, the advertising frame created there is incomplete. It doesn't hold all informations. I checked this Github blob and wrote some code using the NimBLE library and it advertises correct as EddystoneURL.

Here is the iBeacon example code which is beside of one line the same as the original example:

/*
   Based on Neil Kolban example for IDF: https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLE%20Tests/SampleScan.cpp
   Ported to Arduino ESP32 by pcbreflux
*/

/*
   Create a BLE server that will send periodic iBeacon frames.
   The design of creating the BLE server is:
   1. Create a BLE Server
   2. Create advertising data
   3. Start advertising.
   4. wait
   5. Stop advertising.
   6. deep sleep
   
*/
#include "sys/time.h"

#include <Arduino.h>

#include "NimBLEDevice.h"
#include "NimBLEUtils.h"
#include "NimBLEBeacon.h"
#include "NimBLEAdvertising.h"

#include "esp_sleep.h"

#define GPIO_DEEP_SLEEP_DURATION 10		 // sleep x seconds and then wake up
RTC_DATA_ATTR static time_t last;		 // remember last boot in RTC Memory
RTC_DATA_ATTR static uint32_t bootcount; // remember number of boots in RTC Memory

#ifdef __cplusplus
extern "C"
{
#endif

	uint8_t temprature_sens_read();
	//uint8_t g_phyFuns;

#ifdef __cplusplus
}
#endif

// See the following for generating UUIDs:
// https://www.uuidgenerator.net/
BLEAdvertising *pAdvertising;
struct timeval now;

#define BEACON_UUID "8ec76ea3-6668-48da-9866-75be8bc86f4d" // UUID 1 128-Bit (may use linux tool uuidgen or random numbers via https://www.uuidgenerator.net/)

void setBeacon()
{

	BLEBeacon oBeacon = BLEBeacon();
	oBeacon.setManufacturerId(0x4C00); // fake Apple 0x004C LSB (ENDIAN_CHANGE_U16!)
	oBeacon.setProximityUUID(BLEUUID(BEACON_UUID));
	oBeacon.setMajor((bootcount & 0xFFFF0000) >> 16);
	oBeacon.setMinor(bootcount & 0xFFFF);
	BLEAdvertisementData oAdvertisementData = BLEAdvertisementData();
	BLEAdvertisementData oScanResponseData = BLEAdvertisementData();

	oAdvertisementData.setFlags(0x04); // BR_EDR_NOT_SUPPORTED 0x04

	std::string strServiceData = "";

	strServiceData += (char)26;   // Len
	strServiceData += (char)0xFF; // Type
	strServiceData += oBeacon.getData();
	oAdvertisementData.addData(strServiceData);

	pAdvertising->setAdvertisementData(oAdvertisementData);
	pAdvertising->setScanResponseData(oScanResponseData);
	// pAdvertising->setAdvertisementType(ADV_TYPE_NONCONN_IND);
}

void setup()
{

	Serial.begin(115200);
	gettimeofday(&now, NULL);

	Serial.printf("start ESP32 %d\n", bootcount++);

	Serial.printf("deep sleep (%lds since last reset, %lds since last boot)\n", now.tv_sec, now.tv_sec - last);

	last = now.tv_sec;

	// Create the BLE Device
	BLEDevice::init("MeBeacon");

	// Create the BLE Server
	// BLEServer *pServer = BLEDevice::createServer(); // <-- no longer required to instantiate BLEServer, less flash and ram usage

	pAdvertising = BLEDevice::getAdvertising();

	setBeacon();
	// Start advertising
	pAdvertising->start();
	Serial.println("Advertizing started...");
	delay(10000);
	pAdvertising->stop();
	Serial.printf("enter deep sleep\n");
	esp_deep_sleep(1000000LL * GPIO_DEEP_SLEEP_DURATION);
	Serial.printf("in deep sleep\n");
}

void loop()
{
}

And here is the EddyStoneURL example:

/*
   Based on Neil Kolban example for IDF: https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLE%20Tests/SampleScan.cpp
   Ported to Arduino ESP32 by pcbreflux
*/

/*
   Create a BLE server that will send periodic iBeacon frames.
   The design of creating the BLE server is:
   1. Create a BLE Server
   2. Create advertising data
   3. Start advertising.
   4. wait
   5. Stop advertising.
   6. deep sleep
   
*/
#include "sys/time.h"

#include <Arduino.h>

#include "NimBLEDevice.h"
#include "NimBLEUtils.h"
#include "NimBLEBeacon.h"
#include "NimBLEAdvertising.h"

#include "esp_sleep.h"

#define GPIO_DEEP_SLEEP_DURATION 10		 // sleep x seconds and then wake up
RTC_DATA_ATTR static time_t last;		 // remember last boot in RTC Memory
RTC_DATA_ATTR static uint32_t bootcount; // remember number of boots in RTC Memory

#ifdef __cplusplus
extern "C"
{
#endif

	uint8_t temprature_sens_read();
	//uint8_t g_phyFuns;

#ifdef __cplusplus
}
#endif

// See the following for generating UUIDs:
// https://www.uuidgenerator.net/
BLEAdvertising *pAdvertising;
struct timeval now;

#define BEACON_UUID "8ec76ea3-6668-48da-9866-75be8bc86f4d" // UUID 1 128-Bit (may use linux tool uuidgen or random numbers via https://www.uuidgenerator.net/)

#define FLAGS_UID 0x00
#define FLAGS_URL 0x10
#define FLAGS_TLM 0x20

static const char *eddystone_url_prefix_subs[] = {
	"http://www.",
	"https://www.",
	"http://",
	"https://",
	"urn:uuid:",
	NULL};

static const char *eddystone_url_suffix_subs[] = {
	".com/",
	".org/",
	".edu/",
	".net/",
	".info/",
	".biz/",
	".gov/",
	".com",
	".org",
	".edu",
	".net",
	".info",
	".biz",
	".gov",
	NULL};

static int string_begin_with(const char *str, const char *prefix)
{
	int prefix_len = strlen(prefix);
	if (strncmp(prefix, str, prefix_len) == 0)
	{
		return prefix_len;
	}
	return 0;
}

void setBeacon()
{
	BLEAdvertisementData oAdvertisementData = BLEAdvertisementData();
	BLEAdvertisementData oScanResponseData = BLEAdvertisementData();

	const char url[] = "http://giesecke.tk";

	int scheme_len, ext_len = 1, i, idx, url_idx;
	char *ret_data;
	int url_len = strlen(url);

	ret_data = (char *) calloc(1, url_len + 13);

	ret_data[0] = 2;	 // Len
	ret_data[1] = 0x01;  // Type Flags
	ret_data[2] = 0x06;  // GENERAL_DISC_MODE 0x02 | BR_EDR_NOT_SUPPORTED 0x04
	ret_data[3] = 3;	 // Len
	ret_data[4] = 0x03;  // Type 16-Bit UUID
	ret_data[5] = 0xAA;  // Eddystone UUID 2 -> 0xFEAA LSB
	ret_data[6] = 0xFE;  // Eddystone UUID 1 MSB
	ret_data[7] = 19;	// Length of Beacon Data
	ret_data[8] = 0x16;  // Type Service Data
	ret_data[9] = 0xAA;  // Eddystone UUID 2 -> 0xFEAA LSB
	ret_data[10] = 0xFE; // Eddystone UUID 1 MSB
	ret_data[11] = 0x10; // Eddystone Frame Type
	ret_data[12] = 0x20; // Beacons TX power at 0m

	i = 0, idx = 13, url_idx = 0;

	//replace prefix
	scheme_len = 0;
	while (eddystone_url_prefix_subs[i] != NULL)
	{
		if ((scheme_len = string_begin_with(url, eddystone_url_prefix_subs[i])) > 0)
		{
			ret_data[idx] = i;
			idx++;
			url_idx += scheme_len;
			break;
		}
		i++;
	}
	while (url_idx < url_len)
	{
		i = 0;
		ret_data[idx] = url[url_idx];
		ext_len = 1;
		while (eddystone_url_suffix_subs[i] != NULL)
		{
			if ((ext_len = string_begin_with(&url[url_idx], eddystone_url_suffix_subs[i])) > 0)
			{
				ret_data[idx] = i;
				break;
			}
			else
			{
				ext_len = 1; //inc 1
			}
			i++;
		}
		url_idx += ext_len;
		idx++;
	}
	ret_data[7] = idx - 8;

	std::string eddyStoneData(ret_data);

	oAdvertisementData.addData(eddyStoneData);
	pAdvertising->setAdvertisementData(oAdvertisementData);
	// oScanResponseData.addData("Bernd Giesecke");
	pAdvertising->setScanResponseData(oScanResponseData);
}

void setup()
{

	Serial.begin(115200);
	gettimeofday(&now, NULL);

	Serial.printf("start ESP32 %d\n", bootcount++);

	Serial.printf("deep sleep (%lds since last reset, %lds since last boot)\n", now.tv_sec, now.tv_sec - last);

	last = now.tv_sec;

	// Create the BLE Device
	BLEDevice::init("MeBeacon");

	// Create the BLE Server
	// BLEServer *pServer = BLEDevice::createServer(); // <-- no longer required to instantiate BLEServer, less flash and ram usage

	pAdvertising = BLEDevice::getAdvertising();

	setBeacon();
	// Start advertising
	pAdvertising->start();
	Serial.println("Advertizing started...");
	delay(10000);
	pAdvertising->stop();
	Serial.printf("enter deep sleep\n");
	esp_deep_sleep(1000000LL * GPIO_DEEP_SLEEP_DURATION);
	Serial.printf("in deep sleep\n");
}

void loop()
{
}

[beegee-tokyo]

I think I understand now the purpose of BLEEddystoneURL and BLEEddystoneTLM classes. They are to decode data received from an Eddystone beacon, not to create one.

Will try if I can create some code to test these classes. One ESP32 acting as Eddystone beacon, another one to receive the data and decode it using these classes.

[h2zero]

Thanks for doing this, NimBLE has built in functions for creating eddystone URL and UID beacons. Maybe I should add that functionality to the class to use those.

[beegee-tokyo]

That sounds great as long as it doesn't break compatibility with the current BLE libraries. Let me know if I can help.

[h2zero]

I'll have to look at it more, the library seems to be capable of creating a beacon and decoding them so it should be dual purpose. I'll see what I can do to use NimBLE to simplify your example.

[beegee-tokyo]

More tests from my side.
I created an iBeacon, an EddystoneTLM beacon and an EddystoneURL beacon and wrote a beacon scanner app (base source).

I checked the beacons as well with the nRFConnect app on Android.
All three beacons work as expected and are detected by both nRFConnect and the ESP32 scanner as the correct beacon types.

Two findings:

BLE scan callback called twice

In the scanner app I declared the scanner callback as in your example with BLEScanResults foundDevices = pBLEScan->start(scanTime, false); and the callback is called as expected. But it is called twice for each device, but the first time without the data attached.
Serial output:

Scan done!
Found ServiceUUID: 6e400001-b5a3-f393-e0a9-e50e24dcca9e

Device name: EC-BF6C660E

Found ServiceUUID: 6e400001-b5a3-f393-e0a9-e50e24dcca9e

Found an iBeacon!
iBeacon Frame
ID: 004C Major: 0 Minor: 90 UUID: 8ec76ea3-6668-48da-9866-75be8bc86f4d Power: 0
Device name: iBeacon

Found an iBeacon!
iBeacon Frame
ID: 004C Major: 0 Minor: 90 UUID: 8ec76ea3-6668-48da-9866-75be8bc86f4d Power: 0
Device name: TLMBeacon

Device name: TLMBeacon

Found ServiceUUID: 0xfeaa

Found an EddystoneTLM beacon!
Reported battery voltage: 3246mV
Reported temperature from TLM class: 37912.00C
Reported temperature from data: 24.58C
Reported advertise count: 91
Reported time since last reboot: 1844ms

General bug in BLEEddystoneTLM (both Arduino and NimBLE)

EddystoneTLM has a temperature value.
According to Google definition the format is 8.8 fixed-point notation. But in the BLEEddystoneTLM class the temperature is declared as float, and the two bytes of the temperature value is just interpreted as an float value which gives complete wrong values. I found one Eddystone TLM example, and there the setting the temperature values is done wrong as well. The wrong value can be seen in nRFConnect, where it shows impossible temperatures.

The correct conversion from float to 8.8 fixed-point and back is shown in Fixed Point Maths.

The correct conversion on the beacon side would be (as example read temperature from a DHT sensor)

float tempFloat = dht.getTemperature();
int temp = (int)(tempFloat * 256);
...
beacon_data[5] = (temp & 0xFF);					   // Beacon temperature
beacon_data[4] = (temp >> 8);					   //
...

And in BLEEddystoneTLM class the getTemp() function should be

float NimBLEEddystoneTLM::getTemp() {
	return ((((m_eddystoneData.temp) & 0xFF00) >> 8) + (((m_eddystoneData.temp) & 0xFF) << 8)) / 256.0f;
} // getTemp

I will report this bug in the arduino-esp32 issues as well.

[beegee-tokyo]

Further test on double callback, I tested with the default Arduino BLE libraries and the callback is called only once there:

Device name: EC-BF6C660E

Found ServiceUUID: 6e400001-b5a3-f393-e0a9-e50e24dcca9e

Device name: iBeacon

Found an iBeacon!
iBeacon Frame
ID: 004C Major: 0 Minor: 332 UUID: 8ec76ea3-6668-48da-9866-75be8bc86f4d Power: 0
Devices found: 2
Scan done!

[h2zero]

Great catch, thanks for this, I will update the code to reflect asap.
As for the double callback I will have to see what I can do about it. This is a function of NimBLE, it calls the callback once for the advertisment and a second time for the scan response. In the library I just updated the advertisedDevice data with the scan response data when it comes but perhaps I should filter it and only callback when the scan response is received.
Then I would have to make sure active scanning is being performed and the device has scan response capability or there will be no callback lol. I'll put it on my todo list to look into.

[beegee-tokyo]

Ah, now I understand the double callback.

Just a thought (not compatible with ESP32 BLE library)
Have two call backs. One for advertise data and a second one for scan response data.

For the other issue (EddystoneTLM temperature) I need to investigate more. Chegewara gave me some feedback I need to check. Don't push a change yet.

[h2zero]

Yeah I was thinking of making a scan response callback but I wanted to remain compatible as much as possible. Also made more sense to just add that data to the advertisement packet data. However I do believe the callback should only occur with the complete data is available.

[h2zero]

Pushed an update to address the double callback issue, if you could retest with your beacons it would be much appreciated. Also log level selection now works correctly in Arduino and reading/writing descriptors is fixed.

All looks good in my testing so far, more testing and examples next.

[beegee-tokyo]

Double call of scan callback is gone. Scan works as expected.

Sent a pull request for the EddystoneTLM data parsing fix. There was more than just the temperature wrong in that class.

Added 3 examples
Beacon scanner example to show usage of BLEEddystoneTLM class and BLEEddystoneURL class
EddystoneTLM beacon example
EddystoneURL beacon example

[h2zero]

Great work! Thank you!
That saved me a lot of time not having to dig into that and search up examples.
I will merge the PR shortly.

[beegee-tokyo]

If you are interested in a more complex example using custom characteristics, I just converted my
ESP32WiFiBLE to use NimBLE-Arduino ==> ESP32WiFiBLE-NimBLE.

Here a comparison of the identical code, one time using ESP32 BLE library and second time using NimBLE-Arduino library

Memory usage (compilation output)

Arduino BLE library

RAM:   [==        ]  17.7% (used 58156 bytes from 327680 bytes)    
Flash: [========  ]  76.0% (used 1345630 bytes from 1769472 bytes)    

NimBLE-Arduino library

RAM:   [=         ]  14.5% (used 47476 bytes from 327680 bytes)    
Flash: [=======   ]  69.5% (used 911378 bytes from 1310720 bytes)    

Memory usage after setup() function

Arduino BLE library

Internal Total heap 259104, internal Free Heap 91660

NimBLE-Arduino library

Internal Total heap 290288, internal Free Heap 182344

---

This is double the free heap for the application!

---

[h2zero]

That's a cool project. Only thing you need to do is add security so you don't send plain text credentials over the air. It's already configured for just works pairing it in the library all you have to do is turn it on by adding:
PROPERTY_READ_ENC | PROPERTY_WRITE_ENC
to your createCharacteristic() call.

[beegee-tokyo]

Just have to find out how the decoding/encoding on the Android side works.

Last commit doesn't compile. Something is missing. If I comment out the function (not called from anywhere right now), it compiles and works.

[h2zero]

Working on that right now, seems I pushed the wrong file from my working directory.
Is it the int rc = ble_gap_check_conn_params( BLE_HCI_LE_PHY_2M, m_pConnParams); that's causing your error?

[beegee-tokyo]

I think so, it's past midnight here and I thought I left a comment on your commit, but it's gone. Will check tomorrow morning.

[h2zero]

Had some time to play with the beacon examples you made. They work great, I may find them useful for my own projects 👍 . Thanks again! I think this is almost complete as an initial stable release at this point.

[beegee-tokyo]

Glad you liked the Beacon examples.

For adding security, I added PROPERTY_READ_ENC | PROPERTY_WRITE_ENC but I do not see any difference. My Android app just connects and reads the data as it did before.

How would I add pairing requirement to the app?

[h2zero]

Not sure how Android handles pairing in a just works method, might not prompt for user acceptance. I'll test it on my end and see if NimBLE shows the encryption getting initialized.

[h2zero]

I finally got it to compile with IDF lol, arduino libraries can be a pain with it, had to do some edits to arduinojson to get it going. However with those properties added the pairing is working for me. I suspect Android just isn't telling you. As a side note, with IDF v3.2 and your code with my sdk settings I get: Internal Total heap 296852, internal Free Heap 239988 :)

[beegee-tokyo]

Nice!

I saw the automatic pairing after I connected the ESP32 to my phone. The ESP32 was listed as paired device.

Next challenge, I tried to add passcode with NimBLESecurity. But my phone always says connection failed because of wrong pin code.

[h2zero]

Cool, well for passcode testing what you need to do is change the io mode like this:
NimBLEDevice::setSecurityIOCap(BLE_HS_IO_DISPLAY_ONLY ); // set display output capability NimBLEDevice::setSecuityAuth(false, true, true); // no bonding, man in the middle protection, secure connection
That's the NimBLE way, but the old ESP lib way works too.
123456 should be working as the default passcode.

[h2zero]

I never liked how the old library did the security callbacks, defining another class etc. I kept it in for compatibility but implemented the callbacks in the server/client callback classes as well, I need to build some examples of both to show the changes.
I just added those 2 lines to your code after the setPower() call and it worked fine with 123456 as the passcode.

[beegee-tokyo]

Thanks, pin works now when using the default pin.

But if I try to set a different pin with

// Set pin code
BLESecurity * pSecurity = new BLESecurity();
pSecurity->setStaticPIN(300162);

then again I am back to "wrong pin" on the Android side.

How would I set another pin?

[h2zero]

Oops, I didn't complete that part of the backward compatibility, I'll get on that next. For now in your class MyServerCallbacks add:

uint32_t onPassKeyRequest() {
    return 300162;
}

[beegee-tokyo]

Aaaahhh,

ok, no pressure on that one. I am just playing around.

I implemented all security callbacks, but it looks like only onAuthenticationComplete is called.
onPassKeyRequest is never called.

Something for later.

[h2zero]

Thats's odd I'll have another look tomorrow. I fixed the backward compatibility just now and will push the update if you want to try it with the setStaticPIN() call.

[beegee-tokyo]

Super!

setStaticPin() works now! Thanks.

btw., in NimBLEDevice.h lines 88 and 89

    static void             setsScurityInitKey(uint8_t init_key);
    static void             setsScurityRespKey(uint8_t init_key);

is this a typo?

[h2zero]

haha, yes it's a typo from the original library that I copied over I guess, good catch!

Edit: Fixed now

[h2zero]

I think this is all sorted, closing this one for now.