Server limited to 25 characteristics about esp-homekit

[maccoylton]

What chip are you using? Devices like the esp8266 have limited memory, each service & characteristic consumes some memory, you may be hitting the limit of what the chip can handle.

[MrCavemen]

I'm running the ESP-wroom-32. I've attached the details from the chip below
Before the requests, the Free heap was 185608. So it doesn't seems to be a memory issue.

To be absolutely sure, I now also print the free heap when the characteristics are requested.
The lowest I see is 189804. (I've reduced my memory footprint a bit since last time.)

I (27) boot: ESP-IDF v4.3.2 2nd stage bootloader
I (27) boot: compile time 16:29:17
I (27) boot: chip revision: 1
I (30) boot_comm: chip revision: 1, min. bootloader chip revision: 0
I (37) boot.esp32: SPI Speed      : 40MHz
I (41) boot.esp32: SPI Mode       : DIO
I (46) boot.esp32: SPI Flash Size : 4MB
I (50) boot: Enabling RNG early entropy source...
I (56) boot: Partition Table:
I (59) boot: ## Label            Usage          Type ST Offset   Length
I (67) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (74) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (82) boot:  2 factory          factory app      00 00 00010000 00100000
I (89) boot: End of partition table
I (93) boot_comm: chip revision: 1, min. application chip revision: 0
I (100) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=1dcach (122028) map
I (153) esp_image: segment 1: paddr=0002dcd4 vaddr=3ffb0000 size=02344h (  9028) load
I (157) esp_image: segment 2: paddr=00030020 vaddr=400d0020 size=95150h (610640) map
I (380) esp_image: segment 3: paddr=000c5178 vaddr=3ffb2344 size=042e4h ( 17124) load
I (387) esp_image: segment 4: paddr=000c9464 vaddr=40080000 size=1671ch ( 91932) load
I (425) esp_image: segment 5: paddr=000dfb88 vaddr=50000000 size=00010h (    16) load
I (437) boot: Loaded app from partition at offset 0x10000
I (437) boot: Disabling RNG early entropy source...
I (448) cpu_start: Pro cpu up.
I (448) cpu_start: Starting app cpu, entry point is 0x400813b8
0x400813b8: call_start_cpu1 at /HomeKit/esp-idf/components/esp_system/port/cpu_start.c:150

I (0) cpu_start: App cpu up.
I (463) cpu_start: Pro cpu start user code
I (463) cpu_start: cpu freq: 160000000
I (463) cpu_start: Application information:
I (467) cpu_start: Project name:     led
I (472) cpu_start: App version:      f8dcf29-dirty
I (477) cpu_start: Compile time:     Jan  5 2022 12:37:16
I (483) cpu_start: ELF file SHA256:  f84209b702d7d407...
I (489) cpu_start: ESP-IDF:          v4.3.2
I (494) heap_init: Initializing. RAM available for dynamic allocation:
I (501) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (507) heap_init: At 3FFBAC38 len 000253C8 (148 KiB): DRAM
I (514) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (520) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (526) heap_init: At 4009671C len 000098E4 (38 KiB): IRAM
I (534) spi_flash: detected chip: generic
I (537) spi_flash: flash io: dio
I (542) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (606) wifi:wifi driver task: 3ffbc66c, prio:23, stack:6656, core=0
I (606) system_api: Base MAC address is not set
I (606) system_api: read default base MAC address from EFUSE
I (616) wifi:wifi firmware version: eb52264
I (616) wifi:wifi certification version: v7.0
I (616) wifi:config NVS flash: enabled
I (616) wifi:config nano formating: disabled
I (626) wifi:Init data frame dynamic rx buffer num: 32
I (626) wifi:Init management frame dynamic rx buffer num: 32
I (636) wifi:Init management short buffer num: 32
I (636) wifi:Init dynamic tx buffer num: 32
I (646) wifi:Init static rx buffer size: 1600
I (646) wifi:Init static rx buffer num: 10
I (646) wifi:Init dynamic rx buffer num: 32

[MrCavemen]

I found the "issue".
In the server, the endpoint_params has a hard coded ids array of 25.
When having more than 25 characteristics, the characteristics beyond 25 are ignored.
This has a side effect causing none of the characteristics to properly update.

For my use case, I simply increased this to 100. (Causing 300 bytes of extra memory consumption)

To ensure a low memory profile for the esp8266, maybe this should be made dynamic?
We can store the id_count from homekit_server_on_url, store a pointer in the endpoint_params an realloc it's memory when needed.
Alternatively, it could be made user configurable from the menuconfig, but this seems to be less user-friendly.

[maximkulkin]

So, yes, I agree, that it can be made configurable with ESP32 devices getting larger values by default plus allowing overriding defaults if really needed. Will think about automating it (e.g. automatically configuring it to the number of characteristics present).