Under the "Extended image configuration" section of the readme.md it suggests that you can include your own Telegraf config file by using the dockerfile.template option. Currently this appears to not work due to an issue with how entry.sh generates the final exec command.

Per the readme.md file, using this in the template file:

FROM balenablocks/connector:%%BALENA_MACHINE_NAME%%
COPY customTelegraf.conf .
CMD ["--config customTelegraf.conf"]

generates the following error in the container:

<...snip...>
21.03.22 14:22:41 (-0400)
21.03.22 14:22:41 (-0400) balenaBlocks connector version: 1.1.2
21.03.22 14:22:41 (-0400) Changing hostname to c5ab484
21.03.22 14:22:41 (-0400) flag provided but not defined: -config ./customTelegraf.conf
21.03.22 14:22:41 (-0400) Telegraf, The plugin-driven server agent for collecting and reporting metrics.
21.03.22 14:22:41 (-0400)
21.03.22 14:22:41 (-0400) Usage:
<...snip...>

note the missing dash in front of the config option, should be "--config ./customTelegraf.conf"

Changing the CMD line to: CMD [" --config customTelegraf.conf"] (note the space between the double quote and the first dash) generates a different error:

<...snip...>
21.03.22 14:27:38 (-0400)
21.03.22 14:27:38 (-0400) balenaBlocks connector version: 1.1.2
21.03.22 14:27:38 (-0400) Changing hostname to c5ab484
21.03.22 14:27:38 (-0400) /app/entry.sh: line 27: /app/ --config ./customTelegraf.conf: No such file or directory
21.03.22 14:27:39 (-0400) Service exited 'connector sha256:12372dfc2677cb971debefb074f19408c5ed77b7d10d65cce930745220980e99'
<...snip...>

The option flag is now correct but the command to run appears to be "/app/" and not "./telegraf". I suspect there is something new with how this bit of code below from entry.sh is executed which I do not understand:

<...snip...>

If command starts with an option, prepend telegraf to it

if [[ "${1#-}" != "$1" ]]; then
set -- ./telegraf "$@"
fi

exec "$@"
<...snip...>

Use case: I want to pull two different types of data from two completely different endpoints with different authorization.

This was working correctly until yesterday when it suddenly started getting the following error. As far as I know, nothing had changed on the device. It might have lost power at some point, so potentially things got corrupt but this failure seems like maybe something changed on the balena API?

22.11.22 18:43:28 (+0100)  connector  Traceback (most recent call last):
22.11.22 18:43:28 (+0100)  connector    File "/app/./autowire.py", line 96, in <module>
22.11.22 18:43:28 (+0100)  connector      config = autowire.GetConfig()
22.11.22 18:43:28 (+0100)  connector    File "/app/./autowire.py", line 62, in GetConfig
22.11.22 18:43:28 (+0100)  connector      services = self.GetServices()
22.11.22 18:43:28 (+0100)  connector    File "/app/./autowire.py", line 33, in GetServices
22.11.22 18:43:28 (+0100)  connector      device = self.balena.models.device.get_with_service_details(device_id, True)
22.11.22 18:43:28 (+0100)  connector    File "/root/.local/lib/python3.9/site-packages/balena/models/device.py", line 309, in get_with_service_details
22.11.22 18:43:28 (+0100)  connector      raw_data = self.base_request.request(
22.11.22 18:43:28 (+0100)  connector    File "/root/.local/lib/python3.9/site-packages/balena/base_request.py", line 197, in request
22.11.22 18:43:28 (+0100)  connector      raise exceptions.RequestError(response._content)
22.11.22 18:43:28 (+0100)  connector  balena.exceptions.RequestError: b'Unauthorized'

I started up an app with transport, influx and grafana, which couldn't autowire any inputs (because there are none). Should we fall back to enabling something by default?

https://github.com/balena-io-playground/transport-primitive/blob/388046ed0612c1cbd718e4a6bca24ad30647130f/plugins/externalHttpListener.py#L6

Should we assume that if a user sets the name for the external HTTP listener service that they want it enabled?

https://github.com/balena-io-playground/transport-primitive/blob/388046ed0612c1cbd718e4a6bca24ad30647130f/plugins/externalHttpListener.py#L23

Instead of enabling or disabling json_query = "payload_fields", allow the value set in the env var to pass through to this query to allow the user to specify their own value for json_query.

10.02.21 10:52:17 (+0000) connector 2021-02-10T10:52:17Z W! [inputs.mqtt_consumer] Server "mqtt:1883" should be updated to use scheme://host:port format
10.02.21 10:52:17 (+0000) connector 2021-02-10T10:52:17Z I! [inputs.mqtt_consumer] Connected [mqtt:1883]

I'm running a device with the sensor service from balenaSense, I've changed the port to 7575 in the code, tried EXPOSE 7575 in the Dockerfile, and tried setting the port in the compose file. The service is accessible on port 7575 externally (which I don't actually want in this case), but it isn't automatically detected by transport. Could this be because my device is in local mode?

Currently you can provide a custom telegraf.conf file by extending the block image but that disables autowire feature. Consider allowing both behaviours simultaneously.

I tried to run the internetspeedtest fleet on my NUC and got the error

Error: no matching manifest for linux/amd64 in the manifest list entries

The connector service references this block's pre-built images, so it seems they haven't been built for all architectures
https://github.com/balenalabs-incubator/internetspeedtest/blob/d8e94b670971f68919962de19cf24943eae4f942/docker-compose.yml#L17

Hello! This issue is about your example configuration for a mqtt client.

The code you have states that you should be able to connect via the localhost keyword:

client = mqtt.Client("1")
client.connect("localhost")

while(True):
value = GetReading() # code omitted for brevity
client.publish("sensors",json.dumps(value))
time.sleep(5)

However, when I try this, I get the following error:

14.06.21 19:15:52 (-0700) co2 Traceback (most recent call last):
14.06.21 19:15:52 (-0700) co2 File "/usr/src/co2.py", line 85, in
14.06.21 19:15:52 (-0700) co2 mqtt_client.connect("localhost", 1883, 60)
14.06.21 19:15:52 (-0700) co2 File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 941, in connect
14.06.21 19:15:52 (-0700) co2 return self.reconnect()
14.06.21 19:15:52 (-0700) co2 File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1075, in reconnect
14.06.21 19:15:52 (-0700) co2 sock = self._create_socket_connection()
14.06.21 19:15:52 (-0700) co2 File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3546, in _create_socket_connection
14.06.21 19:15:52 (-0700) co2 return socket.create_connection(addr, source_address=source, timeout=self._keepalive)
14.06.21 19:15:52 (-0700) co2 File "/usr/local/lib/python3.9/socket.py", line 843, in create_connection
14.06.21 19:15:52 (-0700) co2 raise err
14.06.21 19:15:52 (-0700) co2 File "/usr/local/lib/python3.9/socket.py", line 831, in create_connection
14.06.21 19:15:52 (-0700) co2 sock.connect(sa)
14.06.21 19:15:52 (-0700) co2 OSError: [Errno 99] Cannot assign requested address

If the transport primitive starts before InfluxDB the database creation times out and is not retried.

This applies at the least to HTTP pull and push, unsure about other inputs.

If we add something like json_string_fields = ['string_*', 'boolean_*'] to the telegraf configuration it means fields in the received JSON with these prefixes will be allowed to pass to the database, without this, strings and boolean values are dropped indiscriminately.

The published Readme refers to Browser on both the BalenaHub published build as well as the source.

The telegraf Azure IOT Hub output plugin is await further development, and may be some time before it's usable. Develop a way to get data to Azure via IOT Hub, with no effort from the Edge developer.

The block is currently a black box after generating the config. If, for example, you push data to MQTT but it's not hitting your data sink - there's no way of knowing why. Implement debug logging of some sort.

It looks like python is not reading packages under /root/.local. I get this error when running autowire

root@cceab6e:~# balena run --rm -ti --privileged balenablocks/connector sh
/app # python3 autowire.py
Traceback (most recent call last):
  File "/app/autowire.py", line 6, in <module>
    from pluginbase import PluginBase
ModuleNotFoundError: No module named 'pluginbase'

This seems to work on pi 3

Currently, when using the External HTTP Listener plugin, data posted to this endpoint from multiple sources ends up in the same place within InfluxDB. This works if you have two different data sources, but if you had two sources the same (two identical sensors in different locations, for example), that post the same data structure, it won't be possible to differentiate between the two.

When the block detects an input (sensor, HTTP, MQTT...etc) it should log the input. The user should be very clear about what the block as auto-wired.

For some HTTP API endpoints the client will be required to authenticate, we should add the ability to specify this header. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization

When using livepush or if the fleet has no releases yet the autowire will fail (https://github.com/balenablocks/connector/blob/9e40e50439db37053b79ac0f0187f18aadbfb61e/autowire.py#L16-L26)

Currently when autowire finds a MQTT server with the name "mqtt" it connects as a data input source using both "sensors/#" and "balena/#" as topics. Consider allowing the user to add/change/delete topics w/o having to disable autowire and using a custom telegraf.conf file.