HiveMQ Edge is an MQTT gateway that enables interoperability between OT devices and IT systems. It translates diverse protocols into MQTT for streamlined communication and helps organize data into a unified namespace, making managing and streaming data across your infrastructure easier.
Home Page: http://hivemq.com
License: Apache License 2.0
![renovate[bot] avatar](https://avatars.githubusercontent.com/in/2740?v=4 "renovate[bot]")
The sidebar nav icons at the moment have not been discussed. We should ensure they have meaning and are distinct. Presently we have duplicates which seem to just have the meaning of "links".
I can image we want iconography to support the various entity/functional areas of the system.
The protocol adapters have links to documentation rendered in the web application. These should link to the wiki docs. In fact they link to the main website which results in a 404
For usage in the workspace an other components, we need to be able to extract the topics defined in every adapter, regardless of their internal structure
While a generic mapping could be delivered as part of the consolidation of the backend's adapter definition (see ?????), this is a job that in fact can be easily implemented in the frontend
The current definition of all known adapters is using a custom format attribute :
/// opc-ua
'mqtt-topic': {
type: 'string',
title: 'Destination MQTT topic',
description: 'The MQTT topic to publish to',
format: 'mqtt-topic',
},
/// modbus
destination: {
type: 'string',
title: 'Destination Topic',
description: 'The topic to publish data on',
format: 'mqtt-topic',
},
Want to be able to specify min and max fields for simulation.
Within the HiveMQ Edge Docker container, hostname host.docker.internal can be used to connect to an OPC-UA server running on the Docker host machine
OPC-UA protocol adapter errors with log message:
ERROR - Not able to connect and subscribe to OPC-UA server opc.tcp://host.docker.internal:49320
java.util.concurrent.CompletionException: java.net.UnknownHostException: {Docker Host Hostname}: No address associated with hostname
Caused by: java.net.UnknownHostException: {Docker Host Hostname}: No address associated with hostname
host.docker.internal as hostnameShow the last started column in a more human-readable for, e.g., "15 days ago", "1h ago"
The column only shows the unit "minutes", e.g., "1234 mins ago" which isn't very readable.
Create a protocol adapter and wait for an hour. The respective component sits in the frontend.
The change is suggested to be implemented it in the frontend since the backend is correctly returning the timestamp.
Method com.hivemq.util.Strings.convertBytes has unit tests.
Method com.hivemq.util.Strings.convertBytes has no unit tests.
No steps to reproduce required.
No steps to reproduce required.
No steps to reproduce required.
Add several improvements to com.hivemq.util.Strings.convertBytes:
rename to toHumanReadableFormat, I think that it's a more readable name
add unit tests
Question. Method FileUtils.byteCountToDisplaySize from the commons-io library (already used in the project) does almost the same thing, more details here, the exception is that FileUtils.byteCountToDisplaySize is less precise (sizes without additional decimal points, e.g. 4 GB). Should FileUtils.byteCountToDisplaySize be used instead?
Affected HiveMQ Edge version(s):
Used JVM version:
In the wiki chapter 'Installing HiveMQ Edge/Windows Systems/Manual installation', the first task ist to run ./gradlew clean hivemqZip but this is no valid gradle task. The actual distribution task should be ./gradlew clean hivemqEdgeZip.
Abstract the adapters further breaking out polling from normal and allowing fine grained status control.
We should allow bridges to be independently started and stopped via API and UI. A new action on the overflow menu to START | STOP | RESTART the connection.
The class com.hivemq.util.Files has unit tests.
The class com.hivemq.util.Files has no unit tests.
No reproduction steps required.
No reproduction steps required.
No reproduction steps required.
Unit tests should be added to com.hivemq.util.Files.
Also, during the addition of these tests, a change behavior was detected (and fixed) for the method call getFileExtension("/home/pa.th/somefile")
Accepting the URL
URL isn't accepted by the regular expression confirmed by https://regex101.com/r/Y1koJj/1
Creating an HTTP adapter with the URL: http://192.168.0.27/production.json?details=1
Probably the regular expression HTTP_URL_REGEX in HttpConstants isn't correct.
The custom schema generator should not generate duplicate title and description fields for wrapped types.
Title and description fields are duplicated on the field and item level leading to form display issues.
Over a number of weeks I have seen behaviour (seemingly after a CRUD event) where the browser suddenly logs out. There seems to be no obvious reason for this, as the JWT is only a few minutes old.
Today I captured the event in logs where a previously issued JWE was presented to the API and rejected (causing the logout event).
Please see the log attached where a JWT is issues at the start of the session, then on a subsequent operation a previous token is presented.
2023-08-18 16:54:37,739 [pool-29-thread-1] JwtAuthenticationProvider ERROR- jwt validation failed, reason JWT rejected due to invalid signature. Additional details: [[9] Invalid JWS Signature: JsonWebSignature{"kid":"00001","alg":"RS256"}->eyJraWQiOiIwMDAwMSIsImFsZyI6IlJTMjU2In0.eyJqdGkiOiJZSkgxaGJ4aFJvZ1VDOHpMZjJfeWdnIiwiaWF0IjoxNjkyMzczODk2LCJhdWQiOiJIaXZlTVEtRWRnZS1BcGkiLCJpc3MiOiJIaXZlTVEtRWRnZSIsImV4cCI6MTY5MjM3NTY5NiwibmJmIjoxNjkyMzczNzc2LCJzdWIiOiJhZG1pbiIsInJvbGVzIjpbImFkbWluIl19.CPwswyJtkeHvOYWTK9Y6DTcEAA-2QeMKfWB-c80bAZbKmLacDjlcekHJM7L66HI1qG4MH3urWljQa4G0zbNztGoMu9NVp6EIAc_UW4YNSx-ovaFTAZLPyfFYvWtSlDjN84A5CZ2FVeKBTiBaO7QsvQ47_ak5dl4CtLZ_yD2HG01GzvkN9Lhk6046P8cT_SO_Bmsij4F7R1RsZAxa1mBTHAS8eKvexwAMzwWsXsvxha5imHsd14aIX8Poe321R_gZkQnooTRduisYsVxyMVaJZu3GSAfYF2xzxZ9UkwzUM4TTn2JiexVPIDm02m4xqgGu3rxe_4Sosz82Hy--MzY_lA]
2023-08-18 16:54:41,707 [pool-29-thread-2] JwtAuthenticationProvider INFO- Generated JWE 'eyJraWQiOiIwMDAwMSIsImFsZyI6IlJTMjU2In0.eyJqdGkiOiIwZmdDX3gzcHJrdHVwbWRLTHhMVUpnIiwiaWF0IjoxNjkyMzc0MDgxLCJhdWQiOiJIaXZlTVEtRWRnZS1BcGkiLCJpc3MiOiJIaXZlTVEtRWRnZSIsImV4cCI6MTY5MjM3NTg4MSwibmJmIjoxNjkyMzczOTYxLCJzdWIiOiJhZG1pbiIsInJvbGVzIjpbImFkbWluIl19.AtdhBfZiLJxiFmZMtFjmQmUhwMaG31ZklIIOdNrrh94C3w4Pr7v-Rn-k0D7VdlkF-LyamZUUAGIr4JG8Xse9NovKX8vBwvSodTOKv-9JBF5PB4Q3Tj_1GHGSTKXwzz6X2W339y18r0kwQp_hBt_Tl9mSHA4reIoAUJuB4SXfYZvHCoIcbnMqVgdZKt2i_xeCWsjvuB8vlsJ7Dm8EIdDrgJLHIVQhIb4Sv4cx0Lk_umiPAa3Kj7Ufyfg2n7G8zbF_VNnoPQdqmdJd1hpn3AYbnx5HwaplFtv5qafCHC572214UwOKqPx8mSrkvSeBtg08qPmolryavsgGjh5E48G2_w' for principal ApiPrincipal{name='admin', roles=[admin]}
2023-08-18 16:54:50,796 [pool-29-thread-1] JwtAuthenticationProvider ERROR- jwt validation failed, reason JWT rejected due to invalid signature. Additional details: [[9] Invalid JWS Signature: JsonWebSignature{"kid":"00001","alg":"RS256"}->eyJraWQiOiIwMDAwMSIsImFsZyI6IlJTMjU2In0.eyJqdGkiOiJZSkgxaGJ4aFJvZ1VDOHpMZjJfeWdnIiwiaWF0IjoxNjkyMzczODk2LCJhdWQiOiJIaXZlTVEtRWRnZS1BcGkiLCJpc3MiOiJIaXZlTVEtRWRnZSIsImV4cCI6MTY5MjM3NTY5NiwibmJmIjoxNjkyMzczNzc2LCJzdWIiOiJhZG1pbiIsInJvbGVzIjpbImFkbWluIl19.CPwswyJtkeHvOYWTK9Y6DTcEAA-2QeMKfWB-c80bAZbKmLacDjlcekHJM7L66HI1qG4MH3urWljQa4G0zbNztGoMu9NVp6EIAc_UW4YNSx-ovaFTAZLPyfFYvWtSlDjN84A5CZ2FVeKBTiBaO7QsvQ47_ak5dl4CtLZ_yD2HG01GzvkN9Lhk6046P8cT_SO_Bmsij4F7R1RsZAxa1mBTHAS8eKvexwAMzwWsXsvxha5imHsd14aIX8Poe321R_gZkQnooTRduisYsVxyMVaJZu3GSAfYF2xzxZ9UkwzUM4TTn2JiexVPIDm02m4xqgGu3rxe_4Sosz82Hy--MzY_lA]
Can you put the helm charts or deployment yaml files for running hivemq-edge on Kubernetes? Kubernetes for edge computing is becoming the standard.
The gradle build should complete succesful and a zip should be the output
Build fails directly with message: Cannot add task 'hivemqEdgeZip' as a task with that name already exists.
Stacktrace:
Build file FOLDERNAME line: 91
Cannot add task 'hivemqEdgeZip' as a task with that name already exists.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
Checkout the project switch to tagge commit 2023.4 or stay on master. Run ./gradlew clean hivemqEdgeZip in console or execute task in Intellij gradle
Current discovery is an advanced adapter feature, which only some adapters support. In anticipation of UI and API calling these, we need to provide an indication as to wwhether the underlying type supports it. e.g. HTTP will never support discovery.
New method added to the type metadata, new abstraction of discoverValues in place for default implementations and strong gatekeeping around the API method.
User Story
As a HiveMQ Edge user I want to test my connection parameters before committing the configuration, to enable me to validate my parameters or understand what parameters might be incorrectly set
Value
Definition of Done
I think the ease of first use is a powerful feature, and we should be compelling people to try it. Given the designs are in place and visually work nicely, we should incorporate a small section on the front GH page which demonstrates the UI.
User Story
As a HiveMQ Edge user, I want to recognise that I'm editing a topic for the bridge and how that topic might relate to previous configuration
Value
Definition of Done
This ticket is part of an initiative consolidating the visualisation, selection and editing of topics across the Edge. See (#138, #139 and #140)
When more than one Edge instance is running, we should be able to visually see and inspect the other nodes, and connect them via a transparent MQTT bridge.
Phase 1
Add configuration which represents a node in the graph from XML
Add a UDP broadcast and response engine that allows dynamic population of the above
Add a real-time model which maintains a network graph of instances
Expose this model through the API
Add a new side bar item "Edge Connect" which when clicks shows edge instances in the network (inluding self).
Phase 2
Add the ability to CONNECT x2 instances of Edge together via a bidirectional wildcard subscription so messages are synchronised across the Edge instances
Phase 3
Add the ability to synchronise/replicate adapter configuration across the instances
Currently all logos are sourced and created in different styles. We should standardise the naming and the design of these. We should work with burkhard & team to come up with a common template which we can add to along with defining common dimensions.
Now we have introduced isolated states for runtime and connection status, it gives us far more control to be able to start / stop individual instances. Depending on the implementation of the adapter, some adapters will NOT try to reestablish a connection on an invalid set of details after a number of backoff retries (they will be transitioned from connection ERROR back to DISCONNECTED - but still running) where they will stay until they are edited. We should allow more control on these, but allowing a START | STOP | RESTART on each instance in the control actions.
We may also wish to consider the fundamental control/state cycle of each adapter. Suggest we detail some rheudamentary state transition diagrams for this work unit.
The font colour for the component should pass the WCAG 2.1 AA colour-contrast threshold test
The current (and default) implementation of the error message below a field uses a red colour that is at times too light for the white background. The contrast is not sufficient from an accessibility point-of-view and breaks the "best practice" guidelines of the WCAG tests
Check any field with a contextual error message, using either the Axe tool or the Cypress tests.
FormErrorMessage to adjust font colourThe UNS CTA is a button that reads "+ Setup a unified namespace". This works great for first use or when there is no UNS defined. We should detect is there is a UNS setup already and change the text of the button accordingly... when one is setup the button text should read:
"Modify unified namespace"
Some Modbus Server only allow reading coils.
In addition to reading holding registers, other modbus functions like reading coils discrete inputs and input registers should be possible.
We should expose and test an AB protocol adapter.
It should not be possible to add a whitespace prefix or suffix to the id field. If you manipulate the form this is possible.
Presently the adapter abstraction is too coarse and all extended types are bound to the polling type which isn't required.
The adapters should be updated to specify the runtime management out of the box using Start and stopInternal methods which manage the runtime state - on the base type.
We should then introduce a polling version which adds polling functionality and binds to polling configuration as standard by its generic typing.
Presently there are no run configurations created for the open-source project. We should create x2 run configurations to make it easy for developers to run in the development environment easily.
Among the issues to improve:
iddrawer and fieldsetThe react-jsonschema-form library is used to create a form out of the JSON Schema specification of an adapter. It introduced a secondary schema to provide information on how the form should be rendered, uiSchema .
See https://rjsf-team.github.io/react-jsonschema-form/docs/api-reference/uiSchema for a description of the schema.
At the moment, the uiSchema is hardcoded in the frontend and contains the combined information of most of the known adapters, by property name. This is not a sustainable situation, for two reasons:
The uiSchema should therefore be located within each adapters, along their configSchema.
The schema is a JSON object, whose attributes are described in the document above. If strongly typed, the description of the UiSchema type can be found here: https://github.com/rjsf-team/react-jsonschema-form/blob/6ffc2ceb1f24e2661d0acbc0c3e09e5e443e9f27/packages/utils/src/types.ts#L784
To the list of default options, we have added a new one to combine top-level properties into separate tabs:
{
"ui:tabs": [
{
"id": "coreFields",
"title": "title",
"properties": ["id", "port", "host", "uri", "url", "pollingIntervalMillis"]
},
{
"id": "subFields",
"title": "Subscription",
"properties": ["subscriptions"]
}
]
}
Update the adapter templates with the new abstractions.
Now we have the abstractions in place, we need to exposer and test the S7 protocol adapter.
An active HTTP adapter should not indicate connected when it is blatant that the server is not running.
Both the frontend and the backend accept a clearly fake URL. Upon creation, the active adapter is updated and the status switches to connected
Creating an HTTP adapter with the URL: http://fake
Other adapters like MODBUS or OPC-UA are shown as disconnected when the "URL" of the service is not a live system.
Note: there is also a question as to http://fake is a valid URL
Class com.hivemq.util.ObjectMemoryEstimation has unit tests.
Class com.hivemq.util.ObjectMemoryEstimation doesn't has unit tests.
No steps to reproduce required.
No steps to reproduce required.
No steps to reproduce required.
Add several improvements to the class com.hivemq.util.ObjectMemoryEstimation:
rename to MemoryEstimator (in my opinion it's a more readable name)
remove method calls that retrieve constant values, here is explained why it's better to use the constant value instead of having a function that returns a constant value
add unit tests
add Java doc
Affected HiveMQ Edge version(s): latest version (master)
Used JVM version: Java 11
User Story
As a HiveMQ Edge user, I want to recognise on the workspace the topics that I've created through the different elements (bridges and adapters).
As a HiveMQ Edge user, I want to be able to filter parts of the workspace out, using the interactive selector as a visualisation of the topic tree
Value
Definition of Done
This ticket is part of an initiative consolidating the visualisation, selection and editing of topics across the Edge. See (#138, #139 and #140)
When the UNS form is filled with a single character in each of the fields, I expect this to be a valid submission.
The UI encounters (what appears to be) an API side error and displays the error in a red toast.
Add single character values to your UNS
The editing of topics in the app is not consistent:
Bridge editor uses an auto-complete select (without any options added)Adapter editor uses a plain text input without any formattingWorkspace shows the topic with a specific (if minimalist) rendering
The discrepancy is partly due to the difficulty in customising the JSONSchema-driven form generation. With the recent mastery of the format option (see #110), it is now possible to customise the form at that level
We should convey a sense of consistency to the users, strongly emphasising that they are creating MQTT topics:
Bridge and Adapter editors should use the same UI widgetTopic input should use the auto-complete select widget, with the list of existing workspace-wide topics as optionstopic tree under constructionThe error message for the validation of a url field should tell humans what the error is about, not regex parser :-)
The standard error message for a pattern attribute of a JSONSchema string is literally transcribing the wronged regular expression. Which, in the case of a url`, can be long and unreadable, therefore unusable
Creating an HTTP adapter with an incorrect URL, like in #106
xxx is not validhttpxyz is not validThe user interface experience should render and be fully available when the device has no access to the internet.
The user interface hangs on the login page
Run HiveMQ Edge on a device with no internet connectivity
User Story
As a HiveMQ Edge user, I want to recognise that I'm editing a topic for an adapter and how that topic might relate to previous configuration
Value
Definition of Done
This ticket is part of an initiative consolidating the visualisation, selection and editing of topics across the Edge. See (#138, #139 and #140 )
UISchema associated with the format mqtt-topic, as extracted from their JSONSchema configuration
JSONSchemaForm: create a custom BaseInputTemplate to deal with mqtt-formatCreatableSelect: create a custom auto-completion selector for the topics
ReactQuery: refactor the extraction of topics from the data sources (bridges and adapters)
JSONSchemaForm: create a custom validation rule to validate topics (e.g. rejecting wildcards, ...)<hivemq xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="config.xsd">
<mqtt-listeners>
<tcp-listener>.....
<hivemq>
<mqtt-listeners>
<tcp-listener>
<port>1883</port>
ahh, makes sense: config.xsd is not there either. But is referenced by default config.xml so....maybe adding this and referencing in the examples xml makes more sense here.
Anyone ever thought about integrating or using Apache PLC4X for additional connectors?
Currently the section for the OPCUA adapter in the documentation is empty.
Add documentation into the section or, if not available, add text indicating this.
When the API detects a request time that sits within an expiry window that is 10% of the token length, it will include a header on all secured API requests with a new JWE.
The UI should detect the existence to the HTTP header X-Bearer-Token-Reissue and update its local cache with this new token.
Will the HiveMQ be available only in Java? Any plans for C/C++, Rust?
When connection to the backend cannot be secured because of network errors, the user should be notified appropriately and certain operations (like creating a new adapter) should be restricted
At the moment, most of the error handling has been focusing on receiving errors (4xx or 5xx) from the backend. But the cases where the requests cannot even be sent due to network issues have not been covered.
The frontend, being a Single-Page Application and using React Query caching facilities, continues to render relatively appropriately. Some of the icons are not properly loaded but the caches still provide all the information to populate the React pages.
It includes accessing and interacting with forms, even if it means sending data through requests that are not operational.
There is not a single feedback about the connection status to the user until a cache becomes "stale" is fetched again and the failure of X retries triggers the error
Access the frontend, navigate through the different pages (to make sure all the requests are accessed once and cached) then shut the backend down.
Alternatively, use the browser throttling capability to simulate "offline" behaviour.
Originally posted by @vanch3d in #95 (comment)
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.