Comments (20)
I did some checking for feasibility. One question pops to mind: if there is authentication in place, do you expect that to happen before or after the mangling? Let's say we're talking about messages on topic a/b/c
:
- In incoming publish: do you want the 'write' permission to be checked on
prefix/a/b/c
ora/b/c
? - In delivering a publish to a subscriber, same question: do you want that 'read' permission to be checked on
prefix/a/b/c
ora/b/c
?
There are some architectural difficulties if you would want the ACL check to happen with the prefix. Logically, I'd say without the prefix makes most sense.
from flashmq.
I can see why you would want it. I'd have to spend some time looking at how complex it would be to make, and what the performance implications are (I tend to avoid adding overly heavy features).
from flashmq.
I hear you. A "strip the first N components of the incoming / outgoing message" feature would work for most if not all users, if that's easier than a comparison; I didn't yet encounter any such link that needed more complex arrangements.
In any case, on the positive side, at least this feature has no performance impact if it's not used.
from flashmq.
This topic mapping would indeed be very appreciated.
@smurfix you indicate topic prefixing when sending, how did you accomplish that as that would already do most of the job in my case, but did not recognise how to do that from the conf file nor from the bridge code
from flashmq.
I currently accomplish topic prefixing with RabbitMQ, but I'd rather switch to something faster + more extensible (FlashMQ has this nice plugin architecture …) if at all possible.
from flashmq.
As long as you're consistent whether the checks are done on the paths as seen by the client or as seen by the server (= every other client that doesn't have mangling applied to it) I can write an ACL to match.
Thus, my answer is "whatever happens to be easier to implement".
from flashmq.
OK, then I'll give it a go. Prefix stripping on receive is easy. Adding a prefix on send is a bit more difficult, but if we can do it at the very last step, meaning after authentication on the original topic string, it's probably doable.
Do note that it will incur some extra overhead, because there are certain optimizations it can't do then.
Probably the next feature you want is multiple bridge connections with load balancing. It's on my list...
from flashmq.
I created a branch with a test version: https://github.com/halfgaar/FlashMQ/tree/PREVIEW-bridge-prefixes. If compiling is too hard, I can probably give you a .deb
package.
The bridge config sections supports a local_prefix
and remote_prefix
option. It should act the same as Mosquitto's, in terms of how it subscribes and what it adds and removes.
The prefix options are defined per bridge, not per pattern. There are some conceptual problems with having them per pattern. I'm not sure how Mosquitto does it...
The ACL checking turned out to be the following:
- Remote prefix is removed and local prefixes is added before ACL
write
checks. So, the ACLwrite
checking is done after the mangling. - The
read
ACL checks (delivery to subscribers) is done before mangling. So, the ACL is done on the path as it appears locally. Then, after ACL, the local prefix is removed and the remote prefix added.
from flashmq.
Actually, I was a bit premature. There are various issues with that branch still.
from flashmq.
Thanks. Looking forward to testing it once you've got the issues ironed out.
Building a .deb
is not a problem.
from flashmq.
OK, then I'll give it a go.
👍🏼
Probably the next feature you want is multiple bridge connections with load balancing. It's on my list...
Heh. Not really. My volume is high but not that high.
What I do want is redundancy, i.e. connect my MQTT servers in an arbitrary mesh and lose no messages (*) if any one of them hangs / fails / whatever. I'm currently using my own protocol on top of MQTT for this and I'd love to be able to bypass the whole thing.
(*) except exact duplicates of course; when a sensor tells me five times that it's 18° out there I don't care how many of those arrive – as long as the number is greater than zero.
from flashmq.
As an update; I was doing some prep work for this change, which kind of opened up can of worms of things that needed fixing. So, I'm working on a new release, coming in a few days I hope, and then I can continue with this.
from flashmq.
Thanks for the update. Not a problem.
from flashmq.
I think I'm close to giving you a .deb
to test. For what OS+version do you want it?
from flashmq.
Cool.
Standard Debian stable should be fine.
from flashmq.
You can download a test version for Bookwork here. It's a release build, so optimized and no symbols. At this point, I don't expect test scenarios I need debuggers for, just whether it behaves correctly. I can always make a debug build if required.
Install with dpkg -i
.
This comment still applies as to how it works.
The more scenarios tested, the better. Like bridges with persistent sessions (non-clean start and non-0 expiry), severed connections, queued messages, using aliases, etc.
from flashmq.
I'm eager to any test results, both positive and negative alike :)
from flashmq.
Yeah and I'm eager for some "free" time in which to test the setup I have in mind …
I'll try to shovel some hours free next week.
from flashmq.
Related Issues (20)
- `flashmq` its command-line options are not documented in its man page HOT 1
- Several protocol violations or bugs in FlashMQ HOT 3
- Another protocol violations or bugs in FlashMQ HOT 18
- Feature request: Allow skipping tests that require a network connection HOT 2
- Is there a maximum number of retained messages? HOT 14
- Build of 1.7.2 fails with gcc 13.1.1 HOT 1
- Support lua scripting HOT 6
- Check if retained message has the same contents don't resend it
- If you connect to the broker with a Last Will and Testament (LWT) and correct username, the LWT is published even if the password is wrong. HOT 2
- docker hub image for flashmq
- Huge amount of rentained messages and restarting HOT 1
- keepalive interval vs paho-mqtt HOT 9
- FlashMQ keeps disconnecting the same socket?
- Last Will not working correctly? HOT 4
- Install on Raspberry Pi HOT 21
- Add armhf and arm64 builds to the apt repo for Raspberry Pi OS HOT 10
- Deal with empty user names some way HOT 1
- FlashMQ randomly dropping incoming SSL connections HOT 6
- Running multiple testcases should not leak file descriptors HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flashmq.