Discussion around modular robots about hrim

[peterpolidoro]

Hi. Very interesting discussion so far, thank you!

I too have been building very similar devices for the last few years. I would like to figure out how to modify my devices to make them compatible with both of your approaches.

My approach so far may be more similar to @nicolas-rabault, where each of my modules can be used by itself, or modules can be combined for more complex functionality. So each of my modules contains some amount of power management, communication, actuation, and sensing. For example, I have one module that controls stepper motors, one that controls solenoid valves, one that plays audio, another that interfaces with encoders, another that controls LEDs, etc. I use these modules to build experimental rigs for scientists. Some rigs may only require one or two modules, some may require many. The modules can communicate with each other over UART, or with each other and other devices and users with digital trigger lines, or to a host computer over USB. They send JSON messages over the serial communication lines. UART is very convenient when only using two modules, but is inefficient for multiple devices. I want to switch to DDS ethernet and ROS messages, but I do still love the convenience of being able to send simple JSON strings written by hand or by host programs and being able to have two devices talk without needing a router.

I do think all of our devices will be more successful if we are all compatible and can collaborate rather than each of us doing our own thing. I open source all of the hardware and software that I create and I am willing to abandon my approach and adopt another if I can figure out how to make it work with all of the projects I am working on and maintaining.

[nicolas-rabault]

modules require certain capabilities that components don't have

I think we don't have this "component vs module" difference at Luos. I'm not sure to understand it properly. How a component is linked to the robot? Through a module?

its main type should be Communication

I agree it make sens. At Luos device type define what it can be used for, but not constraint specific messages type. We just combine device type as we want without priority or importance between them.
Managed messages type only depend of the module type (led, button, dynamixel, ...)

Would it be possible to get access to some of your technology to support you in the process of adapting HRIM?

Yes it could be, we plan to open-source a big part of our sources, but we have to clearly separate things in our code and have a good understanding of what it's mean to be HRIM compatible.

Can you elaborate a bit more on Robus and what HRIM-like implies to you?

Robus is the communication medium we created to transmit and receive data. Robus is a low level piece of software basically managing uart, timers, and GPIO to communicate. It's running on each of our L0 (module mother board). Robus is able to manage multiple virtual module slot and sort input and output messages to each of them.
Each virtual module slot have basic information such as ID, module type, alias, and multicast (topic) interest table.
Robus is able to determine the wiring plan of the robot (serial and star) to find witch virtual module is where on the network. IDs are distributed to virtual modules following its place into the network.
Robus is optimized for realtime communication so it's better for small latency than than big data bandwidth (L0 can manage up to 6M baud communications).

For us a virtual module represent something like a folder with software functionalities on it. A physical module (basically an L0) can manage multiple virtual modules. For example our Dynamixel modules are able to dynamically create virtual modules depending on the number of motors linked to it. Thank to that, in a software point of view there is a virtual module for each Dynamixel motor.

Each virtual module have a layer called Luos (yes I know this is not really original). This is the layer that I previously called "HRIM like". This layer manage messages datas and formatting depending on the virtual module type. Basically this layer deal with structures, data type conversion, and with Robus.

The virtual module manage the device (motor driver, IMU, ...)

The user-space is not available for now for our client because nobody ask about it for now (and the API is not ready). But the idea is to have a space to create something we call "reflex behaviors". A reflex behavior is a basic function managing ultra fast and secure behavior. For example a distance sensor can stop wheels to avoid collision. This kind of strategy is used on a lot of simple living being.

In the history of this project there is a short period of time were we were involved in Rust we create some robots for clients who still using this architecture on Rust and we open-source it :
https://github.com/pollen-robotics/robus
https://github.com/pollen-robotics/luos

[ibaiape]

First of all, our most sincere apologies for not responding sooner, it's been some busy weeks. Secondly, sorry for the incoming wall of text.

I'll try to shed some light on some of your questions:

How a component is linked to the robot? Through a module?

A module is a component. It's a component that contemplates certain guidelines (defined in the still in development ISO-22166 (International Organization for Standardization. Robotics — Modularity for service robots — Part 1: General requirements) to enable modularity. As we define it (in our glossary), a module is a component or assembly of components with a defined interface thus facilitating system design, integration, interoperability, and re-use.

Is the XML the format of data to transit between modules?

The XML models are a way to represent the communications each module could (and partly must) have, they are a "schema" based on which we'd generate platform specific implementations (e.g. our initial ROS 2 implementation), which would be what (I believe) you are talking about. They are our way to define those implementations in a platform-agnostic way, in a machine readable language. The models represent everything that a module "could do", for example a camera might not have a microphone and therefore it wouldn't make use of an audio topic, but said topic is represented on the camera's model.

Do you mean that all data contained in the "generic messages" bracket have to be sent for each messages?

Not with every message, but on their own topics. We follow ROS's (they are common) communication patterns of topics with publishers and subscribers, services and actions. Said "generic messages" define the information we deemed necessary to enable modularity and interoperability, like identification, information on communication status and capabilities, power... They'd get published depending on necessity, there's no need to publish identification at a fast pace as it shouldn't change, but power status can be critical depending on the circumstances. Same with the simulation related topics, makes no sense to publish those if nothing on the network can make use of it.

Purpose specific communication, like camera image or sensor readings (what I understand you talk about with "each message"), have their own topics.

In the common requirement mandatory is their all informations listed on .msg mandatory?
For example for the power section is the current_surplus always needed? veen if the module is a led or a button?

That's our (at least initial) vision, yeah. You do raise a valid point though. I'd say this issue comes from the difference in what we consider modules, unavoidable because of the difference between our focus markets. Some of the information we deem necessary might be excessive with modules on a different scale, and could simply make no sense in "custom" small modular systems. This information can have a lot of value on bigger systems, either for maintenance, diagnostics, or even power-management (talking about the example you mention).

It's something we'll take into account.

Is it allowed to store something like a link to download the model in another module or computer?

It's an idea that has come up before, we don't categorically oppose to this. It's something we have to review more in-depth though, as we can't assume every robotic system has access to the internet, and opening depending on what controller/system to the internet opens the door to a huge amount of security issues. A middle point would be filling that field with a pathname in the system itself where the models would be stored, so even if the modules themselves can't store the models they are still accessible without leaving the system. For us those topics' focus would mainly be to represent the system in a screen/touchpad as a user interface, and it's very possible neither said device nor its controller have a way to connect to internet. From where we stand this is a non-issue as we aren't as limited with storage, but it's good to know there could be a need for this.

OPC UA does something similar with component/software manuals, which are optional attributes containing an URL (or path inside the system itself) to download said manual. However, they do this with manuals, which are documents for human usage, while said models would be processed and used by other machines (even if they'd be "used" by users/operators).

I do have some questions of my own, more specifically on the transmitted data structures between modules, including their respective user-focused abstractions.

@nicolas-rabault I take from your posts and looking at your documentation (very visually appealing by the way!) the identification works "per execution" as an ID integer given by the Robus software that runs on each L0 when starting the system. Is this an identifier for each L0 itself? A local (per L0) identification for each virtual module? A unique virtual module identification for the whole system?

Also, what would the communication of these modules look like at a low level? I'm asking about the data structure. For example, and as I understand it so far, each (virtual?) module publishes its type, ID, alias, a multicast interest table (as in the nodes "subscribed" to that module's publications?) and type-specific status (taken from your docs' quick start page). A LED module exposes 3 integers for RGB, a button a boolean, etc. I understand there's a lot going on under the hood, I'm interested on the data necessary for the module's usage, be it directly used by the user or needed for its correct management (i.e. said multicast tables).

Is there any kind of meta-data? For example, is there a way for me as a user to discover what are the value ranges a potentiometer could give me other than physically trying it out?

@peterpolidoro would you be willing to elaborate on your communication patterns? Also, I understand your modularization is more focused on the quick development of specific solutions and their scalability than enabling and facilitating the modification of said solutions, am I close at all? Could I switch a module of the same type between two of your systems and have it work as-is?

Every piece of data passed between modules in your system is in JSON format? Have you had any issues on this for bigger data (i.e. images/audio)?

As an aside, I believe this kind of conversation is of great value. Having different viewpoints while sharing a vision on modularity is a great chance to get new feedback and ideas!

[kfriesth]

Hi All,
Just continuing this thread to share and continue this discussion together, I’ve been experimenting with robotics since 1983 with multi axis arms and over the past several years working with dual tracks with conventional robotic designs and modular robotics. My efforts with modular robotics initially was using a similar connection method using RS485 but have gravitated to ethercat connectivity.

Who am I ?

I'm CSO/Founder at Haute Fabrication and Saint Robotics (not launched): www.hautefabrication.com

At Haute Fabrication our focus is automated and roboticized mass manufacturing and Saint Robotics is targeting mass produced robots.

With some partners we will be releasing an open source solution once our hardware design and code base is stabilized.

What are the big challenges that modular robots will have to face?

Complete lack of interoperability, use of highly proprietary design using archaic programming techniques with very limited sensor feedback with high data latencies

About HRIM

We do like the HRIM initiative, we had not decided our path but HRIM looks promising. Converting from Machinekit to HRIM and direct messaging to/from modules seems logical but I am still wrapping my head around msg, srv and actions utilization while reading sensors in real time and such to allow this.

About us

Presently we use a ROS to Machinekit connection bridge for ROS connection to the HAL layers and the hardware. Our hardware design is in flux but we are working on settling on a final chipset usage. My robotic development started in 1983 and have evolved over time and morphed with our utilization of ROS which also involved from a converted plotter as a crude fdm printer and various robotic designs to where we are today.

[nicolas-rabault]

So, now that we know each other, I have few question about HRIM.

Is it possible to have a module belonging to multiple device type?
For example an USB module can be Communication and Power?

[vmayoral]

Hello @nicolas-rabault, very happy to see you jumping in. Our team would be delighted to explore the opportunity to collaborate. There're several organizations using HRIM already thereby I strongly believe that adopting it will benefit your business and adoption of technology.

As for your questions:

Is it possible to have a module belonging to multiple device type? For example an USB module can be Communication and Power?

Yes. There are different ways to approach such cases:

• Ideally, a USB component (modules require certain capabilities that components don't have, refer to https://acutronicrobotics.com/docs/modularity/hrim/introduction/glossary for language definitions) mainly aims to communicate and transfer data thereby its main type should be Communication. Power transfer is also, potentially, within its capabilities thereby to us, it's logical to extend the Communication model (which we haven't yet define and where contributions are welcome) with optional fields to reflect the power capabilities.
• Another path would be to use the HRIM meta-model, pick both types (their models) and assemble an implementation (through the MDE process) using both Power and Communication models. With this, your module would be interoperable with other modules corresponding to each one of the types. We wouldn't recommend you this path for this particular example (the USB one)
• One final way would be to use the composite type which we do not encourage for this particular setup because typically, composites are understood as modules (which comprehend a group of modules and components) that altogether bring up new capabilities that each independent module within isn't able to achieve on its own.

A few questions from our side:

• Would it be possible to get access to some of your technology to support you in the process of adapting HRIM?
• Can you elaborate a bit more on Robus and what HRIM-like implies to you?

[nicolas-rabault]

@peterpolidor : I hope my previous reply help you understanding how we manage our modules. We don't use JSON between modules, we only use it between the network and the computer.

We have a python lib allowing users to deal with it easily : https://www.youtube.com/watch?v=ula16zdZgDk

Some of our client use ROS on their system and they use something to convert JSON data into ROS node, I don't have a lot of new about this part of their project (no news, good news).

[nicolas-rabault]

I have a lot of things to say related to the lightness of HRIM. Indeed as mention on the first message we have limited resources on our side, and we can't develop something without thinking of it.
So, my next questions could be not relevant for you. If it is, feel free to let me know.

Is the XML the format of data to transit between modules?
We have face this question at the begining and we choose to have readable data format (JSON) only where data can be easily reeded (on the computer side). So between modules we have optimised bitfield like datas. Those data can be translated into JSON by communication modules so the user never deal with this bitfielded datas.
To debug Robus messages we use USB devices such as salae logic to get serial datas. This tool allow us to unbitfield data to have something understandable using python scripts.

I'm not sure to get the "generic messages" thing. Do you mean that all data contained in the "generic messages" bracket have to be sent for each messages?

In the common requirement mandatory is their all informations listed on .msg mandatory?
For example for the power section is the current_surplus always needed? veen if the module is a led or a button?

I'm agree with the fact that simulation should be mandatory but send the entire STL model could be difficult for us because we don't have a lot of memory.
In the documentation you say "define a way for the user to obtain the 3D model and a URDF" Is it allowed to store something like a link to download the model in another module or computer?
In our model we plan to re-create an approximation of URDF using the network topology (Robus can detect it) and by downloading the 3D model and simulation data from a module database.

[peterpolidoro]

Great questions!

I love the idea of being able to swap modules and have them be able to just work automatically. Is that why you define modules as certain types? How do you make it easily swappable? Am I correct in that you define a set of topics for each type (actuator or sensor say) and then every possible actuator or sensor would respond to that topic? So you have something like a get_position topic that rotary sensors or linear sensors or any other position sensor would use? Or are your topics more general so you have something like get_value so it also works with temperature sensors say? Seems like it would be challenging to think of every possible topic in advance that would work with every possible sensor. You would either have to have lots of types with lots of specific topics or very generic topics. Or am I wrong about how you make your modules swappable?

I do something a little different. I standardized the format of the function call, but not the function names. I am using JSON-RPC, but the strings you send between modules can look like bash commands or like simple lisp s-expressions, with a function strings and zero or more parameter values. Then when you interact with a device, you first ask the device what are its functions and their possible parameters. So one tiny driver on a host machine can work with any device. So for example, one little python driver or one Matlab driver asks each device what it can do, then automatically creates a class with those methods attached to it. This makes it easy for a user to interact with the devices without needing some central library or specific driver. One little client library on each device can allow it to talk to any other device. When modules control each other, however, you do need to know the function names at compile time, so that prevents it from being as swappable as yours are.

I love the idea of being able to connect hardware modules together like UNIX pipes connect small software programs together, at run time rather than compile time. A human can do this with my devices, but when my devices just talk to each other, right now they need to know in advance what devices they are controlling specifically.

[nicolas-rabault]

the identification works "per execution" as an ID integer given by the Robus software that runs on each L0 when starting the system. Is this an identifier for each L0 itself? A local (per L0) identification for each virtual module? A unique virtual module identification for the whole system?

ID are distributed during a specific sequence called "detection". Actually this detection sequence is initiated by the user on the entry point module (a communication module). The entry point module detect the position of each virtual module, give them ID depending on their position in the system, and create a routing table allowing to find ID from different type of data (principally aliases).
So ID's are for Virtual Modules, Robus doesn't care of which VM (Virtual Module) is on which L0.
For users there is different way to find which VM is on which L0. VM on the same L0 are seen like parallel connected modules, habitually modules are connected serially or star. Another way is to get the L0_serial number of a VM.
We can connect multiple Luos network on the same computer and give them names but we don't have any notion of sub networks on Robus for now. We have client facing this problem and they use Aliases to solve it instead of "distance_sensor" they call it "arm1/distance_sensor".

Also, what would the communication of these modules look like at a low level? I'm asking about the data structure.

This is a deep question, and I have multiple answer depending on the level (Robus or Luos)
On the Robus level, each messages have :
Preamble | Header | Data | ACK
Header contain

• Status
• Protocol revision
• command
• source ID
• target mode
• target

On Luos level we manage modules information. The minimal information for a module is Alias, module type, and ID. The module type is really important because it define the capabilities and parameters of the module.
We have a multicast working on Robus but we don't use it on Luos for now.

Is there any kind of meta-data? For example, is there a way for me as a user to discover what are the value ranges a potentiometer could give me other than physically trying it out?

We manage this kind of information as parameters in the module, you can ask the value to the VM and/or set it by yourself.

[ibaiape]

Answering to @peterpolidoro (might need to start sinthetizing these posts, I tend to gravitate towards novel-like responses)

I love the idea of being able to swap modules and have them be able to just work automatically.
Is that why you define modules as certain types?

Our module categorization is semantic, there's no set functions or behaviour that depends on this categorization, it depends on each module type. Meaning, the code itself wouldn't act differently depending on the module category, but depending on the modules themselves (a camera, a torque sensor...). Do keep in mind HRIM is focused on enabling, among other things, modularity and interoperability, but it ain't able to add those to a system by itself, it depends on the software+hardware implementation for that. Any node can take care of discovery, for example, taking advantage of HRIMs identification and self-description structures.

How do you make it easily swappable?

Again, this is largely taken care by the work of my colleagues both on software and hardware. Hardware like H-ROS connector A has a lot to do on with ease of swapping, same with the software part automating discovery on a live system. Sadly my knowledge on both is quite lacking and I don't feel knowledgeable enough on them to delve much on deeper questions, but I'm sure if you are interested on this front I can get someone to provide some answers.

Am I correct in that you define a set of topics for each type (actuator or sensor say) and then every possible actuator or sensor would respond to that topic?

Not exactly, the topics of each module depend on both it's type (a gripper, an arm) to define the maximum capabilities it could have (as in every communication a module of that type could ever have), and it's specific capabilities to know which ones it makes use of (as in a servo without temperature measuring capabilities simply can't publish a temperature topic). As a note, I use category to refer to what you call type, while I use type to define, let's say, sub-types inside those categories. For me, a specific camera's category is sensor, and it's type is a camera.

It does enable what you mention, at least in some way. If one of your modules waits for a temperature reading, for example, with HRIM you'd be able to discover any thermometer module (or you can specifically look for the temperature reading of a specific type, say every servo in your system) and use a single communication message definition, independently of the sensor's manufacturer. You'd use the same message to read any temperature reading coming from other modules. This enables you to hotswap a sensor for another from a different manufacturer, or even a module with different capabilities like a humidity sensor that measures temperature, if discovery is set up for that. Any one of our modules in the system could make use of that if needed (take into account said modules' capabilities would be extended through our SoM).

So you have something like a get_position topic that rotary sensors or linear sensors or any other position sensor would use?
Or are your topics more general so you have something like get_value so it also works with temperature sensors say?

I don't think I follow you. Anything considered necessary for the usage of a module (say, controlling the position of a servo) is present in every module of it's type and it's structure is always the same, independently of extra capabilities the module might have. As an example, we consider that not all servos have acceleration control, so we separated it (it'll probably end up as a service, it's a topic the module subscribes to for now) into it's own topic. This way you can use the same message definition to control any servo (namely hrim_actuator_servo_msgs/GoalRotaryServo.msg) on its node's goal topic and, on acceleration-control capable servos, you'd additionally make use of the acceleration topic to control it.

The great thing about this is if your software is developed enough you'd be able to automatically check for these capabilities (some declared in the communications themselves, others checking for their respective topics' existence). This does get out of hand quick though, and developing a system able to adapt to every possible combination is a bit ludicrous.

Do tell me if I didn't answer the question you had.

Seems like it would be challenging to think of every possible topic in advance that would work with every possible sensor.
You would either have to have lots of types with lots of specific topics or very generic topics.
Or am I wrong about how you make your modules swappable?

We focus on defining data structures that habilitate the usage of the respective module, be it sending orders or receiving information from it. We are conscious of the fact that we can either homogenize these data structures or contemplate every single possible capability a specific module could have, but not both. At least not without brutally upping the design (and possibly it's usage) difficulty.

Trying to compromise between both we end up with a mixture of generic topics for the common capabilities across all modules of that type and more specific ones to open up their specific capabilities to the system. Finding this balance is, I'd say, one of the biggest hurdles in the design of HRIM, and what we try to correct via contact with manufacturers and developers.

No one said it'd be easy.

I love the idea of being able to connect hardware modules together like UNIX pipes connect small software programs together, at run time rather than compile time. A human can do this with my devices, but when my devices just talk to each other, right now they need to know in advance what devices they are controlling specifically.

Funnily enough, today, a colleague had to reset a specific module (specifically the last joint, which also had a connected gripper) of one of our MARAs for some reason. He disconnected the module from the previous joint (exposed cabling on the links between joints while developing), reconnected it, and got working again without having to touch anything else (after a short pause for discovery and initial checks). All this without disconnecting the whole system, cutting power to the arm, having to execute anything manually or even stopping the script controlling the arm (it wasn't moving, of course). This process is comparable to the one we have in mind with the H-ROS connector I mentioned above, as you'd simply unscrew the module from it's link(s) and remove it, switch it if needed, then screw it back in without touching a single cable. After a short delay (powering on, starting threads...) the system is capable of discovering the newly connected module, makes the needed checks, and is ready to work all by its own.

It might be wrong for me to say (:P), but I find this very, very cool!

For an example of direct communication between modules: if you wanted to you could have a rangefinder module look for motors in the network and if its readings passed certain ranges (i.e. there's a wall just in front of the robot) send a stopping order to them. This is possible because of the homogenization of naming and data structures, as independently of what motors said rangefinder would send the order to it knows what to look for to identify motors in its network and knows how to send the desired order, independently of the manufacturer. As long as the middleware permits listing nodes and their topics you get a lot of freedom in this front. Live replacements or addition of modules without code changes.

Of course this works the same in controller-module traditional communications, being able to ignore the specifics of an arms servos for example. One of the arms we took to ROSCon Madrid had 3 servo joints and a gripper all from different manufacturers, we jokingly called it "Franky". You were able to control all joints making use of the same data structures (previously had to physically adapt them for our SoM and code their drivers of course).

[ibaiape]

Also for @peterpolidoro:

I am using JSON-RPC, but the strings you send between modules can look like bash commands or like simple lisp s-expressions, with a function strings and zero or more parameter values. Then when you interact with a device, you first ask the device what are its functions and their possible parameters. So one tiny driver on a host machine can work with any device. So for example, one little python driver or one Matlab driver asks each device what it can do, then automatically creates a class with those methods attached to it.

So, if I understood right, your communication works through methods+parameters that get processed on each module from method-like user calls passed as strings, as in to get a temperature reading from a thermometer module I'd send a string like "get_temperature" and it'd return said reading?

Also, as a user just getting access to one of your system, I can list the connected modules and, from each of them, I can get a list of their methods and needed parameters. As the communications are simplified, a single driver is able to get said method list and generate an interface for the user to interact with the module, as the user's calls and the module's methods would be close to a 1:1 relationship.

Like, if I want an RGB LED to glow red, as a user I'd call something like "led.set_rgb(255,0,0)" and the transmitted message would look something like "{identification, "set_rgb", 255, 0, 0}"?

I love the concept of having a single simple driver able to work with any of your modules. I imagine it being something like: I take a random computer I work with for a quick test, install said single driver (library, whatever), and I'm able to work with any of the modules on any of the systems I work with, independently of their function or usage. All while being able to have extremely specific capabilities on any of them without having to develop a "common ground".

What are your limitations on communication data size, for example? Anything you feel gets limited because of the communication pattern?

I'm curious as to the process you followed to end up at your current state. It's a curious approach to self-description.

[ibaiape]

I believe both of your approaches to user interfaces ("objectifying" the modules for a simpler way to make use of them) are quite similar, am I wrong about this?

[nicolas-rabault]

@ibaiape :

Trying to compromise between both we end up with a mixture of generic topics for the common capabilities across all modules of that type and more specific ones to open up their specific capabilities to the system. Finding this balance is, I'd say, one of the biggest hurdles in the design of HRIM, and what we try to correct via contact with manufacturers and developers.

We are facing the same difficulties. We create basics messages type for each physical unit for example we have a message for rotation_position in degree, this message can be used by potentiometers, encoder, servos, ...

@peterpolidoro:

I am using JSON-RPC, but the strings you send between modules can look like bash commands or like simple lisp s-expressions, with a function strings and zero or more parameter values. Then when you interact with a device, you first ask the device what are its functions and their possible parameters. So one tiny driver on a host machine can work with any device. So for example, one little python driver or one Matlab driver asks each device what it can do, then automatically creates a class with those methods attached to it.

In our side we make some command generic and some other not. Not generic command depend on the module type. For example the usage of a rotation position depend of the module type (encoder, or motor). This could be a little bit confusing because the same command don't do the same thing and can have different parameters. For now this is the way we limit the number of command.

I believe both of your approaches to user interfaces ("objectifying" the modules for a simpler way to make use of them) are quite similar, am I wrong about this?

I believe too, yes. One day a software guys after a talk say to me "You are doing object oriented hardware !"

[peterpolidoro]

Exactly, open-source object-oriented hardware! Each hardware object has a set of methods and stored property values.

My approach has evolved over the last ten years or so in an attempt to handle the hundreds of projects that I have had to create and maintain. So far I have not found a company that can sell all of the components I need for all of these projects, so I have either had to glue together parts from lots of different sources or create my own.

All of my hardware objects run a REPL, so any user can interact with it without knowing what it is in advance. For example, if you open a terminal, connect to a device, and send it a '?' it might respond like this:

{
  "id": "?",
  "result": {
    "device_id": {
      "name": "fly_bowl_controller",
      "form_factor": "5x3",
      "serial_number": 0
    },
    "api": {
      "firmware": [
        "FlyBowlController"
      ],
      "verbosity": "NAMES",
      "functions": [
        "setIrBacklightsAndFansOnAtPower",
        "setIrBacklightsAndFansOnAtIntensity",
        "setVisibleBacklightsAndIndicatorsOnAtPower",
        "setVisibleBacklightsAndIndicatorsOnAtIntensity",
        "addVisibleBacklightsPwm",
        "addExperimentStep",
        "getExperimentSteps",
        "getExperimentStatus"
      ],
      "parameters": [
        "pulse_delay",
        "pulse_period",
        "pulse_on_duration",
        "pulse_count",
        "sequence_off_duration",
        "sequence_count",
        "step_delay",
        "step_duration"
      ],
      "properties": [
        "flyBowlsEnabled"
      ],
      "callbacks": [
        "setIrBacklightsAndFansOn",
        "setIrBacklightsAndFansOff",
        "toggleIrBacklightsAndFans",
        "setVisibleBacklightsAndIndicatorsOn",
        "setVisibleBacklightsAndIndicatorsOff",
        "toggleVisibleBacklightsAndIndicators",
        "removeAllExperimentSteps",
        "runExperiment",
        "stopExperiment"
      ]
    }
  }
}

Functions take zero or more parameters and can return values, properties have values that are stored in non-volatile memory, and callbacks are functions that take no parameters and return no values, so they can be triggered by hardware interrupt lines.

If you want to know more about a function, you can send it 'addVisibleBacklightsPwm ?' and it responds like this:

{
  "id": "addVisibleBacklightsPwm",
  "result": {
    "name": "addVisibleBacklightsPwm",
    "firmware": "FlyBowlController",
    "parameters": [
      "intensity",
      "pulse_delay",
      "pulse_period",
      "pulse_on_duration",
      "pulse_count"
    ],
    "result_info": {
      "type": "long"
    }
  }
}

If you want to know more about the parameters it takes, you could ask it 'addVisibleBacklightsPwm ??' for verbose output and it responds:

{
  "id": "addVisibleBacklightsPwm",
  "result": {
    "name": "addVisibleBacklightsPwm",
    "firmware": "FlyBowlController",
    "parameters": [
      {
        "name": "intensity",
        "type": "double",
        "min": 0.000000,
        "max": 50.000000
      },
      {
        "name": "pulse_delay",
        "type": "long",
        "min": 0,
        "max": 2000000000,
        "units": "ms"
      },
      {
        "name": "pulse_period",
        "type": "long",
        "min": 2,
        "max": 2000000000,
        "units": "ms"
      },
      {
        "name": "pulse_on_duration",
        "type": "long",
        "min": 1,
        "max": 2000000000,
        "units": "ms"
      },
      {
        "name": "pulse_count",
        "type": "long",
        "min": 1,
        "max": 2000000000,
        "units": "ms"
      }
    ],
    "result_info": {
      "type": "long"
    }
  }
}

You can call the function by sending it this string 'addVisibleBacklightsPwm 25 1000 100 50 10', which is just shorthand for the JSON array version: '[addVisibleBacklightsPwm,25,1000,100,50,10]'.

You can get property values by sending 'flyBowlsEnabled getValue' and it responds:

{
  "id": "flyBowlsEnabled",
  "result": [
    true,
    true,
    true,
    true
  ]
}

You can set property values by sending 'flyBowlsEnabled setValue [false,true,false,true]'.

The little python driver asks the devices what its API is, then automatically fills out class methods on its response. So the python way of interacting with the same device looks like:

from modular_client import ModularClient
dev = ModularClient(timeout=0.1) # Automatically finds device if one available
dev.get_device_id()
{'name': 'fly_bowl_controller', 'form_factor': '5x3', 'serial_number': 0}
dev.set_properties_to_defaults(['ALL'])
dev.fly_bowls_enabled('getValue')
[True, True, True, True]
dev.fly_bowls_enabled('setValue',[True,False,True,False])
[True, False, True, False]
dev.ir_backlight_power_to_intensity_ratio('setValue',[5.99,5.59,5.41,5.57])
[5.99, 5.59, 5.41, 5.57]

Then the python version can automatically generate the ROS messages and topics it needs for a ROS system to interact with the python driver, which then creates the string commands and sends them over a serial port.

I do see the advantage of adding ethernet to each device and sending it ROS messages directly. I do like the convenience of being able to use a string REPL though.

[ibaiape]

That's a cool handmade solution for self-description!

The code example helps quite a lot in making a "mental map" of the usage of your modules. It also rises some questions:

properties have values that are stored in non-volatile memory

So if I have, let's say, half of my lights out, that same behavior is kept (if I choose to) after shutdown?

dev.set_properties_to_defaults(['ALL'])

Assuming 'ALL' is a shorthand for every parameter of that device. What are those default values? Are they hard-coded for each device or can they be changed like the parameter's value? I see nothing on the device's self-description relating to default values.

From that same method call I infer there's some common methods for all devices, is that so? I also don't see anything on the previous self-description specific to ir_backlight_power_to_intensity_ratio, is that also automatically generated? From what?

[peterpolidoro]

When I send a '?' to the device, it only responds with a subset of the possible methods. For example, when I send the fly_bowl_controller device the command:

getDeviceInfo

It responds:

{
  "id": "getDeviceInfo",
  "result": {
    "processor": "MK64FX512",
    "hardware": [
      {
        "name": "Teensy",
        "version": "3.5"
      },
      {
        "name": "modular_device_base",
        "part_number": 1000,
        "version": "1.1",
        "pins": [
          "bnc_a",
          "bnc_b",
          "btn_a",
          "led_green",
          "led_yellow",
          "btn_b"
        ]
      },
      {
        "name": "backlight_controller",
        "part_number": 1270,
        "version": "1.2"
      }
    ],
    "firmware": [
      {
        "name": "ModularServer",
        "version": "5.0.3"
      },
      {
        "name": "ModularDeviceBase",
        "version": "5.0.2"
      },
      {
        "name": "DigitalController",
        "version": "2.2.1"
      },
      {
        "name": "BacklightController",
        "version": "4.0.2"
      },
      {
        "name": "FlyBowlController",
        "version": "3.0.1"
      }
    ]
  }
}

This controller has 5 firmware sets, 'ModularServer', 'ModularDeviceBase', 'DigitalController', 'BacklightController', and 'FlyBowlController'. 'FlyBowlController' is the child class and the others are its parent classes. It inherits their methods. Sending a single '?' only shows the methods that belong to the child class. Sending two question marks '??' shows all methods. For example:

{
  "id": "??",
  "result": {
    "device_id": {
      "name": "fly_bowl_controller",
      "form_factor": "5x3",
      "serial_number": 0
    },
    "api": {
      "firmware": [
        "ALL"
      ],
      "verbosity": "NAMES",
      "functions": [
        "getDeviceId",
        "getDeviceInfo",
        "getApi",
        "getPropertyDefaultValues",
        "setPropertiesToDefaults",
        "getPropertyValues",
        "getPinInfo",
        "setPinMode",
        "getPinValue",
        "setPinValue",
        "forwardToAddress",
        "forwardToClient",
        "getClientInfo",
        "setTime",
        "getTime",
        "adjustTime",
        "now",
        "allEnabled",
        "setPowerWhenOn",
        "setPowersWhenOn",
        "setAllPowersWhenOn",
        "setAllPowersWhenOnToMax",
        "getPowersWhenOn",
        "getPowers",
        "setChannelOn",
        "setChannelOnAtPower",
        "setChannelOff",
        "setChannelsOn",
        "setChannelsOnAtPower",
        "setChannelsOff",
        "toggleChannel",
        "toggleChannels",
        "setAllChannelsOnAtPower",
        "setChannelOnAllOthersOff",
        "setChannelOffAllOthersOn",
        "setChannelsOnAllOthersOff",
        "setChannelsOffAllOthersOn",
        "channelIsOn",
        "getChannelsOn",
        "getChannelsOff",
        "getChannelCount",
        "addPwm",
        "startPwm",
        "addRecursivePwm",
        "startRecursivePwm",
        "stopPwm",
        "stopAllPwm",
        "getChannelsPwmIndexes",
        "getPwmInfo",
        "getPowerBounds",
        "setAllIrBacklightsOnAtPower",
        "setAllIrBacklightsOnAtIntensity",
        "setIrBacklightOn",
        "setIrBacklightOnAtPower",
        "setIrBacklightOnAtIntensity",
        "setIrBacklightOff",
        "toggleIrBacklight",
        "getIrBacklightPowersWhenOn",
        "getIrBacklightIntensitiesWhenOn",
        "getIrBacklightPowers",
        "getIrBacklightIntensities",
        "getIrBacklightPowerBounds",
        "getIrBacklightIntensityBounds",
        "irBacklightPowerToIntensities",
        "irBacklightIntensityToPowers",
        "setAllVisibleBacklightsOnAtPower",
        "setAllVisibleBacklightsOnAtIntensity",
        "setVisibleBacklightOn",
        "setVisibleBacklightOnAtPower",
        "setVisibleBacklightOnAtIntensity",
        "setVisibleBacklightOff",
        "toggleVisibleBacklight",
        "getVisibleBacklightPowersWhenOn",
        "getVisibleBacklightIntensitiesWhenOn",
        "getVisibleBacklightPowers",
        "getVisibleBacklightIntensities",
        "getVisibleBacklightPowerBounds",
        "getVisibleBacklightIntensityBounds",
        "visibleBacklightPowerToIntensities",
        "visibleBacklightIntensityToPowers",
        "setAllHighVoltagesOnAtPower",
        "setHighVoltageOn",
        "setHighVoltageOnAtPower",
        "setHighVoltageOff",
        "toggleHighVoltage",
        "getHighVoltagePowersWhenOn",
        "getHighVoltagePowers",
        "getHighVoltagePowerBounds",
        "setAllLowVoltagesOnAtPower",
        "setLowVoltageOn",
        "setLowVoltageOnAtPower",
        "setLowVoltageOff",
        "toggleLowVoltage",
        "getLowVoltagePowersWhenOn",
        "getLowVoltagePowers",
        "getLowVoltagePowerBounds",
        "setIrBacklightsAndFansOnAtPower",
        "setIrBacklightsAndFansOnAtIntensity",
        "setVisibleBacklightsAndIndicatorsOnAtPower",
        "setVisibleBacklightsAndIndicatorsOnAtIntensity",
        "addVisibleBacklightsPwm",
        "addExperimentStep",
        "getExperimentSteps",
        "getExperimentStatus"
      ],
      "parameters": [
        "firmware",
        "verbosity",
        "pin_name",
        "pin_mode",
        "pin_value",
        "address",
        "request",
        "client",
        "epoch_time",
        "adjust_time",
        "channel",
        "channels",
        "power",
        "powers",
        "delay",
        "period",
        "on_duration",
        "count",
        "pwm_index",
        "periods",
        "on_durations",
        "intensity",
        "ir_backlight",
        "visible_backlight",
        "high_voltage",
        "low_voltage",
        "pulse_delay",
        "pulse_period",
        "pulse_on_duration",
        "pulse_count",
        "sequence_off_duration",
        "sequence_count",
        "step_delay",
        "step_duration"
      ],
      "properties": [
        "serialNumber",
        "clientsEnabled",
        "timeZoneOffset",
        "channelCount",
        "powerMax",
        "irBacklightPowerToIntensityRatio",
        "irBacklightIntensityMax",
        "visibleBacklightPowerToIntensityRatio",
        "visibleBacklightIntensityMax",
        "highVoltagePowerMax",
        "lowVoltagePowerMax",
        "irBacklightSwitchingFrequencyMax",
        "visibleBacklightSwitchingFrequencyMax",
        "highVoltageSwitchingFrequencyMax",
        "lowVoltageSwitchingFrequencyMax",
        "flyBowlsEnabled"
      ],
      "callbacks": [
        "reset",
        "resetClients",
        "resetAll",
        "enableAll",
        "disableAll",
        "toggleAllChannels",
        "setAllChannelsOn",
        "setAllChannelsOff",
        "setAllIrBacklightsOn",
        "setAllIrBacklightsOff",
        "toggleAllIrBacklights",
        "setAllVisibleBacklightsOn",
        "setAllVisibleBacklightsOff",
        "toggleAllVisibleBacklights",
        "setAllHighVoltagesOn",
        "setAllHighVoltagesOff",
        "toggleAllHighVoltages",
        "setAllLowVoltagesOn",
        "setAllLowVoltagesOff",
        "toggleAllLowVoltages",
        "setIrBacklightsAndFansOn",
        "setIrBacklightsAndFansOff",
        "toggleIrBacklightsAndFans",
        "setVisibleBacklightsAndIndicatorsOn",
        "setVisibleBacklightsAndIndicatorsOff",
        "toggleVisibleBacklightsAndIndicators",
        "removeAllExperimentSteps",
        "runExperiment",
        "stopExperiment"
      ]
    }
  }
}

[peterpolidoro]

When I send:

setPropertiesToDefaults [ALL]

That means set properties to their default values in all of the firmware sets. I could also send:

setPropertiesToDefaults [FlyBowlController,BacklightController]

Which would only set the properties in the 'FlyBowlController' and 'BacklightController' firmware sets to their default values, while leaving all of the other property values unchanged.

The property default values are hardcoded at compile time, but their values can be changed by a user or by other programs or devices and those values are preserved when the power is cycled. Properties belong to one firmware set and have their own set of functions. For example, if I send:

irBacklightPowerToIntensityRatio ?

It responds:

{
  "id": "irBacklightPowerToIntensityRatio",
  "result": {
    "name": "irBacklightPowerToIntensityRatio",
    "firmware": "BacklightController",
    "type": "array",
    "array_element_type": "double",
    "array_element_min": 1.000000,
    "array_element_max": 50.000000,
    "array_length": 4,
    "array_length_default": 4,
    "array_length_min": 0,
    "array_length_max": 4,
    "value": [
      5.600000,
      5.600000,
      5.600000,
      5.600000
    ],
    "default_value": [
      5.600000,
      5.600000,
      5.600000,
      5.600000
    ],
    "functions": [
      "getValue",
      "setValue",
      "getDefaultValue",
      "setValueToDefault",
      "getElementValue",
      "setElementValue",
      "getDefaultElementValue",
      "setElementValueToDefault",
      "setAllElementValues",
      "getArrayLength",
      "setArrayLength"
    ],
    "parameters": [
      "value",
      "element_index",
      "element_value",
      "array_length"
    ]
  }
}

'irBacklightPowerToIntensityRatio' belongs to the 'BacklightController' firmware set. You can see its default value, current value, and its available functions and parameters. For example you can send it:

irBacklightPowerToIntensityRatio setAllElementValues 7.7

It will respond:

{
  "id": "irBacklightPowerToIntensityRatio",
  "result": [
    7.700000,
    7.700000,
    7.700000,
    7.700000
  ]
}

Saying that all element values are now 7.7.

To get all of the current property values, I can send:

getPropertyValues [ALL]

It responds:

{
  "id": "getPropertyValues",
  "result": {
    "serialNumber": 0,
    "clientsEnabled": [],
    "timeZoneOffset": -4,
    "channelCount": 16,
    "powerMax": [
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000,
      100.000000
    ],
    "irBacklightPowerToIntensityRatio": [
      7.700000,
      7.700000,
      7.700000,
      7.700000
    ],
    "irBacklightIntensityMax": [
      12.987013,
      12.987013,
      12.987013,
      12.987013
    ],
    "visibleBacklightPowerToIntensityRatio": [
      14.800000,
      14.800000,
      14.800000,
      14.800000
    ],
    "visibleBacklightIntensityMax": [
      6.756757,
      6.756757,
      6.756757,
      6.756757
    ],
    "highVoltagePowerMax": [
      100.000000,
      100.000000,
      100.000000,
      100.000000
    ],
    "lowVoltagePowerMax": [
      100.000000,
      100.000000,
      100.000000,
      100.000000
    ],
    "irBacklightSwitchingFrequencyMax": [
      120000,
      120000,
      120000,
      120000
    ],
    "visibleBacklightSwitchingFrequencyMax": [
      120000,
      120000,
      120000,
      120000
    ],
    "highVoltageSwitchingFrequencyMax": [
      12000,
      12000,
      12000,
      12000
    ],
    "lowVoltageSwitchingFrequencyMax": [
      120000,
      120000,
      120000,
      120000
    ],
    "flyBowlsEnabled": [
      true,
      true,
      true,
      true
    ]
  }
}

[peterpolidoro]

I am thinking of redesigning all of my hardware so that each device contains an H-ROS SoM. Are these available for purchase yet? Do you know roughly how much they will cost in quantities of 10-100? Is there any documentation on connecting sets of them together into networks? Do you need a special router? Thanks!

[igorrecioh]

Hi @peterpolidoro!

Thanks for your interest in our H-ROS SoM! My colleagues from the sales team will take care of your request via e-mail.