Experiences, processes and thoughts about exploring Sphero's JS SDK and creating a React/Electron application to control a Sphero bot.
As a part of getting to know Sphero as a company, I borrowed Sphero SPRK and BB-8 Battle Worn bots so that I could play with Sphero's Javascript SDK and experience what it was like to be a hobby-developer who wanted to dive a bit deeper into the capabilities of their Sphero robot.
Because I wanted to understand 1) the basic functionality that is the "building blocks" of controlling a Sphero bot and 2) what had already been done so that what I was doing wasn't "reinventing the wheel", I began by looking through Sphero's own educational docs, as well as developer sites, like StackOverflow. The first thing that I noticed is that most people who were using JS to control their bots had been doing so more frequently a few years ago. There were many blog posts about people receiving their new bots and how they chose to begin their exploration into what they were capable of. Similarly, I found Sphero's sphero.js repo on GitHub, which gives developers a starting point and contains examples of how you might use Javascript to control your bot, but also was last updated 3 years ago. None of this is, necessarily, an issue, but it meant that I may be relatively on my own for this adventure.
As I jumped in to my attempts at controlling the bots at my disposal, I forked Sphero's sphero.js repo to my own GitHub, into a sphero-v3 repo. The required dependencies (serialport & noble) were well documented, but I did spend a good bit of time (Google searches and digging through StackOverflow) arriving at the conclusion that I needed to use sandeepmistry's pull request #26 for my xpc-connection package in order to get everything to play nicely together (as is expounded upon in the README for my sphero-v3 repo). I also had a much easier time connecting to the SPRK (via Bluetooth Classic) than the BB-8 (via BLE), so I opted to start by working solely with the SPRK as I developed my preliminary understanding of how this was all going to go down.
I used the arrow key example from the sphero.js repo as a base for what I wanted to eventually be able to do with the bots and added a printout of various pieces of battery information so that I knew when my bot needed some rest. The times when I had the SPRK on the charger were usually spent digging deeper into tips and tricks for how to connect the BB-8 to macOS.
Some of the ideas that I had for what to have the bots do, based on recognizing that I could change the color of the lights and instruct the bot to go in any direction at a specific speed for a set duration, were to:
- Have the bot spell an input word or short phrase.
- Have the bot play a hot/cold game where the light colors became warmer as the bot was controlled closer to a set location, and cooler as the user steered it away.
As I did more research on the accuracy of the geolocation sensors in the bots at my disposal, it seemed that having the bot follow a pre-set pattern (such as creating a pattern for each letter of the alphabet that could be used to spell out a word/phrase that the user input) would be a much better suited task.
As I set out to create letter patterns (or even just a series of movements off of one command), I began by testing out the roll function. This would be integral to my task, as it was the basis of what I wanted the bot to do. I figured that, especially if I wanted the text to be readable, I probably wanted to decrease the speed at which the bot would roll AND each letter segment would require the bot rolling for a different duration. These durations would need to be played with, as well, so that the letters were of a reasonable size AND so that each letter could be the same size. It also seemed that it would be beneficial to use the capital versions of the letters for simplicity and legibility.
My plan was to create a function for each letter's pattern that mapped to a switch statement called within a loop that moved through the input word/phrase (I would map the user-input string to an array where each character had its own position in the array and then I would loop through the array to create each character).
As I began to play with the roll function, I found that it didn't have a particular affinity for responding to the speed or duration I input. I quickly learned that it was binding orb.roll to roll (for simplicity) that set a speed that could not be overwritten that was contributing to part of my issue. I attempted to remove that simplification and simply use orb.roll in locations where there had previously just been roll, but that seemed to open up a new can of worms in regard to speed and direction, so I opted to accept my single speed fate (which would be applied no matter if I was spelling something or just controlling the bot using the arrow keys).
Another predicament I encountered was with the delay function, or (potentially) lack thereof... I noticed that this function was utilized both in the educational documentation, as well as in the examples in the sphero.js repo, but using both delay and orb.delay caused console errors. I tried using both await before each action/delay (as in the educational docs) and delay().then() (as in the examples in Sphero's sphero.js repo), but neither seemed to be quite the accurate implementation of the delay function, if it does still exist.
After finding that there wasn't an obvious implementation of the built-in function to cause a pause in the code before another action was taken, I tried playing with normal JS functions, like setTimeout and setInterval (I thought setInterval might be a good one for creating an equilateral shape, where I could divide the duration/angle by the number of sides a user input), but was unsuccessful with these, as well (as were others, from my Google explorations).
Upon finding that there were potentially some blockers that required a bit more OG knowledge to break through, I figured I would do another deep dive into roll; this time, into to bind and the available parameters there. The parameters for the binded roll seemed to be in a slightly different order than the roll function itself. The first was, understandably so, binding the function to the orb. The next (second) parameter was for the speed (which was what I had found earlier, as well). The third (new!) parameter was for the angle.
I played a bit with changing the angle and speed to test if there was a relationship between them, but they seemed to operate independently (which is what I would expect). The one caveat to the angle parameter is that, when it is in use, only two of the four arrow keys will cause the orb to roll. I played around to see if, maybe, the keys that most aligned with the input angle would be those used (ie up and down would work for angles closer to 0 or 180/ left and right would work for angles closer to 90 and 270/ up and left would work on one diagonal/etc), but this did not seem to be the case; I had times where only the right and up keys worked (despite changing the angle) and other times where only the left and right keys worked.
Another experiment I did was to add a 4th parameter (duration? nope....). There didn't seem to be any rhyme or reason to this parameter. Only numbers 1-99 worked in this place and having a number here seemed to cause the angle parameter to be entirely disregarded, so I began entering empty single quotes for the third parameter, to reduce the number of variables in the equation. If the fourth parameter was between 1 and 99, all arrow keys caused the bot to move, but only at about a 0 degree angle. The speed parameter was, however, still functional. Using a number over 99 caused the bot not to move at all.
The BB-8 has been my unicorn. As mentioned above, connecting to it (via BLE) proved to be a bit of a challenge and was what I occupied my time with when the SPRK was charging (and about 50% of the time in the beginning, before I decided it would be wise to get at least one of the bots performing tasks). Connecting to the SPRK (and other regular Bluetooth bots) only required scouring your /dev directory to find its port. Connecting to the BB-8 (and other BLE bots) required you to use a noble script to obtain the UUID of the bot. Unfortunately, this script ran indefinitely for me without returning the desired information (and others on the interwebs had encountered similar issues).
I tried searching for others' tips and tricks and did find a few test scripts that allowed me to confirm that there was nothing wrong with the bot; it was likely my noble implementation or my drivers (though I have no reason to believe that it was my drivers, as there is nothing unique about them). I put far too much commenting in the code to see if I could find where the script was getting stuck (which was much more effective for the smaller test script than for the included noble script). I looked for other tools (like cylon, which others had had success with in previous years, however I wasn't able to find an implementation that allowed me to retrieve the desired information (and cylon, in particular, seemed to cause issues with connecting to the SPRK (coincidence?)).
I downloaded the BB-8 mobile application to see if the UUID could be retrieved from it, but was only able to find the MAC address. I tried to see if there was any helpful correlation between the UUID and the MAC address, but soon found that there wasn't a way to obtain a UUID, given a MAC address, especially given the different natures of the two IDs (Bluetooth vs Wifi).
After spending a little over a week exploring what all I could do with the Javascript SDK, I decided to dive in to creating a UI from which I could call commands (rather than relying simply on the command line). This was, after all, what I had set out to do in the first place. Especially since one of the areas of knowledge that it was suggested I garner more knowledge in was Node/Electron, I thought it would be fun to build a React app wrapped in an Electron environment.
In another attempt to go where none had gone before, I figured I would go with a 90s theme (the 90s do seem to be making a comeback and it is the time of my youth; what self-respecting 90s kid could deny their roots and not go with a Saved by the Bell theme?!?). I wanted to be sure I could relatively quickly create a design and leave most of my last few days with my borrowed bots for making the app actually function (especially given the fun I had with some of the initial connections), so I did a quick Google search to ensure I 1) had all the graphics I needed and 2) could find an only-mildly-obnoxious color palette.
My initial design findings only made me more excited for the task at hand, however I put my searches aside to dive into how to begin building a React App with Electron (which can be found in my saved-by-the-ball repo). There were a few people who had some thoughts and guidance on this topic, but I wanted something more recent that, hopefully, also worked with yarn, as I've found yarn to be faster than npm and time was of the essence. I ended up having about 3 people's pieces handy as I waded through the one that most resonated with my goals and got ahead of myself a few times before I realized I was being far too overzealous and needed to start simple and grow gradually 😉.
When I had both React and Electron humming along happily, I turned back to my design, adding in a background, adjusting a few div sizes to make everything more proportional and grabbing a Google font that fit the theme and might stand out against my fun background (though I did end up just using a solid background for each subsection of the page, in the end). Not wanting to spend too much time making things pixel perfect before I actually had a working application, I turned to working on pulling my command line functions into my React app.
To create a more seamless application and use React in all of its glory (rather than just to slap a recognizable name on my app), I went about starting to convert some of the returns to state setting functions and linking my buttons to the functions I had created. I still wanted to display my battery info, so I worked on mapping that to its own page section at the top. Displaying battery info was also part of a larger set of functions that checked for the bot actually being connected, so it was a doubly beneficial feature to attempt to add towards the beginning. Since I wanted information about my bot to be retrieved on page load, I put my initialization function in a componentWillMount, so that the information was readily available and I had an action I could take to retrieve information about the connectivity of the bot.
Working in React, however, posed its own set of challenges, especially since one of the core packages that allows us to connect with the bot (serialport) relies on node and system bindings. I spent a good while working my way through different errors and seeing what others had done to potentially finagle things so that serialport and others were happy, but I ultimately (a little late in the process) landed on the conclusion that I would need to attempt to pass system level communication with the bot between Electron's ipcRenderer to my React frontend.
I lived, breathed and (quite literally) dreamed Sphero bots for about a week and a half. I am so excited about all that I learned and all the puzzles I haven't quite figured out, yet. My Saved by the Ball app doesn't quite work, yet, but I am anxious to keep poking and prodding to get it to where it should be. Working with hardware brings an entirely new level of entertainment to the table and I can't wait for more forays into the space.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent