Two mice, two cursors

I always thought that when you connect two mice to a computer, there should be two mouse cursors, one for each mouse. Unfortunately that's not really how it works on modern operating systems, at least not by default. Software add-ons exist that draw a second cursor for the second mouse (and also perform some tricks to make button clicks work as expected). I'm sure they work fine, but I came up with a pure hardware solution to the problem that requires no special software running on the host. Here's a video demonstrating this solution.

What you can see here is two mice, connected through a USB hub to an Arduino Leonardo with a USB Host Shield. The mice are wireless, but they use 2.4 GHz USB receivers so they're effectively USB devices. The Arduino is connected to the computer and acts as a USB mouse.

The Arduino reads inputs from the two mice, does its magic, and sends the resulting inputs to the computer. What's the trick that makes the computer display two cursors? If you look closely, you'll notice that the two cursors flicker a bit. That's because they're not really two cursors, instead it's just the one normal cursor oscillating quickly between two positions on the screen, appearing as two due to persistence of vision. The Arduino keeps track of the X/Y positions of both cursors and updates them based on the inputs received from each mouse. Then it alternates between the two causing the illusion of two cursors. To be able to do that it uses absolute cursor positioning, unlike a regular mouse, which only sends the X/Y deltas and doesn't know where on the screen the cursor is at any moment. (I have previously used absolute cursor positioning for both good and evil.)

One remaining problem is what to do when you click a button on one of the mice. The user will naturally expect for the click to happen at the screen position of the cursor corresponding to that mouse. So we can't keep alternating between the two cursors. For as long as the button is pressed, we only send the cursor position of the mouse that's doing the clicking. This means that the second cursor disappears for a brief moment, but the clicks work as expected.

If you want to play around with it, I posted the source code on GitHub. It uses the standard USB HID protocol and works with Windows, Linux and Mac, without any additional software needed on the computer.

Bluetooth trackball Mark II

I made a Bluetooth trackball in which all the electronics are inside the ball. The ball is the entire trackball. Here it is in action:

Inside the ball there's a microcontroller and a set of sensors that together with some clever algorithms allow the microcontroller to know the ball's absolute orientation in 3D. When the ball is rotated, it sends the appropriate commands to the computer over Bluetooth to move the mouse cursor. Oh, and the keyboard in the video is a completely separate USB device, it's only used for mouse clicks and doesn't interact with the ball in any way.

This is actually the second iteration of the project. While I was very proud of my previous attempt, it had some undeniable shortcomings and was generally a proof-of-concept quality device. This new version improves upon it in many regards, to the point where I think it could possibly be used on a day to day basis as a replacement for your regular mouse. Specific features include:

• an nRF52840 chip along with LSM6DS33 accelerometer/gyroscope and LIS3MDL magnetometer
• a wirelessly rechargeable 500 mAh battery
• battery level reporting over Bluetooth
• more robust "up" calibration
• fully sealed construction allowing for smooth rolling of the ball

I used Adafruit's Feather nRF52840 Sense, it has all the necessary sensors onboard. The other components are a lipo battery, a wireless charging coil and two reed switches, one of which is connected to the board's "En" pin, allowing us to turn the device off with an external magnet, the other is connected to a GPIO pin and is used to trigger the "up" calibration.

The ball is 65 mm in diameter and is made of two 3D printed halves. After putting the electronics inside I glued the two halves together and smoothed the surface by progressing through increasingly finer grits of sandpaper. It rolls pretty smoothly on 2.5 mm zirconium oxide bearing balls, though it's still admittedly not as smooth as a real trackball. I made a few bases in various shapes for the ball to roll on, they are just pieces of 3D printed plastic and don't have any electronics in them.

To charge the battery inside the ball I also made a special base that has a wireless charging coil inside and is powered with USB (or any 5V source). It also has a strategically placed magnet that can be used to trigger one of the two reed switches inside the ball, one will turn the device off, the other allows for setting the logical "up" directon of the trackball (by itself the trackball doesn't know which direction should move the mouse cursor up, so you can set it by putting a magnet near one of the reed switches and rotating the ball around the vertical axis).

The Arduino code and 3D models are on GitHub.

Lean back to scroll

You know those days when lifting a finger to scroll a website seems like to much of an effort? No, of course not, me neither. But just in case I made this so I can scroll just by leaning back in my chair:

If you look closely there's Adafruit's Feather nRF52840 Sense board attached to the back of the chair. It uses the onboard sensors to figure out the angle of the chair's back and if it's over a threshold it sends scroll down events to the computer (using the standard Bluetooth mouse protocol so it works with any OS). To set the "zero" position you press the "user switch" button on the board.

Here's the Arduino code.

Hat mouse

I've been playing around with orientation sensors and Bluetooth HID recently. Today I'd like to show you what is perhaps my most practical application of those yet. I made a head mouse, also known as an air mouse, or in my case, a hat mouse:

The device is attached to a hat and you move your head around to move the mouse cursor on the screen. It uses the standard Bluetooth mouse protocol so it works with Windows, Mac and Linux with no additional software running on the computer. The board used is Adafruit's Feather nRF52840 Sense. The way it works is it reads accelerometer, gyroscope and magnetometer data, puts it through a sensor fusion algorithm to get orientation and then uses the yaw and pitch values to position the mouse cursor on the screen. It uses absolute cursor positioning so any given head orientation corresponds to a fixed cursor position on the screen. To reset the center position, you press the "user switch" button on the board.

Here's the Arduino code. In addition to the Feather board we need some way to power it. I used this lipo battery, but the board can also be powered over the USB port so a small power bank could work as well.

As with some of my previous custom devices that work as a mouse, unfortuntely the hat mouse has no way of performing button clicks yet. In the video above I'm using a separate USB foot switch for clicking. Depending on the movement limitations of the user, something like a sip-and-puff switch might be more appropriate (same goes for the orientation reset button that's currently located on the board). The Feather nRF52840 Sense board has a lot of sensors in addition to those that we're currently using for the orientation. Perhaps the pressure sensor or the light sensor or even the microphone could be used to make entirely head-operated switches. I have some ideas, but I decided to treat that as a separate project.

Human trackball

Remember Logitech's April Fools' video from 2017? The one where the gym ball works as a trackball? I made that for real:

I did this as kind of a detour while working on the next version of my Bluetooth trackball from last year (stay tuned for that). I found a board on AliExpress that has the nRF52832 Bluetooth chip from Nordic, an MPU-9250 9DOF sensor, a built-in battery charging circuit and comes with a lipo battery attached. Originally a development board for a fitness bracelet, it looked like a pretty good match for my needs.

For this application I just attached the board to the gym ball using scotch tape (and tried to avoid sitting on it or crushing it by rolling the ball).

The software is derived from my previous IMU-based Bluetooth trackball attempt. This time I decided to do the sensor fusion algorithm in software - even though the MPU-9250 is better than the MPU-6050 because it has a magnetometer (3 more DOFs!), its onboard algorithm was not updated to make use of it - it still only uses the accelerometer and gyroscope. So I used Adafruit's library implementing Sebastian Madgwick's sensor fusion algorithm. I also used Sandeep Mistry's Arduino core for the nRF52832 chip and his BLEPeripheral library for Bluetooth. I programmed the board using Nordic's nRF52 DK development kit.

You can find the code here. Also included is a sketch for calibrating the magnetometer that sends the data over Bluetooth serial (I used Adafruit's Bluefruit Connect app to read the data). And since the gym ball itself has no way of performing button clicks, I used a foot switch connected to a Digispark for clicking the mouse. A sketch for that is also included.

Even though it looks like a trackball, the ball works more like a joystick in this case: when you tilt it to the right, the cursor moves as long as the ball is kept in that position and stops moving when it is returned to the neutral starting position. Right now it's not really a practical mouse or trackball replacement, but I believe that with some fine tuning it might become a viable option. Another possibility would be to only use it for scrolling, not moving the cursor, which I haven't tested, but it should be an easy modification.

Can't wait for Logitech's next year April Fools' video!

Bluetooth trackball with all the electronics inside the ball

Update (2021-01-27): I have made an improved version of this project, see here for details.

For a while now I've been fascinated with the idea of making a trackball-type input device in which the ball itself would be the entire device. Unlike a regular trackball, where the ball is just a ball and registering its motion and communicating with the computer is done by external sensors and chips, here everything would live inside the ball. And now I have finally done it. Here's a video:

And here's how I did it. For obvious reasons, the device couldn't use USB, it had to be Bluetooth. For registering the rotation of the ball, a combination of an accelerometer and a gyroscope seemed like a good way of ensuring smooth results. I ended up using the following components:

• Adafruit Bluefruit EZ-Key
• MPU-6050 accelerometer/gyro
• Arduino Micro as the brains of the operation

I designed the ball itself in Fusion 360 and then 3D printed it. It has a diameter of 10 centimeters and consists of three parts, two halves of the ball and an inside part that holds the electronics and the battery. The three parts screw together to form the ball. I also made a stand on which the ball can be rolled, but that's just to make it easier to use as a trackball, the stand is optional and the device works when held in the air as well.

Here's what it looks like inside (the MPU-6050 is under the battery):

For the software part, I was pleasantly surprised when I found out that the MPU-6050 chip can do the sensor fusion magic (combining the outputs of the accelerometer and the gyro) for us and just give us the 3D orientation in the form of a quaternion. What's left then for the Arduino to do is to read the current orientation, calculate the difference between the previous orientation and the current one, translate it to X and Y mouse cursor offsets and send the appropriate commands to the Bluetooth chip that will pass them on to the computer.

Here's the Arduino code.

You may wonder how the ball knows which direction the cursor should go when it's rotated in a certain direction. The answer is of course that it doesn't, so the initial mapping of the directions might not be correct. But it does know which way is up (because it has an accelerometer), so the directions can be calibrated by rotating the ball around the vertical axis.

While I personally consider this project a great success, there's always room for improvement. For example, the trackball would be a little more practical if it had the ability to click, not just to move the cursor. I explored the idea of detecting taps on the ball using the accelerometer, but found it hard to eliminate false positives. This area needs some additional work.

An obvious shortcoming of the current solution is that to switch the device on or off, I have to unscrew the two halves and connect or disconnect the battery. Ideally there would be some way of doing that without disassembling the ball, a magnetic switch of some sort or maybe the device could always be on and just go to sleep when it's not used for a while (like regular wireless mice do).

On a related note it would be really cool if we also didn't have to open the ball to change the battery. Seems like a good use case for wireless charging.

If I ever do a next version of this, I will probably try to use a different Bluetooth board, one that's not discontinued (and perhaps can do Bluetooth LE).

Bluetooth mouse from a Wii nunchuck

I never had a Nintendo Wii, but I got a nunchuck controller to play with. It has a joystick, two buttons, an accelerometer for orientation sensing and the third-party ones are dirt cheap. It turns out that it's really easy to talk to them from an Arduino too, because they speak I2C and as you would expect the protocol is well documented on the Internet. I used Adafruit's Trinket, Bluefruit EZ-Key and a 100 mAh lipo battery to turn the nunchuck into a Bluetooth mouse:



Here's a diagram of the connections, pin 1 on the Trinket goes to the RX pin on the Bluefruit and pins 0 and 2 are used for the I2C communication with the nunchuck:



On the software side, I used the WiiChuck class I found here, but I modified it to use the TinyWireM library for I2C. It is equivalent to the standard Arduino Wire library, but it runs on ATtiny chips like the one used by the Trinket. I also used this SendOnlySoftwareSerial library. Here's my sketch and the modified WiiChuck library.

I haven't found any clever use for the accelerometer inside the nunchuck yet, perhaps it could somehow be used for scrolling.