Hat mouse

I've been playing around with orientation sensors and Bluetooth HID recently. Today I'd like to show you what is perhaps my most practical application of those yet. I made a head mouse, also known as an air mouse, or in my case, a hat mouse:

The device is attached to a hat and you move your head around to move the mouse cursor on the screen. It uses the standard Bluetooth mouse protocol so it works with Windows, Mac and Linux with no additional software running on the computer. The board used is Adafruit's Feather nRF52840 Sense. The way it works is it reads accelerometer, gyroscope and magnetometer data, puts it through a sensor fusion algorithm to get orientation and then uses the yaw and pitch values to position the mouse cursor on the screen. It uses absolute cursor positioning so any given head orientation corresponds to a fixed cursor position on the screen. To reset the center position, you press the "user switch" button on the board.

Here's the Arduino code. In addition to the Feather board we need some way to power it. I used this lipo battery, but the board can also be powered over the USB port so a small power bank could work as well.

As with some of my previous custom devices that work as a mouse, unfortuntely the hat mouse has no way of performing button clicks yet. In the video above I'm using a separate USB foot switch for clicking. Depending on the movement limitations of the user, something like a sip-and-puff switch might be more appropriate (same goes for the orientation reset button that's currently located on the board). The Feather nRF52840 Sense board has a lot of sensors in addition to those that we're currently using for the orientation. Perhaps the pressure sensor or the light sensor or even the microphone could be used to make entirely head-operated switches. I have some ideas, but I decided to treat that as a separate project.

Plotting sensor data with MS Paint and Bluetooth HID

I have recently discovered that the HID mouse protocol supports absolute cursor positioning in addition to the more common relative mode. I think that opens interesting possibilities, one of them being we can plot real-time charts in MS Paint with no additional software running on the computer, using just the standard Bluetooth mouse protocol:

The board used above is Adafruit's Feather nRF52840 Sense, which has Bluetooth LE and lots of sensors. On the video we're plotting readings from the accelerometer. The chart is drawn by simulating mouse and keyboard inputs (keyboard is used to clear the screen when we reach the right edge).

Here's the Arduino code.

USB "rubber ducky" with mouse input

Perhaps you're familiar with a type of device commonly referred to as a USB rubber ducky. It's a seemingly innocent device that looks like a regular USB drive, but when connected to a computer, it acts as a USB keyboard and sends malicious keystrokes to the victim's machine, as if a human typed them (but faster). The commands sent can download some unwanted software or open a reverse shell and do nasty stuff in general.

So I thought why stop at keyboard, why not also pretend to be a mouse, launch Paint and draw something funny:

The main challenge here is that a regular mouse doesn't really know where the cursor is on the screen. It only sends relative position changes like "move the cursor 7 units to the right and 2 units down". Depending on the sensitivity setting on the user's system, it might correspond to a different distance in pixels. When you add mouse acceleration into the mix, it's not really practical to try and guess where the cursor will end up being.

But, as is turns out, the mouse HID protocol also allows for absolute cursor positioning and all the major desktop operating systems support this mode (perhaps because touchscreens and graphics tablets use it?). With that knowledge, the task becomes easy, as we can just say "move the cursor to position X, Y".

I used a Digispark with a modified version of Adafruit's TrinketHidCombo library for the demo above. Here's the Arduino code.

Human trackball

Remember Logitech's April Fools' video from 2017? The one where the gym ball works as a trackball? I made that for real:

I did this as kind of a detour while working on the next version of my Bluetooth trackball from last year (stay tuned for that). I found a board on AliExpress that has the nRF52832 Bluetooth chip from Nordic, an MPU-9250 9DOF sensor, a built-in battery charging circuit and comes with a lipo battery attached. Originally a development board for a fitness bracelet, it looked like a pretty good match for my needs.

For this application I just attached the board to the gym ball using scotch tape (and tried to avoid sitting on it or crushing it by rolling the ball).

The software is derived from my previous IMU-based Bluetooth trackball attempt. This time I decided to do the sensor fusion algorithm in software - even though the MPU-9250 is better than the MPU-6050 because it has a magnetometer (3 more DOFs!), its onboard algorithm was not updated to make use of it - it still only uses the accelerometer and gyroscope. So I used Adafruit's library implementing Sebastian Madgwick's sensor fusion algorithm. I also used Sandeep Mistry's Arduino core for the nRF52832 chip and his BLEPeripheral library for Bluetooth. I programmed the board using Nordic's nRF52 DK development kit.

You can find the code here. Also included is a sketch for calibrating the magnetometer that sends the data over Bluetooth serial (I used Adafruit's Bluefruit Connect app to read the data). And since the gym ball itself has no way of performing button clicks, I used a foot switch connected to a Digispark for clicking the mouse. A sketch for that is also included.

Even though it looks like a trackball, the ball works more like a joystick in this case: when you tilt it to the right, the cursor moves as long as the ball is kept in that position and stops moving when it is returned to the neutral starting position. Right now it's not really a practical mouse or trackball replacement, but I believe that with some fine tuning it might become a viable option. Another possibility would be to only use it for scrolling, not moving the cursor, which I haven't tested, but it should be an easy modification.

Can't wait for Logitech's next year April Fools' video!