Week 9: Crisis & Pivot: A New Path Forward

Crisis struck this week. The custom-designed IMU sensor PCBs I had waited weeks for finally arrived… and promptly shorted the moment current touched them. I’m still not sure whether the issue lies in the manufacturing or my schematic design, but either way, the result is the same: unusable hardware and no time for replacements. With shipping lead times stretching beyond the final project deadline, I had to face the reality that my original plan wasn’t going to fly. So I pivoted, fast, to my backup.

The new system is still a glove-based tracking device, but it now captures finger motion mechanically instead of relying entirely on tiny embedded PCBs. Each finger is equipped with a badge reel-style spring-loaded mechanism connected to a potentiometer. As the finger bends, the reel rotates, and the potentiometer tracks angular displacement. For overall hand orientation, the glove includes a 9-DOF IMU. Since the custom sensors didn’t work, I had to buy a prebuilt development board for the sensor. You may be thinking, why didn’t you use this prebuilt sensor from the beginning? The answer lies in the size. This prebuilt sensor is almost 4 times the size of my custom one, leading it to not fit on the finger. Getting back to the main point, while this covers basic motion, it suffers from the well-known problem of drift: small errors in orientation build up over time, making the data less reliable.

To combat this, the glove also listens. A software module running on the ESP32 microcontroller captures ambient musical notes. These are treated as reference events: if the glove hears a known note while detecting a certain hand orientation, it can use that data point to recalibrate and realign itself, helping correct drift. The ESP32 acts as the brain of the system, collecting all potentiometer values, processing the IMU’s motion data, analyzing acoustic input for pitch recognition, and merging it all into synchronized output.

This new direction combines physical tracking, inertial sensing, and audio-based error correction, and while it wasn’t my original plan, it’s a solid step forward.