Week 10: The final week

Welcome to the final week. It has been quite a journey, with both highs and lows, but it has finally come to an end.

This week was all about improvisation and iteration. I set out to connect an ESP32 microcontroller to a glove-based tracking system using a mix of potentiometers and an IMU sensor. It sounds straightforward in theory, until you hit real-world hardware.

First came the wiring chaos. With no custom PCB and limited soldering tools, I had to get creative. I tried using Dupont connectors to hook everything up, but they wouldn’t stay in place. After some trial, error, and a decent amount of hot glue, I got a semi-stable setup. Not ideal, but functional enough for prototyping. For components without pre-soldered headers, I rigged up temporary contact using male jumper wires and held them down with adhesive. It wasn’t pretty, but it got signals flowing.

Next came communication. I wrote Arduino code to read five potentiometers, then package that data and send it over WiFi via UDP to Unity. Getting the Unity side to listen correctly took some debugging, especially since the Unity receiver didn’t update inside the main thread at first. After adding the proper method calls and synchronization, I finally saw real-time data streaming in. It was a huge relief.

Along the way, I ran into every issue imaginable: analog noise on open pins, serial port lockups, a full laptop bluescreen, and even a moment where I feared I’d fried the board. Thankfully, most of these were fixed with reboots, better grounding, or wiring corrections.

If you were reading properly, you may have noticed that I mentioned an IMU sensor. This was meant to check the overall hand orientation, as all I currently measure are the notes played on the piano and which finger plays them. However, I was unable to get the IMU sensor functioning in time. When experimenting with it early in the week, I somehow managed to fry it, and there wasn’t enough time to order another one with the 4 days shipping time that it had.

Despite the setbacks, I now have a functional glove that sends finger bend data wirelessly to Unity in real time, which then analyzes piano play and offers visual feedback. It’s not yet perfect, and there’s a lot left to polish, but the concept is there for a whole future in real time piano feedback.

I hope you guys enjoyed reading the blog throughout the weeks. Please contact me if you have any comments or questions. See you guys later!