IMU-Based Tennis Swing Tracking, Analysis, and Coaching Feedback
Advisor: Wendel
Internship Location: Salesforce
Onsite Mentor: Mr. Yi Jin, Software Engineer, Salesforce
As a tennis coach for young children, I have observed that many beginners struggle to master the basic swing motion, often due to challenges with coordination and timing. This observation led me to explore whether my interest in technology could be combined with tennis to create a tool that supports skill development. Through researching existing wearable sensor devices, I found that many current options are either bulky and impractical for use during play or prohibitively expensive, making them inaccessible for young players and community programs.
This project aims to develop an affordable, wireless, IMU-sensor-based wearable system that measures tennis swing kinematics and provides feedback that is usable by coaches and players. By collecting motion data from IMU sensors and processing it using Stanford’s OpenSense software, the system will analyze swing mechanics and compare them to those of a high-level player and generate actionable feedback to help users identify areas for improvement and apply specific corrections to their technique.
Project Blog
Week 1: Introduction + Background
Hey everyone! I’m Michelle and welcome to my first blog post! For my senior project, I aspire to create an affordable, wireless, IMU-sensor-based wearable system which tennis players can use to improve their swing.
Tennis can be a rather expensive sport for many people to get into, and coaching lessons are getting more and more expensive by the second. I wondered if I could alleviate this problem by making a somewhat “24-7” available coach to many kids. After researching affordable sensor options, I discovered WitMotion IMU sensors. I placed a bulk order, and once they arrived, I started experimenting with them and quickly saw opportunities to expand the project further.
Through utilizing tools already available on the internet such as the Stanford OpenSim software, I hope to use the features of the software (specifically the scaling, inverse dynamics, and inverse kinematics tools) to develop an interface which the user can upload their own swing data captured by the IMU sensors into the software and compare it with motion with one of a high-level tennis player’s to identify the areas in which they can improve their swing.
This first week, I conducted background research on existing products and performed a literature review on tennis sensor technologies. Although many current options provide valuable feedback, a lot of them are either impractical to wear during match play or extremely expensive. There are also numerous video-based analysis tools, but inconsistent camera angles can lead to perspective errors and less accurate biomechanical feedback. By using WitMotion sensors, I hope to address these issues by developing a wireless device that emphasizes practicality, affordability, and improved motion accuracy.
Then, through the online tutorials provided by the Stanford OpenSim software, I learned how to operate and navigate the main tools I’ll need for my project—specifically inverse dynamics, kinematics, and scaling. I also experimented with the WitMotion sensors, testing movement in multiple directions to better understand how the data is recorded and formatted. In addition, I organized previously collected tennis swing data from my friends and me into structured folders for easier analysis.
My next step is to figure out how to properly format and input my sensor data into the software so I can generate a visual model of my swing. Currently, the Stanford OpenSim software is primarily designed to accept data from more expensive, research-grade sensors, so I will need to adapt and convert my WitMotion data into a compatible format before I can run a full analysis. See everyone in Week 2!
Week 2: Data Processing
Hello everyone, and welcome back to my blog! Last week, I left off on the data processing step of this project. I had previously collected swing motion data from basic tennis movements (such as forehands, backhands, and volleys), and I needed to figure out a way to upload it into the Stanford OpenSense software to generate a 3D model of the swing.
Although this might seem straightforward, it actually required quite a bit of work because the Stanford OpenSim software was designed to be compatible with more expensive research-grade sensors. Even though my WitMotion sensors collected all the data components needed for this project, I had to reformat and rearrange the data into a format that the software could read.
I first studied what specific motion data my WitMotion IMU sensors measured. After recording a forehand swing, I exported the raw data collected from the sensors. The file included timestamps, device IDs for each sensor, acceleration, angular velocity, and quaternion orientation values (Q0–Q3). I also noticed that each row represented a single sensor reading rather than all sensors at one time step.
Next, I looked into what type of file and format the OpenSense software requires. I learned that OpenSense expects the sensor orientation data to be formatted as a .sto file with a specific header structure. The software mainly uses quaternion orientation values, which must be organized in a particular way.
The main differences between the two datasets were how the information was organized and labeled. The WitMotion sensors export data where each row represents a single sensor reading. However, OpenSim requires the data to be organized so that each row represents one time step containing orientation data from all sensors. In addition, the WitMotion sensors are labeled using device IDs (such as WTD1 or WTD2), while OpenSim labels sensors based on body segments. Finally, the WitMotion sensors include multiple types of measurements, such as acceleration, angular velocity, magnetic field data, and quaternion orientation. For this project, however, OpenSim only requires the quaternion orientation data.
To make the data compatible, I needed to extract only the quaternion orientation values, reorganize the dataset so that all sensors appear in the same row for each timestamp, rename the sensors to match body segments, and convert the file into the .sto format required by OpenSense.
To do this, I installed the necessary Python packages for data processing and wrote a script to restructure the dataset. I used Python because it is widely used for data processing and scientific computing, especially when working with sensor datasets. Python has libraries such as pandas that make it easy to read, reorganize, and clean large data files like the IMU sensor exports. The WitMotion data required several transformations, including reformatting timestamps, reorganizing rows by sensor, renaming columns, and exporting a new file format, which are tasks that Python handles efficiently with built-in tools.
Python is also commonly used in biomechanics and robotics research workflows, so it integrates well with tools like OpenSim. Compared to many other languages, Python requires less setup and fewer lines of code for data manipulation, which makes it easier to quickly write a script that converts the sensor data into the required format.
After writing the script and converting my dataset into a format compatible with OpenSim, I attempted to load the orientation data into the OpenSim IMU Placer tool. However, I encountered errors when calibrating the placement of my IMU sensors. Every time I tried importing the file into the IMU Placer tool, the orientation file field turned red and prevented me from running the model. My next step will be to figure out what went wrong during this step.
Thanks for reading this week’s blog. See you guys in the next one!
Week 3: Data Processing (cont.)
Hi everyone, and welcome back to my blog! This week I continued to work on data processing, which is one of the main parts of my project.
Last week I started writing the code to convert my Witmotion sensor data into a format that the OpenSim software can use (read my last blog for more details!). However, there were still some minor errors in my code from last week. One problem was that my headers were incorrect. Instead of naming the sensors body parts (e.g. head, pelvis, etc), my headers were still left in the WitMotion format of WTD1_Q0, WTD10_Q0, WTD11_Q0, etc. OpenSim couldn’t recognize the sensors and wouldn’t read the file properly because of that. To combat that, I added a sensor map that mapped each sensor to a body part as shown below.
The conversion script should be good now. I was able to convert my raw Witmotion CSV data into an OpenSim-compatible .sto file, with properly formatted time columns and quaternion orientation data for each sensor. Each sensor is now labeled according to its corresponding body segment using the sensor map, which allows OpenSim to correctly interpret the motion data. (Shown below)
After fixing up the minor bugs and tweaks, I attempted to load the motion onto the model. However, I still had issues with the IMU Placer tool. Even though I loaded the motion and data into the IMU Placer, the software wasn’t letting me play the run button. I will need to investigate deeper about why that happens.
I’m aiming to finish this data processing step by next week—come back then for an update!
Week 4
Hey everyone, welcome back to my blog!
This week I continued focusing on data processing and trying to get OpenSim to read my data.
Last week I left off stuck on calibrating and running the IMU placer tool. To figure out the issue, I went back and reread the IMU placer and inverse kinematics tutorials from the OpenSense platform. After that, I looked through the example data files they provided and compared them to mine.
I realized the problem was in my calibration file—my code was only taking the first time frame instead of using the full motion data. So I started working on fixing my code to correctly process the data file.
Right now, I’m still trying to convert my full IMU orientation file into a proper initial placement file. I’ve been getting a lot of errors each time I run it, so I’m going through them one by one and debugging slowly.
On a different note, I also tried running the example model from the OpenSense platform to see how the IMU placer and inverse kinematics tools are supposed to work. Even though it ran, the output looked off—the walking simulation turned into a person kind of floating and rotating strangely. The model kept bending at the torso, even though it was supposed to just be a simple walking motion.
I’m not sure yet if this is a calibration issue on my end, something wrong with the software setup, or a problem with the example data itself, so that’s something I’ll keep looking into.
Thanks for reading, and see you in the next one!
Week 5: Debugging
Hey everyone, and welcome back to my blog! This week I’m so excited to update you guys that I’ve finished the data processing step! It’s definitely a small milestone for me as I’ve been working on this step for a couple weeks now.
However, it’s not all sunshines and rainbows quite yet. Even though I managed to get the IMU Placer and IMU Inverse Kinematics tool to read and run my data, there were still many issues with the motion generated.
Problems
1. The motion was a bit wacky (same problem when I ran the example model code…). Even though the recorded motion was just a forehand swing, the motion generated showed the skeleton bent over with one leg diagonally up with the person basically floating.
2. The next problem was that the calibrated model with the IMUs placed on the body were missing some of the sensors, specifically the sensors on arms/head (i.e. head, humerus_l, humerus_r, radius_l, radius_r)
Next Steps to Debug/Brainstorming solutions to these problems
I met with my mentor to discuss the problems I’m currently facing, and together we brainstormed some reasons why these errors were happening. Regarding the first error for the motion being orientated weirdly, we either thought it could be a calibration error or a misalignment error. The calibration error would mean that the first initial pose taken for calibrating the model was not in a neutral stance. Although I thought when recording the motion we held the initial pose for long enough, there might have been other factors that affected it. To fix it, I’m aiming to either manually picking a better frame or re-recording a short “T-pose/neutral pose” clip before the swing. A misalignment error, on the other hand, means that the heading (forward direction) is misaligned and facing a different direction than it’s supposed to. To fix that, I’m going to try rotating/setting the pelvis towards different axes and angles to find the right orientation of it.
Next, regarding the missing sensor issue, I’m going to take another look at my conversion code to see if the upper body data was lost somewhere along the way. Although the data files generated look correct, there could be an incorrect formatting issue that causes the Opensense software to not be able to read it. I will specifically look at the head, humerus_l, humerus_r, radius_l, radius_r sensors. The error could also be coming from bad quaternion data. Reading over the documentation again, I realized that if a sensor has: NaNs, constant values or corrupted orientation the OpenSim silently ignores it. For my data, we did each frame slowly by slowly, which might have caused the data to be constant for too long and throw the orientation off track. Another explanation for the missing sensor issue could be sensor-to-body misplacement. If my real sensors were placed slightly rotated, upside down, or a different orientation, OpenSense assumes wrong alignment and ignores that part of the code, which would explain the missing sensors.
Although I’m frustrated by these errors and bugs along the way, I was prepared to experience some errors along this journey. Next week I’ll aim to at least start looking into some of these errors and hopefully gain a clearer insight of why my motion might be off. I’ve detailed the steps I’m planning to take already, so come back next week to see how the beginning of the debugging process goes!
Week 6: Debugging + Regrouping
Hello Everyone, and welcome back to my blog! This week was a bit different since I was on spring break and didn’t have access to my sensors. So instead of collecting new data, I focused on debugging and trying to fix some of the issues I’d been running into.
One thing I experimented with was rotating the model to better align the torso. I tested a few different rotation configurations: -90, 0, 0; 0, 90, 0; 90, 0, 0; and -90, 0, 180. My goal was to get the torso into a more natural starting position, since it had been appearing misaligned in previous runs. However, all of the rotations yielded no better motion than before—the torso was still twisted in a sitting-like position relative to the legs.
I tried approaching another issue I faced last week: missing sensors. To combat that, I went back through my code to double-check that all the sensors were properly mapped. That’s when I realized that in my placement file the head and upper body sensors were missing, which explained why certain segments weren’t behaving correctly. However, when I tried adding them back, my code started producing errors, which was the initial reason why I deleted it from my original code. This is something that I will have to take another closer look at.
At this point, I think the most reliable next step is to retake the data entirely, since there may be inconsistencies from earlier recordings from either the sensors being misaligned while placed or the data gathered just not being how the OpenSim software wanted it to be collected.
Since I couldn’t collect new data this week, I shifted focus to something else: figuring out why the example code itself wasn’t working. After carefully checking all the file formats and rerunning everything, I was finally able to get it working properly. I think previously I skipped one of the rotating and calibration steps which made the motion produced kind of wacky. Fixing this issue was a big relief and gave me a better reference point moving forward as I now know that the OpenSim software itself does indeed work.
In this following week I’m planning to re-collect data with corrected sensor mappings and hopefully get more accurate motion results by rerunning the new data. Thanks for reading, see you guys in the next one!
Week 8: New Data Collection
This week, I gathered new sensor data to try to run through my software. My previous dataset had a major issue: the sampling wasn’t continuous or fast enough so the OpenSense software tended to fail processing the data. This time I made sure to gather continuous, higher-quality data.
I also expanded the variety of movements I recorded. Instead of just a few isolated swings, I collected a much broader dataset, including: forehands, backhands, serves, warm-up motions, and more dynamic movements like actually rallying with a tennis ball. This should give me a much more realistic representation of actual tennis play, which is important for both accuracy and future analysis.
Next, I moved on to improving my coding script. I made two big changes to my conversion script. Previously, I was converting .csv files to .sto files one at a time, which was honestly painful. So I updated my script to mass convert all CSV files into STO format automatically with each output file having the same corresponding name with the motion. Secondly, I also applied a time offset adjustment, since I sometimes held my starting calibration pose for too long before moving. I coded it so that the motion was detected automatically, and subtracted the corresponding time offset difference leaving only around 4-5 seconds held in the neutral position. These changes made my workflow way more efficient and also helped clean up inconsistencies in the data.
After all that, I was ready to run everything through OpenSense… but I ran into another issue. When I tried to load the new files, the “Run” button was grayed out again, the exact same problem I encountered earlier when initially trying to run my data through the software.
I've already checked all of my data file headers, and made some adjustments there. Now I’m shifting my focus back to the sensor placement and orientation calibration files to see if there are any issues there with the flow.
Week 9: Part 1 Completed!!
I realized that I was having trouble with the data conversion step because my data was glitchy. After tweaking my placement and conversion code repeatedly, I decided to pivot and take a look at my data. That’s when I realized some sensors were missing. That could’ve been due to losing Bluetooth connection momentarily or sensors disconnecting because of the heat (it was pretty hot when we gathered data on the tennis court).
Also, I realized that OpenSim only needed 8 sensors to generate a full motion body so I decided to simplify and go with that instead of the 11 sensors I previously had.
I decided to gather some new data to have a fresh plate to start from. I made sure that my sensors were aligned correctly and taped on securely.
The data conversion code worked when I feed in the new data. I could now run the simulation yay! However, two major problems then arised. One, the arm sensors were not moving. Two, the skeleton was rotated weirdly.
I repeatedly looked over my code to see if my arm sensors got lost somewhere but everything seemed fine. I also checked my data format and saw that the arm sensor values were changing. I shifted the quaternion values by one coordinate to see if maybe the arm sensor followed a different structure than the other sensors. I also added more precision to the code and rounded the Q values less but nothing seemed to work. These changes just made the skeleton stationary or turned weirdly.
I then turned towards checking the sensors. I increased the frequency the data was collected from 10Hz to 100Hz but that just made my quaternion values stay the same causing the skeleton to be stationary. I then played around and altered the bandwidth, axises, and other components of the sensors to see if anything would help but nothing worked.
It was getting extremely frustrating so I decided to recalibrate all the sensors to see if that was the issue. After doing that, the motion started moving!! The lower body and torso data looked correct but the model was still sideways and the arm sensors weren’t moving. I repeatedly tried tweaking my code but sometimes it just made the motion even worse.
I tried rotating the model using the rotation tool in the GUI but it still didn’t work.
I decided to pivot towards another direction and see if maybe the model was the problem. After searching through the models OpenSim had, I realized there was a newer full body model (2016 vs 2015). I tried the data on the 2016 version and everything worked!! All this trial and error just for the fix to be changing the model. 😅
Now that the data conversion process is finally working, I’ll aim to move on to Part 2 of my project, which is generating tips and feedback for users!! See you guys in the next one.
Week 10: Part 2 - Generating Feedback
This week I focused on Part 2 of my project: generating feedback from the motion data. Since the motion generated by OpenSense was not perfectly smooth, I decided to narrow my focus to two major components that could still provide meaningful and broad feedback: torso rotation and knee bend. Instead of trying to analyze every aspect of the swing, I wanted to prioritize movements that are especially important for power and consistency in tennis strokes.
Thankfully, this stage of the project went much smoother than the earlier debugging process. Once the motion data was converted into .mot files, it became relatively straightforward to tweak and analyze the values generated by the code. To test the system, I recorded three different swings: a “good swing” that included proper torso rotation and knee bend, a “no-torso” swing with limited upper body rotation, and a “no-knee” swing with minimal knee bend. After running the code on each trial, the feedback system was able to successfully identify the differences between the swings and generate the expected results!!
Week 11: Concluding Thoughts
Wow. I can’t believe we’ve already reached the end of this project!! This project was definitely not a linear process, and I learned so much through it. There were a lot of setbacks along the way, and trial and error became a huge part of the experience. Even if things didn’t work the first time, I kept pushing forward by troubleshooting bugs, tweaking my code, recalibrating sensors, regathering data, and making sure the entire pipeline became more streamlined and reliable.
One thing I realized is that the first half of the project—collecting, converting, and debugging the IMU sensor data for OpenSense—took much longer than I originally expected. Because of this, I had less time to fully explore the second half of the project, especially the analysis and feedback side. If I had more time, I would want to gather a much larger dataset of swings and analyze additional parts of the body beyond the areas I focused on in this project.
In the future, this system could be improved in several ways. More sensors could be added to capture a fuller representation of body movement, and larger datasets could help improve the consistency and accuracy of the analysis. The feedback system could also become more advanced by incorporating machine learning or AI models that compare a player’s motion to professional biomechanics. Overall, this project showed the potential of wearable technology in sports training and how engineering can be used to make athletic feedback more accessible and data-driven!!
Thanks so much for following along my journey!!