Week 8: Four Models, One Dataset: Finding the Right Classifier

Last week ended with a labeled dataset of over 1,000 spectrograms spanning six swing categories, each one manually segmented and verified against slow-motion video. This week, I put that dataset to work by training and evaluating four candidate architectures to determine which one will serve as the base model for the final classifier.

The four models I selected were a baseline CNN, ResNet-50 with ImageNet-pretrained weights, a Swin Transformer, and MobileNetV2. I chose these because they span a meaningful range of complexity, inductive bias, and practical constraints. The baseline CNN serves as the baseline with a straightforward stack of convolutional and pooling layers followed by fully connected heads. If this model cannot learn anything from the spectrograms, the problem is likely in the data rather than the architecture. ResNet-50 is the natural next step, since its skip connections and pretrained features directly address the two biggest challenges: training on a small dataset and preserving the subtle frequency-domain details that distinguish one swing fault from another. The Swin Transformer tests whether the global self-attention mechanism, which can relate distant regions of a spectrogram without building up through local filters, offers an advantage on data where the timing relationship between the backswing and downswing matters as much as the local texture of each phase. MobileNetV2, a lightweight architecture built around depthwise separable convolutions and inverted residual blocks, is included because my long-term goal is to deploy the classifier on a smartphone. If MobileNetV2 can match the accuracy of the larger models at a fraction of the parameter count, it would be a realistic choice for real-time inference on edge hardware.

I split the dataset into training, validation, and test sets using a 70/15/15 ratio and stratified by swing category to ensure each class was proportionally represented across all three splits. I also stratified by player identity, so that no individual golfer appeared in both the training and test sets. Without this precaution, the model could learn to recognize players rather than swing faults, inflating accuracy on familiar body geometries while failing on new ones. All four models were trained for 50 epochs with an Adam optimizer and a learning rate of 1e-4, using cross-entropy loss. For ResNet-50 and MobileNetV2, I froze the pretrained convolutional layers for the first 10 epochs to let the classification head warm up before unfreezing and fine-tuning the full network at a reduced learning rate of 1e-5. For the Swin Transformer, I used pretrained weights from ImageNet-21K and applied the same two-phase training strategy.

For the results, the baseline CNN achieved roughly 72% overall accuracy on the test set, which confirmed that the spectrograms contain learnable structure but that a shallow architecture lacks the capacity to capture it reliably. It performed reasonably well on the correct swing and on the no-shoulder-rotation categories, where the visual differences are most pronounced. Still, it struggled to separate the two incorrect path faults (inside-out and outside-in), often confusing one for the other. ResNet-50 jumped to approximately 89% accuracy. The pretrained features gave it a substantial head start, and the skip connections allowed it to preserve fine-grained frequency information through deeper layers. Its confusion matrix showed strong performance across most categories, with the main source of error being a modest tendency to misclassify early extension as no hip rotation, two faults that share some low-frequency spectral overlap because both involve restricted lower-body motion. The Swin Transformer achieved around 87% accuracy, which was competitive with ResNet-50. Its attention maps revealed that it was attending to the correct temporal regions of the spectrogram, particularly the backswing-to-downswing transition and the impact zone. Still, the relatively small dataset appeared to limit its ability to outperform the CNN-based models, consistent with the literature’s finding that Transformers require more data to fully exploit their flexibility. MobileNetV2 reached about 84% accuracy, which is remarkably close to the larger models, given that it has roughly one-tenth the parameters of ResNet-50. Its errors followed a similar pattern to ResNet-50’s, with slightly more confusion between the path faults and early extension.

A few observations stood out beyond the headline numbers. First, the choice of spectrogram representation mattered. I ran a quick comparison using both time-Doppler spectrograms and range-Doppler maps as inputs, and the time-Doppler spectrograms consistently outperformed the range-Doppler maps across all four models by 3 to 5 percentage points. This makes sense as the temporal evolution of Doppler velocity is where the biomechanical sequencing of a swing is most directly encoded, and flattening the time axis into a single range-Doppler snapshot discards that information. Second, the player-stratified split was essential. When I briefly tested a random split that allowed the same player to appear in both training and test sets, ResNet-50’s accuracy jumped to 94%, a misleading number that reflected body-geometry memorization rather than genuine fault classification. The drop back to 89% on the player-stratified split confirmed that the model was learning swing mechanics rather than silhouettes.

Based on these results, ResNet-50 is the clear choice for the base model. It achieved the highest accuracy, with interpretable error patterns concentrated on biomechanically similar fault pairs, and its pretrained backbone makes it well-suited for fine-tuning on a dataset of this size. Next week, I will focus on hyperparameter tuning to further improve ResNet-50’s performance, with particular attention to the fault pairs it currently confuses.