Week 10 - Wrapping up

This week marks the official wrap-up of my senior project. My senior research paper titled “Visualizing Word Embeddings Using Dimensionality” is finally completed. What began as a vague curiosity about how AI “understands” words transformed into this deep-dive into high-dimensional vector space and dimensionality reduction techniques. 

During this last stretch, I put my foot on the pedal and finished an astonishing amount of writing. Finalizing touches to my writing, making sure the mathematical formulations are correct, and going over the paper a few times, it felt so good when I finally exported the paper as a pdf. 

Looking back, I felt so lost at times during this project. Back when it was just me and my computer’s terminal, it felt hard to visualize a whole research paper coming out of this. But week by week, I’d build this project piece by piece. It’d be a few cells in my jupyter notebook completed, it’d be some more notes down in my overleaf project, or it’d be just a better foundational understanding of a dimensionality reduction technique. But progress was steady, and I’m proud of the work I’ve done over the past 10 weeks.

Working through all this answered some questions I had about the topic, but brought upon so many more. What happens when we use context-based embeddings? Are static embeddings a statistical amalgamation of the different definitions a word can have? How will clustering change? One of the biggest unanswered questions was the usage of DrLiM, dimensionality reduction by learning an invariant mapping. If we could learn a sophisticated enough mapping from high dimensions to low dimensions using tons of data, what results could we achieve? With only 999 training examples in our experiments, I wonder if performance could improve with way more. But these questions are for another project, another time. For now, I am signing off!

-Kevin Zhou