Week 8 - More Writing!

This week was frankly, really boring. Transcribing jargon and the research I’ve been doing for the past few months into something that’s both rigorous but understandable is quite the task. This week was a full dive into academic writing; I’ve had enough of staring at overleaf.

At first, it was a bit intimidating. I was scared to write statements, fearing the lack of rigor behind my understanding of things. I cross referenced my understanding with the abstracts and introductions of other academic papers, making sure I wasn’t making incorrect assumptions, and then incorporating it into a writing style that’s friendly to those without technical background. I ended up dividing the paper into the standard research paper sections: Introduction, Methodology, Results, Discussion, Future work, etc.  

What I found most interesting in this was dissecting the “personality” of each dimensionality reduction technique. After all, these techniques were invented with trade-offs in mind, whether it be linearity, computational efficiency, neighbor consistency, or others. I noticed t-SNE was the most “aesthetic,” having the nicest clusters, but often messing up distances between clusters. As the statistical analysis suggested, UMAP had kind of the best balance between clustering and cluster distance. This is incredibly imprecise terminology, but it’s a blog so I can say such things

The more I write, the more I think about future work in this field. I’m imagining playing around with contextual models instead of standalone word embeddings, comparing the same word’s embedding in different contexts. Or maybe looking into using more predefined labels or clustering to drive our understanding of linguistics into the word embedding vector space. There’s a lot to think about here! 

Overall though, this week was really just a step back from everything, and looking at what this project is ultimately about: demystifying word embedding space. Lot’s of typing this week, and i look forward to more typing next week! See y’all then!