Week 7: The Merits of Meta-Analysis and Common Misconceptions

Hello everyone! For this week’s updates, I’ll be focusing on a topic my project advisor recommended: the nitty-gritty of meta-analysis (as well as my experience with this process that, at surface-level, seems “easy”). For my meta-analysis, I have put a final close to my data collection process (after many weeks of doubt and rescreenings) and have extracted some of my data. For screening, I have officially completed the PRISMA flow diagram (attached below). 

The whole purpose of the flow diagram is to transparently map the flow of information through different phases of a systematic review/meta-analysis, while outlining the number of records identified, screened, included, and excluded. It also serves to ensure the meta-analysis is authentic and replicable. 

Now, for the crux of my blog, I want to address the misconception that meta-analyses are easy** and shed light on the merits of meta-analysis in research (think pros and cons, alongside the significance of meta-analysis in various fields of research). Albeit a simple concept on the surface (collecting papers, narrowing down papers, and typing code into R for analysis), there is so much more complexity to it. 

Meta-analyses, by nature, are very thorough since they systematically collect and statistically combine results from dozens (and sometimes upwards of hundreds or thousands) of independent studies on the same topic. It’s such a diverse tool, however, and its purpose can’t truly be defined in a few broad terms without it being an oversimplification. But the essence of meta-analyses is to filter out the noise of individual study limitations, such as small sample sizes or methodological specificities, to reveal more reliable patterns in the data. By synthesizing data from multiple sources, the results of meta-analyses are also highly regarded in their given fields since they avoid cherry-picking one-off results. In fact, they can even reveal publication biases within a specific topic.

To me, as a high school student with no access to many paid resources, one of the largest merits or pros of meta-analysis is that they are very accessible. With time and many free resources available online, a meta-analysis can be done by anyone, albeit with some limitations in the application of results. However, with a process so thorough and widely used by funded researchers, there are many hurdles I’ve had to navigate. Although accessible, meta-analysis is by no means easy. 

One of the biggest challenges of meta-analysis is getting started. In the literature search and screening process, for more niche topics (such as Schizophrenia and postpartum disorders), where incidence is already rare and undocumented, there might be a distinct lack of papers to even perform a meta-analysis with. Also, while navigating free tools, many small mistakes can lead to having to redo the whole screening process (i.e., not documenting each phase of your literature search) and complicate replicability. The full-text screening phase can also be very time-consuming, as it includes reading full papers and determining if they have enough data to actually include in the analysis. In my case, this led me to narrow down a full 30 papers to only 9, and I had to backtrack and reference-mine even more studies to get back to a stable number of 20 studies (~10 for each directionality, which is the standard for more niche, unexplored comorbidities such as postpartum disorders and schizophrenia).

Finally is data extraction and analysis (which I’m still navigating). I’ve made a coding form that I have to fix prompts for to ensure that the extraction program that I’m using can properly identify regions in the study that have relevant information. However, that is not the end of it. After, I’ll have to navigate moderator variables and determine what effect size metric I’ll use (which is another realm of confusion). 

Although difficult, meta-analyses are highly regarded in their given fields and can highlight patterns that have significant implications. As I am still figuring it out, I have no doubt my research direction might slightly shift or change along the way, depending on what is accessible to me, but I hope to create a final product that can reveal something meaningful in the field of thought disorders and postpartum care.

Till next time, 

Yalini Kathamuthu

**To clarify, I’m not claiming that many think meta-analyses are easy. In fact, to many researchers, meta-analyses are generally not considered easy, and there are a few interesting papers about the subject, such as Berman & Parker’s 2002 paper “Meta-analysis: neither quick nor easy.” But rather, I am addressing the fact that it’s a process that seems simple on the surface, especially with the advancement of AI tools.