Week 5: Sentiment Analysis

Hello everyone, and welcome back to my blog! This week, I analyzed how sentiment changes across both X and Reddit. By using sentiment analysis, I measured how positive or negative discussions are within each community and compared patterns across topics.

How it was conducted

To calculate these scores, I used the VADER (Valence Aware Dictionary and Sentiment Reasoner) model. VADER works well for varying lengths of text and for detecting nuance in language like sarcasm and intensity. 

First, each comment in the dataset was passed through the sentiment analyzer, which calculated a sentiment score that ranged from -1 (most negative) to +1 (most positive). I then normalized these scores to vary from 0 to 1 since I will be making all three of the scores I will be using stay in that range as well. Next, I calculated the average sentiment score across all comments in each community. Finally I compared the scores across both dataset to find patterns in how sentiment varies. 

Sentiment Scores for X

Political Campaigns and Elections: 0.982575

International Relations and Strategy: 0.719739

National Security and Defense: 0.259813

Education and Teaching: 0.658222

Research and Scientific Development: 0.563059

Economic Policy and Finance: 0.995929

Social Issues and Race Relations: 0.629837

Media, Communication, and Public Discourse: 0.820205

Overall, this dataset had a very wide range of sentiment values, from 0.259813 to 0.995929. Political Campaigns and Elections and Economic Policy and Finance were very positive while National Security and Defense had a very low score of 0.259813. The rest of the communities tended to still stay in a relatively positive range as well.

Sentiment Scores for Reddit

Private Prisons: 0.218447

Tax Policy: 0.412983

Healthcare Policy: 0.361275

Donald Trump Policies: 0.287640

Economic Inequality: 0.254918

Iran Sanctions: 0.302771

Religious Freedom: 0.446512

Middle East Policy: 0.334905

Abortion Access: 0.201389

These results show that sentiment in the Reddit dataset is consistently more negative with all values being between 0.201389 and 0.446512. Topics like Abortion Access and Private Prisons had very low sentiment. Even the most positive topics, like Religious Freedom and Tax Policy, were still much lower than the highest scores on X. 

Overall, this analysis shows that sentiment is also greatly affected by where the discussion takes place. Aspects like context and topic specificity can also influence sentiment scores as well.

Next week, I will calculate Network Isolation for each of the communities, which will finally prove which groups really are echo chambers and which ones aren’t.

Thank you for reading, and I will see you all next week!

Harish