Can Sentiment Analysis Models Increase the Accuracy of Machine Learning Based Stock Forecasting?
Advisor: Swetha Bhattacharya
Onsite Mentor: Jaya Jaware
In this project, I aim to determine if the tone of news articles heavily influences the trends of technology stock market prices. As the stock market changes everyday and is a national market, all data pertaining to it will be purely online. Therefore, I will utilize online datasets, such as NASDAQ and use websites such as Yahoo Finance to gain more information about the market. Additionally, since both my statistical models are online, I will conduct online research and read computer science websites, such as Kaggle and Stack Overflow, to learn how to create and modify my models. Since online news is more common than paper news, I will be using online news outlets, such as The Washington Post and The New York Times as the ones that I am analyzing. Finally, I aim to create a Powerpoint Presentation to present at the Senior Project Presentations as a final product, presenting my findings. If time permits, I want to consolidate my research in the form of a research paper and publish it, however, for now, my main goal is to make a presentation of my findings.
Project Blog
Week 1: Welcome to my Senior Project!
Hello everybody and welcome to my Senior Project! Over the next few months, I will try to determine how much news articles affect the trends of the stock market. My plan is to take stock market data from March 1, 2023 to September 1, 2023 and try to predict stock market prices for the next 6 months using the ARIMA model. Then, I will take news articles (an external influence) from the next 6 months (September 1, 2023 to March 1, 2024) and determine if they can explain the discrepancies between the ARIMA model’s predictions and the real life stock market prices by conducting sentiment analysis on the news. Today, I’ll talk about the background and inspiration behind my project.
If you invest in the stock market, you’ve probably read at least a couple articles on Yahoo Finance or CNN to try to predict which companies to put your money into. There are many theories and observations that the news has some sort of impact on the stock prices, but there isn't any definite research stating the exact effects that these articles have on the market. There are also many different types of news articles that could have varying effects (ex. positive/negative earnings reports, mergers/acquisitions, scandals). In this project, I will focus on 7 stocks known as the “Magnificent 7” that are said to have the biggest influence on the stock market. These stocks are Google (GOOG), Amazon (AMZN), Apple (APPL), Meta (META) , Microsoft (MSFT), Nvidia (NVDA), and Tesla (TSLA).
First, I will create a picture of what the market for my chosen stocks would look like if their prices fluctuated in a mathematical manner. To do so, I will use a statistical model to predict stock prices solely based on the numerical data inputted into it. The ARIMA model (Auto Regressive Integrated Moving Average) is a machine learning model that uses time series data to learn from past datasets to understand and predict future trends. Research has shown that the ARIMA model is one of the best for forecasting stock prices/trends. It has three main qualities that allow it to do so. First, it can look at and learn past trends. Second, it can factor in changes and patterns in the data into its predictions. Third, it has a moving average to consider error terms. However, the stock market is extremely volatile and may therefore unexpectedly change very quickly, a trend which the model may not be able to predict based on past data. This is expected because my results from the ARIMA model will serve as a mathematical prediction of the stock market eliminating the effect of outside factors (which is not the case in real life).
Next, I will analyze news articles using a common algorithm to determine what kind of effect they’ll have on the market. Sentiment analysis is the process of analyzing digital text to determine if the tone is positive, negative, or neutral. Sentiment analysis has been proven a very reliable way to classify words based on their tone as linguists tend to agree 85% of the time (giving the model an 85% average accuracy rate). This shows that it is a much more effective method of word classification than other models, such as AutoML. There are multiple ways to conduct sentiment analysis (from asking LLMs such as Chat GPT to classify words/news headlines to creating your own sentiment analysis model. For the purposes of ensuring my model is accurate, I will create my own to conduct this experiment. I will refer to various articles and research papers such as "News Sentiment Analysis" by Samuels and Mcgonical to create my model, as theirs has a high accuracy rate and has proven to be reliable for the purpose that I need it for. After determining whether the news articles I analyze are positively, negatively, or neutrally skewed, I can make conclusions on whether or not they can account for the discrepancy between the predicted versus real life stock prices.
Many people, including myself and my family, look to the news as the primary source for financial information. However, sometimes the news may not represent the full picture and may even mislead readers who utilize these articles to make their decisions. I want to determine whether primary financial news outlets are really the best method for learning about corporate actions and decisions or whether people should look to other sources instead. I hope that by conducting this project, I am able to help people make better financial decisions by informing them how influential the news is on stock prices. Stay tuned to discover what I find out!
Week 2: Its all about the Data
Hello everybody and welcome back to my blog! This week, I’ve been trying to compile the data needed as input into my ARIMA model. If you remember from my last post, the ARIMA model represents stock forecasts without external influences such as news articles. In this post, I’ll talk about the process of preprocessing and visualizing my training data to prepare for the ARIMA model to run forecasting algorithms.
Acquiring Datasets
My plan at the beginning of this project was to acquire the daily opening stock prices of the 7 companies I chose (Meta, Google, Nvidia, Tesla, Amazon, Apple, Microsoft) from March 1, 2023 to September 1, 2023. I was able to find this information on Yahoo Finance and was able to apply filters such as my desired start and end dates as well as the increment at which I wanted the data. After setting my desired preferences and preprocessing the data, I realized that I only had around 129 data points as there were many dates, such as weekends and holidays, during which the stock market was closed. I wanted many more data points to give the model, as the more training data/data points it has to learn from, the better the model’s predictions will be. This is because having more data allows the model to detect patterns, rather than just one time occurrences which may be the case with fewer data points. Therefore, I decided to expand my time frame for training data to September 1, 2022 to September 1, 2023. This gave me 252 data points which was sufficient for inputting into the ARIMA model. An example data set (Google) that I used from Yahoo Finance looked like this:
Preprocessing Data
As shown above, the data from Yahoo Finance had much unnecessary information and was also formatted in a way that the model would not understand. There were also days such as when dividend payments were made, which I had to eliminate because I did not want those values inputted into the model. Additionally, I only wanted the date and the opening price of each stock. Therefore, I had to eliminate columns such as the highest/lowest price, closing/adjusted closing price, and the stock volume. To do this, I imported the dataset into Google Sheets and manually altered it to fit the requirements of the ARIMA model. At the end, I was left with two columns: the dates and opening prices. The preprocessed dataset for Google looked like this:
Visualizing Data
This week, I executed part of the ARIMA model by creating a graph of the data values I collected over the past few days. To do so, I imported two Python libraries, pandas and matplotlib. First, I called pandas, specifically the read_csv and the datetime functions to read the input dataset. Then, I called matplotlib to graph the values read by the program. I set the x-axis to be the date and the y-axis to be the stock price. The graph for the price of Google’s stocks that I forecasted by my ARIMA model is shown right above the actual charts for Google's stock. As the model only needed to read a spreadsheet, the accuracy is very high, and the two charts look almost exactly the same.
As shown, Google’s price has fluctuated from September 2022 to September 2023. It dipped towards the end of 2022 to under 90 dollars per share, but rose throughout 2023 and was valued at over 130 dollars per share in September 2023.
My internal advisor, Mrs. Bhattacharya suggested that I should look at smaller companies in comparison to the larger ones that I use in my project. Therefore, I decided to look into some smaller startups that recently went public. I chose to look at the stock of Sharps Technology, which is a technology company that went public in early 2022. I compiled and graphed the stock prices for Sharps Technology, which is shown below:
Upon graphing this data, I immediately saw differences between this graph and those of bigger companies, such as Google. The biggest was that Sharps Technology’s stock fluctuated much more than that of Google’s. Some days had extremely large spikes while others had large drops. On the contrary, Google’s stock remained relatively stable. I think that it will be much easier for the ARIMA model, when making predictions, to be able to forecast values based on a relatively stable company that is in its mature growth phase. After September 2023, Sharks Technology’s stock was unfortunately unable to recover from the drop they faced till date. The ARIMA model definitely does not have the capacity to predict this, as the previous data suggests that the stock price will come back up. Therefore, while it will be interesting to see how the model treats both datasets, I hypothesize that it will be able to better predict the stock prices of larger companies as their datasets are easier to make patterns of.
And that’s it for Week 2 of my Senior Project! Thank you so much for reading and following along on my journey so far and I’ll see you guys next week.
Week 3: You win some you lose some
Hello everybody and welcome back to Week 3 of my Senior Project blog! This week, I tried to run my ARIMA model using my data from last week and accumulate my “predictions” of stock prices with no external influence. Unfortunately, I could not get my ARIMA model to work and output forecasts, but in this week’s post, I’ll detail the process I went through trying to run this statistical model.
First, I wanted to run the model on Google Colab, and I was able to run part of it (remember the images from last week where I visualized my data). However, when I tried to run my forecasting function, the platform kept crashing and returning the following error message:
In other words, I had two options - either pay for Colab Pro or find another way to run my program.
After doing some research, I found that I could use my laptop’s terminal to run larger Python programs, so that’s what I decided to do. However, I had never used my terminal to run programs, so I didn’t know how to run this model. After installing Python to run the programs and Anaconda to deploy the program, my laptop’s terminal was able to run the program, however, many error messages still displayed. The ARIMA model that I was making had a running time of 20 minutes. Let me explain how it works. First, the model reads the CSV file and outputs a graph of the visualized data. However, after it outputs a graph of the closed data, it waits for the user’s input to close the graph as a sign to continue. Then, it analyzes the data and creates a forecast of stock prices and outputs a graph for the forecasts. But guess what, I had no idea that the program WAITS for the user to close the graph. Wednesday night, when I first ran this program at 1:00 in the morning, after spending the entire day figuring out how to run a Python program on my laptop’s terminal, I was overjoyed to see that first graph - a visualization of my data. “Okay, this is gonna work this time,” I thought. I patiently waited for 20 minutes to pass, expecting to see a second graph with my forecasts, but that never happened. 20 minutes passed….then 40….and suddenly it was past 2:00 in the morning. I decided to go to sleep and check back again the next morning, and to my surprise, the model was still running 10 hours later! How can a model that takes 20 minutes to run still be running after 10 hours!?!?! At this point, I was a mixture of confusion, stress, worry, and could not figure out what was wrong with my code. At this point, I was trying to run my model on multiple devices, in hopes that one would work. Then, Thursday afternoon, my dad walks into my room, looks at my code, and simply states “Oh, you have to close the first graph.”
Now, I had a model that was finally working, but if you thought it would be smooth sailing from there, you are wrong. Now that I had finally found a way to run my model, I could finally deal with the myriad of errors that arose when I tried predicting stock prices for these companies. In my program, I utilize a module known as datetime. This module can essentially manipulate the date/time of the data I enter. For example, it can read my data points (which are a list of dates - essentially just numbers) and determine when a new month or year starts. This helps greatly with graphing data, as this module can organize data by its date or time. It was an essential part of my forecasting model because it helped me set the increments at which I wanted to forecast stock prices (daily). Well, when I ran my model, it turns out, this module was deprecated. This meant that it no longer existed in newer versions of Python. From here, I could either reprogram my model with a different module to support the newer Python versions, or I could figure out how to get access to an older version of Python.
I decided to use my sister’s old laptop to run my program. This laptop is very old, and does not have a working battery, so I couldn’t disconnect it from power. And by this point, you know the drill. It had an older version of Python installed, so I downloaded my dataset and program and ran it with that. While the program was able to run, it provided me with a brand new list of errors that looked something like this:
Let me explain this picture. On the left is the date that I am trying to predict (or at least some of them as I couldn’t fit everything in one screenshot. On the right is supposed to be a forecasted price, but instead it says NaN, which is an error code, which is definitely not supposed to happen.
What exactly does this mean and how do I fix it? I guess you’ll have to wait until next week to find out!
Week 4: Never back down never what?
Never give up. Hello everybody and welcome back to Week 4 of my Senior Project! This week, I spent a lot of time focusing on trying to make my ARIMA model work after the error messages displayed last week.
For reference, here are the error messages that I encountered last time:
My model had multiple errors in the code. First, this model was based off of one that was used to predict the value of cryptocurrency, which updates every day. However, the stock market is very different. The stock market is closed on weekends and holidays, which is something that I didn’t take into consideration when making the dataset. Therefore, when making predictions, I asked the model to use a daily increment, but it returned error messages when it reached Fridays, as my dataset went straight to Monday from Friday (and skipped Saturday and Sunday). Additionally, I had no idea which format the model was storing predictions in (whether as fractions, decimals, or scientific notation), so that could have also been one reason why it was returning the error message of “Not a Number.” To fix both of these problems, I created a Python program called “dataset modifier” that parsed through my dataset, identified any missing days, and filled it with the last active stock price. For example, if the opening price of a stock on Friday was $101.11 and the opening price on Monday was $105.21, the program would automatically fill in the stock price for Saturday and Sunday to be $101.11. It then created a new dataset of two columns - one with the date (with no missing values this time) and one with the stock price truncated to two decimals so I could control the format of the stock prices. After doing this, I had 8 new datasets with no missing values and stock prices with only two decimal values. Finally, I had a logical error in my program where it was forecasting values on a weekly basis, however this was an easy fix as I just changed the code to forecast stock values on a daily basis like I wanted.
After making those changes to my data and program, I ran the model. It returned its forecasts for my 8 companies (Amazon, Apple, Google, Meta, Microsoft, Nvidia, Sharps Technologies, Tesla). The results are shown below:
While I plan to run accuracy tests next week, I was able to make some observations on the forecasts. First, I was surprised to see that the model was able to predict the decline in the price of the stocks of Sharps Technology. While it couldn’t get the exact price right, the model was able to show that the stock wouldn’t be able to come back up. Small companies’ stock prices are definitely much more volatile, but I hypothesize that the accuracy of the forecast for Sharp’s Technologies won’t be too far off from the accuracy of that of the more stable companies. For the other seven, the ARIMA model was definitely able to predict the overall trend of the stocks, except for Google, Amazon, and Tesla. For these three stocks, the ARIMA model predicted a much larger gain in their price than what happened in reality, and their ending price is almost $100 off from the stock price in reality. However, that is what my project hopes to discover - why did the actual stock price deviate from this forecast? These results were a mathematical prediction of stock prices, so it showed what these stocks would look like without any external influence.
Next week, I hope to run accuracy tests to get a quantitative idea of how much these prices deviated from stock prices in real life. After that I plan to start collecting news articles (my external influence) and determine whether or not they can account for the difference between these forecasts and the real prices. Thank you so much for reading and I’ll see you guys next time!
Week 5: Can I Predict the Future?
Hello everybody and welcome back to my Senior Project! Before I get into my progress for the week, I would like to quickly revisit my timeline that I created at the beginning of this project. Before starting, I made a 12 week timeline with goals that I wanted to achieve at the end of each week. My timeline is below:
Week 1: Research
Week 2: Research
Week 3: Compile stock market data
Week 4: Preprocess stock market data
Week 5: Run stock market data through the ARIMA model
Week 6: Accuracy Tests
Week 7: Compile news articles
Week 8: Build Sentiment Analysis model
Week 9: Preprocess Sentiment Analysis Data
Week 10: Accuracy Tests
Week 11: Conclusion
Week 12: Finish Final Product
Now that I am almost halfway through the project, I decided to revisit my timeline and modify it based on what I’ve done so far. My updated timeline is below:
Week 1: Research
Week 2: Preprocess and Compile stock Market Data
Week 3: Develop ARIMA model
Week 4: Continue Developing ARIMA model and run data through it
Week 5: Accuracy Tests
Week 6: Compile news articles for Sentiment Analysis Data
Week 7: Preprocess Sentiment Analysis data
Week 8: Develop Sentiment Analysis Model
Week 9: Develop Sentiment Analysis Model
Week 10: Accuracy Tests
Week 11: Create Presentation Slides + Rehearse
Week 12: Present at Senior Project Showcase
This new modified timeline uses less time for compiling and processing data (as I realized it doesn’t take much time) and more time for developing the statistical models (as they output many error messages which take much time to fix).
Now, let’s talk about what I did this week. My goal for the week was to run accuracy tests on my ARIMA model to figure out how accurately the model was able to forecast stock prices. Now remember, I inputted data from September 1, 2022 to September 1, 2023 and asked the model to predict the prices from September 1, 2023 to March 1, 2024. Going into this week, I did not expect the accuracy to be very high because the ARIMA model was supposed to represent stock forecasts without any external influence. I ran this model on 8 different companies and surprisingly, the model was more accurate than I thought it would be. The first thing I did was create a graph so I could visually conclude how close/far the model’s predictions were. The graphs for all companies are shown below:
As shown from these graphs, the model was pretty good at predicting the price for Apple, Google, Microsoft, Meta, and surprisingly not very far off from its Sharps Technology forecast. It was not very accurate for Tesla, Amazon, and Nvidia. However, while these graphs were helpful, I wanted a quantitative way to know exactly how far off these predictions were.
I ran two tests on each ARIMA model to find the Mean Absolute Error (MAE) and the Mean Percentage Error (MPE). The MAE is the value by which each prediction that the ARIMA model makes deviates from the actual stock price. The MPE is the percentage by which the prediction deviates from the actual stock price. The MAE and MPE for each company is listed below:
AMAZON:
Mean Absolute Error (MAE): 38.477605580118656
Mean Percentage Error (MPE): 27.31%
APPLE:
Mean Absolute Error (MAE): 23.59470828609305
Mean Percentage Error (MPE): 13.01%
GOOGLE:
Mean Absolute Error (MAE): 6.76478021978044
Mean Percentage Error (MPE): 3.27%
META:
Mean Absolute Error (MAE): 27.357669922490018
Mean Percentage Error (MPE): 5.14%
MICROSOFT:
Mean Absolute Error (MAE): 17.253761822453814
Mean Percentage Error (MPE): -1.16%
NVIDIA:
Mean Absolute Error (MAE): 145.22725207136597
Mean Percentage Error (MPE): 31.58%
SHARPS TECHNOLOGIES:
Mean Absolute Error (MAE): 0.1772688493753724
Mean Percentage Error (MPE): 47.82%
TESLA:
Mean Absolute Error (MAE): 120.094505494509
Mean Percentage Error (MPE): 54.04%
As shown, the model was pretty accurate for Microsoft, Meta, and Google as their mean percentage error was below 10% which means on average the model’s predictions deviated from the actual values by less than 10%. This was very surprising to me because I did not expect the model to be that accurate (since it was only a mathematical forecast). For Microsoft, the ARIMA model only deviated from the actual value by 1% (the MPE is negative because the actual values were higher than the predicted values on average). It was not very accurate for Tesla and Sharps technologies, however this was expected. Sharps Technologies is a startup, so its stock price is very volatile. Additionally, there have been many issues with Tesla’s stock and it has declined very rapidly, so much that since the start of this project, it has left the “Magnificent 7.” If you remember, these stocks were the ones that had the biggest shares in the market which is why I chose to look into them. However, around a week ago, Broadcom replaced Tesla as one of these Magnificent 7 stocks, which wasn’t something the ARIMA model could have predicted. It wasn’t very accurate for Amazon and Nvidia, however, those results were around what I expected (for the model to be between 60-80% accurate) so that I could further investigate whether or not the news could have helped its accuracy.
Overall, I think the results of this ARIMA model were pretty surprising and more accurate than I would have expected, given that there was no external influence on these prices. Would I recommend someone to utilize it to make real life financial decisions? The answer is no. While these results weren’t bad, I still don’t know how much external factors influence these prices. I definitely also did not input enough training data and the time frame for my predictions was very small, so I don’t know how the model will perform in the longer run. However, while the model may not be ideal for real life predictions, it was still very interesting to see how it was able to act with the stock prices that I inputted. Next, I plan to start researching and compiling news articles about these companies to see how they influence these predictions that my model gave. Thank you guys so much for reading and I’ll see you next week!
Week 6: Reading the News
Hello everybody and welcome back to Week 6 of my Senior Project! This week, my goal was to compile news articles for my Sentiment Analysis model. My objective for this project is to figure out how much the news influences the trend of stock market prices. In order to do this, I have to find news articles that match the increments of my forecasts, which are daily.
At first, I just performed a normal Google search for my first company, Amazon (AMZN). However, when I just searched for “News about Amazon Stock Prices” my results were a mix of various news articles from April 2024 and news outlets that looked like this:
However, I wanted news articles that were published in the past because in the real world, a person making financial decisions about whether to invest in September 2023 would obviously look at news articles published around that time. Therefore, upon further research, I figured out that I can filter Google Search results based on the date that articles were published. By typing in this input into Google “Amazon (NASDAQ: AMZN) after:2023-09-01 before:2023-09-02” (news articles that mention the stock of amazon published between September 1, 2023 and September 2, 2023), I was able to narrow down my search results to this:
All these results were published in the time frame that I wanted. For the purposes of this project, I decided to compile data based on the assumption that people would just read the top few articles. Therefore, I decided to take the top three articles from each day and create a dataset with the titles of these articles. If I do this for each day, I’ll have almost 500 articles over the span of the 6 months that I am going to look into per company. News articles can be lengthy, so the model might not be able to handle analyzing 500 full news articles, which is why I chose to use only the titles. I repeated this process for the remaining 7 companies, and created datasets containing the news titles and dates that next week I will spend time preprocessing and getting ready for my Sentiment Analysis model. Thank you so much for reading and I’ll see you next week!
Week 7: Creating Datasets
Hello everybody and welcome back to Week 7 of my Senior Project! This week, I spent most of my time compiling the titles of news articles to put into my Sentiment Analysis model. Unlike my first model (ARIMA), I couldn’t find a dataset online to fit my needs for this project. The dataset I need has to be the news titles of all news articles on the company that I am analyzing on a daily basis. Therefore, I had to create my own datasets. In order to do so, I created 2 columns on my Google Sheets, labeled “Date” and “News Article.” The format for the date was the same as the ARIMA model since I knew the datetime module could read that format. Again, I just decided to use the titles of the news articles because I have over 500 articles per company that I am analyzing (since I’m taking the top 3 from each day). The average length of a news article is around 1,000 words, so if I did the full article, it would be over half a million words for the model to analyze, which I don’t think my laptop can handle. The datasets looked like this (I used Amazon as an example):
Now obviously, there aren’t gonna be 3 news articles for every single company every single day. There were days where I could only find one or two, or sometimes even none at all. For these days I just put a “0” because 0 is a neutral number. When the sentiment analysis model analyzes the news article, it will give the article a score ranging from -1 to 1, -1 being negative and 1 being positive. By assigning a value of 0, that is entirely neutral, which I feel best represents the case in real life. If there are no news articles published, then the opinion of the public can’t be changed.
Before this week, I definitely underestimated how much time it would take me to compile these datasets. Looking on the internet for over 500 news articles and manually copy pasting them into my dataset was definitely extremely time consuming. While I wasn’t able to finish all 8 datasets by today, I definitely want to finish them by Monday morning so I can start developing my sentiment analysis model. One of the datasets that I didn’t get to was Sharp’s Technologies. However, I think this is definitely going to be the one with the most 0’s in my data sheet. All the other 7 companies that I am analyzing are extremely big, and therefore have many news articles published on them every single day. Until I did much research on stocks that went public in 2022, I had never even heard the name “Sharp’s Technology.” I hypothesize that it’s definitely going to be much harder to make a correlation between the stock price of Sharp’s Technology and the influence of news articles than other companies simply because there’s very few news articles on smaller companies. To learn if this hypothesis is true or not, you’ll have to wait until next week to find out!
Week 8: Positive or Negative?
Hello everybody, and welcome back to Week 8 of my Senior Project! This week, I did many different things, but in summary, I finished creating the datasets and developed my sentiment analysis model. If you remember from last week, I had started creating my datasets. This week, I was able to continue working on the same datasets and create the rest of them. As predicted, the dataset for all the news articles with Sharp’s Technology had the most 0’s (which means there weren’t many news articles on this company).
After creating all of my datasets, I needed to develop my sentiment analysis model. However, I realized that in order to compare the results of my sentiment analysis model to the results of my ARIMA model, I would need to save the results of my ARIMA model in a dataset. Therefore, I decided to take a step back and go back to my ARIMA model to save all the forecasts in a dataset. The process for doing that was pretty simple: I just had to add one command to create a new dataset with two columns (titled “Date” and “Forecast”) and store all the values accordingly. However, I did get some error messages as when the ARIMA model tried to graph the data, now the column titles were getting in the way so it couldn’t understand how to graph “Date” and “Forecast” since they weren’t real numbers (obviously). This wasn’t a difficult problem to solve, though, since I just removed the dataset titles and manually added them into the file after the program was done running. An example of the data (from Amazon is shown below):
Next, it was time to develop my sentiment analysis model. From past projects, I had a lot of experience with the NLTK library, which uses the SentimentIntensityAnalyzer and the VADER lexicon to assign a piece of text a sentiment score. Using this algorithm, I was able to assign each news article a sentiment score (from the scale of -1 being the most negative to 1 being the most positive). However, there was still a pretty big discrepancy between the results of my ARIMA model and the results of my sentiment analysis model. My ARIMA model had one data point for each date (which was the daily forecast) but my sentiment analysis model had three (the sentiment score for each news article that day). Therefore, I decided in order to keep just one data point per day, to take the average of the sentiment scores of all three news articles and keep that as the sentiment score of news for that day. Finally, I wanted a qualitative measure of whether the news was positive/negative, not a quantitative score, so I added a conditional statement where if the sentiment score was above 0, it was positive, if it was below then negative, and if it was 0 then neutral. The final sentiment analysis results (for Amazon) looked like this:
Currently, I am in the process of running the sentiment analysis algorithm on each of my companies. While this data can definitely be used for comparison to see the effect news articles have on stock prices, there is one drawback to this sentiment analysis model: I don’t really have a way of measuring how accurate it is. When designing the ARIMA model, I was able to take real life stock prices from September 1, 2023 to March 1, 2024 (the time frame I was analyzing) and compare them to the forecasted prices to determine how accurate the forecasts were. However, since I manually created the dataset for the news article, nobody has used it before me and conducted sentiment analysis on it, so I don’t have any test data to verify the results with. I want to spend the coming week running the sentiment analysis algorithm on the remaining companies and researching how I can find the accuracy of this model (if there is even a way to do so). But will I be able to do so? Stay tuned until next week to find out!
Week 10: Sentiment Analysis
Hi everybody and welcome to my Senior Project! This week, I continued running sentiment analysis on the datasets that I created to output whether the news article titles were positive or negative or neutral. One dataset in particular that was different from the others was Sharps Technology.
As expected, Sharps Technology had almost no news articles pertaining to it. In fact, throughout the entire 6 months, I could only find less than 10 articles, and there were multiple months where there weren't any news articles at all pertaining to the company, so I just had to put a “0” for each day. As a result, the sentiment analysis results for Sharps Technologies looks like this:
As compared to the results for a bigger company (Apple) which looks like this:
The results from the two companies varied drastically. First of all, since there was barely any input data for the sentiment analysis algorithm from Sharps Technologies, it of course returned almost all neutral results. However, since I was able to find at least 3 news articles every single day regarding Apple’s stock, the sentiment analysis algorithm had much more input data and therefore returned a variety of results from positive to negative to neutral.
I had originally hypothesized that the tone of the news articles might be able to influence the stock price of larger companies. For example, if the media is flooded with negative news about Amazon then people would be less inclined to buy the stock, and maybe even more inclined to sell it causing a decline in the price. However, this wouldn’t be the case for the stock prices of smaller companies, like Sharps Technologies. The further along I progress in this project, the more my hypothesis seems to be supported. There are almost no news articles regarding Sharps Technologies, so the sentiment analysis results are extremely stable, with almost all of them stating “neutral” however from the graphs of the stock prices, we can clearly see that they fluctuate a lot. The sentiment analysis model results definitely won’t make up for the discrepancy between the forecasted and real stock prices of Sharps Technologies.
The next thing that I have to do is figure out the relationship between the results from my two statistical models. I plan to do this by comparing whether the stock price of a certain company went up/down to whether the tone of the news articles that day were positive/negative. How exactly will I create this program? Stay tuned until next week to find out!
Week 11: The Last Mile
Hello everybody and welcome back to Week 11 of my Senior Project! This week, I finished running my datasets through my sentiment analysis model and started on the final part of my project. As I mentioned before, I could definitely see a big difference between the results from the larger companies versus Sharps Technologies. For the larger companies, the dataset had much variation, between “Positive”, “Negative”, and “Neutral.” However, since I was unable to find data pertaining to Sharps Technology for most days, the model didn’t have much input and by default assigned each day “Neutral”.
Now, I had three datasets per company: one with my ARIMA model’s forecasting stock market values, one with the real stock market prices, and one with the sentiment of the news per day. The last step in this project is to now connect them. This week, I also spent time developing a program that would determine whether or not the tone of the news can account for the differences between the trends of the forecasted stock market prices or the real ones. Essentially, it would figure out the percentage of times where the tone of the news articles was able to account for the difference between the direction that the forecasted stock was moving versus the real stock.
In order to do this, I first had to import all three datasets into the program. Then, I had to figure out the trend of the stock prices in the forecasted prices dataset and actual prices. To do this, I parsed through the datasets, and if the stock price went up that day, I assigned it a value of 1 and if it went down, then I assigned it a value of 0. I did this for both the forecasted values dataset and the actual values one. Then, I parsed through the sentiment analysis results and wrote the following conditional:
This conditional essentially says that if the sentiment result is negative, the forecasted stock price goes up but the actual price goes down, then return 1. Additionally, if the sentiment result is positive, the forecasted stock price goes down but the actual stock price goes up, then return 1. Otherwise, return 0. In this situation, the value 1 means the news has influenced the stock prices and 0 means it didn’t.
In the coming week, I hope to run all of my datasets through this program to figure out how much these news articles can explain for the difference in the predicted versus actual stock prices and finally answer the question I had at the beginning of this project: How does the underlying tone of news articles affect the trends of the stock market? The end is near, and I can’t wait to see the results of this project. Stay tuned for next week to see what happens!
Week 12: The Final Post
Hello everybody and welcome to my final blog post! Before I get started, I just want to say thank you to everybody who has been reading my blog and following along with my project. It has been an incredible experience for me to see an idea for a project that I had just six months ago come to life throughout the past 12 weeks and I truly appreciate your support and for being part of this journey with me.
The final results from my project showed the percentage of times that the news’ sentiment impacted the price of the stock market. The output is as follows:
Amazon - 22.53%
Apple - 10.99%
Google - 14.84%
Meta - 13.74%
Microsoft - 13.74%
Nvidia - 11.54%
Tesla - 15.38%
Sharp’s Technology - 0.01%
As shown, this correlation was definitely much lower than I expected. The highest was Amazon, which was still below 25%. I definitely recognize that there are many factors affecting this output, such as time. News articles might be published one day, but that doesn’t necessarily mean they’ll impact the stock market immediately; they might take 4,5, or even 6 days to do so. While my project did not take into account how time impacts the stock market, it is definitely something that I want to look into in the future. Another interesting aspect that I could research more about is the difference in various news outlets and articles. While conducting this project, I noticed a lot of differences between various news articles including but not limited to their popularity, frequency of posting, and average article length. There are also many differences between various news articles such as whether they are fact or opinion based that my project didn’t look into as well. This is definitely something that I want to further research, as “news” is a very broad term and I would definitely like to narrow it down to see if different types of news have differing effects on the market.
After finishing this project, I presented it at the Senior Project Showcase in front of my classmates, parents, teachers, and more. A video of my presentation is attached below, if anybody was unable to attend and would like to watch:
Additionally, I know that the slides are a little bit difficult to see in the video, so I have also attached a link to the slides that I used below:
https://docs.google.com/presentation/d/1qLkM0U6RhrRBx5zBy5qZc5ZEcCK5IddUSZon5-svuGQ/edit?usp=sharing
Finally, to end this project, I developed my final product which was a Github Repository of all my programs, including both statistical models, that I developed throughout the course of this project. The link to the Github is attached below: