A Blessing or a Curse? A Causal Study on Artificial Intelligence and Job Search
Advisor: Wang
Internship Location: University of California Santa Cruz
Onsite Mentor: Yuchao "Frank" Li, University of California, Santa Cruz
We’ve all heard about the stories of artificial intelligence (AI) and the job market. However, whereas much research on the economic impacts of AI are oriented around productivity, unemployment, and wages, there is very little analysis of job search and unemployment duration. In this research project, I will exploit variation in industry AI-exposure and use monthly unemployment duration data in the United States to perform an event study and a continuous differences-in-differences analysis on the effect of ChatGPT's release on unemployment duration. I hypothesize unemployment duration to increase due to higher applicant competition and noise.
Time permitting, I will perform robustness tests and further in-depth analysis on the specific mechanisms through which AI affects unemployment duration. By employing cutting-edge econometric research to the analysis of AI’s impact on the economy, I hope to provide a greater understanding of unemployment duration, enabling colleges and companies to develop policies that increase job search and application processing efficiency.
Project Blog
Week 1: Data, The Beginning and End of All Econometrics
Context:
The general sentiment among corporations, news media, and the public is that the job market is brutal. Conventional wisdom points towards Artificial Intelligence (AI) causing these changes, replacing workers in jobs. Meanwhile, simple economic intuition seems to contradict reality, as the increased productivity from AI is supposed to increase wages and employment. Thus, while many economists have studied AI's effects on wages, job vacancies, and unemployment rates, I aim to understand its impact on an understudied aspect of the labor market: job search and unemployment duration. With research showing that long-term unemployment negatively affects health and socioeconomic outcomes, it begs the question: how has the rise of AI affected job search?
I hypothesize that the rise of AI has increased unemployment duration through two mechanisms. First, the use of AI to boost resumes and pass interviews introduces adverse selection to the hiring process, making it harder for employers to differentiate high and low-quality applicants. Second, with AI substituting jobs and creating layoffs, the applicant pool becomes much more competitive. Both force employers to dedicate more time and resources to screen applicants, decreasing job offer rates and increasing unemployment duration.
To test my hypothesis, I am using an extension of the traditional differences-in-differences (DiD) design. DiD is a quasi-experimental econometric method that statistically estimates the effect of an event by comparing changes over time between a group that is exposed to the event and a group that is not. It models a randomized control trial, deriving causation rather than correlation from observational data. Continuous DiD extends the design to multiple groups with different levels of exposure to the treatment. In my study, I will use continuous DiD to analyze the release of ChatGPT and its impact on unemployment duration across groups with different exposures to AI.
Week 1:
This week, I read Callaway et al. (2024), the paper that created continuous DiD, and familiarized myself with the statistical methodology. I also created two datasets. First, I compiled a state-level dataset of monthly unemployment rate and duration from 2019 to 2025 using data from IPUMS. Then, I created another dataset of state-level AI-exposure using Anthropic's AI Usage Index (AUI).
This differs from my original design, as I had considered using industry-level data. Unfortunately, unlike state-level data, industry-level AI-exposure indices solely estimate AI's impact on occupational tasks, ignoring its impact on job applications. However, the state-level approach isn't without flaws either. I couldn't find any pre-2022 estimation for AI usage across US states, so the data used for AUI was endogenous. This means the index was affected by changes in the labor market caused by ChatGPT, becoming a biased approximation of AI-exposure and preventing causal interpretation. After a discussion with my external mentor, I decided to perform two continuous DiD analyses, one on state-level data limited by endogeneity and one on industry-level data limited by its sole focus on occupational tasks.
Next week, I'll compile industry-level data and begin running my analyses. I will also search for an exogenous state-level AI-exposure index if possible.
Week 2: The Good, the Bad, and the Ugly of Economic Data
The Good: Data and Preliminary Results
Continuous differences-in-differences requires data for three variables: a variable of interest, a date variable, and a treatment exposure variable. Fortunately for me, I have all three of these. From the IPUMS database, I can obtain the variables of interest and the date of survey response. In accordance with our last post, I am analyzing the unemployment rate and duration. I will also analyze changes in hours worked at jobs, as AI may affect worker productivity in ways that impact how much they work rather than outright unemployment.
While IPUMS does not outright measure treatment exposure to AI, there is a bountiful number of datasets that do. Last week, I discussed how I used Anthropic’s AI Usage Index, an endogenous index constructed from Claude usage, for state-level analysis. After talking to my faculty advisor, I decided to use a proxy, specifically internet usage/accessibility, to measure AI exposure instead. Given that the functions of LLMs and the internet overlap significantly, it provides a good approximation for which states use LLMs more or less than other states.
Regarding industry and occupation-level exposure, there were several options. Acemoglu et al. (2022)’s study on AI’s impact on the labor market used three indices. While the other two indices, constructed by Webb (2020) and Brynjolfsson et al. (2022), were available, Felten et al. (2019)’s AI Occupation Exposure (AIOE) offers the most direct and flexible measurements, with variations made for industry exposure (AIIE) and geographical exposure (AIGE). However, as mentioned in the previous week, these exposure indices solely focus on changes to productivity, not incorporating changes in job search and application signalling/screening.
While preliminary tests using the endogenous AUI for AI-exposure did not reveal changes in unemployment rate or duration, using AIOE as AI-exposure, we found a statistically significant positive DiD estimator in our TWFE regression, indicating that AI may potentially be causing an increase in unemployment. Nevertheless, in the next week, I will perform a continuous differences-in-differences analysis using the available package in R to receive unbiased estimates.
The Bad: Crosswalks and Crashes
Even though data was available, that did not make working with the data easy. In fact, complications with the data popped up everywhere.
First was the crosswalks. Creating a unified dataset with AI-exposure by state was easy enough. Matching each state’s code from the IPUMS dataset to the state-level AI-exposure index was easy enough. However, when it came to occupations, it was not as easy. IPUMS classifies occupations using occupation codes (OCC), which are 4-digit codes roughly updated every decade. Meanwhile, Felten et al. (2019)’s AIOE uses the Standard Occupation Classification (SOC), a 6-digit code updated every four years, to estimate AI-exposure. Harmonizing these variables requires a crosswalk, essentially a translation book, between these two variables. Even with the help of Claude’s data cleaning abilities in Python, it took hours to troubleshoot issues with naming, matching, and compiling.
Industry-level data is worse. Not only does IPUMS industry-level data use classification codes separate from the AIIE’s North American Industry Classification System (NAICS) codes, but both classifications change frequently, making harmonization even harder. Over the next week, I will focus on harmonizing industry-level variables and troubleshooting potential issues or loss of data. I will likely only use data from 2020-2024, as that only requires me to match one version of classification codes to the NAICS classification.
Harmonization and working with crosswalks were not the only difficult parts of data cleaning and analysis. Though it may take time, the process to harmonize data remains straightforward. However, crashes, on the other hand, were harder to work with. Because I am performing my data analysis in Python on Google Colab, I have a limited amount of memory. With large, nearly decade-long monthly data with hundreds of thousands of datapoints, my code often goes above the memory capacity, crashing my program. I need to limit my datapoints and optimize my code to prevent crashes, which can take hours.
Nevertheless, my research remains promising despite these roadblocks, with preliminary results supporting my hypothesis and meaningful time spent analyzing data.
The Ugly: the Anthropic Labor Market Report
The most surprising and unexpected event to occur over the past month is probably the Anthropic labor market report. On March 5th, 2026, Anthropic released a report on how AI affects the labor market. When my external mentor sent it to me, I was surprised by how similar it was to my research. The research team used a high-exposure and a no-exposure group of occupations, constructed from their AUI, to perform a standard differences-in-differences analysis on the release of ChatGPT in 2022. They found little to no change in unemployment rates and job inflow rates. However, what surprised me wasn’t necessarily the results, but rather the methodology; it was practically identical to the one I was using.
Nevertheless, I am not discouraged, panicked, or frustrated–rather, I am glad. I am thinking along the same lines as top researchers at Anthropic, so it’s a positive signal for my pursuit of research and choice of methodology, not a negative one. Furthermore, my research is still unique, with an emphasis on job search/unemployment duration and continuous differences-in-differences. Considering that Anthropic is using AUI as their measure of AI-exposure for their analysis, their analysis faces endogeneity issues that I will not. Whether Anthropic’s labor market report will be for the better or worse may still be up in the air, but I choose to believe it will only positively benefit my research.
Week 3: O Significance, Wherefore R Thou?
The past week has been full of trial and error, coding and recoding, downloading and redownloading, all in pursuit of evidence of AI’s impact on the labor market.
A Quick Look at Cleaning and Formatting Data
By now, I was already familiar with handling crosswalks, merges, and cleaning data. I was able to use internet usage by state in 2021 from Statista as an exogenous approximation of AI-exposure for my state-level analysis. Similarly, with the help of Claude’s data cleaning skills, I used crosswalks to match the IND variable from IPUMS to the North American Industry Classification System (NAICS) used by the AI Industry Exposure (AIIE) index from Felten et al. (2019). After I saw my raw industry-level dataset with 4,096,004 observations drop to 1,175,333 observations, I converted my AI-exposure indices for both state-level and industry-level datasets into a range between 0 (unexposed) and 1 (most exposed) based on percentile. I finished cleaning my data; it was time to run statistical tests.
The Preliminary Tests: Two-way Fixed Effects and the Before-and-after Interpretation
Causal inference methods like the continuous differences-in-differences (DiD) analysis that I am using require many, often untestable and very strong, assumptions. After all, it’s very difficult to perfectly eliminate all potential confounding variables or reverse causality mechanisms that make an observed change in one variable correlate with an observed change in another variable. Thus, it’s always valuable to run preliminary tests that don’t directly observe causation but provide evidence of a relationship.
Methodology
The preliminary tests that I run are called two-way fixed effects (TWFE). TWFE provides us with the framework to analyze how outcomes change based on exposure to the treatment after controlling for fixed differences between units and time periods. To put it in simpler terms, it provides a before-and-after analysis after controlling for inherent differences between units (e.g., agriculture vs. tech) and time periods (e.g., Covid-19 impacts felt in every state/industry).
Results
When running TWFE on industry-level data for unemployment duration, unemployment rate, and hours worked per week, several interesting trends popped up. In the TWFE analysis on unemployment duration, a 1% increase in exposure to AI increases unemployment duration by 0.006881 weeks after the release of ChatGPT. This is not a large coefficient, and the p-value, 0.5754, indicates that it is not statistically significant from zero. However, tests on unemployment rates and hours worked per week showed statistically significant results. A 1% increase in exposure to AI increased the unemployment rate by 0.0334% after the release of ChatGPT. It also increased hours worked per week by 0.14474 hours after the release of ChatGPT. Both of these coefficients were statistically significant, with p-values of 0.0104 and 0.0003, respectively, indicating that these results were not due to random chance. Further TWFE tests on state-level data revealed similar results, with insignificant coefficients for unemployment duration but consistent positive coefficients for unemployment rate and hours worked.
Implications
Some of these results, like unemployment rates, align with my hypotheses. Meanwhile, others, like hours worked, showed interesting yet contradictory patterns. Interestingly enough, if the TWFE analysis of unemployment rate is true, it would contradict the same analysis conducted by Anthropic, which used their constructed AUI to perform a standard DiD between the most exposed and the least exposed industries. Unfortunately, while many of these results are significant, they cannot be interpreted causally. Before its critique, many economists used similar TWFE methods to interpret causation. Famous papers like Autor, Dorn, and Hansen (2013) have used this method to analyze the effects of imports on domestic production, wages, and employment. However, with Brantly Callaway et al. (2024)’s critique of TWFE, I used more formal methods to obtain a more direct and valid estimate of the causal effect of AI on the labor market.
Trials and Tribulations with Continuous Differences-in-differences
Methodology
Have you ever wondered why it was called differences-in-differences? The standard design only has a treated group and an untreated group. DiD requires the assumption of parallel trends. Prior to the treatment, both groups follow similar trends, providing evidence that one acts as a valid counterfactual of the other. After the treatment is applied to one group, it estimates the difference between the change after the treatment in the treated group and the change after the treatment in the untreated group. The difference between the differences is the causal effect of the treatment.
The continuous DiD design follows a similar structure with similar assumptions. However, given its continuous nature with varying levels of treatments and groups, it has a strong version of the parallel trends assumption to combat potential heterogeneity and requires more complex, non-parametric calculations to estimate the average treatment effect (ATE). Fortunately, Callaway et al. (2024) provide a package in R to perform said complex calculations.
Learning Curves
Even with a package, however, there were several hiccups along the way. The biggest of all was that I did not know the programming language R. Thus, while I have experience coding in Java and Python, I had to rely on Claude to write my code in R. But even with its help, running the analyses was difficult. The first few times I tried to run the continuous DiD package, errors continued to pop up. It took several hours for me to realize that it had to do with my data formatting, and it took several more hours to re-format, re-download, and re-run the code, only to be met with the same error over and over.
Additionally, the R package had significant data restrictions. Given that it was in beta, it could only accept one datapoint per time period per group (state or industry), and it required balanced panel data. By aggregating and eliminating data points to meet these requirements, the number of observations quickly dropped, from the millions to around 5000. Without these additional datapoints, my standard errors would quickly rise, and statistical power would drop drastically.
O Significance, Wherefore R Thou?
Callaway et al. (2024) noted that continuous DiD generally estimates larger effect sizes and greater statistically significant coefficients than TWFE. However, all of the most significant coefficients estimated with TWFE disappeared. These results can likely be attributed to the data loss from meeting the stringent data requirements and the low statistical power of non-parametric estimation. Nevertheless, it would be statistically incorrect and ethically questionable to ignore these shortcomings. Thus, over the next week, I will aim to explore how I can bypass these data requirements. I aim to increase my statistical power or seek alternative methods of estimation, like the discrete DiD design also discussed in Callaway et al. (2024) or the two-stage DiD design introduced by Gardner (2021).
Week 4: One Does Not Simply Estimate Differences (in Differences)
Last week, I began my data analysis after two weeks of collection and cleaning. The preliminary TWFE tests, which many economists have previously used as evidence of causality, revealed that the release of ChatGPT has increased the unemployment rate and hours worked, but not unemployment duration. However, the continuous differences-in-differences tests revealed no statistically significant relationship whatsoever. Given its stringent data requirements and low statistical power, I’ve sought alternatives that better satisfy relevant assumptions and provide higher statistical power. This week, I conducted analysis using discrete differences-in-differences and conducted pre-trend tests.
Discrete differences-in-differences
Discrete differences-in-differences is a lot like continuous differences-in-differences, except instead of continuous treatment exposure, there is a discrete one. To convert my continuous AI exposure indices and proxies into discrete groups, I binned them into quintiles. This means that every 20 percentiles of exposure will be grouped into one “binned” exposure. This has a few benefits and drawbacks. Given that I eventually do not find any statistically significant causal effects on hours worked and unemployment duration, I will pivot my focus to the unemployment rate, which is arguably the most important and relevant variable of interest.
You know what they say about Assumptions!
As discussed last week, continuous/multi-valued discrete DiD require the assumptions of parallel trends (PT) and strong parallel trends (SPT). I didn’t dive into the details of testing these assumptions because SPT is impossible to test, and PT could be visualized on the event-study plots. PT remained relatively unproblematic in continuous DiD, as zero remained within the confidence intervals prior to intervention. However, in multi-valued discrete DiD, due to its higher statistical power, PT becomes harder to satisfy as confidence intervals become smaller.
Fortunately, SPT becomes more plausible in a discrete setting. Binning multiple exposure groups together produces a more stable estimate of the counterfactual trend, as it averages out noise across groups rather than relying on a single exposure group's trend.
Checking for Parallel Trends
When analyzing pre-intervention trends, a visual inspection doesn’t reveal much. PT generally seems to hold across all bins at both the state-level and occupation level. Key note: I switched to occupation-level analysis to obtain a more varied measure of AI-exposure alongside more datapoints for multi-valued discrete DiD. After all, a balanced panel was no longer a data requirement.
Using Callaway’s DiD package, including a function to check for PT, I compared every level of exposure to the lowest level of exposure, my primary counterfactual. At the state-level, without the need to control COVID-19, each exposure passed the PT test except for the second-lowest exposure. This should not be a significant issue, as later on, I will show that there was no statistically significant average treatment effect for the second-lowest exposure. At the occupation-level, even after controlling for COVID-19 by removing data from 2020, there were some issues. The highest and second-highest exposure bins did not pass the PT test. This could be problematic, as it may demonstrate self-selection effects with more AI-exposed occupations inherently following a different trend than non-AI-exposed occupations. There are additional methods to address these violations of PT tests, including a method introduced by Rambachan and Roth (2023) built in a package called HonestDiD. For now, I will continue my analysis with a nuanced causal interpretation of my occupation-level analysis.
With a Shaker’s Worth of Salt
Interpreting a multi-valued discrete DiD regression is very different from interpreting a TWFE or continuous DiD regression. Instead of a single value, we get 4; one for every exposure bin (except the lowest one), each measuring the average treatment effect (ATE) at that particular exposure.
At the state-level, the ATE monotonically decreases across each exposure level, with the fourth and fifth quintiles of exposure having statistically significant (p < 0.05) increases in unemployment. At the highest exposure level, ChatGPT increased unemployment by 0.81%. Similarly, at the occupation-level, the ATE shows an overall decrease in unemployment rate, with the fourth (p < 0.05) and fifth (p < 0.01) quintiles having statistically significant increases in unemployment. At the highest exposure level, ChatGPT increased unemployment by 1.32%.
However, these estimates aren’t without their limitations. The AI-exposure index used for the state-level analysis was internet usage, an imperfect proxy for AI usage. Furthermore, the estimates only included 51 observations, as there was only one aggregated datapoint per group (in this case, fifty US states and one Washington DC). Similarly, the occupation-level analysis, though having more datapoints, did violate the PT test for the fourth and fifth quintiles, which were the only statistically significant quintiles. Thus, it is best to take these results with a grain of salt. Nevertheless, despite seeming worse than continuous DiD, these estimates have higher statistical power, more rigorous PT tests, and greater SPT validity.
The Practical and the Idealistic
At this point, I am starting to step into ethically grey areas. Am I really exploring different, similar methodologies to better satisfy assumptions and obtain higher statistical power, or am I really just trying to chase interesting results? Am I finding real relationships, or are these results driven by randomness, appearing due to the sheer number of statistical tests I am running? These are all questions that are very important to think about. I believe the answer to the first is that I am truly exploring methodologies. Though I wanted to explore unemployment duration, I dropped it when I didn’t find any interesting results. Similarly, I believe I can run discrete DiD without aggregation, increasing statistical power to ensure results are unlikely to be caused by spurious relationships. Nevertheless, I will continue asking these questions over and over again. In the following week, I will re-run multi-valued discrete DiD, test for parallel trends with the HonestDiD package, and reflect on the validity of my methodology.
Week 5: Everything Everywhere AI at Once
Last week, I attempted to run multi-valued discrete differences-in-differences (DiD). However, in my analysis of occupation-level and state-level data, I used aggregated data, which significantly diminished my t-statistics. This week, I realized I made an error when creating the econometric specification. By using interaction terms, I was able to use de-aggregated data for my analysis. To ensure my analysis is free from ethical issues, I rigorously specified my methodology and tested for assumptions prior to analysis. In the following sections, I will ensure full methodological transparency.
Multi-valued Discrete Differences-in-Differences
Data Selection
There are many ways to choose what type of data I will be using for my analysis. Previously, I had used state-level, industry-level, and occupation-level data. To measure exposure to AI, I used internet proxy, AI Industry Exposure (AIIE), and AI Occupation Exposure (AIOE) data, respectively. To remain consistent with my previous analyses, I decided to use all three for my analyses. I used monthly data from 2018-2026 for occupation-level and state-level analyses, while I used monthly data from 2020-2024 for the industry-level analysis.
Assumptions: Strong Parallel Trends and Parallel Trends
As I discussed before, the two main assumptions for multi-valued discrete DiD are strong parallel trends (SPT) and parallel trends (PT).
SPT is an untestable assumption stating that the average trends of the observed units in each exposure group is representative of the overall average treatment effect of all units if they were to be treated at that particular exposure group. Using quintiles, I am able to bin multiple units into each exposure group, making the sample more representative compared to using a single unit for each exposure group. Therefore, I believe SPT is reasonably satisfied for all three analyses.
PT is the assumption that the trend for treated groups, if left untreated, would have followed the trend of the control group. Although I do not have a true untreated control group, Callaway et al. (2024) mentions that the next logical option would be to compare it with the lowest exposed group. To test PT, I once again used the DiD package provided by Callaway to compare if each treated group’s pre-intervention trends were statistically different from the lowest exposure group.
Checking for Parallel Trends, Again
In the state-level analysis, PT held true for all exposure groups. In the occupation-level analysis, PT was violated in most exposure groups. However, after removing volatile time periods like 2020 and 2021, PT held true once again. Unfortunately, in the industry-level analysis, PT was violated even after removing time periods like 2020 and 2021. Therefore, I will not be performing multi-valued discrete DiD on industry-level data, and only on state-level and occupation-level data.
Using 95% confidence intervals, on average, parallel trends tests should report a false positive (statistically significant difference when there isn’t one) once every 20 tests. Furthermore, with low statistical power often caused by low numbers of datapoints, there is a higher probability of reporting false negatives. In my 8 tests, plausible parallel trends were reported as true in all of my tests, each with very high statistical power (>200,00 datapoints per exposure group). Therefore, the parallel trends assumption is very plausible for my analysis.
Letting the Data Speak for Itself
The following table shows the raw results for the multi-value differences-in-differences test using state-level data.
The main coefficients to note are the interaction terms, those being the ones labeled “post:q_n”. In this scenario, “q” stands for quintiles, and “n” represents the nth quintile. In the state-level analysis, the coefficients remain consistently negative across all interaction terms and generally decrease across each quintile. This indicates that unemployment increases more in more AI-exposed states after the release of ChatGPT, supporting my hypothesis. Additionally, the p-values are very statistically significant (p < 0.01) for all quintiles except the third, so this is very unlikely to be due to random chance.
The following table shows the raw results for the multi-value differences-in-differences test using occupation-level data, after removing 2020 and 2021.
In the occupation-level analysis, the coefficients remain consistently negative across all interaction terms and also generally decrease across each quintile. Once again, this indicates that unemployment increases more in more AI-exposed states after the release of ChatGPT. Additionally, the p-values are very statistically significant (p < 0.01) for the third and fifth quintiles, while being slightly statistically significant for the second and fourth quintiles (p < 0.10). Therefore, this is also very unlikely to be due to random chance.
In both of these analyses, my hypothesis is supported: ChatGPT and the use of AI substitutes physical labor, increasing unemployment. However, to further verify this result, I need to run robustness tests.
Robustness Checks: Where Chaos Turns to Order
Robustness checks are extremely important in empirical work to ensure that results are not the cause of specific specifications. For instance, a causal relationship may be identified using occupation-level and state-level analysis, but with more granular data, the relationship disappears. Similarly, a causal relationship may be identified using quintiles, but it disappears once deciles are used instead. Thus, those are exactly the questions I will ask for my robustness checks.
County-level Analysis
An alternative specification to state-level, occupation-level, and industry-level analysis is to use county-level analysis. Because it is more granular than state-level data, there may be variation in unemployment rates within counties that state-level data cannot capture. To run my county-level analyses, I went through the same process as for the state-level, occupation-level, and industry-level analyses.
Data Selection
Interestingly, I will be performing two county-level analyses: one where AI-exposure is measured by 2022 internet usage, and one where AI-exposure is measured by the AI Geographic Exposure (AIGE) index, once again produced by Felten et al. (2019). Just like internet usage vs. AIIE/AIOE for state-level and industry/occupation-level analyses, there is a tradeoff of an imperfect proxy for general AI-usage or an overly specific measure focused purely on AI task-level impacts. Once again, I used monthly data from 2018-2026 for both analyses.
Assumptions: Strong Parallel Trends and Parallel Trends
First, the SPT assumption, being untestable, is reasonably satisfied, and more so than the state-level analysis due to a greater number of counties being more representative of the average county if given a certain exposure to AI. Second, the PT assumption is tested across all quintiles for both county-level AI-exposures. Across every quintile for both analyses, results remain negative. The parallel trends assumption is plausibly satisfied, even with high statistical power (>400,000 datapoints per exposure group).
Tricky County-level Results
The following tables show the raw results for the county-level multi-value differences-in-differences tests using internet usage and AIGE for AI exposure, respectively.
Using internet usage as a proxy, the coefficients remain consistently negative across all statistically significant interaction terms and also decrease across each quintile. Once again, this indicates that unemployment increases more in more AI-exposed states after the release of ChatGPT. Additionally, the p-values for the fourth and fifth quintiles are very statistically significant (p < 0.01).
Interestingly, using AIGE instead, the results are very different. The coefficients are mostly positive instead and generally increase. This indicates that unemployment decreases in more AI-exposed states after the release of ChatGPT, opposite to my hypothesis. Furthermore, the p-values for the second and fifth quintiles are very statistically significant (p < 0.01).
These two differing results highlight exactly why it is very important to perform robustness checks. If I had only performed the county-level analysis using AIGE, I would have concluded that AI decreases unemployment. If I hadn’t performed county-level analysis using AIGE, I would have lost the nuance that comes with differing results. At this moment, the majority of tests still indicate that AI increases unemployment, but it is worth performing additional specifications to get a better picture of the data. Thus, in the next section, I will be doubling my number of analyses by using deciles instead of quintiles.
Decile Exposure Groups
Using deciles, as opposed to quantiles, has benefits and drawbacks. On one hand, I can identify greater treatment effect heterogeneity with deciles and have an, on average, lower exposure control group. On the other hand, I will have less data in each group, lowering statistical power and making SPT harder to satisfy. Once again, for industry-level, occupation-level, state-level, and both county-level analyses, I will be going through the entire process.
Data Specification
The datasets for each analysis will remain the same. The only difference is that instead of grouping AI-exposure into 5 groups based on percentiles, I will be grouping them into 10 groups instead.
Assumptions: Strong Parallel Trends and Parallel Trends
Once again, SPT is untestable. However, the same argument as before still generally stands, even though it is weaker. However, PT still needs to be tested. After performing parallel trends tests on every exposure group for every analysis, I could not perform occupation-level or industry-level analysis due to parallel trends not being plausibly satisfied. However, state-level and both county-level plausibly satisfied PT, with state-level analysis only having one exposure group testing positive. Given the number of parallel trends tests performed, this could very likely be a false positive, though to be sure, I removed data from 2020, and all exposure groups for the state-level analysis plausibly satisfied PT.
Decile Results Summary
For the sake of time and space, I will not be showing the raw results for each of these tests. However, across every specification, the coefficients remained consistently negative and also generally decreased across each quintile. Additionally, most interaction terms had p-values that were very statistically significant (p < 0.01). Surprisingly, once deciles were used in the county-level analysis with AIGE, the results changed drastically. Instead of showing that ChatGPT decreased unemployment, it showed the opposite, remaining consistent with most of my other analyses.
Everything Everywhere AI At Once
This week was full of running statistical tests. These included tests for parallel trends for each exposure group for industry-level, occupation-level, state-level, and both county-level analyses. This also included multi-valued discrete differences-in-differences tests for each dataset twice: once with quintiles, once with deciles. I critically analyzed my assumptions, specified robustness checks, and interpreted the results. Nearly every single result showed the same effect: the release of ChatGPT caused a rise in unemployment.
But while my main analysis may be mostly complete, there is still a lot more to do. First is heterogeneity. Are women more negatively affected by ChatGPT’s release than men? Do younger workers face greater risks of unemployment caused by ChatGPT than older workers? Then comes further robustness testing. Would placebo testing reveal divergent trends prior to the release of ChatGPT? What would sensitivity analysis reveal about how COVID-19 affects my analysis? These are the questions that I hope to answer in the following weeks.
Week 6: A Farewell to Stars
Last week, I was able to use multi-valued discrete differences-in-differences to estimate the average treatment effect (ATE) of the release of ChatGPT on the unemployment rate. Validating assumptions using the parallel trends test, I found strong, robust evidence to support the claim that AI caused unemployment.
This week, most of that can be thrown out the window because I forgot half a line of code.
Clustered Standard Errors
Standard errors are important to obtain the p-value and statistical significance of an estimated coefficient. A high standard error indicates that a particular sample is not a precise estimate of the true population parameter. Meanwhile, a low standard error indicates that a sample is a very precise estimate of the true population parameter. Thus, a lower standard error allows you to be more confident that, given the same estimated coefficient, the true population parameter is statistically distinguishable from zero. This provides high statistical significance.
The problem with standard errors is that they assume data points are independent and identically distributed. Sometimes that’s not true, especially in differences-in-differences where the same unit (occupation, county, state, etc.) is being resampled. Thus, clustered standard errors adjust for this, ensuring that standard errors aren’t artificially small when they shouldn’t be.
This was my mistake. All it took was half a line of code to implement. The Wikipedia page for clustered standard errors explicitly mentions differences-in-differences, too. But I didn’t realize it until it was too late. Now, when I reran my regressions, my standard errors were more accurate– and much, much larger. Very few results remained statistically significant, even at the 10% level. Hence, zero stars. My estimated ATEs were likely due to random chance, and AI never had any impact on unemployment.
Reflecting Back on the Literature
This result isn’t surprising. In fact, it is expected. Nobel prize-winning economists like Daron Acemoglu and prominent AI companies like Anthropic have found that AI has had little to no impact on the economy, from productivity to employment. Though not many papers or reports have utilized cutting-edge empirical causal inference methodology, the same point has been made abundantly clear in nearly every attempt to re-analyze the same situation. Research isn’t about finding something different; it’s about finding the truth. And maybe sometimes the truth is disappointing.
Finding Stars in an Empty Void
Even if the ATE of the release of ChatGPT on unemployment is unclear, there are still a few dimensions to explore.
The first is by exploring heterogeneity: how is the ATE different for different demographics? I briefly mentioned this last week, but I wasn’t able to explore it fully due to the change in circumstances. Maybe there is no substantial change for older workers, but there is for younger workers. Or maybe the direction that AI affects unemployment is opposite for low education workers and high education workers, averaging out to zero. All these questions are important to explore, even if they yield null results.
The second is by exploring other variables. Though I briefly explored unemployment duration and hours worked, I could take a closer look to see if the results yield anything interesting. Similarly, other variables like wages and productivity could be explored as well. This approach will border on data dredging, so I will use it as a last resort.
There is potentially another lens. Hampole et al. (2025) found that AI often substitutes physical labor, but these effects are mitigated by its effects on productivity. I believe I can differentiate the two effects by adding covariates to my DiD specification. However, I have not yet studied literature on DiD with covariates, as researched in Caetano and Callaway (2024).
Week 8: Testing, Testing, 1 2 3
Last week, I reran my primary analysis with clustered standard errors, only to find null results. This week, I explored potential heterogeneous treatment effects in my data to understand the differing impacts of AI on subgroups of the US population. In particular, I examined differences among four variables: sex, nativity, education, and age.
The 1 2 3’s of DiD
My methodology to uncover how AI affects unemployment rates differently across different demographics follows three key steps. They are similar, but slightly different, to the methodology used in my main analysis of AI’s overall effect on the unemployment rate.
Step 1 is to group the raw data into categories based on the variable of interest. For instance, sex is categorized into male and female (the dataset does not include data on non-binary individuals). Nativity is split into native-born and foreign-born. Education is split into professional/graduate, undergraduate, high school, and no degree. For age, I split the data based on common survey norms: 18-24, 25-34, 35-44, 45-54, 55-64, and 65+.
Step 2 is to validate assumptions. This means satisfying parallel trends (PT) and strong parallel trends (SPT) within each group. Just like in my main analyses, I ran parallel trends tests between each exposure level and the lowest exposure level within each particular group. While performing multiple tests poses the potential for false positives, I ensured that all tests must be negative to ensure PT is not violated. SPT, like before, is untestable, but I believe that it is once again satisfied for the same reasons.
Step 3 is to run the regression. In this scenario, I used dummy variables and interaction terms in my county-level AIGE DiD analysis to identify statistically significant differences between different subgroups in the population. Thus, the DiD estimator for each AI exposure level only estimates the ATT for the base subgroup. The base subgroups are female, native-born, no degree, and 18-24 years of age, for each variable, respectively. I will be examining the signs and estimated p-values for both the DiD estimators and the interaction terms to interpret my analysis.
Heterogeneous Treatment Effects
Sex
As mentioned before, sex will be split into male and female, with female being the base subgroup. When testing parallel trends for sex, it was plausible for every AI exposure when data from COVID-19 (2020 and 2021) was removed.
Interestingly, when analyzing the data, the DiD estimators changed from negative signs to positive ones, indicating that the ATT of AI on employment is actually positive, decreasing unemployment. The interaction terms were also positive, indicating that the negative aggregated estimates may have been the result of Simpson’s paradox. Simpson’s paradox is when a relationship in individual groups is one direction, but when pooled, it is in the other direction. Nevertheless, because none of the estimates were statistically significant, I cannot infer AI’s positive impact on employment with enough confidence.
Nativity
Just as I performed my analysis of heterogeneous treatment effects by sex, I did the same with nativity. As mentioned before, the two subgroups of interest are native-born and foreign-born. Unfortunately, when testing PT, a large portion of tests indicated that PT was not plausibly satisfied. Even after removing 2020 and 2021 from the dataset, I could not confidently claim that the PT assumption holds. Thus, I did not perform my analysis on how AI affects unemployment differently based on nativity.
Education
To analyze the heterogeneous treatment effects by education, I used interaction terms to estimate the difference in ATTs between those with no degree and those with high school degrees, undergraduate degrees, and graduate/professional degrees. Fortunately, for each of the subgroups, the PT assumption was satisfied after removing 2020 and 2021 from the dataset.
When analyzing the data, a very interesting trend comes up. While the ATT across each exposure level is not statistically significant from zero for any of the subgroups, some of the differences are. The regression shows that AI has an insignificant negative effect on employment for those without any degree. However, the estimated ATT is statistically significantly higher (p < 0.01) across all exposure levels for those with high school degrees. While both are not statistically significant from zero, the difference is. For those with undergraduate and graduate/professional degrees, this difference decreases and is no longer statistically significant.
The next question is obvious: what does this mean? Intuitively, it indicates that AI increases unemployment for those without degrees much more than for those with high school degrees. However, it is not immediately obvious why, even after I talked with my external mentor and faculty advisor. I believe that this is simply an artifact of how the unemployment rate and labor force are calculated. After all, a large portion of those without degrees are in school. They will be counted in the labor force and unemployment calculations only if they are employed. This could skew calculations of the unemployment rate and affect the estimates.
Age
My analysis for heterogeneous treatment effects by age is very similar to the one by education, in both methods and results. I used interaction terms to estimate differences in ATTs across age ranges, with ages 18-24 as my base subgroup. Testing revealed that PT holds for every age group after removing 2020 and 2021 from the dataset.
This time, when analyzing the data, the estimated ATT across each exposure level for the base subgroup was positive, but once again not statistically significant. However, the estimated ATT is statistically significantly lower (p < 0.01) for those ages 24-35, but not for any other subgroup. Just as before, it is not immediately obvious how to interpret. Both conventional wisdom and economic intuition suggest that younger individuals should be more negatively affected than older groups due to less work experience. Thus, I believe that this is also an artifact of labor force and unemployment rate calculations.
Taking a Step Back
After examining different subgroups of the US population, I do not find evidence that AI affects unemployment. Despite using methods with greater statistical power and exploring heterogeneous treatment effects, my results follow those of the literature. While there seem to be a few interesting results when analyzing different age groups and education levels, I believe it can be largely attributed to how the labor force is calculated. Though there may be alternative explanations and methods to test my conjecture, that is outside the scope of this project.
For the next few weeks, I will focus on organizing and writing my paper. I believe I can effectively demonstrate the use of cutting-edge econometric methods and share insights into the present job market. Though my results weren’t novel, my presentation doesn’t have to be as well.
Week 9: A Series of Unfortunate Events
Last week, I performed an analysis of the heterogeneous treatment effects of AI on unemployment. Despite analyzing multiple variables, like my primary analysis, the results were all null. This week, I focused on organizing my research, compiling regression results, and holistically reflecting my results in the context of existing literature and public sentiment.
How DiD we get here?
It started with public sentiment. The release of ChatGPT rocked the world and changed many dynamics. From emails to code, many tasks could be automated. Numerous companies cited AI as the reason for layoffs, and navigating the job market has become a nightmare. The rhetoric from both tech leaders and the general public seem to point towards one very simple chain of logic: AI is replacing workers.
But basic economic intuition seems to point towards the opposite direction. If the average employee can use AI to do so much more, companies will hire more workers to produce a greater profit. In the long run, even if AI does replace current jobs, new jobs will be created, and productivity will increase.
This contradiction is what makes research into how AI affects the economy so intriguing. It’s what inspired many economists to tackle the question.
Not All that Glitters is Gold
However, when many economists tried to answer the question, they all arrived at the same, confusing conclusion: it’s not doing much. Nobel prize winners like Daron Acemoglu found no detectable changes in employment and wages. Professors from top universities have contradicting claims of whether AI will increase or decrease inequality. Even total factor productivity was predicted to increase by only 0.71%. Nothing added up to the drastic claims of tech CEOs, anecdotal reports by average individuals, or reasoning provided by companies that laid off workers.
This is when I decided that I wanted to take a shot at the answer myself. I decided to use new, cutting-edge econometric methods that increase statistical power to detect the previously undetectable. I wanted to look at what conventional wisdom had to say about the heterogeneous effects of AI across different subgroups, especially across education levels and age. Despite having high hopes, my analysis corroborated the literature: no significant effects.
A Series of Unfortunate Events
Why is the job market so tough? Why does it seem that AI is affecting the economy so much? What is really happening? There are a few potential explanations, but the general consensus is relatively simple: there was simply a series of unfortunate events. ChatGPT was released during a volatile time, just a few years after COVID-19 strained supply chains and sent the world into a recession. Since then, interest rates have remained high to curb inflation at the expense of employment. Even as COVID-19 died down late 2024, new threats to the labor market began to arise. President Donald Trump’s unpredictable tariff policies have put many on edge, making hiring risky for companies, quitting risky for employees, and investing risky for investors.
So What about AI?
While AI certainly has significant potential, its lack of impact has a few explanations. First, AI may be suffering the same productivity paradox as IT in the 1970s. A mix of adjusting to technology, IT mismanagement, and other factors may be stifling growth. Second, AI’s long-term productivity effects may be offset by short-term substitution effects. As Hampole et al. (2025) found, there is causal evidence to suggest that augmentation and automation are effectively canceling each other out. Whatever the case, AI has not had a significant effect on the economy.
Week 10: You can Teach a Man to Understand Econometrics…?
Last week, I framed my research in the context of existing literature, public perception, and the political reality. This week, I focused on using that framing in the presentation for my research. More specifically, I tried to strike a balance between engagement, comprehension, and technical rigor.
Watching My Tone
Economics isn’t something most people get excited about. There’s no physical economy to touch, no visible results to see. This makes it very difficult for people to stay engaged, especially when the tone and theme of the presentation is just another white slide with bullet points. Thus, I focused on making the presentation interestingly themed. Quotes from news headlines as a hook, file-like visuals for each stage of my methodology, sticky note-like composition of my literature review. Additionally, limiting the use of bullet points in exchange for oral explanations, as well as large, in-your-face graphs, ensures that the audience doesn’t just look, but also listens. Each step was deliberate to make my presentation a little more interesting and engaging.
Seeking the "Uh-huh" Moment
Engagement from the audience isn’t everything. Just because they’re listening doesn’t mean they’re understanding. This was especially difficult considering economics is already counterintuitive, let alone econometrics. This problem was most evidently tackled in my methodology. Instead of immediately diving into multi-valued discrete differences-in-differences, I started with the simple linear regression. After a brief explanation of why correlation doesn’t mean causation, I explain how causation is actually identified through methods like differences-in-differences (DiD). That’s when I bring up Callaway et al. (2024) and continuous differences-in-differences, which extends the standard DiD framework to incorporate continuous exposures. I then explain why I chose multi-valued discrete DiD instead, being a more flexible yet equally valid approach to tackling my research question. By bringing the audience step-by-step into my research, I can share my passion for econometrics while breaking the often opaque black box of research.
Time for Me to Get a Watch
But as much as I may be passionate about bringing the audience along my research journey, 15 minutes is not enough time to discuss every little detail. Thus, I need to cut down on my time while keeping my explanations interesting and understandable. These included lengthy discussions on the assumptions of multi-valued discrete DiD, large tables of regression coefficients, and explanations for why some seemingly interesting results may not actually be interesting. Nevertheless, these topics can be brought up in the appendix as part of the QnA session if requested.
You CAN Teach a Man to Understand Econometrics!
Through a balance of engagement, comprehension, and rigor all within a tight timeframe, I believe I made my research project from something a select, knowledgeable few could watch into something everyone can. Econometrics and economics research is one of my greatest passions, and through my project, I hope to be able to share it with many others.
Week 11: Paper, Please
Last week, I focused on creating the presentation, balancing aspects of engagement, comprehension, and rigor. This week, I finished writing my paper. Unlike the presentation, I no longer faced the same time limit restraints or the need to simplify. Thus, in each section, I focused on giving as much relevant detail as possible.
Data
The data section was split into three sections: unemployment data, AI exposure indices, and summary statistics.
Unemployment Data
In the first section, unemployment data, I introduced the variables and data that I obtained from IPUMS. These include the data for the primary analysis, EMPSTAT, YEAR, MONTH, and COUNTY. This also includes the variables for the robustness tests and heterogeneous treatment effect analysis, like OCC2010, SEX, EDUC, and AGE.
AI Exposure Indices
In the second section, AI exposure indices, I introduce the variables that I used to measure AI exposure. Most importantly, this contains the AI exposure indices developed by Felten et al. (2021). I briefly explained how it was calculated; first, using a public dataset, each task was given an AI application-ability relatedness score. Then, using a weighted sum of task importance and prevalence within each occupation, the AI Occupation Exposure (AIOE) score was calculated. Finally, after using O*NET data, the AIOE scores were successfully mapped onto individual counties, creating the AI Geographic Exposure (AIGE) score. In addition to the AIGE score, I explained how I would use the Local Estimates of Internet Adoption (LEIA) by the UC Census Bureau as a robustness test.
In the last section, I gave a brief description and a table for the summary statistics of my dataset, including averages, standard deviations, quartiles, and number of observations.
Methodology
Like the data section, the methodology section was also split into three sections: identification strategy, robustness checks, and heterogeneity analysis.
Identification Strategy
In the first section, I talked about the methodology I used in my primary analysis. I introduced the methodologies proposed by Callaway et al. (2024) as an extension of the traditional differences-in-differences (DiD) design. I briefly explained the problems with TWFE as demonstrated in the paper and weighed the pros and cons of using multi-valued discrete DiD as opposed to continuous DiD. After choosing my design, I explained the exact specification I used, binning AIGE scores into deciles with the lowest exposure decile as the baseline. I also explained the assumptions of parallel trends and strong parallel trends.
Robustness Checks
In the second section, I introduced the alternative specifications I used for my robustness checks. I justified the need to utilize alternative specifications and introduced the three robustness checks I would analyze. The first was identical to the primary identification strategy, except AIGE scores were binned into quintiles instead of deciles. The second was identical to the first robustness check, but using LEIA estimates instead of AIGE scores. The last was also similar to the primary identification strategy, except that, instead of county-level data, occupation-level data was used, with AIOE being split into deciles.
Heterogeneous Treatment Effects
In the last section, I introduced the models I would use to analyze heterogeneous treatment effects (HTEs). I used a generalized model that could be applied to all three specifications for my HTE analysis. The three analyses focused on different treatment effects between sexes, educational attainment levels, and ages.
Results
The results section was split into the same three sections as the methodology section: primary analysis, robustness checks, and HTE analysis. In each of these sections, I start by explaining the results of the parallel trends tests, then explain the regression estimates.
Primary Analysis
In my primary analysis, the parallel trends tests passed after removing data from 2020. However, when analyzing the data, there were no statistically significant estimates, indicating that AI has little to no effect on the unemployment rate.
Robustness Checks
In my robustness checks, the first regression I analyzed is the AIGE quintile analysis. I found that parallel trends once again hold after removing data from 2020. However, once again, there were no statistically significant estimates, indicating that AI has no significant effect on unemployment.
The second regression I analyzed is the LEIA quintile analysis. I found that parallel trends hold only after removing data from 2020 and 2021. However, this time, the estimates showed that AI seemingly increased unemployment; however, I also mentioned that since LEIA is less direct a measurement for AI exposure, it may provide biased estimates due to measurement error. The results of the LEIA quintile regression demonstrate why robustness checks are so important; if I had only analyzed the results of the LEIA quintile regression, I would have concluded that AI causes unemployment, something that does not show up in the other regression estimates.
The last regression I examined is the occupation-level decile analysis. However, even after removing data from 2020 and 2021, the parallel trends assumption was not plausibly satisfied. Thus, I could not perform the DiD regression.
Heterogeneous Treatment Effects
In my HTE analysis, the first variable I analyzed was sex. The parallel trends tests were plausibly satisfied without the need to remove any data. However, the regression estimates did not show different treatment effects between males and females.
The second variable I analyzed was education. The parallel trends tests were plausibly satisfied only after removing data from both 2020 and 2021. The regression estimates did show statistically significant differences between the baseline no degree group and the high school degree group. However, I elaborated that due to labor force calculations not counting high schoolers without jobs in school, it was the likely cause for the statistically significant difference, not because of HTEs.
Similarly, in the third variable I analyzed, age, similar results showed up. Parallel trends passed without the need to remove any data, but the regression estimates only showed statistically significant differences between 18-24 year olds and 25-34 year olds. A similar reasoning can be used to justify the differences, as many undergraduates and high school seniors are in school, distorting labor force calculations.
Conclusion
In the conclusion, I had two sections: discussion and limitations.
Discussion
In the discussion section, I interpreted my results. Most interestingly, my null results indicated that AI did not significantly affect the unemployment rate, and it did not particularly affect any group different from other groups, or at least to an interesting degree. I provided two potential explanations for this phenomenon. The first was the repeat of the Solow paradox, where implementation lags and learning adjustments increased costs temporarily as novel technologies were introduced, preventing significant changes in productivity and thus employment. The second was a clash between automation and augmentation, as suggested by Hampole et al. (2025). Automation effects of AI substituted physical labor, increasing unemployment; however, augmentation effects of AI increased productivity, decreasing unemployment. These two effects work against each other, leading to minimal changes in the unemployment rate.
Limitations
In the limitations section, I introduced three potential avenues for improvement. The first was data; by incorporating other, alternative AI indices that directly measured AI exposure, new estimates could be derived that may paint a better picture of the full story. Additionally, waiting for newer data to be released in the upcoming years could lead to analysis of the long-term treatment effect as the implementation lags, learning adjustments, and automation effects become less prevalent.
The second avenue of improvement was analyzing causal mechanisms. I was able to estimate the effect of AI on unemployment, but not provide empirical evidence to back the “why”. This could be improved in future studies by using empirical or mathematical models that analyze the two potential explanations I proposed.
The last avenue of improvement was using alternative methodologies. I relied on using ChatGPT as an exogenous shock that significantly affected the use of AI in the industry. However, developments in AI over time, both before and after the release of ChatGPT, may be better analyzed using other methods, like instrumental variable analysis, albeit with their own tradeoffs.
Introduction
Looping back to the introduction, I began my discussion on my paper by briefly discussing the public opinion on artificial intelligence. Using a poll by the Pew Research Center, I cited growing concerns about AI becoming more commonplace in our daily lives. Then, I transitioned into the literature review, explaining existing research on the effects of AI on the economy.
Funneling into more labor market-specific papers, I briefly explain their estimates of AI’s impact on employment and provide my own explanation for how their methodologies had room for improvement. Interestingly, many papers did not use causal inference, or at least used causal inference methodologies that suffered from endogeneity issues. Additionally, I also explain how very few studies have analyzed HTEs. Thus, I was able to explain how I would improve upon the literature by using a new, more accurate empirical strategy and explore the HTEs of AI on unemployment.
Week 12: A Retrospective on the Senior Project
This is the last week that I spent working on my senior project. I finished up slides, polished my script, practiced, practiced, and practiced. I will spare you the details, but my final project slides are linked here. The recording of my presentation is here:
A Rollercoaster of Emotions
First off, I would like to thank my faculty advisor, Dr. Wang, and my external mentor, Frank Li. Over the past 12 weeks, I have felt a rollercoaster of emotions. From anxiety to joy to confusion to frustration, every week was different, and they helped me navigate it all. I still remember the joy I felt when my regressions showed statistically significant estimates. However, I also remembered the dread I felt when I realized I had to cluster my standard errors, making my estimates worthless. Every week, it felt like there was either a pleasant surprise or I made a devastating mistake. You can see it on my blog. Without them, I would still be lost.
While I have some research experience before the senior project, none has ever been as polarizing as this one. In my previous projects, one used linear regression, and the other used instrumental variables; everything went my way. In this project, however, I had to pivot my research question, reconsider my methodology, and clean and wrangle millions of datapoints. It was a project at a much larger scale, with much more guidance, than anything that I’ve done before. I faced many issues, but that also meant I had many great learning opportunities. And learn I did.
Learning Through Doing
Throughout this project, there were many aspects of economic research that I encountered for the first time. There were data cleaning problems, like calculating the labor force, using crosswalks, and making raw data usable. There were methodological ones, like learning how two-way fixed effects are biased, how continuous differences-in-differences (DiD) estimated parameters, and when to use clustered standard errors.
But what I learned the most from this project was the process of failure in research and how to adapt to it. Parents, teachers, and mentors have repeatedly told me over and over again that research fails. Like a professor and former mentor of mine told me, if academic literature is a huge circle, my contribution would only be a dot. Of course, this was relatively obvious; revolutionizing entire fields is what’s awarded to Nobel Prize winners, not high schoolers. In fact, most research doesn’t even make any meaningful contribution. However, even though I knew this fact, I hadn’t felt it. This was, of course, largely in part due to my limited experience with academic research; however, even in the papers I wrote, I hadn’t had an inconclusive result or an impossible hurdle.
A Rocky Road to Research
This time, it was different. It felt like roadblocks were everywhere. Why do Felten, Raj, and Seamans (2021) use NAICS codes while IPUMS uses SOC codes? Why do I have to have a balanced panel to run continuous DiD? Should I pivot to focus on the unemployment rate rather than unemployment duration? Why do I have to cluster my standard errors? Why did the two-way fixed effects estimate statistically significant effects, while the discrete DiD finds close to none? Would a logit model have provided more accurate results? How should I handle multiple hypothesis testing? Why do the lowest subpopulations of age (18-24 year-olds) and education (non-degreed) seem to be so different from the next subpopulation (25-34 year-olds and high school degree holders)? How do I interpret the results of this one specification that doesn't match up with the rest? Why doesn’t AI affect employment?
I am incredibly thankful for the guidance that my faculty advisor and external mentor provided for me. Each question forced me to think about my next steps carefully and justify every decision I made. It was a humbling experience. I learned that I could pivot to a derivative of the original research question when the original one doesn’t work out; I realized that results sometimes just aren’t novel to the literature, even if the methodology is; I discovered that each model has its issues, and I just had to work with them. But I also learned that that’s alright. It’s part of the research process. There will always be failures and headaches, feelings of doubt and futility, but at the end of it all, there will be that dot added to the academic literature, that one piece of previously unknown knowledge that you discovered. And that’s the satisfaction of research.
My Journey Through Economic Research: Econometrics
There is another aspect of my senior project that, despite being talked about in every blog post, I still feel like I didn’t do justice to: econometrics. I never really explained what econometrics was, but it’s pretty easily deducible. Econometrics is the use of statistics in economics. But I don’t feel like that definition does it justice. I first discovered econometrics in 10th grade, as I was studying for the National Economics Challenge. At the time, I was fresh out of AP economics, and I was unfamiliar with how it worked, so I skimmed past it. Simultaneously, I had built up a collection of socio-political-economic questions. After all, I loved political philosophy at the time, and I read books like the Communist Manifesto. However, I also only had one method to try and answer them: speculation.
But entering into 11th grade, I learned AP Statistics. I studied the basics of hypothesis testing, linear regression, and p-values. It was also in that class that the same age-old idea of “correlation, not causation” was drilled into me, as it had been in all the previous math classes I had taken. Essentially, only experiments could yield causal results; observations could only yield predictive ones. However, near the end of the school year, I rediscovered econometrics and finally understood how statistical tests worked. What I found changed my understanding of economics.
The Beauty of Econometrics
The vast majority of econometrics was dominated by causal inference. The idea of deriving causal estimates and conclusions from observational data. The methods were more complex than a simple linear regression, a “line of best fit”, but they were far from incomprehensible. Instrumental variables, differences-in-differences, regression discontinuity, interrupted time series, the list goes on and on. Though not immediately, I realized the implications that econometrics had. It contained the toolkit to analyze and answer nearly every question I had at the time, from gun control to minimum wage to taxation. Being a student interested in socio-political-economic questions with a skillset that fits squarely in STEM, I grew a passion for econometrics.
Through what I could only chalk up to divine coincidence, I was given the option to take an econometrics capstone course (it was almost cancelled!) in my senior year. At the time, I had relied on reading Wikipedia over and over again until I understood the concept, but through the course, I was able to formally develop my knowledge in the field. Having a more rigorous and nuanced understanding of econometrics, the only thing left was to use it. The perfect opportunity came in the senior project, where I analyzed how AI had affected employment.
And Now, It's My Turn
The senior project was the second time I used econometrics to answer a question. My econometrics capstone project was the first, but the senior project was the first time I engaged with econometric theory. Econometric theory is a subfield of economic research that focuses on developing the statistical tools to answer economic questions. Oftentimes, it developed new methods or extended existing methods of causal inference to handle different kinds of data, different kinds of assumptions, or different kinds of questions. In my senior project, this came in the form of continuous DiD and multi-valued discrete DiD, both extensions of the traditional DiD design.
Through this senior project, I was able to not just learn to apply and understand econometric theory, but also share it with others. Not many people understand how economists conduct research, nor what questions they answer. Although my project addressed a highly relevant question, I found myself more engaged with the methodology rather than the question itself. With the help of my faculty advisor and external mentors, I was finally able to shed some light on what I find to be the most intriguing part of economics.
Feelings and Facts
Speaking of the research question, my exploration of how AI affected the job market also taught me a lot about the economy. I never made it explicit what my hypothesis was, so I figure I'd do so now. I hypothesized that AI increased unemployment, consistent with the narrative from news media, tech CEOs, and personal anecdotes. After all, I thought that previous methods simply suffered from lower statistical power and lacked true causal estimation, which would be fixed by my methodology. It was not. I've spoken about my thoughts and interpretations at length in my week 9 blog post, but nevertheless, it surprised me. Thankfully, my external mentor had made it clear that, whatever the case, I always needed to remain transparent. I could not put my feelings above the facts. If I kept searching for statistically significant effects, I would be p-hacking, a serious violation of research integrity.
I learned the hard way that not everything needs to be novel. Even though my results corroborated most other studies, it still adds to the literature in other ways. A new methodology was used to provide less biased causal estimates. Heterogeneity analysis indicates AI isn't affecting any particular subgroup more or less. Changes in unemployment caused by AI still aren't felt in the economy, even after 4 years. Even if nothing else, I was able to bring a better understanding of the job market to my audience.
So, What Now?
Now that my project is done, the obvious question would be, what's next? The fun part of the answer is that this project will be the beginning of a long journey. Going into college, I look forward to pursuing research and graduate school. This project engages with three different subfields of economic research: macroeconomics, applied microeconomics, and econometric theory. Thus, I will use it as a stepping stone to continue exploring these three subfields, all of which use econometrics. The senior project has already provided me with invaluable research experience, which will help tremendously when finding research opportunities. Eventually, I may decide to tackle this question again, but with a larger toolkit and much more knowledge of econometrics.
Thank you all for joining me.