Last week, I was able to use multi-valued discrete differences-in-differences to estimate the average treatment effect (ATE) of the release of ChatGPT on the unemployment rate. Validating assumptions using the parallel trends test, I found strong, robust evidence to support the claim that AI caused unemployment.

This week, most of that can be thrown out the window because I forgot half a line of code.

Clustered Standard Errors

Standard errors are important to obtain the p-value and statistical significance of an estimated coefficient. A high standard error indicates that a particular sample is not a precise estimate of the true population parameter. Meanwhile, a low standard error indicates that a sample is a very precise estimate of the true population parameter. Thus, a lower standard error allows you to be more confident that, given the same estimated coefficient, the true population parameter is statistically distinguishable from zero. This provides high statistical significance.

The problem with standard errors is that they assume data points are independent and identically distributed. Sometimes that’s not true, especially in differences-in-differences where the same unit (occupation, county, state, etc.) is being resampled. Thus, clustered standard errors adjust for this, ensuring that standard errors aren’t artificially small when they shouldn’t be.

This was my mistake. All it took was half a line of code to implement. The Wikipedia page for clustered standard errors explicitly mentions differences-in-differences, too. But I didn’t realize it until it was too late. Now, when I reran my regressions, my standard errors were more accurate– and much, much larger. Very few results remained statistically significant, even at the 10% level. Hence, zero stars. My estimated ATEs were likely due to random chance, and AI never had any impact on unemployment.

Reflecting Back on the Literature

This result isn’t surprising. In fact, it is expected. Nobel prize-winning economists like Daron Acemoglu and prominent AI companies like Anthropic have found that AI has had little to no impact on the economy, from productivity to employment. Though not many papers or reports have utilized cutting-edge empirical causal inference methodology, the same point has been made abundantly clear in nearly every attempt to re-analyze the same situation. Research isn’t about finding something different; it’s about finding the truth. And maybe sometimes the truth is disappointing.

Finding Stars in an Empty Void

Even if the ATE of the release of ChatGPT on unemployment is unclear, there are still a few dimensions to explore. 

The first is by exploring heterogeneity: how is the ATE different for different demographics? I briefly mentioned this last week, but I wasn’t able to explore it fully due to the change in circumstances. Maybe there is no substantial change for older workers, but there is for younger workers. Or maybe the direction that AI affects unemployment is opposite for low education workers and high education workers, averaging out to zero. All these questions are important to explore, even if they yield null results.

The second is by exploring other variables. Though I briefly explored unemployment duration and hours worked, I could take a closer look to see if the results yield anything interesting. Similarly, other variables like wages and productivity could be explored as well. This approach will border on data dredging, so I will use it as a last resort.

There is potentially another lens. Hampole et al. (2025) found that AI often substitutes physical labor, but these effects are mitigated by its effects on productivity. I believe I can differentiate the two effects by adding covariates to my DiD specification. However, I have not yet studied literature on DiD with covariates, as researched in Caetano and Callaway (2024).