Building an MLOps Platform for Predicting Opinion Shifts
Advisor: Mr. Dievendorf
Internship Location: Stanford University, 616 Jane Stanford Way, Stanford, CA 94305 (Hybrid)
Onsite Mentor: Mr. Samuel Tong
Deliberative discussions play a critical role in shaping public opinion, yet current machine-learning models for predicting opinion change remain difficult to reproduce, scale, and compare across studies. Although prior research has shown that transformer-based and neural network architectures can capture linguistic and behavioral signals associated with opinion shifts, these studies are typically limited by incomplete data pipelines, inconsistent preprocessing, and a lack of standardized experimental infrastructure. This project proposes and implements a cloud-based Machine Learning Operations (MLOps) platform designed to automate, scale, and improve the reproducibility of machine-learning experiments for predicting opinion shifts within deliberative conversations. Using large-scale deliberation datasets provided by the Deliberative Democracy Lab, the platform integrates data preprocessing, feature engineering, text-embedding storage, model training, evaluation, and version tracking into a unified and automated pipeline built on Amazon SageMaker and DynamoDB. Structured features are managed through SageMaker Feature Store, while high-dimensional textual embeddings are stored and queried using a custom vector database architecture. PyTorch-based models, including neural and transformer-based architectures, are trained and evaluated through SageMaker Pipelines. The platform quantitatively evaluates model performance using accuracy, stability, and robustness metrics, supported by cross-validation and bootstrapping to mitigate overfitting and small-sample bias. Results are benchmarked against prior deliberation-prediction studies to assess whether automated MLOps workflows improve both predictive performance and experimental reproducibility. The final outcome is an MLOps system and a research paper that demonstrates how scalable infrastructure can quantitatively advance the study of opinion dynamics and cognitive science in structured dialogue.
Project Blog
Week 0: Entering a World of Opinions
Ask someone why people change their minds, and you will get answers ranging from “good arguments” to “group pressure” to “emotions.” Ask how to measure that change, and things suddenly get much more complicated.
Public opinion shapes elections, policy, and social movements. However, the tools we use to study opinion change often lag behind the complexity of the phenomenon itself. Over the past decade, researchers have turned to machine learning (ML) and natural language processing (NLP) to analyze deliberative discussions, structured conversations where people debate, reflect, and sometimes change their views regarding difficult and complex issues. These models are powerful, but there is a major problem: they are hard to reproduce, hard to compare, and even harder to scale.
Welcome to my Senior Project! My name is Rishi Gupta, and over the next ten weeks, I will be stepping beyond individual models and into the infrastructure that supports them. My project, “Building an MLOps Platform for Predicting Opinion Shifts,” focuses on designing a system that makes machine-learning research on deliberation more automated, reproducible, and scalable.
The main question driving this work is simple but ambitious: How can we build machine-learning systems that don’t just produce results, but produce results in an efficient manner that we can trust, repeat, and extend?
I will be working with real deliberation datasets provided by the Deliberative Democracy Lab, combining conversation transcripts with pre- and post-discussion survey data to predict opinion shifts. I already have experience with analyzing such datasets, but this time, I have a new goal in mind. Instead of treating this project as an isolated experiment, my goal is to build an MLOps platform that can handle data preprocessing, feature storage, model training, evaluation, and version tracking. This will be conducted in such a way that allows future researchers to plug in new models and data without rebuilding the entire system from scratch.
Now for the big question that everyone always asks. Why does this even matter?
In fields like computational social science, models are often impressive but fragile. Small datasets, inconsistent preprocessing, and undocumented pipelines make it difficult to tell whether improvements come from better modeling, different experimental choices, or just the specific dataset itself. By focusing on infrastructure rather than just algorithms, this project prioritizes reproducibility throughout the research process, making it easier to compare results across studies and move the field forward in a more systematic way.
This blog will serve as my research log and reflection space. I will document the technical challenges of building scalable ML pipelines, the tradeoffs between model performance and stability, and the insights gained from applying engineering principles to social science research. Expect honest discussions of what works and what doesn’t, as well as reflections on what it means to study human behavior with computational tools.
Opinion change is messy. Human conversations are nuanced, emotional, and unpredictable. My goal is not to oversimplify that complexity, but to build systems robust enough to study it responsibly.
Excited for the project to begin!
Week 1: Starting with Data Preparation
Hi everyone, welcome back! This is the first official week of the project! When planning this project out, I decided to dedicate the first two weeks to something that doesn’t always get much attention in machine learning: the data. Before any models can be trained or evaluated, the datasets have to be cleaned, structured, and prepared in a suitable way for the models to use. This is typically overlooked because models are usually the center of attention in such research projects. Additionally, working with data can be very annoying, as it involves a lot of tedious work.
I have been working with deliberation data that combines conversation transcripts with pre- and post-discussion survey responses. While this sounds straightforward at first, working with real data quickly makes things more complicated. Transcripts need to be cleaned without losing important context, survey responses have to be consistently formatted and matched to participants, and small inconsistencies can easily carry through the rest of the pipeline if they aren’t handled early. I have currently taken a first stab at this data-cleaning process by understanding the data and determining what I truly want to keep. The upcoming week will involve actually removing the data I don’t need so that I am left with a final dataset that can be used for training.
Alongside this, I have been reading Hands-On Machine Learning with Scikit-Learn and TensorFlow, which has been helpful in putting these early steps into context. One idea that comes up repeatedly in the book is how much impact data preparation has on the final results of a model. This is something I have already seen myself, especially from other machine-learning projects I have worked on. Many issues that look like modeling problems actually come from decisions made during data preprocessing. The reading has also reinforced the idea that preprocessing isn’t just a setup step, but a design choice. Decisions about how to handle missing values or represent survey scales need to be made carefully if results are going to be reliable and comparable.
Overall, this first week has been more about understanding the data and involved a lot of reading. It has made clear how much careful data preparation matters for reliable results. With this foundation in place, week 2 will focus on turning these decisions into code by cleaning and refining the datasets for training and experimentation. See you then!
Week 2: Structuring the Pipeline
Hi everyone, welcome back! This week involved a mix of reading and reconsidering how the technical workflow should actually be structured.
One of the main sources I worked through was Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation by Jez Humble and David Farley. Even though this book is about software engineering, many of its concepts apply surprisingly well to machine learning systems. One of the concepts that was presented in the book is the value of structured pipelines. Basically, they are systems where processes move through clearly defined stages that can be repeated consistently. It is important to keep in mind that for this project, instead of running tasks manually each time, the goal is to build automated processes that make development more consistent and easier to manage and maintain.
This book also made me reconsider how I originally organized my project timeline. Initially, I had separated the work into two distinct stages: I would prepare the data first, and then I would construct the modeling infrastructure. However, as I thought more about how real machine-learning systems and MLOps pipelines operate, I realized that these steps are much more tightly connected than I initially thought. In practice, data preparation, preprocessing, model training, and evaluation all exist within the same pipeline, where each stage feeds directly into the next. Due to this, decisions made during data preparation don’t just affect the model itself, but also how the entire system processes, tracks, and reproduces experiments.
Therefore, I updated my project syllabus slightly. Rather than treating data preparation and model development as completely separate stages, the early part of the project now focuses more on understanding and designing the machine-learning pipeline that will connect these stages. This includes planning how data preprocessing, model training, and evaluation will interact within the overall system before fully implementing each component. This shift better reflects how real MLOps systems integrate these stages into a single pipeline. The detailed cleaning of the transcripts and survey data will now happen alongside building the model infrastructure that will ultimately use that data.
On a technical level, I am still focusing heavily on the data itself, but I also started thinking more about the modeling process and how it will be integrated into the overall architecture. Specifically, I started designing preprocessing pipelines that will convert the raw deliberation data into structured inputs that the models can use later. Although these pipelines are not developed as of right now, designing them early ensures that the data flows smoothly into the rest of the system.
This week was more about planning and architecture rather than coding, which helped clarify how the various elements of the project will be incorporated with each other. For week 3, I will proceed to clean the data as I develop the machine-learning pipeline infrastructure. See you next week!
Week 3: Data Preparation
Hi everyone, welcome back! This week focused on preparing the dataset so it can be reliably used for analysis later in the project. The goal was to take a large and messy collection of survey and participant files and transform them into a cleaner, more organized dataset.
The first step was selecting the right source data. Instead of starting with the raw survey export, we used the cleaned version of the main survey file. The raw export contained formatting issues such as broken characters and inconsistent text, which would have made the workflow unnecessarily complicated. Starting with the cleaned version provided a much more stable foundation.
Next, the column structure of the dataset was reorganized. The original survey export used the full question text as column names, which made the dataset difficult to read and difficult to work with in code. To address this, the columns were renamed using shorter and more consistent labels while still preserving the survey structure. For example, the names still indicate whether a question came from the pre-survey or the post-survey, making the dataset much easier to navigate and use programmatically.
We also removed columns that were clearly not useful for analysis. The survey export included several administrative fields and sign-up information that would not help with modeling. Removing these fields simplified the dataset while keeping the information that is actually relevant to the project. At the same time, many topic-related survey questions were kept so we can later decide whether they should be used to train the model.
Another important decision was to avoid over-cleaning the survey responses. Most responses were left in their original form so that the dataset retains as much of the original information as possible. Blank responses were kept blank and text responses were preserved. The only time temporary numeric values were introduced was when calculating the prediction target, which measures the difference between a participant’s pre-survey and post-survey answers. When necessary for this calculation, missing responses were temporarily represented by the benchmark midpoint value of 5.5.
Finally, the survey data was connected to the discussion session information. Using participant assignment files and transcript metadata, participants were mapped to their session, room, display name, and transcript user ID. This makes it possible to link survey responses with discussion transcripts later in the project.
Overall, this week focused on creating a cleaner and more consistent dataset while preserving as much useful information as possible for future analysis. Next week, the project will move into the model-building stage. See you then!
Week 4: Structuring Data for Modeling
Hi everyone, welcome back! This week focused on building the internal structure required to make the dataset consistent and usable within the pipeline. Last week was focused on preparing the data, so the next step was to formalize the manner in which the dataset is organized and interpreted by the model.
A key part of this process was creating a system to standardize column names. The original survey data has long and inconsistent headers, which makes it hard to work with in code. To handle this, a mapping system was created to convert the raw column names to a more organized and clean system. This system will give a standardized name to each column based on its role in the dataset. For example, survey questions are categorized into pre-survey and post-survey, while demographic information is stored separately. Columns that are not useful for the model and analysis, such as administrative or duplicate fields, are marked to be removed. This creates a consistent schema that makes the dataset easier to understand and use throughout the pipeline.
The other valuable element that was developed this week was a systematic method of dealing with categorical data. Many surveys contain categorical responses, such as multiple-choice answers, and the dataset I am working with is no different. Instead of relying on automatic encoding, which is prone to numerous errors, a mapping system was created to specify the valid categories for each field and the order in which they should be encoded. This allows for categorical variables to be represented consistently and preserves the meaning of the original responses.
With these systems combined, they bridge the gap between the raw dataset and the structured format needed for modeling and analysis. This is the phase at which the data preparation process starts to have a direct impact on the model, because these mapping decisions define how the data is organized, and ultimately, what the model sees as the input. The division of column standardization and categorical encoding makes the workflow more transparent and easier to maintain.
Overall, this week was devoted to the creation of the foundational components that will guarantee consistency across the dataset while also shaping how the data will be used by the model. Next week, the project will move further into the stages of model building. See you then!
Week 5: Designing the Model Architecture
Hi everyone, welcome back! This week was the start of the model-building phase of the project. Having spent the last several weeks organizing the dataset and defining the structure of the pipeline, I shifted my attention toward the architecture that will be used for training and experimentation.
The main focus this week was designing how the model workflow would be structured. Instead of considering the model as a single script, I began to think of it as part of a larger system made up of separate stages. This is a significant advantage of MLOps, since the project is not just about training one model once, but about creating a framework through which various modeling decisions can be experimented with in a controlled and repeatable manner.
One of the key concepts I worked on was separating stable pipeline logic from experimental modeling choices. In practice, this means creating an architecture where the fixed parts of the workflow remain consistent, while specific choices can still be changed and tested. For example, the system should make it possible to try different transformation strategies or transcript embedding methods without having to redesign the entire pipeline each time. This form of modularity is particularly valuable in machine learning experiments, as even minor design choices can have a major effect on results.
I also started planning how the model will interact with the rest of the infrastructure. Instead of treating inputs as something passed directly into a notebook, I am designing the architecture so that processed outputs can move cleanly between stages and be stored in cloud systems when needed. This will make the workflow more scalable and easier to reuse as the project grows. In other words, the model is being designed as one part of a broader MLOps system rather than as an isolated piece of code.
This week was interesting because even though the actual training process has not fully begun yet, it laid the foundation for how model experimentation will be conducted moving forward. The work was less about results and more about making sure the system is set up in a way that supports repeated testing, comparison, and improvement.
Overall, this week focused on defining the architecture that will shape how the model is trained and evaluated. Next week, the project will move further into experimentation. See you then!
Week 6: Building the Modeling Pipeline
Hi everyone, welcome back! This week focused on moving from architectural planning to actual implementation. The main goal was to begin building a fully connected pipeline that links preprocessing, feature engineering, model training, and evaluation into a single system.
A major component of this work was setting up the SageMaker Feature Store. The Feature Store acts as a centralized system for storing and managing features used in model training. Instead of recomputing features each time a model is trained, the pipeline now generates structured feature datasets that can be stored and reused. This not only improves efficiency but also allows for consistency across experiments, since the same feature definitions are used each time. Additionally, storing features in this way makes it easier to track how different feature sets impact model performance.
With the Feature Store in place, I began implementing the pipeline. This connects the preprocessing stage, transformation logic, model training, and evaluation into a continuous workflow. Rather than running each step manually, the pipeline is designed so that outputs from one stage automatically feed into the next. This allows experiments to be executed in a more reproducible and scalable way, which is essential for comparing different modeling strategies.
Another key focus was integrating feature engineering directly into the pipeline. Instead of treating feature creation as a separate step, transformations such as encoding categorical variables, handling missing values, and constructing target variables are now embedded within the workflow. This allows for any modifications to feature engineering to be applied to all experiments. It also makes it easier to test different configurations, since transformations can be modified without restructuring the entire system.
Finally, I began the initial process of training baseline models to establish initial performance benchmarks. I have started looking into models such as Logistic Regression and Random Forest, which are relatively simple but provide a useful reference for evaluating future improvements. Running these initial experiments helped verify that the pipeline is functioning correctly and that data is flowing properly through each stage of the system.
Overall, this week focused on the transition from system design to execution. Next week, the focus will be on refining model performance and exploring more advanced architectures. See you then!
Week 7: Model Training and Experimentation
Hi everyone, welcome back! The main goal of this week was to scale up the modeling process by training more models and initiating more structured experimentation. With the pipeline now fully connected, the goal was to explore how different modeling approaches and feature configurations impact model performance.
A major part of this week was training and comparing multiple models. In addition to the baseline models from last week, I started testing more sophisticated models like Support Vector Machines (SVM), XGBoost, and Neural Networks. Each of these models captures different types of patterns in the data, so comparing their performance provides insight into which modeling strategies are the most effective and which are not. Running these models within the pipeline also helped make sure that the system can be scaled to a range of architectures without any significant modifications.
I also spent time experimenting with different feature sets and transcript embedding strategies. Since the data includes both structured survey responses and transcript data, the way this information is represented matters a lot. I researched variations in which features are included, as well as different ways in which the transcript embeddings might be incorporated into the model. Even small changes here can have a major effect on the results, so this part was more about testing and observing what actually makes a difference.
Along with model and feature experimentation, I started doing hyperparameter tuning within the pipeline. Instead of adjusting parameters manually, the pipeline is programmed to test and try various settings in a more systematic way. This allows the analysis of a combination of parameters in a more controlled and reproducible manner. By integrating hyperparameter tuning directly into the workflow, the system can efficiently search for better-performing model configurations.
Overall, this week was devoted to the transition from the initial model training to more sophisticated experimentation. Combining multiple models, different feature choices, and parameter tuning gave a better sense of how these decisions impact model performance. With this experimentation framework in place, the next step is to analyze the results more deeply and figure out which approaches are worth dedicating more time to improving further.
Next week, the model will focus on evaluating model performance and comparing results more systematically. See you then!
Week 8: Model Evaluation and Refinement
Hi everyone, welcome back! This week was originally planned as a buffer period to either catch up on modeling or get an early start on the research paper. However, since the pipeline and initial experiments are now in place, I decided to fully dedicate this week to evaluating and refining the models.
The primary goal was to better evaluate the models in a more rigorous, systematic, and reliable fashion. Rather than just splitting the data into training and testing datasets, I began to use methods such as cross-validation to look at the performance of the models on various subsets of the original data. This really helps to avoid overfitting, one of the most common problems with machine learning models. It also gives a much more accurate picture of how well the models generalize, which is basically how well the models perform on unseen data, and not simply just on the data they were trained on.
Apart from cross-validation, I also worked on robustness testing. This helps to determine how sensitive the models are to changes in the data and configuration. In other words, by making slight changes to the inputs and observing changes in performance, it is easier to tell which models are robust and which are more prone to changes in data and configuration.
Another crucial aspect of this week was to expand the number of evaluation metrics. Instead of simply relying on only accuracy, precision, and recall, I experimented with some other metrics to better capture different aspects of the models’ performance. I also started to think more about stability and generalization as key factors, because a model that performs well on average but is not stable across multiple runs is very unreliable. As for the specific metrics, I will give myself some extra time next week to experiment with and finalize them so they are ready to be included in the paper.
Based on the evaluations I conducted this week, I started refining both preprocessing choices and model configurations. This included adjusting feature selections, revisiting how certain variables are handled, and modifying model parameters to improve performance. The goal was not just to find the best-performing model, but to understand why certain approaches work better than others.
Overall, this week was focused on moving from experimentation to deeper analysis. Instead of just running models, the emphasis shifted toward interpreting results and improving the system based on evidence. With a clearer understanding of model behavior, the next step will be to finalize results and begin structuring the findings for the research paper. See you next week!
Week 9: Final Analysis and Starting the Research Paper
Hi everyone, welcome back! This week marked a shift from model development into analysis and documentation. With most of the experimentation now complete, I started focusing on understanding the results and beginning the research paper.
Much of this week was devoted to comparisons of model performance. I examined the performance of the more advanced models compared to the baseline models of the previous weeks, as well as looked at the comparison of the results with those of previous studies. This helped put the results into context and made it easier to realize whether the approaches I was using were actually producing any meaningful improvements.
I also took time to analyze the effect of the various preprocessing and feature engineering options on the results. Because the pipeline was created to evaluate various settings and multiple configurations, I was able to compare how changes in feature selection, handling of missing values, and transcript embeddings influenced model performance. In some cases, small changes in how the data was represented led to noticeable differences in results, which reinforced how important these early design decisions are.
Another major area of focus this week was to interpret the findings more closely. Instead of simply gazing at measurements, I began to think about what the results really signify in the context of opinion change and deliberation. As an example, I thought about which aspects appear to have the most predictive value and how the various modeling procedures respond to the arrangement of the data. This aspect of the process was more qualitative, but it was useful to connect the technical results back to the initial research question.
Alongside the analysis, I began drafting and writing the research paper. I started organizing the overall structure, outlining key sections, and drafting parts of the methodology and results. The goal was to clearly depict and explain both the modeling process and the reasoning behind different design choices.
Overall, this week was about moving from experimentation to explanation. As the final week approaches, the focus has now shifted toward understanding what the results show and how to communicate them effectively. With the main analysis now underway, the next step will be to continue writing and refining the paper. See you next week!
Week 10: Final Analysis and Project Completion
Hi everyone, welcome back! This week focused on completing the final stages of the project and bringing together the experimentation, analysis, and overall findings.
A major part of the week was finishing the final research paper. I refined the methodology, results, discussion, and conclusion sections while making sure the paper clearly explained both the technical workflow and the reasoning behind the modeling decisions. I also finalized the experiment summaries, tables, and evaluation metrics so the different models and feature configurations could be compared more clearly.
In addition, I reviewed the overall pipeline and workflow developed throughout the project. Looking back at the earlier stages, it became clear how much preprocessing, feature engineering, and evaluation decisions influenced the final model behavior and results.
Overall, this week marked the completion of the project. Looking back on the project as a whole, one of the biggest takeaways was realizing how much machine-learning performance depends not only on the models themselves but also on preprocessing, feature engineering, evaluation strategy, and overall workflow design. Throughout the project, I gained a much deeper understanding of how reproducible machine-learning experiments are structured and how difficult it can be to model something as complex as human opinion change. The experience provided valuable insight into both computational social science research and the broader process of designing and evaluating machine-learning systems. Thanks for following along throughout the project!