Designing an Algorithmic Framework for Biomarker-Based Disease Detection
Internship Location: Lawrence Berkelley Natural lab (Remote)
Onsite Mentor: Charles Lively
With a growing emphasis on non-invasive methods to detect diseases with molecular biomarkers comes a greater demand to find more efficient ways to select primers for singleplex and multiplex assay formulation. While primer selection tools like IDTPrimerQuest and APE alleviate the strain of manually searching for primers, there are numerous requirements for primer selection depending on the disease or target biomarker that these tools do not take into consideration. Further, with the increasing relevance of machine learning in bioinformatics, I hope to explore the potential of applying advanced computing techniques like machine learning and predictive models in primer selection to identify optimal primer pairs, reduce off-target amplification, and adapt to varying or repetitive genomic sequences with greater sensitivity and efficiency.
Hence, the tool I plan on developing would be able to significantly increase the sensitivity of detection tests, as well as decrease resource costs and streamline complex processes to develop non-invasive tests.
Project Blog
Post 1: Blog 0: Introduction
Hey everyone!
My name is Joy, and I’m an aspiring engineer with a deep interest in biology. Welcome to my first blog post!
My project focuses on increasing the efficacy of developing multiplex assays (made up of small DNA segments that match that of cancerous DNA).
My motivation to work on this project stems from my own experience creating a methylated multiplex assay to detect colorectal cancer more sensitively so that it could pick up cancerous DNA even at small concentrations. While working on the project with my teammates, I came across the problem of the primers we had designed binding to each other or themselves. Biology classes have taught us that DNA sequences or primers are made up of four nucleotides: adenine, guanine, thymine, and cytosine. Adenine binds to thymine, and guanine binds to cytosine. This fundamental principle of base pairing, essential to life, ironically made it difficult for us to design unique primers—some of the nucleotides of the primers bound to other primers or themselves instead of the cancerous DNA. We tried almost everything but to no avail—the conditions for multiplex formulation were too specific, and manually selecting primers had a high human error. How can we improve the primer selection process?
That’s the question I’m tackling with my senior project. I’ll be developing an interface that allows users to efficiently select primers based on a set of adjustable parameters. The orientation of my project will be practical and research-oriented, aiming to streamline multiplex assay formulation to ensure high sensitivity and efficiency across diverse experimental conditions. To bring this idea to fruition, I’m working with Dr. Charles Lively, an engineer with a rich background in performance modeling and optimization of scientific applications in HPC environments, and with the support of Dr. Dimitri Akulov, my incredibly insightful history teacher and internal advisor.
I’m ready to take on this challenge, so come along with me as I work to design and implement the algorithm, discover the intricate world of primer selection and primer-primer interactions, and explore the application of advanced technologies such as machine learning to primer selection!
Stay tuned, and I’ll see you later.
Post 2: Week One: Preparation
Hey everyone, and welcome back to my blog!
I’ve been using this week to ease myself into the project by preparing and mapping out the logistics of my algorithm. I already have experience designing, formulating, and testing primers in the lab, but applying my interest in computer science to automate this process is a new challenge.
So far, I’ve been conducting research on machine learning in bioinformatics and how I can increase the efficacy of primer selection. A significant portion of my project is data collection, so I’m working on finding different parameters I need to take into consideration for primer selection and quantitative ways for me to measure them. What makes a primer “good”? What are some things I need to take into consideration? And how can I actually measure how “good” the primers will be?
Before actually writing out my algorithm, I hope to make a framework in which I incorporate these different parameters into primer selection so I’ve been reading about existing primer design algorithms and downloading the necessary libraries and applications needed to implement my code. I’ve been brushing up on my knowledge of the different programming languages and applications I’ll be using throughout my project so I can streamline the development process as I go deeper into this project.
I’ve additionally been reviewing the fundamentals of machine learning—specifically looking at and learning about different search algorithms that could improve my project. It’s been fascinating learning about the different strengths and weaknesses of each search method and I’ve been considering the relevancy of these algorithms to my project as I read through them.
So far, I’m still in the introductory stages of my project, so this week has been a lot of mulling, reading, and researching, but I can already see the pieces coming together! I’m still a novice at machine learning, so I hope to make sure I build a solid understanding of machine learning before trying to tackle my project blindly.
That’s it for me, folks! See you next week!
Week 2: Data Collection
Hey everyone!
Did you miss me? I can’t believe it's already week two of my project!
This week, I’m continuing to review the basics of ML and its implementation in Python—it’s been such a rewarding and exciting experience learning about the different types of machine learning models and algorithms—especially supervised learning techniques I’ll be using like decision trees and random forests. I’m still trying to get more familiar with the different Python libraries I’ll be using, like Scikit-learn, Pandas, and more.
Further, I mapped out the logistics of my algorithm—compiling primer selection requirements, finding successful primers to use as testing data, and quantifying specific parameters that I need to take into consideration when evaluating a primer’s effectiveness. A large part of my project will consist of compiling data for my models to train on, so that’s what I’m focusing on now!
I’ve just been prioritizing these two tasks at the moment. Though it’s just one step, compiling data will be the factor that makes or breaks my model—I’m exploring different resources like publicly available primer databases, published research studies, and clinical trial data to include in my training data set. The more diverse and representative my dataset is, the more reliable my model will be. Finding data has been going relatively smoothly, and I’m grateful for the different resources I have to find validated and experimentally tested primers since I won’t be stepping into a lab. It’s been really interesting working on data analysis without the more hands-on, wet-lab work I’m used to. Relying solely on in silico methods is definitely a new experience, but I’ve already learned so much despite it just being week number two.
Next week, I anticipate continuing to collect data for the first half of the week, then regrouping with my external advisor to discuss the actual implementation of my model. I'm excited to dive into model training!
See you guys next week!
Week 3
Socrates once said: “The beginning of wisdom is the definition of terms.”
Hey everyone, welcome to blog #3! (Can you believe it’s already week three? :O Time flies so fast!)
Starting a machine learning project with limited basic knowledge of the field has not been easy, so I am continuing to take it slow.
This week, I explored more about decision trees and ensemble methods like random forests, as well as how they could improve model performance. I focused on the logic and pseudocode of my project to clarify and help me understand my model’s predictive capabilities. I’ve heard that due to how many resources like open-source tools and libraries exist in the field, it’ll be relatively easier for me to implement my code, so I’m just going to focus more on optimizing my data and feature selection for now.
I know I said that data collection has been going smoothly, but it has been a bit more difficult—particularly with the high-sensitivity assays used in clinical settings, like Epi ProColon. Many of them don’t release their primer sequences as public information due to proprietary laws or intellectual property concerns (I’m assuming). So for these primers, it’s been unfortunately more difficult to directly integrate certain biomarkers into my training data.
That being said, I can’t stress enough the importance of gathering reliable and well-documented data in a project like this. I’ve been continuing collecting primer sequences, their reported sensitivity levels, and the original biomarkers they target by searching through resources like the UCSC Genome Browser. Ensuring accuracy in this early stage is critical!
This week has been exciting with different small victories (and unexpected obstacles), but I can already see how much I’ve learned compared to week 0. Excited to see what next week brings!
Until next time!
Week 4: A new development...
Hey guys!
What. A. WEEK.
This week was spent troubleshooting some issues with the core foundations in my project. As many of you may know, I need a significant amount of training data for my model to accurately predict primer sequences from a larger and broader genetic biomarker (approximately several thousand data points!). Despite searching for extensive data sets with primers on reputable databases and research sites, I was unable to find a dataset that met my model’s needs and the goals of my project. I had two choices to overcome this obstacle: either compile months' worth of data from an already limited field with the remaining six weeks I had, or go in a different direction.
After a lengthy discussion with my external advisor, I decided to continue compiling training data, but I anticipate that I'll have to spend more weeks on this process. I was looking forward to starting work on my algorithm, but I know that the ending result will have been worth it! In the long run, I'll be able to gain a deeper understanding of biological datasets and machine learning methodologies.
This week was pretty rocky, and I felt pretty lost at various times—I’m still a novice researcher. Regardless, I’m pushing forward and adapting to challenges as they come. I anticipate that I’ll be able to learn invaluable things during this project that will help me grow as a researcher. I’ve learned that research isn’t as straightforward as I perceived it to be. It is what it is though—the best I can do now is to take this as an opportunity to broaden my technical skills in other areas too!
In the coming weeks, I’ll focus on putting together my training data, creating and refining my algorithm, testing its accuracy against real primer data, and ensuring that it aligns with established primer selection criteria.
See you guys next week!
Week 5
Can you believe we’re already halfway through the project? Time sure flies, doesn’t it?
This week, I explored using the UCSC Genome Browser to get the original genetic sequences of the different primers in the data. Unfortunately, this step proved to be more challenging than I anticipated due to a lack of information on where many of the researchers sourced the original genetic sequence—specifically, the exact genomic coordinates and indices of the sequences used to design the primers. I created a helper method to modify reverse primers and used BLAT search to match primer sequences to their corresponding genomic locations and verify their accuracy. Many of my datasets were from CpG islands within the biomarker, so these more specific search parameters made it easier for me to export the broader genetic sequences from which the primers were extracted.
Other than that, I mostly continued searching for primer sequences and reviewing papers on different ML models I could use. Initially, I hoped to implement a decision tree model to optimize the primer selection tool based on sensitivity and specificity, but after a more in-depth review, my external advisor and I came to the consensus that a hybrid model combining different ML models like random forests and LSTMs would be more effective and accurate. Thus, I’m still in the process of reviewing papers on different ML models that other researchers have used in their respective projects.
Every week comes with its fires that need to be put out, and these past two weeks have been especially demanding. But I’m taking these new challenges as learning opportunities for me to grow in my knowledge of bioinformatics and adapt to obstacles by sharpening my problem-solving skills.
Well, that’s it for me! Bye for now and see you guys next week :)
Week 6
Hey everyone!
How was everyone’s week?
This week, I continued searching for the original genetic sequences for a few primers cited in the literature—it’s been a little harder than I thought it would be!
Additionally, I’ve been looking forward to the model selection and multiplex primer design—topics I’ve been researching through different papers that introduced the different models they used. One of the tools I read about—NGS-PrimerPlex—stood out to me through its multi-step approach and flexibility in challenging design environments. Through literature review, I’ve been compiling a list of things I hope to implement into my algorithm—such as the ability to save primers generated under strict parameters or specific primers for more unique types of PCR like nested or anchored PCR. I’ve also finalized my list of evaluation criteria for primer quality which I’ll be using to assess my primers! Several widely-accepted standards for these metrics exist, so I’m hoping to compare my analysis against those as well.
Overall, this week laid important groundwork for the next stages of the project. With a better grasp on design parameters and a growing pool of resources, I’m excited to start pulling everything together.
Data collection—especially for the primers—still remains a pervasive problem. Time is running short and finding primers has been getting increasingly difficult. Wish me luck!
See you all next time!
Joy
Week 7
Hi guys!
I can’t believe it’s already week 7—time flies fast when you’re having fun, I guess!
This week, I continued searching for primers to add to my training data, and I’ve been expanding the scope of the primers and resources I’ve been looking into to help identify and categorize primers more effectively. I realized something that I haven’t been talking about is the process in which I am finding these primers.
Typically, I look for common biomarkers of different diseases (ex. SEPTIN9 for colorectal cancer) and find previously created primers on PrimerBank, a database of experimentally validated PCR primers. Because these primers have been used in multiple settings and the sensitivity of the primers are not listed in PrimerBank, I paste the primer sequence (ex. GCAGAGCGGCTTGGGTAAAT) into the search engine of Google Scholar or PubMed to find papers in which this specific primer sequence has been reported, as published studies would typically provide a sensitivity/specificity value. I then open almost every paper associated with this primer sequence and search for their sensitivity/specificity value. Though these attempts are often fruitless, they provide me with an opportunity to learn about different projects and the diverse diagnostic contexts in which these primers are used.
Other than primer selection, I’ve been organizing the layout of my model, as well as thinking through how I want to structure the input features and performance labels for training.
I hope that my explanation was able to give you guys a bit more insight on the primer selection process! The work is extremely laborious but it’s the backbone of this entire project.
See you all next time :) I'm excited to share more with you all!
Week 8
Hey guys!
This week was a big one. After weeks and weeks of searching through what felt like an infinite number of research papers for possible primers to use, I finally started building my machine learning model! It’s still in the very early stages, but just getting to this point already feels like a huge milestone.
These past few weeks have been honestly frustrating—for every hundred or so papers I sifted through, I would probably only find around one or two valid data points for me to use. As I mentioned in previous weeks, my current strategy has been to look for different biomarkers online (I’ve been using COSMIC), input into PrimerBank, and search on Google Scholar. It’s been a slow and extremely laborious process, but I can finally see the finish line! Of course, I anticipate working on this project past the May deadline, but for the symposium, I feel like I’m getting close to something I’ll be proud to share. I’m so excited for my dear readers (yes, you!) to see all the hard work I’ve put into this project.
As for the model, I’ve been working with different algorithms and different features like GC content and secondary structures. It’s been really rewarding being able to apply what I’ve learned when I designed primers in the lab into this tool. For those of you guys who don’t know, I previously worked almost exclusively in the wet lab, which meant that when designing assays, I would physically conduct PCR experiments and validate assays by hand through trial-and-error. Now, working on the dry lab side, it’s been really rewarding being able to apply what I’ve previously learned to potentially help others skip some of that time-intensive work.
Until next time,
Joy
Week 9
Aristotle once said, “For the things we have to learn before we can do them, we learn by doing them.” And that’s exactly what I did this week!
HI EVERYONE!! Welcome to blog 9!
I’ve finally finished collecting my data (thank goodness!) and now I’m working on coding my algorithm. It’s been really enlightening learning about the different packages like scikit-learn, pandas, and even exploring biology-specific packages like Biopython. I’ve encountered different trials throughout the process but each obstacle has taught me something new.
This week was a rollercoaster of frustrations, excitements, and stress. But despite all the chaos, there’s something really satisfying about solving a difficult problem that motivates me to keep pushing forward and see just how far I can go.
I can’t believe it’s the second to last week! This senior project has been challenging and stressful at times, but I wouldn’t trade this experience for anything. I’ve learned so much throughout this entire process—how to analyze bioinformatics data, design a machine learning pipeline, and even how to motivate myself to keep working when all hope seems lost (hint: copious amounts of matcha does wonders!)
Until next time :(
Joy
Week 10: A bittersweet farewell
GOOOOOOOOOOOOOOOOD MORNING FREMONTTT!!!
Can you believe that it is the last week of senior project? Time sure flies when we’re having fun, huh? What a whirlwind! I feel like it was just yesterday when I was writing my first blog (blog 0, for those who just joined my journey!).
This week, I finished building up my model using k-means clustering and random forests to predict the sensitivity/specificity of my training data.
This project has been filled with challenges and mini-victories, but I wouldn’t trade a second of it for anything. I’ve had to navigate creating my own dataset after failing to find one online, parsing through hundreds upon thousands of research papers to collect data points, code (as a somewhat novice programmer) an ML model, and learn to adapt quickly every time something didn’t go as planned.
Along the way, I’ve gained technical skills by reading numerous research papers, taking online courses on machine learning, consulting my mentor, and referencing online documentation. There were definitely moments when I felt stuck or overwhelmed, ready to give up, but each hurdle made the final results that much more rewarding. Of course, more work needs to be done (a lot more 😞) so I intend to keep working on the model following the senior project symposium.
As I wrap things up, I can’t help but think about the people and resources that have helped me grow in these ten weeks. My advisors—Dr. Akulov and Dr. Lively—have given me invaluable guidance in my project and the confidence to persevere through ambiguity and uncertainty. Thank you to my peers, my advisors, Ms. Nagpal, and everyone—I would not have been able to do any of this without your support!
I can’t wait to share my final presentation with you all (be there or be square!!) next week!
Thanks for following along with my journey. I’ll miss you all, my dear readers :(
Until next time.
-Joy