Identifying reproducible tau uptake patterns using positron emission tomography(PET) scans with an ML model.
Advisor: Moshofsky
Internship Location: Kaiser Permanente
Onsite Mentor: Dr. Naina Limbekar, Kaiser Permanente
Chronic traumatic encephalopathy(CTE) lacks an in-vivo diagnostic biomarker, limiting the ability for doctors and researchers to detect trauma-related neurodegeneration whilst the patient is still alive. Currently, it is diagnosed by examining the brain post-mortem under a microscope, however there have been developments to identify some characteristics before death. Tau positron emission tomography(tau-PET) provides quantitative, spatially resolved measurements of tau pathology within neurodegenerative diseases, and is mostly used in studies of Alzheimer’s disease(ALD). However, its ability to detect reproducible patterns of tau misfolds/aggregation remains uncertain. This project proposes to develop a multi-dataset machine learning framework to determine whether reproducible tau spatial signatures of tau uptake can be identified in individuals with a history of exposure to brain trauma(playing contact sports at an elevated level for 5-10 years).
Aggregating publicly available tau-PET neuroimaging datasets in three categories will be crucial to this project; regular patients with normal tau structure(control), patients with ALD, and patients with confirmed/suspected CTE. Both region-based machine-learning classifiers and three-dimensional convolutional neural networks will then be trained using cross-validated, dataset-stratified evaluation to ensure generalizable pattern learning. Model interpretability techniques, including saliency mapping and feature attribution, will be applied to localize brain regions contributing most strongly to classification and to assess biological plausibility relative to known tau distribution patterns in neurodegenerative disease. The central research question guiding this work is: Can machine learning identify statistically significant and biologically meaningful tau-PET spatial patterns associated with repetitive head impact exposure, despite the absence of a definitive in-vivo biomarker for CTE?
Project Blog
Week 1: Prelude, Project, and Preparation
Hi all! My name is Aditya Nautiyal, and my project is on Chronic Traumatic Encephalopathy, specifically using Tau Positron Emission Tomography and Machine Learning to detect, define, and differentiate it from other tau-pathological diseases.
A 2023 Boston University study found that 92% of studied brains of NFL players reported that they had brain trauma of some kind. Chronic Traumatic Encephalopathy, usually referred to as CTE, plagues professional athletes who play contact sports. Currently however, there is no way to diagnose CTE within athletes while they are alive. This stems from a multitude of factors, specifically being the in-depth analysis required of identifying a specific tau protein pattern unique to the disease. Tau proteins stabilize microtubules in the brain, essentially providing a track for transporting nutrients and components. However, trauma to the brain induces mechanical shear, which causes these proteins to misfold and thus hinder their function. This leads to reduced brain function, loss of memory, and other symptoms we commonly see in athletes with long-term exposure to brain contact/trauma/injury.
Alzheimer’s disease, also referred to as ALD, is similar in that it is a tauopathy(tau-related neurodegenerative disease). Currently, there is research that experiments with a process known as Tau Positron Emission Tomography(Tau PET). Tau PET is a hotspot detection system that highlights tau proteins within the brain, helpful for identifying a tau-related neurodegenerative disease. It works by injecting a biomarker, usually 18F-flortaucipir (Tauvid), that tracks and binds to the tau proteins, and then gives an emission detectable by the scan that determines hotspots of Tau. These hotspots are particularly useful as “clumping” is an indicator of the disease.
With these scans existing in datasets, I want to analyze them and use machine learning to determine what quantifies the difference between ALD and CTE in tau protein buildup. The difference will most likely be in the regional clumping of tau, providing insight to how they stem from different patterns/locations. My final product will be a developed machine learning model that can differentiate CTE from other tau-related neurodegenerative diseases like ALD.
As for my activities, Week 1 helped me gain a baseline understanding of Machine Learning and how I will leverage it in my project. I spent around 10 hours watching videos, taking quizzes, and doing online labs as part of the 3 week online course I have enrolled in. The other 6 hours were spent reading the literature outlined in my syllabus.
Week 2: Literature and Learning
Welcome to my second blog post! This week, I finished up my literature reading and constructed a literature review for you guys to read to better understand how I will go about my project and what needs to be done to create the final product. Along with that, I also completed the second week of my machine learning course(one more week to go!) in order to have the sufficient knowledge necessary to construct the final product. After you guys read my literature review, I will go on to talk about my plans for the next coming weeks. Happy reading!
Background of Machine Learning and Its Applications to Tauopathy in Neurodegenerative Diseases
Machine learning is present within our day-to-day digital world. Every Google search, every scroll on Instagram, every shopping site, they all leverage machine learning within their individual algorithms. The textbook definition of machine learning is as follows: a subset of artificial intelligence (AI) that enables systems to automatically learn, identify patterns, and improve performance from data without being explicitly programmed for every task (Bishop; Murphy). It accomplishes these tasks through a variety of mathematical functions, such as gradient descent and a cost function (Bishop; Murphy). Specifically, for tauopathies, we have seen an ML model that is able to identify Alzheimer's through brain imaging, working at a 92 percent accuracy and contributing to the field of diagnosis/treatment.
Even though it technically requires no explicit instructions, there still is a multitude of setup methods the model must go through before being able to accurately complete its task(s). These setups are called training, and are usually based in three main forms: supervised learning, unsupervised learning, and reinforcement learning (Bishop; Murphy). In supervised learning, the model is trained using labeled datasets where the correct output is known (Bishop). This way, the model learns relationships between inputs and outputs and improves its predictions over time when tested within unlabeled datasets.
In contrast, unsupervised learning works with unlabeled data, allowing the algorithm to independently discover patterns and groupings within the dataset (Murphy). Both of these learning modes are widely used in scientific research and data analysis, particularly when dealing with large and complex datasets, and most leverage both to train their ML model to the fullest. Lastly, reinforcement learning is an approach where an autonomous agent learns to make optimal, sequential decisions by interacting with an environment, using trial-and-error to maximize cumulative rewards (Murphy). We see these in environments involving robotics, game playing, autonomous driving, or optimizing complex systems.
As for tasks that an ML model can complete once they undergo significant training, it usually stems from one of three tasks: classification, clustering, and regression(it is worth noting that a lot of projects use multiple) (Bishop). Classification is the process where the model determines whether the input possesses the similar traits of the inputs it was trained on, and then classifies it into a group based on said training. Clustering, often paired with unsupervised learning, is when the model groups the inputs based on similarity. Lastly, regression is when the model predicts values based on inputs it received in its training, and is mostly used in fields like economics, weather, and finance for its forecasting capability.
Now, how can machine learning be used in neurodegenerative disease? Specifically, how can it be applied to the tau related neurodegenerative disease known as Chronic Traumatic Encephalopathy? To understand this, we first must understand the background behind tauopathies.
A 2023 Boston University study found that 92% of studied brains of NFL players reported that they had brain trauma of some kind (McKee). Chronic Traumatic Encephalopathy, usually referred to as CTE, plagues professional athletes who play contact sports. Currently however, there is no way to diagnose CTE within athletes while they are alive (McKee). This stems from a multitude of factors, specifically being the in-depth analysis required of identifying a specific tau protein pattern unique to the disease (McKee). Tau proteins stabilize microtubules in the brain, essentially providing a track for transporting nutrients and components (Goedert). However, trauma to the brain induces mechanical shear, which causes these proteins to misfold and thus hinder their function (McKee). This leads to reduced brain function, loss of memory, and other symptoms we commonly see in athletes with long-term exposure to brain contact/trauma/injury (McKee).
Alzheimer’s disease, also referred to as ALD, is similar in that it is a tauopathy (Goedert). Currently, there is research that experiments with a process known as Tau Positron Emission Tomography(Tau PET) (Goedert). Tau PET is a hotspot detection system that highlights tau proteins within the brain, helpful for identifying a tau-related neurodegenerative disease. It works by injecting a biomarker, usually 18F-flortaucipir (commercially known as Tauvid), that tracks and binds to the tau proteins, and then gives an emission detectable by the scan that determines hotspots of Tau (Goedert). These hotspots are particularly useful as “clumping” is an indicator of the disease.
In the study of tauopathy and CTE, machine learning can be leveraged in order to differentiate it from other tau-related diseases, and possibly providing a front as a new method to diagnose CTE in vivo. Machine learning models are well suited for analyzing complex imaging data, such as the Tau PET scan discussed earlier that are being used for ALD (Bishop; Murphy). They show where tau proteins accumulate in the brain, and by training algorithms on labeled datasets of brain scans from different tau-related diseases, models can learn to recognize subtle differences in the distribution, density, location/origination, etc. of tau protein buildup (Bishop; Murphy). Once trained, these models could classify new scans and help distinguish between diseases such as Alzheimer’s disease and CTE, providing a gateway for an alternate method to diagnose CTE and other tauopathies.
The next two-three weeks will be spent gathering and training the model with various datasets, specifically grouped into three groups: control, CTE-specific, and tauopathic specific. These three types of datasets will be fed into the ML model, and trained to identify differences between them and in turn, isolating the key indicators and differentiators of CTE tau buildup. With this in hand, I will then continue to develop the model to take in scans, and then have it provide its prediction as well as an accuracy/confidence level.
REFERENCES
Brown, S. (2021). Machine learning, explained. MIT Sloan School of Management.
Read article
Ng, A. (2018). Machine Learning Yearning.
https://www.deeplearning.ai/machine-learning-yearning/
Ng, A. (2022). Supervised Machine Learning and Regression Models. Coursera / Stanford University.
https://www.coursera.org/learn/machine-learning
McKee, A. C., Stein, T. D., Nowinski, C. J., Stern, R. A., Daneshvar, D. H., Alvarez, V. E., Lee, H. S., Hall, G., Wojtowicz, S. M., Baugh, C. M., Riley, D. O., Kubilus, C. A., Cormier, K. A., Jacobs, M. A., Martin, B. R., Abraham, C. R., Ikezu, T., Reichard, R. R., Wolozin, B. L., Budson, A. E., Goldstein, L. E., Kowall, N. W., & Cantu, R. C. (2013). The spectrum of disease in chronic traumatic encephalopathy. Brain, 136(1), 43–64.
https://doi.org/10.1093/brain/aws307
Goedert, M. (2004). Tau protein and neurodegeneration. Seminars in Cell & Developmental Biology, 15(1), 45–49.
https://doi.org/10.1016/j.semcdb.2003.12.015
https://pubmed.ncbi.nlm.nih.gov/15036206/
Guo, J. L., & Lee, V. M. Y. (2014). Cell-to-cell transmission of pathogenic proteins in neurodegenerative diseases.
Nature Medicine, 20, 130–138.
https://doi.org/10.1038/nm.3457
https://www.nature.com/articles/nm.3457
Week 3: Datasets and Directives
Hi all! This week, I compiled various datasets of tau-PET scans across different platforms/websites, and began requesting them with the help of my advisors. Specifically, the biggest find had to be a website that detailed close to 50 datasets and where to find them. This was a HUGE help, and I am extremely excited about this find. In the next couple weeks, I will begin requesting from all of them in order to use their data for my project. As for my Machine Learning course, I have completed the third and final week, and now am beginning to start setup of the ML model through applications like Google Colab, VSCode, and Jupyter Notebooks!
Now, where do these datasets fit in the grand scheme of things? With these datasets, I plan to use a 80/20 training-testing split, after generalization of course. Generalization here is probably the most important part of making sure the model works, as if it isn’t generalized, the model might interpret the data completely wrong(it might differentiate between different platforms used to make the brain scan rather than structures solely). In order to accomplish this, we must use tools like dcm2niix and NiBabel. They sound fancy, but in reality are actually simple, but powerful processes! Dcm2niix is a conversion tool, NiBabel is a python library for neuroimaging that has a few key formats(hence the reason for the Dcm2niix converter). Once these scans have been standardized, I will compute SUVR(Standardized Value Uptake Ratio) using the cerebellar reference region as well as the z score for the brain mask. ComBat Harmonization will also probably be used to remove any scanner/site effects. After this, we feed the standardized data into the model, and interpret its results through the 80/20 split. Overall, I am excited to begin this process, however I must still get approval to these datasets in the first place. Until next time!
Week 4: Preliminary Generalization and a Problem
Welcome to Week 4! In-case this is your first time tuning into my senior project blog posts, I’ll quickly summarize what has happened the last few weeks. I’ve finished my ML online-course as well as compiled a series of texts to explain my project in context in the form of a literature review. Within this literature review, I’ve also outlined the steps that need to be taken to stay on course in order to meet the final project deadline. Currently, I am in the “dataset searching/requesting” mode, and have figured out a rough idea of what datasets I will use for the first week of preliminary training. These are publicly available datasets that do not require any requests for access, hence the reason why I chose them for the first “round” of training. They are as follows: NACC, OpenNeuro, Neu3Grid, OSF, and Centaur. I will subject these datasets to some conversion methods briefly, in order to gain experience on what it would look like on a larger scale with more variability and complexity of datasets in the near future.
But why must we do this? Well, the formats that we see on websites like Kaggle(JPEG/PNG) are vastly different from the ones we see from medical institutions(DICOM/NIfTL), however most have one thing in common. They are usually registered to the MNI152 standard brain(brain imaging template), or usually are able to be easily converted to this, allowing us to remove head size/shape differences and enable voxel-wise comparisons, needed for CNN generalization. However, there is still a difference present, which is in the file itself. This is where the converters I mentioned earlier come into play(dcm2niix and NiBabel). Dcm2niix allows me to convert any raw DICOM files into the desired NIfTL files, which will then be processed into the CNN using NiBabel.
In the process of requesting, I did run into a few problems. The biggest was the ADNI data-use agreement, specifically not being able to use AI in projects like mine. I had to stop preliminary testing/conversion processes to address this, double checking to make sure all of the publicly available datasets allowed me to use AI with them. The biggest concern with AI when it comes to Tau-Pet scans would be the risk of data-sharing, as it is still sensitive medical information. In order to get around this hump, I did various research on similar projects to mine(ALD detection using tau-Pet ML), and found that they did use datasets like ADNI despite the user agreement. With this in mind, my advisor sent an email to ADNI explaining our situation and asking for their outlook and decision on our case, and I sent an email to Google CoLab confirming that their website did not involve data-sharing whatsoever. We are still waiting to hear back, and hopefully when we do, we are able to go forward with this specific dataset. Worst-case scenario, I can still analyze the data independent of using ML, and then input those findings into the model without compromising any specific data that is sensitive. Going forward, I will continue to run some preliminary generalizations for the datasets + set up the ML Model in CoLab, as well as continuing to contact medical institutions for more datasets. Until next time!
Week 5: Finalizations and a Foundation
Hi all! This week, I finally finished gathering and finalizing datasets and began on making the processing pipeline for the different datasets that I will be using to both train and test the model on. Specifically, I was able to finish the foundation of the control dataset processing pipeline and generate graphs and statistics to compare the other two datasets(ALD and CTE) to. The specific statistics used here were SUVR(Standardized Uptake Value Ratio), which, put simply, means the concentration of tau protein within a certain area. The results are as follows below on the left(first ten subjects only).
So how exactly did I establish the groundwork for this pipeline? The control dataset I used for the base was ds006756 from OpenNeuro, which consists of 33 patients with normal, young individuals with healthy tau distribution across their brain. This served as a great control, and the fact that it was from OpenNeuro helped even further as they have a specific API within Google Colab that allows my pipeline to be directly integrated with their datasets. This means that no manual download is needed, and that datasets can be downloaded and processed simply by coding a cell using the API. This is great as it allows for new datasets to be processed and integrated seamlessly within the entire ML model, often needing only one more cell of code to download and process it.(Shown below)
As for the actual flow of the pipeline, it is made up of 5 main cells. The first cell handles imports and installs for the actual software we use(OpenNeuro Downloader, NumPy, etc). The second cell involves a tracer configuration/converter, as some datasets use the MK6240 tracer, which needs to be standardized back to the most common tracer(Flortaucipir). This is shown below.
In cell three, I established session folders(these folders are created since the CoLab session is reset after you close the application) and provide a summary of all subjects within the session(this will soon involve more than just the control). Then in the fourth cell, we process each subject and calculate SUVR and ROI(Regions of Interest) means, as well as any other important statistics, and export them to those session folders created in the previous cell. Lastly, in the fifth cell, we extract and convert our findings into boxplots, as depicted below.
The biggest finding, which is also very crucial to this project, was the unevenness of tau concentration within healthy individuals. This is crucial for the project as now we have a baseline to compare datasets of unhealthy tau buildup, and truly note any tauopathic disease traits that might be present within the test scans as compared to general unevenness. In the next few weeks, I will continue to expand this pipeline into a multi-dataset, fully-functioning, and accurate ML model. Until next time!
Week 6: The Cerebellar Mask Issue
This week, I addressed a problem and troubleshooted the pipeline in order to have it ready for the last step of creating a foundational ML model. The core problem was in the get_cerebellar_mask() function in the preprocessing pipeline. It was designed to identify the inferior cerebellum as a reference region for computing SUVR (Standardized Uptake Value Ratio), but it had a fundamental flaw in how it calculated MNI z-coordinates.
The original code was:
mni_z = affine[2, 2] * zi + affine[2, 3]
if mni_z < -40:
mask[:, :, zi] = 1
This simplified calculation only used two elements of the affine matrix's third row, which is only correct when the image is perfectly aligned to MNI axes. However, for most datasets, they are not standardized perfectly. This produced an incorrect cerebellar reference region, and in some cases a completely empty one. This is shown in the below output in the control datasets, where SUVR is far from the “1.0 standard.”
First 5 values: [1.23617827 1.72605648 2.31010386 1.44630158 1.26450816]
The consequence was that when I moved to the ADNI dataset and compared the control SUVRs to the AD group, the control SUVR values came out unusually high (1.6–1.9 across most regions), when healthy young adults should be close to 1.0.
The PET scans were being resampled into native T1w space (the subject's own anatomy) rather than into MNI152 standard space first. The solution to this was:
Resample PET -> MNI152 first (using load_mni152_template from nilearn), then apply the z-coordinate mask
Use a proper cerebellar atlas mask file in MNI space, resampled to each subject's volume, rather than a hard-coded z-cutoff
The results were as follows:
The values were much more stable now, and ready to be compared against the control. I then extracted the same SUVR-ROI values from the ADNI dataset(which contains tau-PET scans of those diagnosed with ALD), which led to the resulting data.
This indicates that the model is properly extracting those values given the raw data, as it is much more skewed than the control group. All that is left is to expand the number of datasets within the pipeline, which I intend to do within the next coming weeks.
Week 7: Refining and Restructuring
Hi all! This week was quite uneventful, as I kept expanding my pipeline with datasets, as well as refine the entire JNB in order to break less. Recently, I had been getting all sorts of problems after just minor tweaks with the JNB, causing the entire thing to stop working and break down completely. At one point, it even created a corrupted file path which led to me having to manually go through my Google Drive folder and delete that file path. To put it simply, I was pretty “fed up” with having to deal with these constant fixes, and so I decided to work on rewriting the entire JNB. The main fixes include basic reordering and cell-splitting(splitting big chunks of code into smaller cells), as well as cleanliness/aesthetic fixes.
We start off by importing and downloading specific functions that are needed for our JNB to work all at one place, fixing the problem that my previous JNB had of having specific imports all located within different cells. This was primarily due to the sequential nature of this project, where datasets are added piece by piece along with their respective imports needed.
I then established safeguards to ensure that no corrupted pathways would be created, and end up breaking the project entirely. This also included defining files to their location within the google drive early, and this will be frequently updated as more and more datasets are added.
I then established the shared utilities across the project, which were previously used across multiple dataset cells. This was done for uniformity, and we see an example of this below where the visualization tool for graphs for each dataset is established once.
After this cell, the rest of the datasets are integrated and processed cell by cell, but overall, this is a huge quality of life improvement. I’ve tested my code by renaming files, entering wrong outputs, etc, and error messages show instead of corrupted pathways forming. This is crucial as we expand the volume of datasets being used in the future. Currently, the pipeline data is as follows:
Combined pipeline total: Approx. 1,463 subjects with full data (tau + amyloid + MRI) Total Subjects: 1,918 tau subjects with full or partial data
Next week, I intend to start more intensive testing and integrating CTE/contact-trauma specific datasets. The biggest challenge I foresee in this stage would be the accuracy of the model, which will need to be in the 90% range to be considered remotely credible for a base(MVP) neuroimaging model. I hope you enjoyed this blog post, and until next time!
Week 8: Preliminary Training
This week, I transitioned from preparing data and building the pipeline to training models. I started training our machine learning model on two types of neuroimaging data: healthy controls and patients with ALD(Alzheimer's Disease). We trained our model and measured its accuracy using various classification metrics (based on cross-validation) to evaluate the performance of the different neuroimaging modalities (those being processed through the pipeline). While the model's accuracy has not yet reached a level sufficient for clinical use, it is producing a significant signal. The model is detecting true patterns within the data and is therefore not randomly guessing. This is a good sign and a solid starting point to work from.
I also spent time developing materials for the final presentation. This included drafting slide decks to communicate the project's rationale and methodology. This process also helped us refine how we wanted to frame the project's goals and the biological questions we wanted to answer with it. Once I receive my final results, I will incorporate them into the slideshow.
Looking forward, the most important next step is submitting a data access request to the DIAGNOSE CTE study. If approved, this dataset of tau PET scans from former contact sport athletes will add a third and scientifically critical class to the model, enabling it to distinguish not just healthy versus pathological tau, but also to detect the distinct tau distribution patterns associated with CTE. This is a major addition, completing my dataset search and setting up my model for the last phase of rigorous training.
Week 9: Access Denied
Week 9
The access request we submitted to DIAGNOSE CTE-CORE has been denied, which is a significant setback to the original scope of the project. The CTE dataset was designed to be the third, most clinically-distinct class in the model. It was to represent a unique pattern of tau deposition as seen in former athletes of contact sports, in contrast to both healthy controls and tau pathology specific to Alzheimer's disease. The model will not be able to address the problem of CTE classification as planned until this dataset becomes available.
In addition, it is not uncommon in neuroimaging research to incur data sharing restrictions due to ethical and legal issues surrounding data access(especially projects that involve the use of AI and ML), and the work done so far remains valid regardless of the fact that we cannot access this data. We will continue to try to obtain this dataset using different channels, including networking with researchers associated with DIAGNOSE CTE-CORE, identifying collaborators with access to the available data, or identifying other (adjacent) datasets that contain information about CTE. This type of access request usually requires some form of an institutional or co-investigator relationship, which may take time to establish. I will continue with the datasets in hand (the healthy cohort) as well as investigating alternate datasets that may exist.
Regardless, I will still present what I have found so far and my methodology as to what I have done up until this point. The key notion here would be to include a “future plans” section next to the results section of the presentation. I have already notified my advisors as well as the Senior Project coordinator of this setback, and they have approved my plan of presenting without conclusive results.
Week 10: Presentation Practice
In this post, I won't be saying nearly as much because most of my effort has gone into preparing for the final presentation of my senior project at the upcoming Senior Project Symposium. I spent quite a bit of time preparing the presentation to ensure that I properly represented the project and what decisions I made regarding its overall design (technical aspects) and how successful each piece was (as far as meeting the required timelines). Another piece to my final project submission was created as a video demo; this video would allow me to provide some explanation as to how the system works while demonstrating the core features. I've also included the confusion matrix from the ML model within my demo, which illustrates how well the ML model has differentiated between the various objects (target classes), where there continue to be issues with misclassification, and how I derived this information.
Week 11: The End?
Hi all! Presentations went great! For those who missed mine, I've attached a link to view my slides. For those who attended, your support means a lot to both me and my fellow peers. We dedicated upwards of a full trimester to this presentation, and it's wonderful to see so much support and recognition for our efforts. Personally, I think the Senior Project was a fantastic experience. The ups and downs taught me a great deal about the research process, as well as specific technical and academic areas of interest like Machine Learning and Neuroscience. Before this project, I was quite unfamiliar with code and didn't know the first thing about tauopathies; however, as I progressed, I filled these knowledge gaps and built a machine learning model from the ground up. To anyone considering a Senior Project, I highly suggest you do it, as it has taught me more in the span of three months than any other period of my academic life.
https://docs.google.com/presentation/d/1RZZ20O2nYqdCDI7q3cEqP5PqDsbmC96SwG3tyB5PhIw/edit?usp=sharing