Fine-Tuning Large Language Models for Rare Disease Diagnosis
Advisor: Ms. Rangoli
Internship Location: Sunnyvale(working with external advisor not organization)*
Onsite Mentor: Junfeng Pan
Rare diseases, defined as conditions affecting fewer than 1 in 2,000 people, often evade timely diagnosis due to their rarity and shared symptoms with more common illnesses. While human physicians frequently struggle with these diagnoses, large language models (LLMs) have demonstrated remarkable potential in medical assistance. Inspired by a case where ChatGPT correctly diagnosed a rare disease after 17 doctors failed, this project aims to fine-tune open-source LLMs for rare disease diagnosis. Using QLoRa, a parameter-efficient transfer learning technique, the model will be trained on datasets such as ReDis-QA covering rare disease symptoms and evaluated using benchmarks such as RareBench. Furthermore, I hope to build a user-friendly website interface that allows physicians to input patient symptoms and access the model's suggestions. This project seeks to create a diagnostic tool that supports, rather than replaces, human physicians, improving diagnostic accuracy for rare diseases and reducing patient suffering.
Project Blog
Post 1: Blog #0: How to Train Your AI
Hey everyone!
I’m Luoxi, and welcome to my blog. I’ll be fine-tuning large language models (LLMs) for Rare Disease diagnosis.
One striking example of AI’s potential in medicine is the story of four-year-old Alex. While seventeen human doctors tried and failed to diagnose him, ChatGPT succeeded. Over three years, he suffered through repeated misdiagnosis and painful symptoms, until his mother turned to ChatGPT as a final effort. With a quick entry of Alex’s MRI notes, the large language model (LLM) suggested tethered cord syndrome. Human doctors ultimately confirmed the diagnosis as correct, allowing Alex to finally receive the care he needed.
Situations like Alex’s show the potential of LLMs in medical assistance. Rare diseases, defined as conditions affecting fewer than 1 in 2,000 people, are especially difficult to diagnose because most human physicians don’t have the experience and training to identify them. Many of these conditions further share symptoms with more common diseases, making them even more challenging to identify.
Trained on vast amounts of case data, LLMs are a promising new tool to help identify rare diseases. They may offer scalable and accessible support to human doctors. For my senior project, I plan to fine-tune (specialize with transfer learning) a general-purpose LLM specifically for rare disease diagnosis, posing the question: How accurately can a LLM identify rare diseases? Furthermore, can it effectively communicate its reasoning with transparency?
However, advancing this technology is intended to aid, not replace, human physicians. While large language models can offer data-driven insights quickly, they lack the empathetic, nuanced judgment that human doctors bring to each patient interaction. While developing a complementary tool to augment physicians’ knowledge and assist with challenging rare disease diagnoses, I hope to explore LLMs and fine-tuning techniques in greater depth.
Thanks for joining me on this journey, and see you until then!
- Luoxi
Post 2: Blog #1: Garbage in, Garbage out
Hey fellow human,
Welcome back! Today, we’re looking at data.
When it comes to improving an AI model, we have two main options:
a) Improve our data
b) Improve our model architecture
For now, we’re focusing on the data side. There are a few things we can do: collect more high-quality data, clean our existing data, or augment it.
I know, working with data doesn’t sound as cool as designing an AI model. But trust me, it’s just as important! If our data isn’t good, our results won’t be either (hence the classic saying: garbage in, garbage out).
So, let’s develop a feel for the data. For this project, I’ll be using two public datasets from real-life patients - RareBench and ReDis-QA.
RareBench Dataset:
Dataset({
features: ['Phenotype', 'RareDisease', 'Department'],
num_rows: 1122
})
Sample Row (Patient Info):
{'Phenotype': ['HP:0001522', 'HP:0001942', 'HP:0003210', 'HP:0003819'], 'RareDisease': ['OMIM:251000', 'ORPHA:27', 'CCRD:71'], 'Department': None}
Phenotypes are the presentation of a disease. They’re represented in the Human Phenotype Ontology code (here, HP:0001522 stands for “Death in infancy”).
Likewise, OMIM and ORPHA (we won’t be using CCRD) are the Online Mendelian Inheritance in Man and Orphanet codes (OMIM:251000 = Vitamin B12-unresponsive methylmalonic acidemia/Methylmalonic aciduria due to methylmalonyl-coa mutase deficiency) for rare diseases. Note that a patient can be suffering from multiple diseases.
The RareBench researchers also provided two JSON files to map phenotype and rare disease codes to their actual names. Some codes weren’t included in the mapping (possibly because the phenotypes are obsolete), so I manually typed them in (I need as much data as possible - more on that later).
To make things cleaner for this LLM (large language model) project and to avoid duplicates, I added another property with the corresponding names of the codes.
Here’s the frequency graph showing how often each disease appears in the RareBench dataset:
ReDis-QA Dataset:
Dataset({
features: ['question', 'opa', 'opb', 'opc', 'opd', 'cop', 'rare disease', 'input'],
num_rows: 1360
})
Sample row:
{'question': 'All are features of Abetalipoproteinemia, EXCEPT:', 'opa': 'Plasma levels of cholesterol and triglyceride are extremely low', 'opb': 'Manifest in early childhood with diarrhea', 'opc': 'Progressive pigmented retinopathy seen', 'opd': 'Neurological manifestation as ataxia in first decade', 'cop': 3, 'rare disease': ['Abetalipoproteinemia'], 'input': 'Question: All are features of Abetalipoproteinemia, EXCEPT:\nChoices:\nA. Plasma levels of cholesterol and triglyceride are extremely low\nB. Manifest in early childhood with diarrhea\nC. Progressive pigmented retinopathy seen\nD. Neurological manifestation as ataxia in first decade\nAnswer:'}
Unlike RareBench, ReDis-QA is a question-answering dataset for rare disease diagnosis. Each sample consists of a question related to a rare disease, four answer choices, and the correct option (cop).
One thing we’ll is that many diseases have less than five instances. That’s… not ideal. With so few examples, the model might struggle to generalize. Since most medical data is private and expanding the dataset isn’t likely, I’ll need to look into few-shot training techniques to make the most of what I have.
Hopefully, this gives you a better idea of what I’m working with. Stay tuned for the next blog!
- Luoxi
Post 3: Blog #2: Patience
Hello and welcome back!
This week, I set up Llama 2 and read about prompt engineering and the current state of ML in rare disease diagnosis.
To get started, I needed access to Llama, so I submitted a request and got approved to download. Then came the (slightly more) fun part: importing it. I chose Llama 2 7B Chat, since it’s fine-tuned for dialogue and would be more useful in a rare disease Q&A setting. At the same time, it’s small enough that I can run it on a M3 chip. Getting everything set up took some patience (downloading it from Hugging Face alone took a little over 30 minutes.)
Now that I’ve got Llama 2 loaded, I started running test prompts and evaluating how well it performs before fine-tuning. For example, when asked, “What is Alpha Thalassemia,” the model responded:
“What is the Alpha Thalassemia? Alpha thalassemia is a genetic disorder that affects the production of hemoglobin, a protein in red blood cells that carries oxygen to the body's tissues. It is caused by mutations in the genes that code for the alpha-chain of hemoglobin, which is responsible for the globin subunits of the protein. There are two main forms of alpha thalassemia: 1. Delet [sic].”
It took 55 minutes to generate this response. Despite cutting off awkwardly (which is something I’ll need to fix), the model’s description matched the OMIM entry, showing a solid baseline knowledge.
Beyond setup, I spent time reviewing ML and rare disease diagnosis, especially on its data scarcity. I also explored PromptingGuide.ai and studied methods like few-shot prompting and chain-of-thought. Alongside fine-tuning, having good prompt engineering can significantly improve results.
The literature review has been slow, but it will pay off.
That’s it for this week.
See you next time! :D
Luoxi
Post 4: Blog #3: Garbage in, Garbage out 2
Hey again CS/LLM enthusiast ✨
This week, I dealt with some logistics after realizing 7B models are way too slow on the M3 chip with 16 GB RAM. The best option? Renting a GPU from the cloud, and I’m going with an A100 (80 GB RAM) on RunPod
Additionally, I looked into more data processing. Each row of the RareBench dataset looks something like this:
{'Phenotype': ['Motor delay', 'Generalized hypotonia', 'Death in infancy', 'Pneumonia', 'Elevated urinary carboxylic acid'], 'RareDisease': ['Succinic semialdehyde dehydrogenase deficiency/Succinic semialdehyde dehydrogenase deficiency'], 'Department': None}
As nice as it would be to just feed the raw data to the LLM and expect readable and more accurate output, sometimes life doesn’t give you want.
To format the data correctly and provide the reasoning, ideally, a human doctor annotates all 1122 cases. Unfortunately, I don’t know a rare disease specialist who has the time. Hence, I decided to use a large LLM to write it.
“… Wait, you’re using LLM-generated data to fine-tune an LLM?” You might be thinking.
Yep.
LLMs are trained on the whole internet’s medical data, giving them a vast knowledge base and strong reasoning abilities. While not perfect, their output is usually solid enough to work with. I’m trusting their output.
Going by the LLM leaderboard, DeepSeek-R1 currently has the best reasoning and is much cheaper than GPT-o1. So, I’m using DeepSeek-R1 to process the data.
The API calls are still running on my computer. Fingers crossed.
Thanks!
Luoxi
Post 5: Blog #4: Attention Please!
… And the API call has finished. This week, I manually checked and translated parts of the data still in Mandarin (a remnant of the training data). But before jumping into implementation, I was kinda curious about the theory behind LLMs and QLoRA. So, here’s an overview :D
Essentially, LLMs work by predicting the next work. Over the past few years, they’ve blown up following the development of transformers, which is much better at handlingcontext than older architectures like RNNs. The "Attention Is All You Need" paper introduced transformers in 2017, and ever since, they've been the foundation of everything from GPT to DeepSeek.
Transformers are built from two key components: attention layers and MLP (feedforward) layers. At the end, a softmax layer normalizes the output into probabilities for the next word.
(Attention Is All You Need)
What’s transformative about transformers is that they incorporate attention, which captures context. Transformers don’t just read words one by one. They weigh the importance of each word to describing the others. Each word is transformed into a Query (Q) and a Key (K) using weight matrices Wq and Wk. Their dot product determines how relevant a word is to another, effectively capturing relationships across the text. These weights are then applied to the Value (V) vectors, which are added to the original word vectors. Now, the word vectors don’t just hold information on the meaning of each word but also accounts for context.
(3B1B)
This is just one head of attention. In LLMs, many of them are running in parallel in each layer.
The sad thing is that large language models are, well, very large, having billions of parameters. 7b models require ~84 GB RAM (https://modal.com/blog/how-much-vram-need-fine-tuning) to fine-tune, way beyond what most consumer GPUs can handle. Fortunately, QLoRA (Quantized Low Rank Adaptation) deals with this issue.
Part of the reason why fine-tuning is expensive is because models use 16-bit or 32-bit precision by default, which is not always necessary. Quantization reduces memory usage by rounding and truncating weights down to 4-bit representations. Instead of storing massive weight matrices at full precision, we use block-wise quantization to compress them without major accuracy loss.
(https://www.youtube.com/watch?v=fQirE9N5q_Y)
Low Rank Adaptation further reduces memory usage. The thing about model parameter matrices is that many rows/columns are redundant - you can remove them without effecting performance that much. This happens when they are not linearly independent and so do not store more information. Low Rank Adaptation works by learning a smaller set of updates stored in two low-rank matrices. This way, instead of tweaking the entire model, we only adjust the most important parts.
Even with the reduction in memory (~⅓ of vanilla fine-tuning) and time, performance doesn’t significantly decrease!
Wow, that was a lot. I hope you got something out of my crazed rambling! Next week, if all goes well, I’ll finally put this into practice!
See you next time!
- Luoxi
Post 6: Blog #5: Implementation
Hey everyone!
Now that we’ve talked about the theory behind QLoRA, let’s dive into the code implementation.
Thankfully, while there’s a lot going behind QLoRA, I don’t have to worry about the nitty-gritty details. Libraries provide short abstractions that are much easier to use.
From the QLoRA GitHub repo (https://github.com/artidoro/qlora?tab=readme-ov-file), here’s how quantization and LoRA are configured:
Quantization:
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
Low Rank Adaptation:
config = LoraConfig(
r=8,
lora_alpha=32,
target_modules=["query_key_value"],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
There are some parameters I can control here:
r is the rank of the low-rank matrices
lora_alpha is the LoRA scaling factor (weights the LoRA contributions to the original model)
target_modules are the layers LoRA is applied
lora_dropout is the dropout probability of the LoRA layers (a proportion of nodes are dropped out during training for better generalization)
bias indicates whether the bias parameters are trainable
But what exactly are the best values to use? Alongside research, I might have to experiment a little too.
- Luoxi
Post 7: Blog #6: In Which LLaMa Loses It
Hey everyone!
I’m back. This week, I finished my first try at fine-tuning LLaMa 2.
Before I get started, I want to note that without any fine-tuning, the model has a zero-shot top-10 recall of 23.81%. This is the baseline to improve from.
Moving onto fine-tuning, I chose r = 8 and a dropout of 0.05. I picked r = 8 as a modest low-rank approximation to start with, hopefully keeping memory reasonable while still enabling the model to learn. The dropout of 0.05 helps combat overfitting by adding some noise (but not too much) to training, forcing the model to generalize more. I also included a validation set, since watching validation loss lets me monitor generalization.
Here are the training losses and validation losses of the model over fine-tuning!
I have some good news and bad news. Good news: validation loss is going down, and the model is learning the task. Bad news: training loss is dropping faster than validation loss, which smells like overfitting.
Indeed, running the model, it sometimes outputs gibberish!
Example:
[INST] <<SYS>>
{{ You are a specialist in the field of rare diseases. You will be provided and asked about a complicated clinical case; read it carefully and then provide a diverse and comprehensive differential diagnosis. }}
<</SYS>>
{{ This rare disease patient suffers from symptoms: ['Diabetes mellitus', 'Abnormal nervous system physiology', 'Impaired glucose tolerance', 'Death in early adulthood']. Enumerate the top 10 most likely diagnoses. Be precise, listing one diagnosis per line, and try to cover many unique possibilities (at least 10). The top 10 diagnoses are: }} [/INST]
As a rare disease specialist, I understand that you want me to help you with your complex clinical case. Here is my analysis after reading the clinical details with you.
Here's what we know so far based on the information given to us by our client:
The patient has been experiencing several symptoms including diabetes mellitus, abnormal nervous system functioning, impaired glucose tolerance, and death at an early age. These signs make up a broad range of possible illnesses. Here are ten potential diagnoses for further investigation:
Mitochondrial disorder - A group of genetic conditions affecting the mitochondria, which can cause various neurological and metabolic problems, such abc as muscle weakness, seizures, and developmental delays.
Klinefelter syndrome- A condition caused by extra X chromosphere, leading to hormone imbalances and reproductive issues.
Hypothyroidism- A condition whereby the thy abc- gland produces too little of the hormones, can lead to fatigue, weight gain, cold intolerant, depression, and other symptoms.
Chronic kidney disease(CKD)- longstanding damage to the kidneys, which may result in permanent loss of kidney function or even complete failure.
Fabry disease- Genetic disorder that results fromacle deficiency in enzyme needed to break down specific substance in body.
Glycogen storage disorders- Conditions in which there is abnormal accumulation of polisacarette carbohydrates in certain tissues of the body.
pituitary dwar abc- Pituitary gland abnormalities, which could impact growth and development in children and also hormonal regulation in both children and adults.
Ehlers-Danlos syndrome- Group of connective tissue disorders that affect the collagen fibre fibres in the skin, joints, and other tissues.
Marfan Syndrome- Torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades torsades…
I have a few suspicions.
The model might be overfitting on the training data and forgetting it’s initial knowledge of English (Google “catastrophic forgetting” for an interesting read). Multiple times, it says “abc” out of the blue, maybe overfitted to the ReDis-QA MCQs in my training set. Furthermore, the learning rate might be too high and overwhelming the weights, the LoRA rank might be too low and not expressive enough for the data, or my code for tokenizing the training data is buggy.
I'm also considering switching to just LoRA for accuracy.
Just thinking of these many factors gives me a headache! But I guess this is all part of the process. In a sense, fine-tuning is like tutoring a genius middle-schooler – you never know if you’re about to get brilliance or gibberish.
Signing out,
Luoxi
Post 8: Blog #7: Learning Curve
Hi!
I hope you’ve had a good week. Sit down, pour yourself a cup of tea, and enjoy my blog post :D
This week, I made some key changes to my model’s training setup: I fixed tokenization issues by adding a pad token, lowered the learning rate to 5e-5, reduced the number of training epochs to 3, and added L2 regularization. These tweaks all help with generalization.
And it works now! The output is finally sensible.
[INST] <<SYS>>
{{ You are a specialist in the field of rare diseases. You will be provided and asked about a complicated clinical case; read it carefully and then provide a diverse and comprehensive differential diagnosis. }}
<</SYS>>
{{ This rare disease patient suffers from symptoms: ['Seizure', 'Spastic diplegia', 'Encephalopathy', 'Death in infancy', 'Fever', 'Spastic tetraplegia', 'Tachypnea', 'Glutaric aciduria']. Enumerate the top 10 most likely diagnoses. Be precise, listing one diagnosis per line, and try to cover many unique possibilities (at least 10). The top 10 diagnoses are: }} [/INST]
Carnitine uptake defect
Carnitine palmitoyltransferase II deficiency
Methylmalonyl-CoA mutase deficiency
Medium chain acyl-CoA dehydrogenase complex deficiency
Hepatic mitochondrial trifunctional protein deficiency
Pyruvate dehydrogenase complex deficiency
Ketone body metabolism disorder
Propionic aciduria type I
Methylmalonic acidemia type A
Methylmalonic acidemia type B
In this case, it gave the right diagnosis - Medium chain acyl-CoA dehydrogenase deficiency
To dig deeper, I took a look at the model’s (literal) learning curve:
Something weird you probably notice here is that the validation loss is less than the training loss, especially in the earlier steps. After double-checking for data contamination and finding nothing suspicious, I was pretty confused too, so I turned to my trustworthy-friend, Google. Apparently, this discrepancy is normal when adding regularization - The correction term is only applied to the training loss and not the validation loss. As the correction term decreases with more steps, so does the difference.
Otherwise, this graph looks pretty solid! Validation loss is going down with training loss, and they converge at the end, indicating good fit. In the last steps, the training loss has stopped decreasing and plateaus, which means I chose the right number of epochs.
Once I had a working model, I started grading its responses. One interesting observation - the fine-tuned model performs better on diseases that are more frequently represented in the dataset. For instance, it’s accuracy for PKU rose to 62.5% from a baseline accuracy of 50% (even then, some of the zero-shot predictions determined as “correct” only captured the broader class, Hyperphenylalaninemia).
As for the overall top-10 accuracy, the fine-tuned model hits 25.48%, increased 1.67% from the baseline of 23.81%. I’m happy with improvement, no matter how small, but I think I can do better if I try hard enough. The (figurative) learning curve starts slow, but we can get there!
Back to work 💻
- Lucy
Post 9: Blog #8: MORE DATA :D
Heya!
We’re now on a countdown to the last blog post.
This week, I revisited prompt engineering, specifically chain-of-thought. That’s just a fancy way of saying I added “Let’s think step by step” to the prompt, in hopes of encouraging intermediate steps that lead to better accuracy.
After running my fine-tuned model with this improved prompt, I have yet another spreadsheet of model outputs. From inspecting the first ten responses, they don’t seem significantly different, though I can’t know for sure until I grade them.
Throwing back to Blog #7: remember how the fine-tuned model performed better on more represented diseases? While that’s good and might imply the model learned from the data, the accuracy for less represented diseases didn’t improve as much.
To combat this lack of data, I went looking for data. I couldn’t find many real clinical cases online, but while playing around on the Orphanet website, I discovered Orphadata’s “Phenotypes Associated with Rare Diseases” XML file. One thing I found interesting: instead of just listing the phenotypes for each rare disease, it also states how frequently each phenotype is expressed. I cleaned and reformatted the data a bit, added it to my training data, and ran fine-tuning. With the test data, I again asked the model to generate responses.
I’ll probably grade the two sheets of model responses this weekend.
Thinking of catching these smaller cases made me think of using a retrieval-augmented generation (RAG), so I dived into another research rabbit hole. Essentially, the model retrieves a relevant document from a database and refers to it while answering, usually increasing accuracy and lowering hallucination. Though the aim of my project is just fine-tuning, I’m kinda curious how using RAG on top of the fine-tuned model would turn out…
Welp. Art is never finished, only abandoned. Starting next week, I’ll begin wrapping it up, building a website, and working on the poster.
Thanks for reading!
- Luoxi
Post 10: Blog #9: Results and Housekeeping
Hello and welcome back!
The end is in sight. Thanks for being along for the ride. This week, I finished grading the model outputs, and here are the results:
Zero-Shot | Finetuned 1 | Finetuned 1 with Prompt Engineering | Finetuned 2 with Prompt Engineering
Hit@10 | 23.81% | 25.48% | 27.86% | 27.86%
Macro-Averaged Hit@10 (with Multi-Label Credit) | 24.79% | 23.42% | 25.06% | 28.19%
For a reality check, it would have been nice to have human doctors diagnose the patients in the test set. Since I couldn’t connect with rare disease experts, I approximated the Hit@10 from randomly guessing. There are 445 classes (rare diseases) and 420 patients in total. To simplify my calculations, I made two ideal assumptions:
a) Each patient has one rare disease
b) The rare diseases are evenly distributed in the test set
So, if I randomly guess 10 diseases per patient, there’s a 10/445 chance of being correct. The expected number correct guesses is 10/445*420, giving a Hit@10 of about 2.25%.
Through fine-tuning and prompt engineering, the overall Hit@10 increased by 4.05%, or 17.01% of the baseline. Not bad. There were other techniques I wanted to try, like more hyperparameter optimization and RAG, but I probably don’t have the time or budget for them. Still, they might boost performance further.
While we treated the rare diseases as evenly distributed in our random diagnosis scenario, this assumption isn’t true, even if it’s good enough for an approximation. Because of imbalanced datasets, Hit@10 doesn’t fully reflect performance (as an extreme example, if 75 out of 100 test patients all have the same disease, a model diagnosing that one disease every time would get >75% Hit@10, even if it isn’t helpful).
To account for the accuracy across different diseases, I calculated the macro-averaged Hit@10 values with multi-label credit. When a case is marked as correct, the success is attributed to all disease labels of the patient (explaining why some macro-averaged Hit@10 values are higher than the plain Hit@10). Multi-label credit isn’t ideal, but it suffices for our purpose. If I had the time and patience to annotate all of the correct LLM guesses in each case instead of just marking “TRUE/FALSE” at the patient-level, I would have preferred using Hammings Loss.
Anyway, I found it interesting that overall Hit@10 increased while Macro-Averaged Hit@10 dropped after my first round of fine-tuning without prompt engineering. This suggests that while the LLM did better on more represented rare diseases, it got worse for the rarest. Similar to the 75-patient scenario, I suspect the model overpredicted more represented diseases. It likely picked up this behavior from the training set, which has a similar distribution of rare diseases. Instead of purely learning the patterns for diagnosis in the training set, it might have learned which diseases were more common and was rewarded for guessing correctly.
Fortunately, I hopefully suppressed this behavior through incorporating Orphadata rare disease-phenotype associations to better balance the training set. After the second round of fine-tuning with prompt engineering, macro-averaged Hit@10 rose to 28.19%, the highest across all models. While its plain Hit@10 matches the previous model, it performs better across all rare diseases.
With fine-tuning finished, I zoomed out and realized my repository was a bit messy. CSV files were everywhere and the lack of (or poor) commenting made the project hard to read. So, I spent a good amount of time on housekeeping - reorganizing files into folders, adding comments, renaming variables, etc.
Now that I have a working model, running it from the command line started feeling too clunky. I built a basic website to host it. It’s simple - just vanilla HTML/CSS/JS with a Flask API. This week, I focused primarily on the backend to get the model up and running, though the user interface is still kinda ugly.
For the rest of the time, I’ll probably continue polishing - improving the UI, writing the README, and putting together the final poster.
See you next week!
- Luoxi
Post 11: Blog #10: Journey’s End
Hey All!
Welcome to my final blog post. I’m pretty sad this will be over soon, but come right along - I have so much to share this week.
This week, I laid the finishing touches: designing the UI, commenting, and creating a poster.
For the UI, I mainly fixed the CSS (I figured React is probably overkill for a textbox and a button). I don’t know about you, but I think this is pretty good:
(Okay it might be giving 2010’s Raz-Kids… I can’t quite put my finger on the problem though…)
As crusty as the UI is, at least the backend works. Basically, on POST, Python Flask sends the input to the RunPod endpoint (where my model is stored and ran) and gets the output as a JSON file. Meanwhile, vanilla JavaScript displays a cute loading GIF and then the output text.
And because open source is awesome, I uploaded my code to GitHub and my fine-tuned model to Hugging Face.
With all the logistics sorted out, I began working on my poster. Aside from writing and designing a flow chart, it wasn’t much work beyond compiling graphs onto one ppt.
And voilà. This is my project for the time being.
Before I go, I want to shout out Ms. Rangoli, Dr. Pan, Stack Overflow, the Hugging Face Forums, r/LocalLlama, the Runpod Discord Community, and you, reader, for supporting me along my journey. I couldn’t have done it without you guys.
Truly, I enjoyed the project so much. This sounds cheesy, but it was never just about attaining a working product. For an amateur high school student (me), the journey and the little things were equally important. Exploring the nooks and crannies, playing around with code, learning random facts from papers, and the kind. Some of my favorite moments include:
- Reading the docs of my first cloud compute service, RunPod
- Playing Tetris while waiting for my model to finish uploading to Hugging Face
- Googling PKU
- Staying up until 2 a.m. watching YT videos about RAGs because I have free will and know how to use it
Of course, it wasn’t all fun and games. Spending an hour debugging only to find out you forgot to keep everything on the same device (= torch.device(“cuda”)) isn’t exactly the most pleasant experience. However, these mistakes sure still were experiences, and I’ve definitely learned to be more thorough through them.
Welp. I guess this is the end (of the beginning). I hope you found this blog worthy of your time, and I’ll see you around on this vast ocean of an Internet!
Signing out,
- Luoxi