Detecting Fast-moving Near-Earth Objects with a Novel Deep Learning Algorithm
Advisor: Dr Sanhita Dixit
Internship Location: University of Florida
Onsite Mentor: Dr. Jian Ge
Near-Earth Objects (NEOs) pose a significant risk to humanity. Just over 10 years ago, the Chelyabinsk meteorite collided with Earth, releasing 30 times the energy of the Hiroshima atomic bomb and injuring ~1500 people. NEOs like the Chelyabinsk meteorite are estimated to collide with Earth every 10-60 years, yet only 0.1% of these NEOs have been tracked. Major factors contributing to the low detection percentage include the time-intensive pipeline for NEO detection and current softwares’ insensitivity to faint, fast-moving NEOs. Especially fast-moving near-Earth objects that pass dangerously close to Earth leave a visible streak in telescope images. Existing research has already proven the efficacy of utilizing synthetic streak training data to train convolution neural networks (CNNs) for NEO detection. In this project, I seek to further increase the accuracy and efficiency of existing algorithms in detecting NEO streaks in images from the Zwicky Transient Facility (ZTF). I also aim to develop an efficient postprocessing pipeline to further augment the NEO identification and verification pipeline.
Project Blog
Week 0: No One Likes Meteorites
Hello earthly specimen! I know my title and abstract are a little jargony, but my project essentially boils down to — finding the asteroids that can hit Earth fast! Asteroids that have a possibility of hitting Earth are labeled as near-Earth objects or (NEOs). More specifically, asteroids are classified as NEOs if their distance of closest approach to the Sun is less than 1.3 AU (where AU is the distance from the Earth to the Sun). According to the B612 Foundation, a nonprofit foundation dedicated to planetary science and planetary defense, only 0.1% of an estimated 3 million NEOs have been detected. Now that’s scary! Many of these NEOs remain undetected due to their small size as well as their high rate of motion.
NASA recently completed its Double Asteroid Redirection Test (DART) mission, proving that we can effectively divert asteroids headed on a collision course with Earth. All we gotta do now is find those suckers! And I think I might just have the right tools and insight to do so. My solution contains three main components:
simulating training data
developing an improved machine learning model for detecting near-Earth objects in telescope data
innovating an improved post-processing pipeline to streamline manual identification and verification of detected streaks.
Fast-moving NEOs that pass dangerously close to Earth leave a streak in telescope exposures. I’ll be trying to identify these streaks in telescope data from the Zwicky Transient Facility (ZTF). I will utilize convolutional neural networks (CNNs), a leading algorithm in computer vision, to identify streaks in telescope images from the ZTF database. CNNs and computer vision algorithms in general require a lot of training data to accurately classify new data. Since there is a limited number of confirmed detections of near-Earth objects, I will simulate training data streaks using the Gaussian Point Spread Function. This allows me to both have a larger training dataset as well as target specific demographics of streaks (e.g. short/faint streaks).
Current algorithms for near-Earth object detection utilize the EfficientNet architecture, a high-performing, popular model for computer vision, but EfficientNet is specialized for processing 3-channel RGB images while data from ZTF is black-and-white. Using the standard Conv2D layers that Keras, a popular machine learning library, offers, I hope to develop a more lightweight and accurate model to process telescope data.
Lastly, when the model finishes processing through full nights of data, it will output a bunch of images in which it thinks there is a streak. Going through these one-by-one manually is both challenging and time-consuming. I want to code up some type of interface to make the process more intuitive and efficient.
And yeah, that’s my Senior Project as well as the objectives I hope to complete. I’ll see you all next week!
Week 1: Deepfake Meteorites
Hello fellow inhabitants of this pearly blue planet! If you’ve forgotten what my project is already due to the extremely wordy and unmemorable title and description, I’m essentially working on finding asteroids that might collide with Earth using machine learning.
In order to detect near-Earth objects (NEOs) with artificial intelligence, we need large amounts of training data to teach the computer algorithm to accurately recognize which images depict asteroids and which images do not. For this project, I’m using data downloaded from the Zwicky Transient Facility (ZTF), which is a telescope observatory based down in Caltech. ZTF data is quite special in two ways — it has a relatively short exposure time of 15-30 seconds and it surveys the entire night sky every two days. This makes it optimal for identifying asteroids across multiple nights of data and extrapolating a more accurate orbit from the detections.
Oh, and that brings up another topic — how will I determine if the asteroid will collide with Earth after I detect it in telescope exposures? Using a program called find_orb, I simply plug in the positions of the observations and the program will attempt to fit orbit parameters to the detections (with lower error when more detections are inputted). So yes, theoretically I could detect a civilization-ending asteroid and yell at NASA to redirect the asteroid and save the world (but highly unlikely).
Anyways, back on topic, training AI to find spooky, scary NEOs. A common tactic in computer vision is to have a large set of real, human-labeled training dataset to teach algorithms what in an image is a bicycle and what’s not, for example. But, unfortunately for us, NEOs are rare and hard to find, meaning that there is a limited number of real detections to train an algorithm on. In addition, if we were to use previous detections as training data, the algorithm would be biased to detect brighter NEOs as those are easier to spot and identify. The algorithm would not be exposed to as many fainter and harder-to-detect NEOs and would ultimately fail to detect these objects in real data.
But, fortunately for us, when NEOs pass by quickly in telescope exposures, they leave a streak that have a nice, predictable spread of brightness, or more quantitatively defined as the Gaussian Point Spread Function. Using this already defined distribution, we can simulate streaks, effectively creating deepfakes of asteroids to generate more training data (though technically it’s not generated with deep learning). How smart! This week, I’ve been implementing this streak simulation, and its mostly working. I am making improvements to the efficiency of the simulation algorithm so that it can generate large training datasets.
Until next week! Sayonara!
Week 2: The Modern David and Goliath — Small, Lightweight AI Model vs. Big, Bad Asteroid
Hello my fellow cosmic mariners helplessly stranded on this spherical rock we call Earth! So, brief rundown: big, bad asteroid zooms around in space; I try to find big, bad asteroid with little AI model; but, oh no, not enough training data to teach little AI model to be smart enough to find big, bad asteroid; but, oh yes, I will “deepfake” asteroid images to teach little AI model how to find big, bad asteroid.
So last week I successfully generated the “deepfake” asteroid training images to train the little AI model, now I need to build the little AI model. Before going into the details of the type of model I’m going to be using, let’s first talk about the objective that the AI is trying to complete.
The AI will be fed two 80 by 80 pixel, black and white photos of the night sky, one reference image and one science image. The science image is a single snapshot of the night sky at a certain time, which can possibly include an image of an asteroid. On the other hand, the reference image is a bit more complicated, but it can be thought of as an average/median of multiple science images, which removes the possibility of having any transients in the image (e.g. if an asteroid were to pass through that region of the night sky in one science image, it is highly unlikely to remain in the same position throughout other science images, and when the median is taken, the image with a streak will be considered an outlier and eliminated).
The reference images are provided by the Zwicky Transient Facility (which is the observatory I’m acquiring all my data from). Below is an example of the data being fed into the machine learning model (on the left is a science image which happens to have an asteroid streak in it and on the right is the reference image).
In this case, I want my machine learning algorithm to output 1, or “yes there is an asteroid in this image.” In the case where the science image does not have a streak, and the science image and reference are similar, I want the algorithm to output 0, or “no there is no asteroid in this image.” This may seem like a pretty simple task, but due to noise, cloud coverage, and heat fluctuations in air, comparing the two images becomes a much more complex problem.
This type of image classification task has recently been dominated by a machine learning algorithm called a Convolutional Neural Network. And Convolutional Neural Networks are a powerful subset of deep learning algorithms (title drop! — “Detecting Fast-moving Near-Earth Objects with a Novel DEEP LEARNING ALGORITHM”).
This blog is already getting quite long, so I’ll end it there. I’ll continue talking about what convolutional neural networks are, what specific type of convolutional neural network I’ll be using, and what makes it novel next week!
Until then, happily continue your deterministic journey through space and time!
Week 3: CNN — Convolutional Neural Networks, Cable News Network, or Confusing Noisy Nuisances?
Hello my fellow collections of atoms, so uniquely arranged that we classify one another as “humans.” Where were we again? Oh right — asteroids and computer vision and convolutional neural networks!
You might be asking, what really is a convolutional neural network? And I’m here to answer that question — a convolutional neural network is simply a type of neural network that uses convolution layers to process images! Wow, I’m truly an expert.
Neural networks — neurons, brain-emulation, bunch of nodes (circles) with weights (lines) between them, value of a node is the sum of the weights times values of nodes from the previous layer of the network, weights are tuned as the network is trained, eventually result in an output node from the network (that’s hopefully close to the desired output).
Now imagine trying to input a 2D image into the linear input — that’s kinda difficult right? Especially when you’re trying to maintain info like color groups and relative positions.
Instead, what if we tack on something to the beginning of this neural network to smash a 2D image down to a 1D array while maintaining most of the important 2D information? We do this by using a “2D weight” between layers instead of the multiple linear connections between the layers in a traditional neural network. These “2D weights” are called filters or kernels. Essentially, they select (or filter) out specific characteristics from an image in its 2D representation and represent it concisely in a more compressed output format (these filters can be adjusted and tuned just like a regular weight). Eventually, this compression simplifies the 2D input matrix into a much smaller 2D matrix (that hopefully, if all went well, contains most of the important 2D information from the original input matrix). This smaller 2D matrix is smushed or flattened and fed in as an input to our traditional neural network. Now why is this method called a convolutional neural network? Because some of filters applied to the input image are called convolution layers (there are other layers called pooling layers but they’re less cool sounding than convolution layers). Here's a nice diagram:
I know that was a mouthful, and just know that you don’t need to understand any of that to understand what I’ve done in my project.
Now, there’s some real smart computer scientists that made some real good convolutional neural networks for finding some real complex stuff in real high resolution images. The current best-performing models out there are those from the EfficientNet family. These models are able to very accurately complete extremely difficult image classification tasks — computer scientists are sometimes kinda very smart!
But, thing is, the image classification task I’m trying to tackle is not all that difficult, essentially it boils down to “is there a bright line in image?” Hence, the EfficientNet model might possibly (foreshadowing!!) be a bit too bulky and smart to be efficient in detecting asteroids. Imagine using an axe to cut paper (it’ll cut it but it’s not as efficient as using a pair of scissors).
So, I’ve settled on creating a more simple, dumb-downed architecture using the convolution and pooling layers from the popular Keras Library. After some extensive testing, I’ve created a model that is both lightweight and efficient, more efficient than EfficientNet believe it or not! So, in a way, I beat the top machine learning computer scientists in a contest that they didn’t sign up for! But we take the win!
Anyways, that’s about it for this week, I’m gonna be applying the model(s) I have generated to some real data (remember everything that I’ve done so far is on deepfake simulated data) and reporting the results.
It’s been your fellow human being smashing their life away on a plastic keyboard! Until next time, peace!
Week 4: Donkey
Hello my fellow insignificant quantum wave functions oscillating fruitlessly in the great abyss that we call our universe!
Soooo. How’s it going? Genuinely, how are you doing? (not that you can respond, but you know, just take a second after reading this sentence, to just take a deep breath, a nice long inhale and exhale, 'cause this blog might be slightly more difficult to bear with than previous ones, but trust me, it’s interesting)
What? Are you telling me to stop stalling and to get to it? Jeez, stop rushing me, I’m just checking in on my dear readers’ wellbeings and definitely not trying to stack up on the word count for this blog.
Alright, so last week I mentioned that I had very possibly created a machine learning model that was better at detecting asteroids than the machine learning models that the very smart computer scientists had developed. And this week, I tried to prove it!
But before I show you the cool data that proves my model is slightly, potentially, maybe, possibly better, you might be wondering — “So how did you create this model? Did you make it from scratch? Did you steal it from another person’s github without abiding by the copyright licenses they had on their repo? Did you pull it out of your donkey (replacing all instances of donkey with a popular synonym will provide for a much more colorful read for the rest of this blog)?”
No. I’ve done most of those things before, but not for this project.
Getting a machine learning model to perform the task at hand is kinda like getting a perfect sculpture of a donkey. EfficientNet (the big, smart, crazy-good model that the big, smart, crazy-good computer scientists created) is like a hand-sculpted marble sculpture of a horse (that looks really nice and could pass for a donkey, but is ultimately is not a donkey). And this marble sculpture you or I can just order, already completed and pre-packaged.
On the other hand, there’s this library called Keras, which basically provides components of a machine learning model that you can assemble by yourself to create your ideal model — or to continue the analogy, lego blocks that you and I can order and assemble together into a donkey. Of course, lego blocks are no donkey, but after enough work and polishing, perhaps the lego blocks can be built and shaped into something which looks a lot like a donkey, more like a donkey than a marble horse.
So, for this project, I used the lego blocks and constructed numerous different models to see which one would look the most like a donkey. You might be wondering (you’re probably not actually wondering) just how many models did you create and test? 40+ models, and multiple configurations for each model. And it’s not just like testing these bad-boys is easy work, each test takes about a day to run. You might be doing a little math in your head — it’s Week 4, 7x4=28 — 28 < 40+ — something ain’t adding up! (keep this down low but I might have done a little testing work before the official start of the senior project)
Alright, anyways, so after a whole lot of building and testing, here’s the most donkey-resembling model I found.
Looks pretty simple but just know that it took a lot of work to get here (just a special note of gratitude to my mentor and my fellow mentees if they ever read this — they deserve quite a lot of credit).
Dang this blog got way too long way too fast! Guess the juicy data that you were waiting for to prove that my model is more donkey-looking than EfficientNet will just have to wait till next week!
Until then, donkey out!
Week 5: More Efficient than EfficientNet?
Hello fellow consciousness — have you ever wondered, is it really you reading this blog or is it just that blob of consciousness floating around in the fixed region that is your skull?
Ok, finally I’m gonna reveal the juicy data that proves that my AI model is better than the big, bad (bad meaning good) EfficientNet model invented by big, bad computer scientists (if you have no clue what I’m talking about, take a look at my Week 4 Donkey blog, trust me it’ll make much more sense).
Anyways.
Boom. Table. Trainable parameters are basically the parameters in an AI model that are trainable! If you imagine a machine learning model to be a brain, the trainable parameters are essentially the number of neurons (or perhaps inter-neural connections) in the brain. More neurons means smarter right? Yes…technically, BUT, that doesn’t necessarily mean EfficientNet is better because it has more trainable parameters and is therefore “smarter.”
More trainable parameters also means that the model is more clunky or that the brain is heavier. Imagine if you wanted to exploit a brain to calculate “1+1” (a calculation that honeybees can allegedly do — https://www.cnn.com/2019/02/08/health/honeybees-learn-math-study-trnd). You wouldn’t want to carry around a 3-pound human brain just so you can calculate “1+1” would you? You would much prefer a conveniently-sized 2-milligram honeybee brain! Less trainable parameters means a more lightweight model that is faster to train and utilize.
Imagine, for a moment, a heckling computer scientist yelling, “BET YOUR MACHINE LEARNING MODEL IS AS DUMB AS IT IS LIGHTWEIGHT! ”
Bam. Graph. On the y-axis is the true positive rate or TPR (technically it’s detection completeness on previous confirmed asteroid detections from the Zwicky Transient Facility) and on the x-axis is the false positive rate or FPR. True positive rate is essentially my model’s ability to recognize real streaks, while false positive rate is how often my model is dumb and thinks an image is a streak when it’s not.
You might be wondering, why is it forming a line? Shouldn’t each model just be a point, with a fixed TPR and FPR? And that’s when I applaud you as a reader for paying such close attention and being so acute.
The reason why I can plot a line is because the output of the machine learning models is not simply true or false, it’s actually a decimal number from 0 to 1. Now, I can set a "threshold" let’s say at 0.6, where any number outputted above 0.6 will be a positive while any number outputted below a 0.6 will be a negative. If I set the threshold at 0, then my model will successfully identify all real streaks, with a TPR of 1, while labeling all images without streaks as having streaks in them, meaning a FPR of 1 as well. If I set the threshold at 1, on the other hand, then my model will never successfully identify a real streak, with a TPR of 0, while labeling all images without streaks correctly, meaning a FPR of 0 as well. Taking a range of different thresholds will generate a line plot, called the ROC curve, and that’s what’s displayed in the graph.
An accurate model will maximize TPR while minimizing FPR. And that corresponds to a curve that stretches as close to the top-left corner as possible. So a “higher” curve on the ROC graph means a more precise model.
And guess what? You know which model is “higher” and stretches the most to the top-left corner? The blue one. The self-built one. And you know which model doesn’t stretch as far to the top-left corner? EfficientNet. Mic drop.
My model is more lightweight and more accurate than EfficientNet. End of Proof. QED. ATCGW.
Next week, I’ll be talking about — actually, I don’t know what I’m gonna be talking about. Maybe life’s not about figuring everything in the future out. Maybe it’s just about living in the moment, enjoying the pure, refreshing flow of time — until it slips by us, ever so silently.
Week 6: Darkness & Murder
If you think about it, the space which your brain inhabits is probably pitch black, surrounded by a bony, opaque skull. Your consciousness is fixed in darkness — I’m communicating with your consciousness. Hello darkness my old friend.
You know sometimes I sit down in front of my laptop and tell myself, “I’m gonna write the best blog I’ve ever written.” I get to typing the first word, and I realize — I have absolutely zero inspiration. So I flip back to some of the blogs from previous weeks, read them, and then I come to the conclusion that I’ll never be able to top my former glory. I’ve peaked too early.
Anyways, self-doubt and anxiety aside, last time I talked about the juicy reasons why my model is more efficient than the EfficientNet model. Over the past week, I worked on feeding an entire night of ZTF (Zwicky Transient Facility) data into my machine learning model. The night I chose — January 25th, 2020. Why that day, you may ask? Is it cause Boeing completed the first test flight of their 777X that day? Is it cause 01252020 just looks like a nice number?
Well, according to ZTF’s discovered streaks catalog, the days from January 20th-29th, 2020 have the highest frequency of detections, meaning that it’s more likely for my model to find NEOs (and hopefully previously undetected NEOs as well). Why the 25th? Cause it’s a perfect square, of course!
After applying my model to telescope data from the night of 2020125, I was left with 3825 positive detections. Compared to the number of images that were fed into the model, this number is quite small. However, 4k images is still a lot to process manually. The current algorithm for verifying streaks considers linking together pairs of streaks, and the total number of valid pairs is exactly 17,245. Now if each pair of detections took me about 2 seconds to verify, it would take me about 10 hours to go through all pairs. You know what’s better than spending 10 hours manually sorting through detections? Spending 100 hours improving my algorithms and then spending 1 hour to manually sort through the detections.
There are a few methods I’ve thought of to make my algorithm run faster: (1) finding a way to sort through detections 1-by-1 while maintain info about relative positioning and context, (2) increasing the accuracy of my algorithm further by retraining the model on more targeted data, and (3) increasing processing speed by threatening to murder my algorithm and all its loved ones. Over the next week, I’ll try to implement all three methods and see if any of them are successful.
Anyways, that’s all I’ve got for now! Catch y’all later!
Week 7: Msitakes
Hello itnllegint bineg. Cna yuo raed thsi? Yse? Ins’t tath crzay? Hwo yoru brani porceseess tpyos liek itss nthinog? Maeks me wnoder…do we raelyl nede to splle crrorectyl? Waht iff ew acectped hour typso adn msitakes in eevrydya comunication, fluly ebmracd oru flawsw? Wuoldnt’ teh wrold bee os muhc mroe coolrful adn welcming?
Um. So. You know people make mistakes from time to time. No one’s perfect. Our flaws are what make us all interesting, unique, and, most importantly, human. This week, instead of making progress, I decided to be quirky and different — I decided to make mistakes instead.
Mistake 1: Telescope data takes up space — space as in bytes on a hard disk. Even though I am working on a remote computer with 15 external drives, each with multiple terabytes of space, I somehow still managed to run out. And you know what you naturally do when you run out of storage space? You clean some stuff out! But you know what you don’t want to do when you clean stuff out? Remove important files.
Well um. Ok technically, I didn’t remove any important files. I just deleted a few terabytes of telescope data that I thought I would never use. If I needed the data in the future, I would just download it again, no biggie. Little did I know, when I processed the data, I sorted the telescope exposures based on the time the files were downloaded to the computer — that timestamp, and hence the order of files, will change if I download the telescope data again.
Now this change in order wouldn’t be a problem if I simply saved a list containing the initial ordering. But you know I’m not smart or careful enough to do that!
Turns out, I have to use some of the data that I deleted. Lucky for me, it’s nowhere to be found! Now I have to download the 1.5 terabytes of data, process it all again, and then continue where I left off. Downloading the data takes about 2 days and processing it all again, about 3 days. THAT’S A WEEK GONE!! Anyways. I’m not mad about it, you are.
Mistake 2: Telescope data is complicated. Well, it’s not that bad, but today I’m venting all my frustrations. Essentially, before being able to actually process the image, you need to subtract the background variation, divide by the standard deviation of the background, and then clip off pixels that are either too bright or too dim (usually really bright stars or camera pixels that have gone haywire).
Anyways, I was looking through my image normalization function and discovered that I forgot to subtract off the background variation! I could significantly improve the accuracy of my algorithm if I simply added the code to subtract off the background variation. So I added it, then spent 2 days regenerating training data and evaluating the model. The accuracy dropped by 20%. And not just any 20%, from 99.8% to 80%. Absolutely tragic.
How could this be? Turns out I had already subtracted the image background, right before I called the normalization function in my code, so I was effectively subtracting the background twice. Why didn’t I put this subtraction code in my pre-defined normalization function? Because of the coding etiquette that I never learned.
I’ll end my blog here — before the salty tears, streaming down my face onto my keyboard, short-circuit and explode my computer.
Week 8: The New Hit Gameshow — “Asteroid or Some Random Junk Flying in Space”
Hello stranger. I’ve spent a lot of time brainstorming entertaining introductions. But sometimes I wonder, who am I doing this for? Am I benefitting myself or anyone else in this process? Is anyone actually reading what I’ve written? What is my goal in life? Am I even working towards that goal through typing up this blog right now?
Besides questioning the purpose of my existence in this world while staring blankly at my bedroom ceiling, over the past week I’ve been working on manually filtering through the data outputted from my machine learning model. Last last week, I discussed some possible improvements I could make to speed up the manual filtering process — technically, I tried implementing some of them (few of them worked, few of them didn’t). But I got too impatient so I decided to sit down for a solid 10 hours to process all of the original outputs anyways.
Alrighty, let’s play a game called “Asteroid or Some Random Junk Flying in Space,” where you try to guess whether the image I show you is of an asteroid or some random junk flying in space!!
To make this easier, let me first show you what an asteroid can look like:
That’s an asteroid named 2020 AE5 (you’d expect astronomers would be a bit more colorful with their names).
And that’s an asteroid named 2019 LQ1, just another banger name for a huge, rocky behemoth flying space.
Now, are you ready? Let’s begin! Is this following image an asteroid or some random junk flying in space?
And the answer is…drumroll please…a Russian scientific satellite designed to perform research on the structure and dynamics of radio sources, a.k.a some random junk flying in space!!
Alright, next image:
And that’s an image of…the upper stage of a Titan IIIC Rocket launched in the mid-1900s! Also often referred to as some random junk flying in space.
This games harder than you think, ay? We’ll do two more:
That one is an image of…an asteroid…just kidding. Did you really think that one was an asteroid? You silly goose. That’s obviously the Explorer 50 satellite, launched in 1973 to study the magnetosphere!
Alright, last one.
Now that one…is hopefully an asteroid. What do I mean by hopefully? Well that’s a great question.
As you might have discovered, streaks due to asteroids, satellites, and space junk all look pretty gosh-dang similar. So how do I know if what I’ve detected is an asteroid or not? Well it’s all thanks to my favorite Github user, Bill-Gray, who developed two programs, astcheck and sat_id, which automatically check if an object is a previously confirmed asteroid or an artificial satellite, respectively. Now you might be asking (I know you’re not asking but I just love using this transition), what if both these programs output nothing, as in the object is neither a previously confirmed asteroid nor a satellite? Well this could mean three things. (1) This object is simply not in the catalogs of astcheck or sat_id, but is cataloged elsewhere (relatively unlikely cause my man Bill-Gray is just built different). (2) It’s another piece of random uncataloged junk flying in space. (3) It’s a novel freaking asteroid.
And the last image I showed above was neither a previously confirmed asteroid nor a satellite according to astcheck and sat_id. Exciting! Now how will I know if it’s an asteroid or junk? Orbit fitting. But that’s a topic for another day.
Anyways — again, it’s been a pleasure smashing my keyboard at you across the world wide web. I’ll catch you next week.
Week 9: Burnout
Hello my benefactor. If you’ve followed me for this long, and you’ve read all my blogs, I want you to know that I appreciate you. You’ve been my companion through my highest highs and lowest lows in this endeavor that is my Senior Project. Though I don’t know your name and I’ve never seen your face, I thank you for being a faceless beacon of motivation.
Hmm. So. Over the past week, I’ve been pretty locked in, working on a lot of different aspects of this research project. But I’ve found that none of them are interesting or substantial enough to talk about…perhaps it’s a metaphor for life. Each and every day we take on so many pursuits — getting out of bed, brushing our teeth, eating meals, walking around — but when we sit down at the end of the day, we can still sometimes feel so unproductive, so unachieved. If only we could value our each and every second, our each and every action, wouldn’t we feel so much better about ourselves?
And with that mindset, I’ve settled on talking about my explorations in the world of training data manipulation. If you imagine a machine learning model to be a student studying for APs, the training data is the study materials the student uses, and the accuracy of the model is the resulting AP score the student gets. Obviously, if the machine learning model is fed bad learning materials, like a student studying for AP Physics 1 by binging Big Bang Theory, it’ll perform poorly.
This week, I’ve been trying to feed the AI model different learning materials — Barron’s, Princeton Review, Khan Academy — and seeing how the results pan out. In total, I tested four different training demographics:
(1) Lower Dropout — so technically in this situation, I didn’t really manipulate the learning materials, I mutilated the student’s brain instead (changed the machine learning model architecture). So in a lot of machine learning programs, programmers add a thing called dropout, which essentially makes the machine learning model forget a random selection of the stuff it’s learned. You must be thinking, that doesn’t make much sense, why would you want your model to forget the stuff it’s learned?
I’m sure there’s a complex mathematically rigorous explanation, but essentially it’s to prevent this thing called overfitting where the machine learning model fits too well to training data while the training data is not necessarily representative of all possible data demographics, hence lowering general accuracy.
In my initial machine learning model, I had a dropout of 0.2 after each convolution layer, meaning I would essentially make the model forget about 20% of the knowledge it learned and make it relearn it through new training data. I lowered that value to 0.1.
(2) More Uniform Training — ok the previous section was a little bit too jargony and long, so I’ll make this one brief. Essentially, instead of having a more limited demographic of streaks in my training data, I showed it all different kinds of streaks, extremely short and faint ones, extremely bright and long ones, everything.
(3) Centered Streak Training — initially the streaks that I used to train the model had their center within 5 pixels (Manhattan distance) of the center of the 80px by 80px image. I reduced that value to 1 pixel.
(4) Fewer Hard Negatives — hard negatives are images where streaks are implanted in both the science and reference image (you can refer back to Week 2 for a concise definition of these terms). I reduced the number of hard negatives, increased the number of regular negatives, and kept the number of positives the same.
And here are the results (ignore the red dots). Week 5 blog explains in detail how to interpret this graph, but essentially the higher the line, the more accurate the model.
Well, looks like the "fewer hard negatives" model scored the highest on their AP exam. Over the next week, I’ll probably apply this model to real nights of data and see how it does.
You know it’s a sad thing in observational astronomy research — you can develop a great technology, but no one gives a damn until you get substantial results. I’m not burnt out, you are. I’ll see you all soon.
Week 10: The End
Hi friend! Can you believe it? We’re here at the end.
When I wrote the first blog in this series, I didn’t know where I was gonna go to college, I didn’t know how three months of school-less life would go, and I didn’t know if I would be able to complete my senior project.
Today, I know where I’m going to college, I know that the past three months of school-less was probably the most carefree era of my entire life, and I know I’m not going to be able to complete my senior project on time. Damn. The fruits of retrospection.
I could talk about what I did this week, but it was essentially just a bunch of coding, paper-writing, and head-bashing, nothing extremely interesting. So instead, I’ll talk about something I’ve been developing here and there throughout the entire senior project.
STREAK TRACER. I wanted to come up with a cool name for my innovative program — clearly I failed. Ok basically, remember how my project is like using an algorithm to detect asteroid streaks in the sky? So assuming that that algorithm works and some streaks have been detected, this streak tracer program helps identify any missed detections of the same streak on the same night.
How does it work? Well, I’m glad you asked. So these asteroids go vroom vroom through the sky, and in one night, it’s motion is generally a pretty straight line at a pretty constant velocity (ignore the convenient generalizations I’m making here). Now, if I have two or more detections of an asteroid at different places and different times, I can plot these two points in a “spacetime cube” with the x and y axes (RA and Dec axes for you nerds out there), and a time axis. Wow.
Quick math review: a line can be uniquely defined by any two points on that line (in like normal middle school geometry — I’m not mathematician don’t come after me). Now if we have two separate detections, we can draw a 3D line between those two points in the “spacetime cube.” If you plotted the asteroid moving through the sky in this spacetime cube coordinate system, it should, ideally follow that line.
To find possibly missed detections of the asteroid, we can just find intersections between the 3D line and the small squares where telescope images were taken! And here’s a few plots of it in implementation:
Now, I just gotta find some images of novel streaks and apply this algorithm to maximize detections!
I know this blog wasn't the most satisfying resolution to ten weeks of hard work, but trust me, I'm still gonna be working on this project over the next few months. I've almost completed a polished pipeline for processing full nights of data, and once that gets running, the new detections will (hopefully) come pouring in. I'm also writing a paper that I'll be submitting to MNRAS (Monthly Notices of the Royal Astronomical Society) to report my findings.
This is the last senior project blog I’m ever going to write. It’s truly been a pleasure sharing my highs, lows, and everything in between with you for the past 10 weeks. Signing out. Peace.