Week 6: Darkness & Murder

If you think about it, the space which your brain inhabits is probably pitch black, surrounded by a bony, opaque skull. Your consciousness is fixed in darkness — I’m communicating with your consciousness. Hello darkness my old friend.

You know sometimes I sit down in front of my laptop and tell myself, “I’m gonna write the best blog I’ve ever written.” I get to typing the first word, and I realize — I have absolutely zero inspiration. So I flip back to some of the blogs from previous weeks, read them, and then I come to the conclusion that I’ll never be able to top my former glory. I’ve peaked too early. 

Anyways, self-doubt and anxiety aside, last time I talked about the juicy reasons why my model is more efficient than the EfficientNet model. Over the past week, I worked on feeding an entire night of ZTF (Zwicky Transient Facility) data into my machine learning model. The night I chose — January 25th, 2020. Why that day, you may ask? Is it cause Boeing completed the first test flight of their 777X that day? Is it cause 01252020 just looks like a nice number? 

Well, according to ZTF’s discovered streaks catalog, the days from January 20th-29th, 2020 have the highest frequency of detections, meaning that it’s more likely for my model to find NEOs (and hopefully previously undetected NEOs as well). Why the 25th? Cause it’s a perfect square, of course!

After applying my model to telescope data from the night of 2020125, I was left with 3825 positive detections. Compared to the number of images that were fed into the model, this number is quite small. However, 4k images is still a lot to process manually. The current algorithm for verifying streaks considers linking together pairs of streaks, and the total number of valid pairs is exactly 17,245. Now if each pair of detections took me about 2 seconds to verify, it would take me about 10 hours to go through all pairs. You know what’s better than spending 10 hours manually sorting through detections? Spending 100 hours improving my algorithms and then spending 1 hour to manually sort through the detections.

There are a few methods I’ve thought of to make my algorithm run faster: (1) finding a way to sort through detections 1-by-1 while maintain info about relative positioning and context, (2) increasing the accuracy of my algorithm further by retraining the model on more targeted data, and (3) increasing processing speed by threatening to murder my algorithm and all its loved ones. Over the next week, I’ll try to implement all three methods and see if any of them are successful. 

Anyways, that’s all I’ve got for now! Catch y’all later!