Week 1 – Introduction

Welcome to my Senior Project! This first post will cover the basics of the project I hope to build, and my posts in the following weeks will detail my steps in creating a model that can colorize film and refine audio. My proposed timeline is to take the first 4 weeks to build a graphical alteration model, the next 4 weeks to build an audio alteration model, and the final 4 weeks to integrate the two together.    

Background 

A similar undertaking in the past couple of years inspired the main idea of this project. Peter Jackson, director of the documentary They Shall Not Grow Old, colorized black and white WWI footage of the British Armed Forces. For audio, he utilized voice actors and sound effects to mimic the sound environment of the WWI scenes. My model seeks to build upon his work by incorporating audio refinement as well. Rather than using his hyper-specific approach of creating new audio for the film, my model will work to refine the audio it is given to produce an overall improved video.   

Graphical Alteration 

In order to colorize black and white film, one part of my model will splice the video into individual frames, colorize each frame, then join them together into a colorized video. Grayscale images are nothing more than arrays of integers ranging from 0 to 255, with numbers closer to 0 representing darker shades and numbers closer to 255 representing lighter shades. Colored images (RGB) are not that different. Instead of one layer of pixels, there are 3 layers of pixels with integers ranging from 0 to 255, each determining how red, green, or blue an image is. The combination of these layers produces the final colors we see. The particular function I will use would be self-optimized by a neural network as it looks for features that link grayscale to RGB. 

Audio Alteration 

Through a method known as signal processing, one can be able to separate different sounds in a scene based on their frequencies. One can also apply filters to each sound to refine the sound (eliminate the noise behind it). Through this, the audio alteration aspect of my model can not only eliminate the static commonly found in old audio but also refine the existing sounds. These effects combined would produce an overall modern piece of film.  

Updates 

This week, I began work on the colorization model. As of now, it can colorize singular images. In other words, the model can colorize a frame of a sample video but is not advanced enough to colorize all the frames together. Next week, I will work on developing the model to colorize entire videos (multiple frames at a time). Thank you for reading this post, and I hope you have fun reading the blog posts in the coming weeks!   

Citations 

Billington, A. (2022, May 10). Tons of vintage 35mm movie trailers scanned & uploaded to YouTube. FirstShowing.net. Retrieved November 13, 2022, from https://www.firstshowing.net/2022/tons-of-vintage-35mm-movie-trailers-scanned-uploaded-to-youtube/  

Bria, B. (2022, October 31). How a gremlins 2 deleted scene directly tackles one of cinema’s great controversies. /Film. Retrieved November 13, 2022, from https://www.slashfilm.com/1073324/how-a-gremlins-2-deleted-scene-directly-tackles-one-of-cinemas-great-controversies/  

Hart, M. (2021, March 30). Artist uses AI to make realistic pics of historical figures. Nerdist. Retrieved November 13, 2022, from https://nerdist.com/article/artist-uses-ai-realistic-pictures-historical-figures/  

McCarty, N. (2021, January 11). Ai can’t color old photos accurately. here’s why. Scienceline. Retrieved October 12, 2022, from https://scienceline.org/2021/01/ai-cant-color-old-photos/  

Rothmann, D. (2021, August 6). The promise of AI in Audio Processing. Medium. Retrieved October 12, 2022, from https://towardsdatascience.com/the-promise-of-ai-in-audio-processing-a7e4996eb2ca