Week 9 – Fixing Issues With Colorizing And Audio Models

Overview 

This week, I tried to fix some of the issues I found with my models from last week. Although I failed to break new ground with my audio alteration model, I did find ways to improve my colorization model.   

Audio Alteration 

For this post, I’ll start off with my progress with the audio alteration model before moving on to the good news with the colorization model. For my noise reduction method, I sought to find a way to differentiate sounds from background static. My original idea was to plot the standard deviations of blocks of a certain number of frames of frequencies and observe any patterns. By doing this, I hoped to find areas where audio fluctuated the most, as the model would confuse those areas to be static the most. Below you can see the graphs of the audio vs the standard deviations of the audio data.    

However, I wasn’t able to find any major patterns in the data and had to follow another approach, one that I will describe in more detail next week.   

Colorization 

As for my colorization model, I was able to get the model to run on images of dimension 512×512 pixels, quadrupling the size of images it ran on before. I aim to see how large of an image the model can run on in the coming days. Any improvements to the size of the image it can run on would translate to better resolution of videos produced, as input images would not have to be shrunk more to be colorized and more textures and details can be preserved.    

Next Steps 

My main next steps lie in audio alteration. I’ll continue working on a new approach to better detect certain sounds and distinguish them from static. As for my colorization model, I’ll keep limit-testing how large of an image the model can run on and note any changes in the resulting videos.