Week 3 – Improvements To Video Alteration

Overview 

This week, I improved on the video colorization and tried to output a more acceptable video format. Last week, I outlined three major areas for growth: frames per second conversion, model colorization, and video dimensions. As of now, these three attributes are for the most part rectified. Let’s delve into how I approached these problems.   

Error correction 

First of all, the issue of frames per second conversion was simple enough.

I simply had to change certain parameters with the method I was using. In this code snippet, I use the OpenCV method Video Writer to append multiple pictures from a certain folder into a video. When merging these pictures together, one can directly control how many frames are present per second. Since the default fps of most videos today is 24, I used 24, preserving the fps of the original video.   

Secondly, to tackle the issue of video dimensions, I decided to approach the problem differently. Rather than waste time trying to get the model to accept pictures of different dimensions, I would transform the image into its original dimensions after the model colorized it. So, if the video had dimensions of 1500 × 1000, the model would transform each frame into 256 × 256 for colorization. Then, the colorized images would be transformed back to 1500 × 1000 and merged into a video. This means that while the resolution would decrease, the video dimensions would stay the same. Furthermore, as I will show in an example later on, this drop in resolution does not hinder video quality by much.   

Finally, the last problem I tackled this week was the colorization of the model itself. In order to have a more accurate colorizing model, I tried training the model for more epochs. Last week, I only trained the model for 3 epochs. This week, I trained the model for 20. I tried training the model for 50 epochs but saw a decrease in performance (the model became worse at colorizing). This is probably due to the fact that the model became overfitted, where the model became too accustomed to colorizing specific images and in turn became worse at colorizing a general image.

The culmination of all these corrections is shown below. I have also added last week’s video and the original video for comparison.  

Next Steps 

As you can see, the colorization of the current video is still not perfect. In order to achieve better performance, I will have to look into other colorizing models. If I still cannot make progress on that front, I will simply move on to the next step of this project: audio correction.