Week 8: Working Through RAM Issues

This past week, I’ve spent most of my time working on problems with gradient descent. While trying to solve that problem, I also ran into another one: my random access memory (RAM) was being used up too quickly in Google Colab (the coding editor I was using).

In Google Colab, you are only allowed a limited amount of memory for use. Because my model was performing thousands of operations on 60,000 data points (as each 30-second EEG recording sample was recorded at 2000 Hz), my RAM depleted almost instantly whenever I ran the program. This caused an error that would completely stop my program.

There are a few methods to solve this issue. One is through the use of the garbage collection (GC) module in Python. Python’s primary way to manage memory is through something called reference counting, in which Python counts how many variables are pointing to the data point in question. Once that count reaches zero, Python deletes and frees up that memory. However, this default method does not account for circular references (when two variables are pointing to each other). The GC module looks for these circular references and deletes them, which is especially useful for memory-intensive programs like neural networks. Using the GC module, I was able to free up a lot of RAM.

Another method is through the very basic signal processing technique of downsampling. As the name suggests, downsampling involves reducing the sampling rate of a signal by a certain factor. Naturally, having less data would allow me to decrease my memory usage.

My main goal next week is to find more ways to reduce my memory usage while also improving my model’s backpropogation.