Week 1: Just the beginning...

Hello everyone!

Welcome back to my senior project blog. I spent this week mostly absorbing information and planning my initial approach to the problem. Let’s get straight into what I learned and why setup is crucial before jumping into experimentation!

Best Linear Algebra Online Course Ever

Important to neural networks is linear algebra, an area of math that I can’t seem to grasp no matter how many times I learn it. Linear algebra seemed like a bunch of random arbitrary rules that have no meaning. However, after going through Gilbert Strang’s MIT linear algebra lectures(highly recommend if you want to self learn), I found that there is a lot of intuition behind concepts like matrix multiplication, row echelon elimination, invertibility, linear combinations, spans, etc. He would refrain from stating rules until the intuition behind the rules are realized, which made learning a lot easier. On the website, there are even practice questions for students to do.

If anyone wants to follow these lectures, I recommend watching at 1.5x speed.

Reviewing Neural Networks

I went through the neural networks course I took on Coursera to remind myself of the structure of neural networks and the different kinds of activation functions such as ReLU, logistic regression, and softmax, and different loss functions like mean square error(not really used in practice), and binary cross entropy.

Setting up Github and virtual environments for jupyter notebook

Some set up before doing the project.

I will use Github to keep track of progress and virtual environments and jupyter notebook to set up the right python version and libraries for the project. This project will be coded using Pytorch.

Planned Approach

I will be solving the system of ODEs numerically. The neural network will be trained on a specific system, and will take in parameters as inputs and the missing variable(s) as the output.

Why this is important

The last time I went into a CS project without good planning, I had to reprogram everything on the last day. That was super painful…because I also had a presentation on my program that day.  While simply guessing and checking all different combinations of neural network structures, activation functions, and loss functions might work, that won’t give me a good understanding on what the neural network is doing. If I don’t understand what I am doing, it will take me ages to figure anything out.

Thanks for reading and see you in the next blog!