Week 9: Testing Matters

Hello and welcome back to another edition of my blog.

If there’s one thing I’ve learned through this senior project, it’s that no project ever goes smoothly. I’ve realized the importance of testing again and again, and giving ample time to modify ideas and debug. As the end of my project approaches, I’m going to focus less on adding new features and more on ensuring my existing ones can work together properly. I’ve decided to scrap the extension idea and stick with the original plan. 

This week, I’ve been putting all the parts together and testing the whole system. I’ve realized that many of my ideas were overly theorized and I probably should’ve done this earlier, but all I can do now is learn from that and test as extensively as I can. I’ve been mimicking a model student, and asking the system if it can help me with my weaknesses. It does well when explicitly asked to refer to memory, but fails to account for it when not asked to. I’ve changed the prompting process to reflect this, adding more explicit pointers and guiding the model to act as I want it to. 

I’ve also decided to change up my memory retrieval algorithm. Right now, the process is still consuming too many tokens, which is why I’ve changed to having an automated script search for keywords instead of having an LLM comb through the entire AP data document. This should help the entire system run faster and save on tokens.

As the project deadline approaches, I’ll continue testing and iterating. See you next week for a final product!