Ashley H. 2023 | BASIS Independent Silicon Valley
- Project Title: AP Research: Creating a Music to Image Program
- BASIS Independent Advisor: Jon Noble
AP Research
Music is a large part of people’s lives and is a big factor in influencing people’s feelings. Artists commonly release album artwork along with their musical albums and singles, which represent the music within, and complement the music to add to the experience. An album’s cover is typically the first impression before people listen to the music, and as the music industry is very competitive, making a strong first impression is important. However, it can be expensive to create album art for small independent musical artists who may not have the resources they need. To help these musicians, I aim to create a program that takes lyrics of the songs and analyzes the musical mood to generate images that closely represent the song, and can be used directly or as inspiration for album art. First, the program uses a Vector Quantized GAN and Contrastive Language-Image Pretraining to generate an image from the lyrics and improve the image to match the lyrics. Next, it retrieves musical features such as tempo, valence, and instrumentalness from Spotify and classifies the mood of the song from the options happy, sad, calm, and energetic. The mood is used to tune the image to be more representative of the song. The program will also ask the users for optional additional keywords that can be used to personalize the image. To measure the program’s functionality, I will conduct a survey with 50 respondents, in which they will be shown multiple examples and asked questions about how successful they believe the program is.