
Vincent Y. 2025 | BASIS Independent Fremont
- Project Title: Evaluating Large Language Models through AI Red Teaming
- BASIS Independent Advisor: Dr Sharma
From ChatGPT to Google Gemini, over the past few years, the adoption of AI in the form of large language models (LLMs) Has become ubiquitous, and is continuing to spread, bringing potential for innovation alongside significant risks. Limitations and vulnerabilities of AI concerning privacy, hallucinations, toxicity, bias, etc. can potentially harm users, especially in critical sectors like healthcare. In the next few years, AI governance and regulation such as the EU AI Act in 2024 will become increasingly relevant. For such regulations to work properly, reliable third-party testing of LLMs is imperative. This project seeks to address this need; I will be conducting a literature review to familiarize myself with conventions in the field and using prompt engineering and AI-red teaming techniques to systematically evaluate LLMs against established metrics and identify their vulnerabilities, with the goal of writing a research paper summarizing my results, ultimately contributing to the responsible development and usage of AI applications.