SafeSense: Advancing Road Safety Through AI-Powered Driver Monitoring Technology
Advisor: Dr. Araci
Internship Location: Micron Technology
Onsite Mentor: Jason Zhao
Drowsy driving is a critical issue, leading to 109,000 injuries and 6,400 fatalities annually, with societal costs ranging from $12.5 billion to $109 billion. To solve the problem of drowsy driving, this project introduces SafeSense, an AI-powered device that monitors driver alertness and prevents accidents. The device Has a compact design with a portable camera that captures a livestream of the driver’s face. Advanced computer vision techniques, using Java and Google Mediapipe, analyze head posture, eyelid movements, and facial expressions for real-time detection of drowsiness and distraction in drivers. SafeSense provides instant alerts to drivers, helping to prevent potentially fatal lapses in attention.
My approach begins with creating a CAD design in Fusion 360, followed by 3D printing a custom case to house the device's components, including a Raspberry Pi board and camera. The detection system is paired with a mobile app, designed in Figma, which allows users to customize alerts, locate nearby rest stops, and notify emergency contacts if necessary. By developing SafeSense, I aim to go through the lean startup process while creating an innovative, user-friendly solution to improve road safety and encourage good driving practices among all.
Project Blog
Week 0: It Makes Sense to Stay Safe.
Hey you, you’re finally awake.
For many drivers, waking up behind the wheel is terrifying, because sometimes, it doesn’t happen at all. Every second spent distracted or drowsy on the road is a second where lives are at risk. Welcome to my SafeSense development blog, where I’ll share my journey to create an AI-powered solution to combat these dangers and make driving safer for everyone.
SafeSense is an AI-driven device designed to monitor a driver’s alertness in real time. Using advanced neural networks and facial tracking, it will detect subtle signs of fatigue or distraction, like drooping eyelids or wandering eyes, and provide timely alerts. My vision is to make SafeSense compact, affordable, and intuitive—something that seamlessly integrates into a driver’s routine.
So, why this project?
My father, though an experienced driver, has a habit of dozing off, especially on long road trips. More than once, I’ve found myself on high alert in the passenger seat, watching him struggle to keep his eyes open, forced to pinch him every few minutes. It’s unsettling, knowing that a small lapse could have serious consequences. And then there was the case of my friend who, tired after a long day, carelessly left their car in neutral while on a busy street. Experiences like these made me realize how common the risks of driver distraction and drowsiness are, not only just for long-haul drivers.
Throughout this project, I have the honor to have Dr. Zeynep Araci and Mr. Jason Zhao as my mentors. By the end of this trimester, I aim to develop a fully functional prototype that’s ready for testing.
- James Gu
Week 1: The Roadmap Unfolds
Hey again, still along for the ride?
This week, I officially launched my survey on Qualtrics. I sent it out to friends and family, asking them all about their thoughts on drowsy driving: what they rely on to stay awake behind the wheel (caffeine, music, AC blast?), how often they feel drowsy while driving, and what they’d be willing to pay for a device like SafeSense. There’s also a question about features they’d love to see (because of course, a device that just screams “WAKE UP” won’t cut it).
Meanwhile, I’ve been neck-deep in Raspberry Pi research. So many cameras, sensors, and add-on breadboards to choose from—it’s like walking into a candy store, except every candy has a spec sheet. I’m currently looking at the Arducam IMX519 Autofocus Camera Module (up to 16MP resolution) for crystal-clear 120 FPS facial tracking. For the core, I’m looking at the CanaKit Raspberry Pi Starter Kit, which conveniently includes the Pi board, a power supply, microSD card, and a protective case. And for all those real-time alerts, I’m considering the Adafruit Speaker. I’m juggling trade-offs in camera resolution, processing power, and price to make sure my SafeSense device can run smoothly. It’s exciting and a bit nerve-wracking, but seeing these pieces come together makes it all feel real.
Next step? Wait for survey responses and parts to roll in. Feels like I’m finally starting to put the puzzle together.
- James Gu
Week 2: Sleepy James
Pura Vida! I spent the first half of this week with my classmates in Costa Rica, forming bonds that will last a lifetime. On the bus, I always snoozed away, which explains why the tour guide started calling me “Sleepy James.” That got me thinking: if I’m out cold in the passenger seat, how does the driver stay awake? Cue SafeSense.
Back home, I was pretty excited to analyze the survey results which collected a total of 183 responses. 33% of respondents said they would actually buy an AI-powered camera to keep them alert while driving. People really liked features like automatic pull-over warnings when you are detected to be drowsy, a voice assistant to silence the alarm once you confirm you’re awake, and a detector that knows if you’re on your phone. Not surprisingly, caffeine is everyone’s go-to method for staying awake—61% said they rely on it. Also, 53% of drivers feel tired at least once a week. Yikes.
On the technical side, I set up my Python virtual environment by installing PyCharm as my primary IDE and integrating key libraries. OpenCV will be responsible for real-time image processing, enabling face and eye detection. MediaPipe, Google's framework for high-performance face tracking, will allow for precise facial landmark detection, which is critical for monitoring eye openness and head position. NumPy will handle fast array-based computations to process image data efficiently, ensuring the detection pipeline runs smoothly. With these tools in place, I can now start coding SafeSense. Stay tuned!
- James Gu
Week 3: Blinking Into Focus
This week, I finally got SafeSense to function on PyCharm. I created the first iteration of the AI model using Google MediaPipe and OpenCV. What's the goal? To monitor eye movement and head posture in real time and detect when a driver is distracted or dozing off.
I created a Python script that takes live video, maps out facial landmarks with MediaPipe, and estimates where the head is looking—left, right, up, down, or straight ahead. I also added a feature called EAR (Eye Aspect Ratio), which detects whether someone's eyes are open, blinking, or closed for too long.
Here’s what the system can do so far:
Detect head tilts using 3D face landmark positions
Project eye movement and head direction onto the screen
Track blink duration to distinguish between normal blinking and possible drowsiness
Display messages like “Looking Left,” “Forward,” or “Sleeping” directly on the screen
I even added a line that draws from the tip of your nose to show where your face is pointing. It’s been really cool watching it all work live, especially when the frame rate holds up and facial recognition responds smoothly.
Now that I’ve got basic detection working, I’ll start building out the alert system next.
- James Gu
Week 4: Hardware Hiccups
The main focus this week was getting SafeSense's hardware component operational. I configured everything for remote access via SSH and VNC, loaded the SD card with the operating system, and set up the Raspberry Pi. The Pi booting up and the camera module coming to life was a cool experience—until it didn't!
I encountered a hardware issue while I was attaching the Pi Camera's 15-pin ribbon connection. The metal connectors on the ribbon cable were stripped as a result of my improperly loosening the camera port latch. The new cables that I bought should come this weekend. I'll be able to complete testing the live camera feed with my program once they arrive.
During this down time, I improved the frame rate performance of the current Python code during the downtime in addition to the accuracy of the head-direction tracking. Additionally, I included a feature that records head direction over time and creates graphs to have a better understanding of user behavior. This feature will allow me to perform statistical analysis on factors like how frequently and for how long users look away from the road.
- James Gu
Week 5: Hardware Integration and Algorithm Refinement
This week, I mainly worked on configuring the SafeSense device. After the Raspberry Pi 5was assembled, a 16 MP camera was linked to obtain sharper images for facial recognition, and a monitor display was added. It wasn't too hard to put everything together overall, but it took some attempts to get the camera positioned correctly. After I eventually figured it out, it began to reliably produce clean video feeds.
As soon as I started testing the system, I saw issues: the facial data was noisy and inconsistent, particularly metrics like pitch, yaw, roll, and EAR (Eye Aspect Ratio). This makes accurately detecting driver distraction or tiredness tough. To fix this, I started researching algorithms that can smooth out noisy data, like exponential moving averages and Kalman filters. I spent time looking through online guides and examples, trying to figure out what might work best for SafeSense.
I also purchased a speaker and amplifier to provide the device aural notifications. But this means I'll have to learn how to solder, which I've never done before. To ensure I was ready, I set aside time to watch lessons and read safety instructions.
At last, I began using Fusion 360 to create a unique housing for all of this hardware. Making a small, well-organized device that neatly fits the Raspberry Pi, camera, cooling system, and speaker was the aim. I experimented with a number of layouts, modifying placement and size to determine which worked best.
I intend to continue prototyping the case design, test out the smoothing techniques I've studied, and begin soldering next week.
- James Gu
Week 6: Prototype 2 Design and Improvements
This week, I focused on building out Prototype 2 of SafeSense. I started by finishing the updated CAD design in Fusion 360. The new version is smaller, better ventilated, and includes cleaner cable routing for the camera, cooling fan, and speaker. After printing the case, I realized a few ports were misaligned, so I made tweaks and reprinted the enclosure to get a tighter fit.
With the case done, I soldered connections between the Raspberry Pi, MAX98357 I2S amplifier, and speaker. It was my first time soldering on a live project, so I went slow to make sure the joints were strong and safe. Once everything was in place, I powered it on and confirmed the speaker worked with simple test audio.
On the software side, I integrated BlazeFace, a faster facial detection model that significantly cuts down on latency compared to FaceMesh. It runs more smoothly on the Pi and picks up facial landmarks quickly, even in dim lighting. However, the yaw and pitch readings still had some jitter, so I implemented a Kalman filter to clean up the data stream. I tuned the parameters to strike a balance between responsiveness and stability, and the results were much more usable for detecting distraction events.
I also started collecting annotated data with EAR and head pose values for specific drowsy or distracted behaviors. This will help train and test the event detection pipeline going forward.
Next week, I plan to connect the speaker notifications to real detection events, explore ways to save data logs, and begin preparing for pilot tests with actual users.
– James Gu
Week 7: Creating the UX
This week, I created the web interface for SafeSense from the ground up with Flask. I began by creating a clean, mobile-friendly layout with a modern card-style UI featuring rounded corners, shadows, and responsive elements. I developed all of the HTML and CSS myself and arranged it such that the app could scale across multiple devices.
I implemented essential functionality such as login, signup, and personalized greetings. Once logged in, customers are taken to a main menu where they may establish a wake-up name, locate local rest locations (convenience stores) using Google Maps, and customize how alerts are sent. Each action leads to a new Flask endpoint, with user data retained throughout transitions.
One problem was coordinating the layout pieces. The space between the logo and the welcome message felt odd, so I changed the margin and gap settings while learning how to use "rem" units in responsive CSS. Small adjustments made a significant difference in overall polish.
I also built a dynamic alert settings page so users may select from a default name callout, a custom TTS message, or an annoying song. The interface refreshes in real time utilizing JavaScript, which I configured to conditionally reveal input fields without reloading the page.
With the entire UI flow in place, I'll spend next week on final testing, which includes sending alerts depending on detection events, gathering interaction logs, and evaluating system accuracy through trial sessions.
– James Gu
Week 8: Final Integration and Testing
This week, I focused on integrating all of SafeSense's primary components into a completely functional system. I began by connecting the real-time drowsiness detection pipeline to the alarm system. When SafeSense detects indicators of tiredness, such as EAR (eye aspect ratio) dips or aberrant head tilts, it now immediately sends a wake-up message via text-to-speech or a pre-selected song.
After completing the basic integration, I proceeded to debug the entire flow. I tried various scenarios, such as purposefully closing my eyes for a few seconds, turning my head, and even disconnecting the camera mid-session, to ensure that SafeSense could handle problems without crashing. Fixing tiny edge situations like these improved the system's stability and demo-readiness.
I also improved the backend settings logic to ensure that user preferences—such as wake-up name, alert style, and custom messages—are properly stored to JSON files and persist across restarts. This required a lot of careful inspection to avoid overwriting data and to reload it cleanly when the program launched.
Finally, I focused on quality assurance, stress-testing SafeSense over longer sessions, fine-tuning UI feedback, and ensuring that the overall experience is smooth for the end user. With this final round of integration and polishing, SafeSense is now ready for pilot testing.
– James Gu
Week 9: Feature Expansion, Data Insights, and Business Planning
This week felt like a shift from building core functionality to making SafeSense feel more like a complete product. I kicked things off by adding an emergency contact feature to the app. Now users can store a trusted contact who can be notified if they’re repeatedly flagged as drowsy. It’s a small piece of the puzzle, but it adds real value.
After that, I spent a day refining the data visuals for head pose and eye aspect ratio (EAR). The old plots were messy, so I cleaned them up with better histograms and angle distributions. The new graphs make patterns way easier to spot, especially when trying to understand drowsy behavior over time.
Midweek, I dove into the financial side. I created a 3-year model to put together a P&L statement, looking at things like how much I'm spending on marketing versus how many customers I might actually convert. It was a different kind of thinking—less code, more spreadsheets—but super helpful to understand how SafeSense could grow sustainably.
To wrap things up, I wrote this blog post and reflected a bit. It’s cool to see SafeSense go from "here's a working prototype" to "here’s a product that actually makes sense, financially and functionally." Now only the visual parts of this project are left.
– James Gu
Week 10: End of a Ride
This is it. The final week.
I spent the first half of the week refining the user interface, adding smoother screen transitions, enhancing the color scheme, and adjusting button placement after finishing the majority of the features. Even though it's minor, it has a big impact on how the app feels. For the first time, SafeSense seems like something I would want to use rather than just something I created.
I completed everything by assembling the demo video and final presentation slides. It was strangely emotional to see all the code, diagrams, and UI screenshots in one location. This began as a sketch on my whiteboard. It's real now, and it works.
I had doubts about my progress ten weeks ago. I now have a working business plan, a driver alert system, and a renewed understanding of how much effort it takes to realize a vision.
Thanks for following along, and safe driving!
– James Gu