Dynamic Models Using Smooth Translucent Level Surfaces for Learning 4D Visualization
Advisor: Ms. Vibha
Internship Location: NVIDIA, 2788 San Tomas Expy, Santa Clara (Remote)
Onsite Mentor: Dr. Yongxi Hou
4D is closer than you think: take any 3D structure, say a pipe system, and slap on a variable for pressure. Yet, most experts at all levels of academia often portray the visualization of 4D space as impossible, in large part because the most accessible learning aids require significant effort to understand. My project aims to address this by implementing a new model for representing 4D objects using level surfaces with varying transparency and color, allowing users to explore 4D space.
I’m working with Senior Computer Vision Engineer Dr. Yongxi Hou at NVIDIA to develop an interactive, dynamic display integrating existing wireframe techniques, as well as my new transparent level surfaces, to identify which requires the least reading and effort from the user to understand. I’m learning how to use the 3D transformation matrix, what potential tricks exist to simplify its 4D counterpart, and which Python libraries would make such a program run efficiently.
I will develop a quiz to investigate the accuracy with which students exposed to the previously mentioned models are able to identify transformations, rotations, and other geometrical operations, compared with how much additional reading is needed. The reading will be increased from explaining what the model represents to a full lesson with 2D-3D analogies as the student progresses. With a stronger grasp of 4D space, students will develop the skills to understand abstract concepts with visual tricks I hope to integrate into classroom learning.
Project Blog
Week 0: Blog - Introduction
Hello everyone!
I’m Samuel Yang, and welcome to my first senior project blog! I’ll be updating this blog weekly with progress on my Senior Project, as I tackle the challenge of developing a tool for teaching 4D Visualization.
Higher-dimensional space is one of the topics that most fascinates me in my study of math, especially because of its relevance to the Physics of spacetime. But despite finding many online resources to help me learn just the most basic of 4D behaviors, I often found myself facing mind-twistingly complex and indistinct animations. The visual resources available make learning 4D much harder than it needs to be, often serving only as supplements to textbook-level explanations, and often requiring readers to put in great effort and precision to build mental models that may not be accurate.
But why is this even important? In addition to being a stepping stone for learning the physics of spacetime, 4D visualization also offers a way to understand variables in 3D space, things like temperature or pressure, potentially opening the door for applying mathematical and geometrical identities for efficient calculations. That’s why I’ll be focusing my project on making 4D more approachable to students by making stand-alone animations and interactive software for improved learning and accuracy.
To start my project off, I’ll jump right in by bringing my unique approach to 4D visualization, color, and transparency variation based on the fourth-dimension coordinate and using it to recreate existing wireframe animations. Through the next five weeks, I’ll create animations for other existing models and expand my own method to use full cross-sectional slices in addition to wireframes. By the end of those five weeks, I’ll have a set of animations and a quiz to test the effectiveness of each visualization model in teaching 4D.
After sending that quiz to gather results, I’ll spend the remaining 5 weeks turning my non-interactive animations into an interactive space for users to explore 4D shapes and operations, and if I have time, I’ll add graphical inputs and possibly contact physics.
With this plan set, I look forward to seeing you all next week, where I’ll hopefully have some animations to show. For now, here’s a graph for any curious minds among you to explore, featuring a wireframe and cross-section: https://www.desmos.com/3d/bnp8l5lbic.
Week 1: Blog: Diving Right In
Hello readers, welcome back to my senior project blog!
This week, I’ve taken my first steps toward rendering 4D objects for a teaching tool, which is rendering 3D objects. I’ve begun building the code for handling vertices, edges, and surfaces, all of which will carry into the fourth dimension. I’m using Python for this project because of the many math-handling conveniences it offers.
To start off my code testing, I’ve chosen the cube. Its vertices can be easily defined in simple x, y, z coordinates, with just three 1s and negative 1s. Meaning, when I get to 4D, all I’ll need to add is an extra pair of 1s.
Here’s a quick rundown of things I’ve done and encountered this week:
Development: I’ve hardcoded in the vertex and edge arrays for a cube, creating a wireframe that can rotate with yaw, pitch, and roll.
Problem: By drawing the lines in order, lines that are meant to be in the back end up drawn in front.
Solution: By finding the furthest-back point and tracing all edges simultaneously from back to front, no such placement problem occurs
Development: I’ve created a way to fill in the whole cube by connecting these traced edges to wrap around the surface. Kind of like a yarn ball
Problem: A cube isn’t the only thing you can make with a few sets of edges. A square can easily turn into a kite, and a hexagon into some weird space invaders shape.
Solution(s): I’ve defined the cube’s triangle mesh and created a method to fill it with a gradient. I’ve also brute-forced the edge combinations that produce valid cube shapes.
Unfortunately, these solutions aren’t scalable to other shapes, much less 4D. So the first thing on next week’s agenda is to build a method to standardize generating valid triangles and edge combinations for any shape, not just the cube.
Coding chaos aside, I’ve also been reading up on 4D tessellation and slicing in preparation for stepping into the next dimension, and if anything, experiencing every mathematical quirk firsthand, albeit in the form of very annoying bugs, has helped me internalize what each concept represents geometrically.
With next week’s programming plans ready, I’m looking forward to seeing you all next time.
Week 2: Blog: One Step at a Time
Hello readers, welcome back to my blog!
Following up on last week, I started this week by deciding on how I would build upon my current foundation for 4D. Last week, I mentioned that I brute-forced the edge combinations for a valid cube. However, I realized I was actually encoding which edges shared a face, something extremely relevant to scaling up.
I already created an internal Graph class to form any number of two-way links between points, but this week I revisited this class to make it number these links with negative indices, so that these edges had numbers to sort them by later. I also made a ThreeShape class to form the basis for all 3D shapes and modified it to track not just the edges between its vertices, but also the faces between its edges.
Unfortunately, this means I will need to continue manually defining the 1D edges, 2D faces, and even 3D cells as I move onto 4D, since a set of edges alone isn't enough to unambiguously determine a 3D shape.
Here’s a rundown of how this panned out throughout the week:
Development: Continuing with tracing edges from back to front, I’ve improved it to generate polygons between edges, not just lines. I’ve also fixed last week’s rendering problem by designing the code to find the faces in contact with each edge. It now follows that connection over to the next edge, wrapping around the whole shape.
Problem: The Edge class I’ve been using to follow edges along their ends doesn’t quite work for completely flat faces. It currently stores a source, target, and location, but it breaks when the source and target share the same depth.
Solution: By coding this case into the back-end logic for rendering shapes, it will draw polygons between the vertices.
Development: I’ve integrated last week’s gradient lines and triangles to create a 4D cube as the first example for my model.
Problem: The coloring works for each layer. But despite each line having low A (opacity) values, things are still barely visible through even just dozens of layers. These layers quickly stack up to make anything under them invisible, concealing the cross-section that's meant to be highlighted.
To solve this problem, I’ve added methods for handling transparency and layering to my reading resource list. Together with my weekly reading on 4D, this week on scaling, measurement, and quaternions, I’ve laid out the steps ahead.
With next week’s goals decided, I’ll see you all next week!
Week 3: Blog: A New Perspective
Hello readers, welcome back to my blog!
This week, I started by fixing the transparency for my layer model from last week. However, I quickly realized I was actually just configuring PyGame incorrectly! But I also learned that representing transparency with the Alpha value in RGBA is not the best way, regardless. The A value isn't commutative with respect to drawing order, red over blue and blue over red yield different purples. It's very common that multiple 3D cross sections of a 4D shape tend to occupy the same depth, so this causes one color to take priority depending on how the edges in my back end are sorted.
After discussing this with my external advisor, though, I’ve received a whole new perspective. The work done around 3D rendering already exists, so I also have the option to do w-layering in the 4th dimension instead of z-layering using depth. This way, the 3D shell of each cross-section can be generated and stacked on top of each other to create the same effect I’m aiming for. However, this still requires a robust definition for 4D shapes, with edges, faces, and cells.
Here’s how that development went:
Development: I adapted edge-tracing logic to follow from the most negative w to the most positive. After storing the intersections and the polygons drawn between them in .ply files, I used CloudCompare to render them together.
Problem: Without prior familiarity with .ply files, I kept sending invalid polygons to storage because of redundant vertices. My backend indexing also caused other mishaps when translating over.
Solution: After reading up on .ply files, I was able to fix these issues and render shapes with CloudCompare
Development: To further familiarize myself with CloudCompare, I created the animations for block-face highlighting and wireframes with perspective.
Problem: My color-layering model is far too slow. Since shapes are defined by their 3D cross-sections, the edges of the 4D shape are jagged at low resolutions. However, it's sending way too much information at the higher resolutions that mitigate this issue.
I could just let the simulation run for minutes per animation, but I still need to tune colors and transparency. I’ll need to properly think through what changing from z-layering to w-layering actually did. Additionally, I don’t think either is currently representing the model I’m aiming for accurately.
Perhaps my vision for this model will emerge with a bit of tuning. But for next week, I’m planning on fully fleshing out the control models, wireframes with perspective and block-face highlighting. My target for Week 5 is to have the animations for each model ready to share, so I’ll have to make these before optimizing my code.
For this week’s reading, I’ve read in depth on rotation in the 4th dimension, especially in relation to defining a quaternion's angle in space. A vector in 3D has three angles for each pair of axes, xy, xz, and zy. But a 4D vector has six, emerging from the combinations of x, y, z, and w. Additionally, rotation in interactive spaces is often defined instead by roll, pitch, and yaw, based on heading instead of axes. After I’ve prepared my animations for Week 5, I’ll review this again to see how I can best represent these angles for intuitive interactivity.
For now, though, see you all next week!
Week 4: Blog: Caught in the Matrix
Hi readers, welcome back to my blog!
Continuing from last week, I’m continuing to play around with CloudCompare to get my 4D model to work. Previously, I created a few animations for wireframes and block-face highlighting to have them ready to share week 5. However, I started off this week by discovering a few major problems.
Here’s what went down:
Development: I implemented multi-highlighting for the block-face model, applying different colors to each 3D cell.
Problem: Faces for blocks underneath other blocks would appear to show up in front or completely override blocks below.
Development: Last week, I hardcoded rotation along a single plane and called it a day. This week, I attempted to implement user controls and more complex operations.
Problem: Rotations were executed in the wrong order, causing confusing double rotations and wobbling. Additionally, lines were colored incorrectly, instantly flipping between red and blue when color modeling was applied.
After talking to my external advisor about the issues I was experiencing, I unfortunately discovered that It was my math, not my code, that needed reworking. Think of a 3D fighter jet that can rotate in three directions. This jet can yaw, turn left and right, pitch, turn up and down, or roll around a central axis. It seems nicely matched: three angles for three dimensions, just like x, y, and z. However, these angles are actually better thought of as operations.
Imagine a plane pitch down 45 degrees and then yaws 180 degrees. Following along with your arm, this would mean the plane is looking 45 degrees upward, and behind where it was. But if the plane turns 180 degrees first, then pitches 45 degrees, it will face down instead. Changing the yaw angle no longer represents the plane turning in its own left and right. Instead of spinning in a circle, it's spinning in a cone.
Translating into my project, I started by animating the x-w rotation, which looks like a linear operation. With x, y, and z as the normal 3D coordinates, this x-w rotation should look like cutting a deck of cards. Rotating in 3D shouldn’t change the fact that the deck is flat. However, my original math turns this deck of cards into a weird, angular slinky.
I originally linked my 3D and 4D rotations together with one 4x4 matrix, using roll, pitch, yaw, plus three more angles. But now I know that I’ll have to unpack my rotation matrix so that 3D doesn’t compromise 4D. Since CloudCompare’s UI can handle 3D operations on its own, I just need to get 4D behaving how I want it to.
I’ve already done this week’s scheduled reading, learning how 4D objects project 3D and 2D “shadows”. With the knowledge gained, I might be able to optimize my layer-stacking model by hiding certain faces or simplifying complex 3D cells into a few flat surfaces and equations.
Fortunately, I still have one more day left this week to tackle my bucket list of fixes.
Wish me luck and see you all next week!
Week 5: Blog: Wrapping Up for Presentation
Hi readers, welcome back to my blog!
Last week, I was able to partially resolve my user control issues with an improved handling for 4D and 3D operations in my model. However, it’s still a bit too confusing for my liking, and is definitely too slow to run interactively during the quiz. Instead, I’ve decided to just use the operations to generate the animations needed in the back end.
This week’s software includes two developments:
I solved last week’s line flip-flopping by changing the Edge class to actually walk in 4D, which I somehow forgot to do. This way, lines actually change color when traversed instead of using their start and end points’ colors.
I also attempted a new face-tracing algorithm to speed up block-face highlighting and not generate redundant surfaces, but it ended up being just as slow. However, I still modified Edge-tracing to overshoot a tiny bit, to patch up small gaps in both wireframe and this new method.
Now, I have models for wireframe and block-face highlighting that work well, and a functional, though slow, experimental model. I unfortunately don’t have a quick solution for defining 3D cells of a 4D object, so I spent quite a lot of time manually defining new sample shapes. In addition to a 4D Cube and a 4D Triangular Pyramid, I also made a sphere and some irregular structures. This also thankfully confirms for me that my code works for shapes that aren’t symmetrical in some way.
Moving away from the back end, I’ve also started on the textual supplements I’ll be using for my quiz. I've structured the quiz into three experimental groups and question blocks. The three groups are orthogonal wireframe, selective block-face highlighting, and my model with highlighted w. Each group will have two textual supplements, a rudimentary “guide“, and a full-blown textual lesson.
Volunteers will answer questions like in what plane is this rotating, or how is Shape B transformed from Shape A. After a few questions, they are given the guide to read over, and after a few more, the lesson. This way, questions are broken into three blocks: pre-reading, post-guide, and post-lesson. By tracking accuracy throughout these three blocks, I hope to find which model is most effective.
I’ve started with a rudimentary “guide” of what my model represents. Scoping my explanation to cover what my animations are doing without outright teaching 4D was challenging. But with my reading this week, a proper textbook introduction to 4D, I’ve had plenty to take inspiration from. The textual lesson, however, is shockingly proving to be more challenging. Rather than confronting my volunteers with walls of text, I need to get the point across in the most relevant and engaging way. The thing I’m most afraid of is my volunteers giving up after struggling earlier, then skimming through the lesson. If needed, I may also have to try tracking how long volunteers spend on each of the textual supplements.
With next week's full-quiz logistics goals defined, I’m almost ready to start collecting results.
Thank you for reading, and see you all next week!
Week 6: Blog: Testing the Waters
Hi readers, welcome back to my blog!
I’ve started this week by completing drafts for each group’s written supplements. After finishing, I decided to meet with a few students early to see how they handle a prequiz. I’m aiming to see which model for 4D Visualization takes the fewest written text to teach 4D accurately. However, to test accuracy, I need to establish a consistent set of controls for answer options.
I’ve settled on using an origin with 4 axes for the four dimensions, animated independently from my test objects. Each question has a rotating object to explore, and four sets of origins and axes, as well as a good-faith “I don’t know” option.
Here’s the software work that went into implementing that:
I’ve extended my FourShape class to create an Origin with five points: an origin, and a point on each axis. Additionally, I added a hasText tag to so that each axis can be labeled independently without interfering with layer-based rendering.
I’ve modified the controls so that users control 4D rotation speed instead of angle. This way, users can explore the object by moving it around in 3D without first stopping the object’s 4D rotation.
I made the code open multiple windows, one for the question and one for each answer option. They’re all linked to the same back-end 3D rotations and 4D speed, but can differ in their 4D rotations.
To test out this control scheme, I met with a few students as a sort of calibration test for origin-based answer options. Unfortunately, it seems tossing my underclassmen into the deep end didn’t result in them magically grasping my control scheme. I was met with resounding confusion at what I was even asking. In light of this, I’ll be spending the next work week developing a 3D tutorial to help get my quiz-takers on track.
This week’s reading couldn’t be more apt, consisting of a chapter on visualizing quaternions (4D arrows) and interactive visualization. I even considered using a quaternion to represent controls, but the subsection on rotator dynamics made it clear that just one vector wasn’t enough. I ultimately settled on using an origin to distinguish between rotations because of how much information four axes can encode in one.
Next week is spring break, so I’ll instead be posting again the week after next.
With my work cut out for me, I’ll see you then!
Week 7: Back on Track
Hi readers, welcome back to my blog!
Last week’s spring break was quite exhilarating, with many of my schoolmates and me competing in a robotics competition and taking home a finalist medal for the team! Now that I’m back and ready to work on this project, I’ve left a few tasks for myself.
Originally, I’d intended to work on user controls after completing animations for my quiz using each model. But now that I’ve met with students, it's become clear to me that just an animation won’t be enough. Without letting students master the 3 dimensions they are already familiar with, a fourth dimension is too much to ask for.
Instead, I will be trying to copy a control scheme that each student should be more familiar with, 3D Desmos.
Here’s what I changed:
Instead of using sliders for roll, pitch, and yaw, I changed the control scheme to track the mouse button on a right-click hold. Moving the mouse up and down pitches the z-axis toward and away from the screen. Moving the mouse left and right rolls the object about the z-axis.
Scrolling with the middle mouse button normally performs a zooming action; however, holding control while scrolling toggles the special 4D control for each sample group: orthogonality for the wireframe, transparency for cell-highlighting, and target-w for my model.
Unfortunately, I’ve run into another obstacle. My current model animations were all done in CloudCompare, for good reason. Using PyGame means I have to generate my objects 2D layer by layer, instead of 3D shell by shell. However, CloudCompare can’t link up to the Google Form I was working on for my quiz. This means I’ll have to quickly slap together a different solution or find a way to downgrade my CloudCompare model to use PyGame. Much of my time this week was spent trying to downgrade my model, but PyGame truly is too slow for a good user experience.
The reading I’ve assigned for myself is the same as last week’s, focusing on visualizing quaternions. However, I’ve additionally given myself a few articles on using GPU architecture in 4D visualization. In other words, a potential key for making my code more efficient and less reliant on CloudCompare. Just in case, though, I’ve also started reading about the steps it’ll take to embed CloudCompare windows into Google Forms.
If things work out well, I might even have an interactable embed to show you all next week!
Wish me luck, and see you all next week!
Week 8: A Site to See
Hi readers, welcome back to my blog!
This week was quite hectic for me, with this project coupled with my house move. Nevertheless, I’ve continued working on transferring my project onto a quiz platform. However, I immediately ran into a huge issue. Google Forms can’t run PyGame, which has all my code.
To work around this, I had to use Pygbag, a library that effectively translates the Python logic in my code to HTML and Java web interfaces. I also had to program the buttons and stages needed to recreate a quiz experience. Fortunately, exporting responses to a Google Form is fairly easy, so all I needed to do from there was find somewhere to host it. Using GitHub to publish my final website, I’m glad to show you: https://thunderrhombus.github.io/4DVisualizerQuiz/
But this skips over the development challenge that went into it.
Here’s how that went down:
Pygbag runs on terminal commands to begin translating, and doesn’t exist as a GitHub extension. This means I have to resend commands and troubleshoot using long terminal and console logs I’m not familiar with. This led me to discover some strange errors in Pygbag itself, where its output file contained a site-crashing typo.
I also accidentally packaged all of Python into Pygbag, causing my file size to explode literally a thousandfold. After experimenting with the .gitignore file and terminal commands, I ultimately moved everything down a folder.
My previous classes and models were not configured to handle new shape inputs and random answer options. I had to create a brand-new interface class for my own testing, in addition to a main class for Pygbag to read over.
In addition to my trouble with Pygbag, I’m still handling a few UI mishaps, especially with the quiz formatting. The link I sent earlier is noticeably missing the option to take the quiz. However, there’s still a free exploration function using the three models and a few shapes. For any of you too eager to wait for the full quiz experience, it's on there for you to explore.
Continuing to read about GPUs and math has been nice, but this week’s true lifesavers are the Pygbag instructions. Without them, I’d be completely, utterly lost trying to get PyScript to work instead.
For now, though, I’m just a step away from reaping all the data I’m looking for.
Wish me luck, see you all next week!
(Early next week, please, I need all the data I can get)
Week 9: The Last Link
Hi readers, welcome back to my blog!
Before we start, my website is fully up and taking responses! Please give me data: https://thunderrhombus.github.io/4DVisualizerQuiz
This week has somehow managed to be more chaotic than the last, with my final push to finalize the code portion of my project coinciding with my flight out to Texas for visiting colleges and the Robotics World Championships!
Unfortunately, this was not the simple task I thought it was. I ran into a linking error I’ve never heard of before preventing my code from interacting with Google Sheets. In order to read from Sheets to equalize my sample group, I had to learn new syntax for making a Google App Script. Additionally, to link Forms together, I needed to learn a lot about console logs and entry IDs.
Here’s what went down:
Google has an anti-bot measure of some sort preventing websites like GitHub from directly accessing systems like Google Sheets. Working around this requires an App Script, which acts as your Google account, running code to perform actions mimicking your account's credentials. I’m not sure if it’s something to do with being the Sheet’s creator or not, but it works.
I also had to figure out quite a bit more about Google Forms formatting, as I realized I needed sections to actually remember my randomized shapes and questions (some shapes are much harder than others). I’ve been randomizing correct answers within a few difficulty groups by applying inversions and offsets to create incorrect options. Since the 1-5 options themselves wouldn’t be the most useful, I had to research the proper formatting for Short Answers.
I’ve sent my form for posting after finalizing it on Thursday, so now I’m just awaiting results. Through further exploration reading of my sources, I might try adding a few features for our upcoming final Symposium showcase.
For now though, please click the link and give me data, it’s just 10-15 minutes!
Thanks for reading, and see you all next week!
Week 10: A Wild Ride
Hi readers, welcome to the final blog!
It’s been a long ten weeks, but we’re finally here! Since the last update on robotics, we’ve won our division as first pick and made it three rounds into the Einstein playoffs of the FIRST Robotics Competition World Championships. Needless to say, I was quite busy as a core pit crew member. Hence which is why I didn’t notice that my form stopped gathering data two days in.
“Wait, what?” you might ask. Yep, I wasn’t gathering data for all four days of the competition. Despite testing my website after pushing new code, I didn’t know to distinguish between new and old instances of Chrome. Apparently, there’s a special kind of cookies that my website relied on to publish to a Google Form for data. As soon as I discovered this, I changed my App Script and code to directly write onto a Google Sheet.
Additionally, I added a prequiz and postquiz asking participants redoing the quiz to rate their previous experience, and compare it to their experience using a new model. However, the data I lost on their accuracy would never come back.
But while robotics is what distracted me from this project, it ironically saved me, too. Our team’s technical mentor conveniently owned a robotics design discussion server and was willing to advertise it. Thanks to our exceptional performance, I found numerous willing volunteers, giving me a whopping 57 responses to work with!
This does change my participant demographics from my previously desired 9th grade level to a relatively scattered group. Because of research ethics around minors and privacy, I couldn’t simply ask what grade they were in. However, I could largely assume them to be high schoolers unfamiliar with 4D, with some middle school and college outliers.
Now onto the data analysis! I will be concealing the final results for my symposium presentation, but I can share some thoughts on the process. Using several Google Sheets formulas, I compiled data on overall accuracy, improvement after reading, outlier removal based on abnormally short analysis or answering times, and more. The following results were quite unexpected, with individual performances ranging wildly, and averages showing surprising trends. However, I’ve gained a pretty clear picture of how I think each model and each reading supplement affected my participants.
Overall, I’m proud of how far I’ve taken this project in ten weeks, and definitely look forward to presenting my findings soon.
If you’d all like to explore what I've made for yourselves, or even take the quiz to try, the link is still here: https://thunderrhombus.github.io/4DVisualizerQuiz
Otherwise, thanks for reading, and see you at the symposium!