On the design and making of a single-stage self-correcting inverted pendulum using LQR control
Advisor: Daniel Hollingsworth
Internship Location: University of Victoria (Remote)
Onsite Mentor: Zuowei Daniel Yin
My Senior Project will focus on building the pendulum and implementing firmware to minimize errors and respond to external disruptions. Specifically, the core engineering objectives are to build a physical test bench, develop a high-fidelity digital simulation, and deploy optimal control algorithms such as LQR and LQI. The sim-to-real gap is one of the most pressing issues in modern automation, as the real world introduces noise in the form of friction, heat, and other factors often neglected when building a simulation. I intend to research the extent to which these imperfections affect a system. To achieve this, I will be quantifying the performance degradation between the idealized simulation and the real-world firmware through comparing the effects of steady state errors and measuring the recovery time when adding a disturbance. Remotely, I will frequently meet up with Dr. Shi, who specializes in control theory, to discuss any problems that might arise from my work and to gain insights from a professional who has been in this field for decades. Heading into Electrical Engineering, this seemed like the perfect project for a transition into collegiate-level academics. It combines my interest in embedded systems, math, and automation.
Project Blog
Week 0: On the Sim-to-Real Gap Regarding Self-Regulated Single-Stage Inverted Pendulums.
One of the most frustrating things about engineering is that the system you spent a great deal of time calculating and simulating suddenly stops working when you bring it into the real world due to unforeseen factors that you couldn’t have predicted, or that simply didn’t exist in an ideal simulation. In the field, this is referred to as the Sim-to-Real Gap; it is one of the most pressing issues in modern automation, as the real world introduces noise in the form of friction, heat, and other factors that are often neglected in simulations.
My name is Zixi Yin, and I’m a senior at Basis Independent Manhattan Upper. My project is titled On the Sim-to-Real Gap Regarding Self-Regulated Single-Stage Inverted Pendulums. In this project, I will build both a Simscape simulation and a real-world self-balancing inverted pendulum and compare their performances.
Specifically, I will compare the real-world model with the simulation to find out where a standard Linear Quadratic Regulator (LQR) fails. And from there, how a modified version, a Linear Quadratic Integral (LQI), helps mitigate the problems introduced by real-world imperfections through the use of an integral term.
Heading into Electrical Engineering, this seemed like the perfect project for a transition into collegiate-level academics. The key to this project is designing the actual controller for the mechanism. This branches into control theory, the field of applied mathematics that deals with automated systems, specifically the algorithms behind automatic regulation and correction through feedback. My background in linear algebra and multivariable calculus, understanding of analog circuits, and proficiency with modeling software, such as MATLAB, are all core skills that would contribute to the success of this project. I‘m looking forward to getting started with my project and working alongside my contact at the University of Victoria.
Week 1: Building Foundations
The joy of learning again. After years of grinding through standard, textbook questions, I envied change, an opportunity to apply that theoretical knowledge into practice. This project provided exactly that. While it still relies heavily on that foundational math, the context is entirely different. It’s a fresh, complex challenge that makes all those years of theory finally feel justified and worth spending time on
For the first few weeks, I will focus on the simulation aspect of my project; after that, I will return to China and stay there until the end of May. Delaying the construction of the physical model both saves the hassle of transporting across borders and gives me access to a wider range of options for parts.
My goal for this week is to completely understand the physics behind this system and derive the governing equations of motion that will allow me to automate the balancing process. These equations will make up the foundation of my project, as both the “sim” and the “real” will rely on the physics of this system to function.
The first, most obvious thought that came to mind was using Newton’s laws to derive equations through balancing forces, Newtonian Mechanics. This idea was quickly scratched when I realized the sheer complexity of the formulas required for this method. Then, borrowing knowledge from my Quantum Physics class, I thought about another way, Euler-Lagrange. This method ignores forces and focuses solely on the energy of a system, eliminating the calculations required to find the forces on a tilting rod on a moving cart.
But before any equations are first derived, the knowns and unknowns of the system need to be established.
System Parameters:
M: Mass of cart
m: Mass of pendulum
I: Moment of Inertia of the pendulum about its center of mass
x: Horizontal position of the cart
θ: Angle of the pendulum
l: Distance from pivot to pendulum center of mass
F: External force applied to the cart
From these parameters, some basic equations, such as the location of the pendulum's center of mass and the Lagrangian, can be derived. These are crucial to the control system, as some of these parameters are the exact values the controller aims to minimize to achieve balance.
The core idea behind the Euler-Lagrange method is to look at the system's total energy—specifically, the Lagrangian (L), which is just the difference between the total kinetic energy (T) and the potential energy (U):
By mapping out the kinetic energy of the moving cart and the swinging pendulum, and subtracting the potential energy of gravity pulling on the pendulum's center of mass, we can run the Lagrangian through a set of partial derivatives to find our equations of motion.
However, the equations that we are left with are full of non-linearities, like the theta^2 term. A Linear Quadratic Regulator (LQR) does not like those very much; therefore, we need to linearize them. Luckily, we can use small-angle approximations to eliminate some of those terms, giving us clean, linear equations.
With these 2 equations, the physics of the simulation is ready; the next step is to embed them into a State-space model for the controller to use. See you next week.
Weeks 2-3: Transforming Viewpoints
Welcome back, everyone. I’m really sorry for the delay. It has been a few busy weeks as I'm currently moving. Last time we left off right after linearizing the equations, today, we are going to see how these equations translate into a State-Space representation.
But first, what even is a State-Space?
Classically, in Physics classes, we often use sets of equations to describe the evolution of a system, such as F = ma or V = IR. They work perfectly fine for a limited number of variables, but as the number of unknowns grows, it can be difficult to keep track of all equations, their inputs, and outputs separately. This is where the state-space representation comes in handy. Instead of having equations scattered around, unrelated to each other, we can embed them into a single matrix that describes them all at once.
In reality, they are just another way to represent the system that we are dealing with.
Typically, 2 equations define a representation.
The State Equation:
And the Output Equation:
Where u(t) is the input, y(t) is the output, and A, B, C, and D are matrices describing the system. x(t) is a little special. Notice how it appears as both an input and the result of the first equation as a derivative. This is because the first equation continuously updates the system’s state based on the current state and the input. Hence, x(t) is also given the name State Vector.
For the pendulum, x(t) is a column vector of 4 elements, the horizontal position of the cart and the angle of the pendulum. These are the fewest variables required to track the state of the system.
u(t) is simply F, as it is the only input of the system, the force on the cart.
Using the equations from Last week,
we can derive A and B to be the following, respectively.
Where D is
Please keep in mind that this D is not the same as the D in the output; this is merely a placeholder for the long string of variables above.
We do not currently need to derive the Output Equations, as our primary goal is not to evaluate the output but to constantly update the cart's position based on the state of the system.
With this done, our system of the cart and pole can now be described solely in terms of matrices and vectors, allowing for much cleaner math, especially when implemented on a computer. Tune in next time as I begin the first important part of the project, the simulation, and use it to find the optimal gain matrix for our controller.
Week 4: Finishing Preparations
Last week, we left off with a novel way to describe our system, the State-Space representation. This allows our variables and equations to be represented by a collection of matrices. This week, we are going to put this property to use, specifically by “finding” the optimal gain matrix. Finding is quoted because there is a caveat to this: the optimal matrix can only be found given the physical properties of the pendulum that we are dealing with, i.e., the mass of the rod, the speed of the cart, etc. Without the actual physical parts of the pendulum yet, there is no way to find THE optimal gain matrix.
However, in the meantime, we can prepare to make the calculation easier once we have the numbers.
A bit of background, Linear Algebra refers to the study of, well, lines, specifically linear equations, such as . A system of linear equations can be represented using matrix multiplication. For example, the system
Is the same as
This allows us to connect multiple equations together and perform operations on them simultaneously. As the complexity of the operation grows, i.e., integrations, this method is considerably simpler to implement. This is also why we transitioned to State-Space.
Back on track. Remember, our goal for the pendulum is to keep it from tipping over. One way to mathematically describe “not tipping over” is by keeping the angle between the pole and the ground at 90 degrees. In essence, the difference between the ideal state, 90 degrees, and the actual state of the pendulum is a cost(J) that we want to minimize, a cost that can be described by this integral
Notice that I’m using the full State Vector (x) rather than just the angle. This is because not only are we controlling the rod to stay upright, but we are also trying to make the cart stay in the middle, so that, in case of a disturbance, it has the most leeway to correct its state. Q here is the weight matrix, it tells the system which variable of x we care the most about, that is, the change in which variable will increase the cost the most. Similar to the state of the system, the cost of the input also has to be considered, hence the second half of the integral, where the input (u) imposes a cost for sub-optimal control choices.
Using A, B, Q, and R, we can solve the Continuous-time Algebraic Riccati Equation(CARE) to find our optimal gain matrix K. Then, we can feed it into
which provides us with the necessary input to keep our system in check.
With this finished, all the preparatory math is done, it is now time to move on from the theoretical and begin actually setting up the pendulum. See you next week.
Week 5: Visualising Theory
If you somehow churned through my blogs and still kept interest, this is your reward. Today, we officially move away from the headache-inducing math, into something easily grasped: the simulation.
We’ve spent the last few weeks going over the mathematical foundation of this project, that is, deriving the governing equations, using them to formulate a state-space representation, and finally, using those matrices in our cost function and the Riccati Equation. Now, it’s time to put those theories into practice.
I’ve spent the last week learning how to use Simscape, the software used to make the simulation, and actually building it. At first, all the moving pieces and their respective ports seemed intimidating. However, after playing around with it, it became clear to me that it was no harder than building a LEGO set; you simply drag and drop the pieces, connect them, and use Rigid Transform if any of them misbehave.
The first thing that I did was build the physical pendulum itself. I wanted to make sure that the Physics functioned as expected before I proceeded with the build. My first attempt looked something like this.Right, remember those Rigid Transforms? This is what happens when they are not properly used. I've also ended up with the rod sticking right out and in all directions except upright. Because of how some of the blocks (the prismatic joint) were designed, they don't have an internal option to change their orientation, so I had to use those Rigid Transforms to get them aligned with the axis I needed.
In this form, the pendulum behaved like a physics problem, no friction, no energy loss, perfect energy conservation. In this state, letting it fall from an angle and watching it is already satisfying enough.
Video: https://www.datocms-assets.com/221631/1790629347-video-project-2.mp4
But we can do one better. By taking position and velocity outputs from the joints, we can wire the system up to our control mechanism. Remember all that math we'd been doing was in preparation for this? This is where it comes into play. Luckily, most of the numbers we crunched do not transiently evolve with the system, so the implementation is surprisingly simple.
Where K is calculated from the matrices we derived, and x is the state vector. Because our only input to the system is the force acting on the cart, we link the controller back to it.
By doing so, the inverted pendulum is now officially working.
Video: https://www.datocms-assets.com/221631/1790629347-video-project-2-1.mp4
(The kick around 5s is intentional)
Right now, the pendulum can balance upright only when it starts upright; it can’t swing itself up from a downward position. Because we used small-angle approximations in deriving the equation, the LQR controller can only reliably restore balance over a limited range of angles around the upright position. To enable it to swing up, I added another controller, this time for energy.
It calculates the system's current energy and compares it to the energy of the system if it were upright. If the system hasn't gained enough energy yet, it rocks the cart back and forth until it has. Adding this gives us a very nice swing-up animation.
Video: https://www.datocms-assets.com/221631/1790629348-video-project-3.mp4
Under the ideal conditions of a world recreated in a computer, this has no flaws. However, in a real-world scenario, LQR will break under the slight tilt of the table, the friction of the track, and the non-ideal behavior of the encoder shaft, as a Linear Quadratic Regulator does not account for imperfections. But an LQI does. The Linear Quadratic Integral has an additional integral term compared to LQR. This allows it to recognize errors that have accumulated since the pendulum began running. For example, over time, it will note the table's slant and apply additional force to correct that error. Adding it gives us the final version of the simulation. Functionally, it is identical to an LQR, but provides a more robust countermeasure against steady-state errors.
We have now officially finished the simulation. I am currently gathering supplies to finally build the physical pendulum. Check in next week to see the progress!
Week 6: Introducing Imperfections
Last time, we successfully balanced the pendulum, provided a swing-up mechanism, and implemented LQI to make it robust to imperfections. Right now, the pendulum operates in ideal conditions: a perfectly flat surface, zero friction, and drag. However, in the real world, conditions aren’t perfect, the table could have a slight lean, and for sure, there will be friction between the rail and the cart. Today, we are going to start bridging the sim-to-real gap by converting our ideal feedback system without delays and buffers into something more realistic, where the motor can’t generate infinite force, and the encoder returns discrete values instead of a continuous function. After this, we are going to put the simulation to the test, to determine the absolute worst scenarios it can perform in.
The first step is to completely revamp the Simscape model. Right now, the controller returns Newtons directly to the joint; this is clearly not what happens in real life. That’s why redesigning the system to be controlled by voltage is necessary. Luckily, because I know the specific mechanism that is going to drive the cart, a timing belt system, we can directly convert from Newtons into Volts. The voltage applied to a motor produces torque, which is related to force by the pulley's radius. I already bought the pulleys I’m going to use, so the equivalent force on the cart can be easily calculated. While this is the main gist, there are specific factors like back EMF and such that slightly complicate the conversion, and those have to be accounted for.
Now the Simscape looks like this.
Functionally, it is identical to the previous iteration. Mechanically, the system now sees itself operating the prismatic joint using gears controlled through the voltage provided to a motor.
The main imperfection I wanted to test is table slant, how much the track can tilt before the pendulum stops working.
To do this, I set the initial position of the rod to be perfectly upright and gradually changed the angle of the track to measure its influence on the system, specifically the time it takes for the Integral term to kick in and realize the slanted surface and push the cart back towards the center (+- 1 cm).
The following are the results I got
The limit of testing is 3 degrees because beyond that, the pendulum doesn’t restore fast enough with the tuning that I’m using. This tells us that the real pendulum will likely survive a table slant of up to 3 degrees. Considering how tables are relatively level, this is quite a lenient system.
With this done, the next step is to put together the actual pendulum and rigorously tune it to make it work. Tune in next week to check on progress.
Post 7: Blog 7: Expressing Gratitude
Over the past month, I have been gathering parts to build the physical plant. In engineering, many unforeseeable things can happen, like not paying attention to the dimensions of screws and products not living up to advertisements, which wastes time waiting for the parts to be reordered. Also, sometimes there simply isn’t a premade part that fits my needs, in which case I have to spend even more time designing and waiting for them to be machined. All in all, I’m trying to get to the point that I don’t want to bore you with the details of my arduous journey buying components. The plant is almost ready, and the numbers I generated in Simscape should be accurate enough to make it work.
As the project slowly but surely approaches its end, I want to take some time away from the hardware and software and instead talk about some of the people involved. In this blog, I would like to express my sincere gratitude to my correspondent at the University of Victoria, Yang Shi, PhD, for his help in making this project a reality.
Recently elected as a Fellow of the Royal Society of Canada, Dr. Shi is among the most knowledgeable individuals worldwide on Control Systems. It was learning from his works that I first got into the world of Control. In fact, he was the one who inspired me to tackle this project.
Long-time family friend, Dr. Shi, was eager to help as soon as he heard about my project. He provided me with the foundational materials on LQR and LQI, and helped me verify solutions along the way. Although I said in my blog that I learned Simscape on my own, I’ll have to admit I couldn’t have done it without the materials he provided.
Thank you, Dr. Shi. Without you, I wouldn’t even know where to start this project, let alone advancing it this far.
Post 8: Blog 8: Applying Theory
Hey, sorry for the long delay. While testing, I discovered that the motor I’m using wasn’t powerful enough, so I had to order another one. In the meantime, let’s discuss how I came to that conclusion.
Following what I’ve finished in Blog 6, I’ve begun testing the different electronic parts to see if they work.
The most important of them all is an STM32 microcontroller, the brain of the build. While I got it almost a month back, I didn’t really want to work on it until I had the hardware almost complete.
Speaking of hardware, it took me a while to gather all the pieces to even build something that resembled a pendulum, let alone something that works to balance itself. While on the surface, it may seem easy, just buy a rail and cart, something to bolt them to, a motor, something to connect the motor to the cart, the encoder, and the pendulum itself. In reality, most of these did not want to work with each other from the get-go. I had to look out for specific parts that helped them do so, and this was the part that took me a while to get right.
But anyway, back to electronics. The STM board had a built-in LED and button, so I wrote a program that turns the LED on and off with the press of the button to confirm that the board itself works. Then I tried both the rotary encoder for the pendulum angle and the encoder for the motor's position at the back of the motor, and luckily, both worked. Then, the last step is to test whether the motor spins and whether the motor driver controls its speed.
Because the power supply provides a constant 12V, the motor needs a regulator to adjust that voltage so it spins at different speeds. The driver does so by using a technique called Pulse Width Modulation (PWM). Basically, it outputs square waves that, after filtering, the motor sees as a constant voltage fraction of 12V.
After I’ve tested and confirmed that all the components work, I finally put them together into a pendulum.
Then comes the hardest part of this project, the Sim-to-Real gap.
Initially, I tried using the exact numbers that I calculated in MATLAB as the K for LQI, but the results were underwhelming.
After more testing, I was able to get it to almost work.
Video: https://www.datocms-assets.com/221631/1790629352-img_1622.mp4
But see that shake? That is where I began suspecting the motor was the problem. Those oscillations seem to come from the fact that the controller can’t pull the cart back quickly enough after an overshoot. This is a torque problem. Initially, I chose a motor with a low gear ratio for a faster maximum speed, but it seems like, for a project like this, higher torque is necessary.
And this brings us to where I am right now, waiting for the new motor to arrive. Tune in next time to see if the new motor makes any difference!