Achieving Pareto Efficient Results with Legally Binding Contracts in Prisoner’s Dilemma Situations
Advisor: Zoey Wang
Onsite Mentor: Scarlet Chen
How can we achieve Pareto Efficiency in games like the Prisoner’s Dilemma? In Game Theory, the Nash Equilibrium is the outcome of a game if both players act in their best interest, and an outcome is Pareto Efficient if no other outcome is better for both players. In many games, the Nash Equilibria are not Pareto Efficient, leading to inefficiencies in micro scale market transactions, but also overall macro scale effects, such as negative externalities like climate change or pollution. There have been many attempts to solve the issue, from repeating the game to introducing cooperation—yet most are ineffective as they rely on the game organizer to corroborate the solution.
To address this, I will look at the effects of legally binding contracts as a mechanism to a solution. I will investigate the theoretical basis for using legally binding contracts, and test the theory through experimental data. I expect that the legally binding contracts will be effective, potentially offering a solution to both micro and macro scale inefficiencies.
Project Blog
Week 1: An Introduction
Hello everyone!
I am Larry, and I am happy to welcome you to my first senior project blog post. I will be introducing my project, its purpose, and its applications to the real world.
Adam Smith, the man most people proclaim the founder of modern economics, based a large portion of his beliefs in the assumption that people acting in their own personal benefit would create the maximum value for the whole of society. And while many of his theories and contributions are widely accepted today, this premise has been proven wrong. One of the best counterexamples of this premise is the Prisoner's Dilemma, a classic example in game theory in which the strictly dominant strategy for each player leads to a worse outcome for both. Being able to find a way for each player to achieve the optimal result for both of them, or what is called the Pareto efficient result, would have implications in many places, including shoring up inefficiencies in business transactions and potentially offering an incentive-based solution to negative externalities like climate change.
My project will be analyzing ways to make this happen. I will be designing and testing a framework based on legally binding contracts that may allow for a solution to the Prisoner's Dilemma to become Pareto efficient. I will design the framework, work through the mathematical theory behind the idea, and design an experiment to analyze the effectiveness of the framework. Previous studies have indicated that repeated instances of the Prisoner's Dilemma could achieve Pareto efficiency, yet no such solution exists for single instance Prisoner's Dilemmas.
This week, I read through some of the existing literature within the field. While I did not do an extensive literature review, I found some cool ideas from some of the papers I read, which I may integrate into my project, as a potential extension to games beyond the Prisoner's Dilemma. I will continue researching the existing literature, and come up with a full literature review in the next few weeks. Stay tuned for that and more details in my coming posts!
Week 2: Literature Review
Hello everyone,
Welcome back to my blog! Today, I will be sharing the literature review I completed throughout this week. In doing research for this project, I first looked into the details of cooperative game theory, learning new concepts like Shapley and Banzhaf values. Then, I looked into ways other people have used to reach the Pareto efficient outcome.
The first paper I looked at was this: https://intelligence.org/files/ProgramEquilibrium.pdf. The paper went into how programs would be able to reach a Nash Equilibrium, which they coined "Program Equilibrium." This would spawn solutions that would in fact reach the cooperative equilibrium at certain points. For example, a program which cooperates when the source code of the other program is the same would be a basic case of this, although programs who cooperate much more often are possible, as shown in the paper.
The second concept I investigated was the idea of a repeated equilibrium being different from the Nash Equilibrium of a one-shot game. This concept is very well-documented, and in a large part is the origin of the field of evolutionary game theory. Here's the idea: let's say that a strategy is comprised of a piece of code, whose input is the previous results (cooperate or defect) of the other player. Then, strategies that cooperate will generate more value over time, as a player who is defecting will likely keep facing defecting, while a player who is cooperative would more likely have their counterparts cooperative.
The third concept I looked into was the idea of using quantum mechanics to model games. There were many papers that investigated this topic, but I particularly focused on https://arxiv.org/pdf/1010.0047.pdf and https://arxiv.org/pdf/quant-ph/9806088.pdf. The premise was as follows. If we model cooperate and defect as the two orthonormal vectors of a Hilbert space, H, we can view the strategy profiles of each player as a unitary operator within this space. Then, once the players decide upon their strategy, an arbitrator can take a measurement of the tensor product of those operators, then delegating payoffs to each player. Under this model, it turns out that the strategy profile of always defecting turns out to no longer be a Nash equilibrium, resulting in a Pareto efficient outcome. However, this model has many drawbacks, such as the unrealistic requirement of an arbitrator, and is largely infeasible to implement into everyday life.
Next week, I am planning to work through the math behind my ideas and see if I can extend them to games beyond the prisoners' dilemma, and also see how its drawbacks and strengths stack up to some of the other strategies I have researched.
That's all for today, make sure to tune in next week to see some of the theoretical mathematical concepts I investigate!
Week 3: Mathematical Foundations
Hello everyone!
Welcome back to my blog! Today, I will be discussing the mathematical concepts behind my project.
In Game Theory, we often use payoff matrices to define the parameters of two-player games. In a payoff matrix, each cell represents a potential outcome; the first number representing the payoff of the first player, the second representing the payoff of the second player. In the prisoner's dilemma, the payoff matrix will look like this:
where T > R > U > S.
Traditionally, the Nash Equilibrium, or optimal strategy for both players, would be for both players to defect. We would call defecting a dominant strategy, since no matter what the other player chooses to do, it gives more payoff for you to defect, rather than to cooperate. Yet, you may also notice that both players cooperating would lead to a better outcome for both players. So, we would call the outcome of both players defecting to be Pareto dominated (in other words, worse for both players), and the outcome of both players cooperating to be Pareto efficient (none of the other outcomes Pareto dominate it).
The basis of my project is to try to reach that improve the Nash Equilibrium and find ways to reach the Pareto efficient outcomes. In my previous blog post, I have explained ways in which people have managed to reach this Pareto efficient outcome; either through doing repeated games, through designing programs to perform your strategies, or through using concepts like quantum entanglement. In my project, I will be offering a different solution, which involves allowing the players to modify the payoff matrices to their benefit.
Imagine, for a moment, that the first player could offer a promise to the second player, that "if my payoff is at least T, I will only keep A of it and give you the rest." Then, our new payoff matrix would look like this:
Assuming that S+T-A > U, a dominant strategy no longer exists for either player. Thus, the new Nash Equilibrium must be a mixed strategy, where each player chooses to cooperate with a certain probability, and to defect the rest of the time. Given S+T-A > U, it is clear that S+T must be greater than 2U in order for this new situation to be productive.
The expected payoff for the first player turns out to be \frac{U+R-A-S}{UR-AS}. Taking its derivative, we find that the first player's expected payoff increases as A increases. For this reason, we can split this into two cases:
If S+T > R+U, then, if A > R, it becomes a dominant strategy for player 1 to defect, but for player 2 to cooperate. The expected payoff of player 1 would be A.
If S+T < R+U, then, A approaches S+T-U. In this case, the new payoff is \frac{U+R-S-T+U-S}{UR-(S+T-U)S}. After some algebraic manipulation, it can be shown that as long as S+T > 2U, the expected payoff is greater than U, resulting in a better situation for both players.
The math shows that the first person can in effect "sacrifice" some of their utility to bring the outcome of the game closer to being Pareto efficient, resulting in increases in expected payoff for both players.
That's it for this week's post. Stay tuned for next week, where I will be diving slightly deeper into the underlying mathematics, as well as talking about the practicality and implications of these ideas. See you then!
Week 4: Significance and Experimentation
Hello everyone!
Welcome back to my blog! Today, I will be explaining a bit more of the mathematics behind the model, while going through the new work that I have done this week.
One potential criticism of the effectiveness of the framework is as follows: In order to do the contract, you need to make legally binding decisions; however, if you could make legally binding decisions, you would just mutually agree upon choosing the Pareto efficient result.
To resolve this concern, we can dive back into the math of the situation. Instead of abstractly viewing the prisoner's dilemma as a payoff matrix, we can also view it in terms of the costs and benefits of decision making.
For example, consider a real life example, such as global warming. In two densely forested countries, governments may have a decision to cut down the trees to make room for more economic development, or keep the trees, helping preserve the climate of the globe. In this simplified scenario, let's say that the economic benefits of cutting down the forests amounts to $3 million, but you cause $5 million in environmental damages to the other country.
It is pretty easy to see that this boils down to a very similar prisoners' dilemma situation, where both countries are incentivized by the prisoner's dilemma to cut down their trees. Yet, presuming that we use a similar contract as previously, we do not need to know what the decision of the other country is in order to fulfill our side of the bargain—a much stronger premise than simply agreeing upon both choosing one particular decision.
This problem with simply agreeing upon choosing a single decision gets compounded when the model is more complicated; whether its because there are more players (or in this case, countries) participating in the game, or whether its because the payoffs yield a more complicated game. And all of these issues can be remedied using these contracts, giving a way to definitively enforce cooperation even when the decisions of other parties are not known.
Beyond continuing to dive into the math, I also began to work towards conducting an experiment last week. I wrote up instructions for the game, and by the end of the next two weeks, I am planning to have finished developing the mathematical foundations of my ideas, and to also be ready to conduct the experiment.
Thanks for tuning in today, make sure to catch the blog next week for more details about the progress of the blog!
Week 5: Game Design
Hello everyone!
Welcome back to my blog! Today, I will be talking through the progress I have made throughout the past week.
Experimental Design:
I have finished two rough drafts of my experiment, and considering the details about which version of the experiment I would run.
In the first version, it is both players may take the initiative to modify the payoff matrices, allowing for a symmetric set of instructions. And while this method may be more practical in real life, it has many limitations in an experimental setting, prompting me to prefer to use the second set.
In the second version, only player 1 can take the initiative to modify the payoff matrices. While the experiment would have asymmetric instructions, it would be much easier to implement, allowing the experimenter (me) to get all of the necessary information from a subject in just one poll.
In the coming weeks, I will still review my instructions: thinking about the budgeting and neutralizing any words that may cause participants to choose one action over another.
I will also draft instructions for a control experiment, where I run the same game, except without the framework, to see the difference in cooperation between the two experiments.
Report:
I have also started to write a comprehensive report about my project. I have written up all of the math behind the ideas in the framework, and I am thinking about adding more examples of games, beyond just the Prisoner's Dilemma, which will help illustrate the effects of the framework.
In writing the report, I have also thought through many new scenarios, seeing how the framework will be realistic and better than other trivial solutions (such as agreeing to a legally binding contract to both cooperate), and trying to come up with real world scenarios (that may be more complicated than the Prisoner's Dilemma) to use my framework to solve.
Make sure to tune into my blog next week, where I will be sharing additional updates about my experimental progress.
Week 6: Experimental Setup
Hello everyone!
Welcome back to my blog! Today I will be talking through my experimental design, as well as the progress I have made towards executing the experiment.
Within my experimental design, I am planning to have a control group (just a standard prisoner's dilemma), and an experimental group, which consists of the two parts I labeled within the previous blog post.
For the experiment, I am planning to hold it on Prolific. Prolific offers a way to reach a wide variety of participants and give payment to them.
However, as I was designing the instructions for the experimental group, I ran into a bit of a problem: depending on the first player's decisions, the instructions for the second player would be different. The only remedy I would have for this would be to make the contracts multiple choice, although even that would require many logistical challenges (along with the potential of introducing many sources of bias into the experiment).
Throughout the next week, I will be addressing this problem, finalizing the experimental setup, and getting ready to budget and publish the experiment on Prolific. Make sure to tune in then for more updates about my project!
Week 7: Finalizing Experimental Setup
Hello everyone!
Welcome back to my blog! Today, I will be going over the progress that I have made throughout the last week.
Last week, I talked about the format of how I am planning to run my experiment and the problems I ran into, today I will talk about the solutions I came up with for those problems, and how the statistical analysis of the data might look like after the experiment is conducted.
The main issue I ran into the previous week was budgeting and the logistics of creating various separate forms. To tackle these issues, I worked with my advisor to modify the format of the form: instead of having separate forms for each side to fill out, I have combined each of them into a single form, determining whether a subject is Player 1 or Player 2 randomly after the form is sent. This allows me to do a few things:
- It helps clarify the instructions of the form and allows players to think about how other players may view the situation
- It generates double the data for the experiment, helping ease the budgeting issues.
Now, to solve the other problem, I figured that by choosing payoffs that are small enough, I can simplify the options that each player has into a few different options, allowing me to fit everything on a singular, pre-set form.
After the data is collected, I will randomly assign players into pairs, randomly choosing which players to be Player 1 and which players to be Player 2. Then, after assigning and distributing the payoffs through Prolific, I will run t-test on the payoffs for Player 1, seeing whether they resemble the traditional Nash Equilibrium, or whether they resemble the payoff expected by my framework. Using this model, I can assess whether the model has had a practical effect on how the players managed the situation, and also whether it significantly improved upon the traditional, logical, line of reasoning. I will also see if demographic data had an effect, gathering data based on demographic information such as age, gender, or ethnicity.
Next week, I (hopefully) will have officially started the experiment on Prolific; under my current estimations, it will cost around $100-125 to conduct the experiment for 100 people. Tune in then for another update about my project!
Week 8: Trying to Begin Experimentation
Hello everyone,
Welcome back to my blog, where I will be fleshing out even more details about the experiment, including the instructions and logistics of the experiment.
In the previous week, I have rewritten and reviewed the instructions of the game. There were places where the instructions were slightly unclear, or there was unnecessary repetition. I have worked with my advisor to iron out those issues, doing the following:
- Within the form, I do not need to explain the game multiple times. Instead, I can explain the game on the first page and run through a complete example of how the game may be run. On the later pages of the form, I can merely explain what they need to do, allowing them to return back to the first page if they ever need a reference. This helps avoid repetition, and allows me to be more clear in what I am communicating to the participants.
I have also reviewed the logistics of the experiment; the cost of the experiment should be around $150, which is within the budget I have allocated to the project. As soon as my advisor and I are happy with the instructions, I will be able to publish it immediately, which should happen within the following week.
I have also begun working on the final slides for my presentation. Next week, I am planning to finish the basic structure of the slides (fleshing out the theoretical portions and leaving the statistical analysis for later), and begin experimentation.
Check back again next week for more details about my progress in sending out the experiment and writing up the slides. See you then!
Week 9: Beginning Experimentation
Hello everyone!
Welcome back to my blog!
This week, I have begun to send out the experiment on prolific! Below, I will explain exactly what my experiment looks like, and how much I expect it to cost.
Within the experiment, I am running two studies: a control group and an experimental group.
Within the experimental group, I will have a google form linked to the survey, where the instructions are explained and the participant can make their decisions.
- I will be paying participants a certain fixed amount before they fill out the survey, and then based on their responses, an additional amount.
The control group will be just a simple prisoner's dilemma, formatted the same as the experimental group, with the same payoff matrices. Everything will work the same, except the payoffs will be much lower (as it has much less complexity).
I am planning to spend approximately 200$ across both the control and experimental group.
Thanks for tuning back to the blog, I will give more updates about the progress of experimentation and results next time!
Week 10: Wrapping Up
Hello everyone!
Welcome back to my blog! Today, I will be going through the progress I have made this past week, as well as some of the interesting features of the data that I have gathered.
During this week, I let out the experiment on prolific. As of now, all of the data has been collected and I have sorted out and paid everyone their appropriate payoffs. I will go over some of the interesting pieces of data here.
First, the control group had a ginormous amount of cooperators (about half); this was very surprising, and may have pointed to an inappropriate rewards system. It seems as if the participants were more focused on simply completing the form as quickly as possible (given that it was so short), rather than trying to read the instructions and maximize the amount of money they got from completing it. In experimental game theory, this is what happens when the reward system is not dominant. Some proof of this was built into the design of it, where people would choose different choices even though they were in the same scenario.
While the control group's result was unexpected, the experimental group had much more representative results. Many of the results mirrored what I had predicted in the theory:
- Almost everyone chose to make some sort of intervention, with the most common one being (4, 1), which was the one that I suggested in my theory was the most effective.
- The split of cooperating and defecting also resembled the mixed strategy nash equilibrium that I wrote up in my theory.
While the results of the experimental group may suggest that the theory is "right," it does not provide definitive proof. Rather, it shows that the framework can be effective; even if the average person did not understand what was going on, the average of their decisions corresponds to the theoretical Nash, allowing it to improve Pareto Efficiency.
Thank you for tuning in this week, next week I will provide a general summary of how the senior project has gone as a whole, as well as some concluding thoughts about the statistical analysis and theory. I will be presenting the senior project on Saturday, so make sure to show up to see the final product or ask questions!