Someone I know, Carson Loughridge, wrote this very nice post explaining the core intuition around Shapley values (which play an important role in impact assessment and cooperative games) using Venn diagrams, and I think it's great. It might be the most intuitive explainer I've come across so far. 

Incidentally, the post also won an honorable mention in 3blue1brown's Summer of Mathematical Exposition. I'm really proud of having given input on the post.

I've included the full post (with permission), as follows:


Shapley values are an extremely popular tool in both economics and explainable AI.

In this article, we use the concept of “synergy” to build intuition for why Shapley values are fair. There are four unique properties to Shapley values, and all of them can be justified visually. Let’s dive in!

A figure from Bloch et al., 2021 using the Python package SHAP
A figure from Bloch et al., 2021 using the Python package SHAP

The Game

On a sunny summer day, you and your two best friends decide to run a lemonade stand. Everyone contributes something special: Emma shares her family’s secret recipe, Liam finds premium-quality sugar, and you draw colorful posters.

The stand is a big hit! The group ends up making $280. But how best to split the profits? Each person contributed in a different way, and the success was clearly due to teamwork…

Luckily, Emma has a time machine. She goes back in time — redoing the day with different combinations of team members and recording the profits. This is how each simulation went:

Who was involved?Total profits
Only Emma$20
Only you$30
Only Liam$10
Emma and you$90
You and Liam$100
Liam and Emma$30
Emma, Liam, and you$280

Individually, Emma makes 20 dollars running the lemonade stand, and you make 30 dollars. But working together, the team makes 90 dollars.

The sum of individual profits is 20 + 30 = 50 dollars, which is clearly less than 90 dollars. That extra 90 - 50 = 40 dollars can be attributed to team dynamics. In game theory, this bonus is called the “synergy” of you and Emma. Let’s visualize our scenario as a Venn diagram.

Note: Synergies can also be negative (e.g., if Liam and Emma fought it could hurt profits).

The synergy bonuses in the Venn diagram are “unlocked” when the intersecting people are part of the team. To calculate total profit, we add up all areas relevant to that team.

For example, when the team consists of just you and Liam, three portions of the Venn diagram are unlocked: the area exclusive to you (30 dollars), the area exclusive to Liam (10 dollars), and the area exclusively shared by you and Liam (60 dollars). Adding these areas together, the total profit for team “You and Liam” comes out to 30 + 10 + 60 = 100 dollars.

Referring to our Venn diagram, the same formula holds true for every other team:

Team membersSum of synergiesTotal profits
Emma$20$20
Liam$30$30
You$10$10
Emma, Liam$20 + $30 + $40$90
Liam, You$30 + $10 + $60$100
You, Emma$10 + $20 + $0$30
Emma, Liam, You$20 + $30 + $10 + $40 + $60 + $0 + $120$280

Emma and Liam are impatient and want their fair share of money. They turn to you, the quick-witted leader for help. While staring at the Venn diagram, an idea strikes!

Take a moment to look over the visual. How would you slice up the Venn diagram fairly? Pause here, and continue when ready.


You decide to take each “synergy bonus” and cut it evenly among those involved.

Doing the math, each person’s share comes out to:

Emma and Liam agree the splits are fair. The money is handed out, and everyone skips happily home to dinner.

In this story, the final payouts are the Shapley values of each team member. This intuition is all you need to understand Shapley values. For the adventurous reader, we now tie things back to formal game theory.


The Formalities

Shapley values are a concept from cooperative game theory. You, Liam, and Emma are all considered “players” in a “coalition game”. Every possible “coalition” (or team) has a certain “payoff” (or profit). The mapping between coalition and payoff (a.k.a. which just corresponds to our first table of profits) is called the “characteristic function” (as it defines the nature, or *character*, of the game).

We define a set of players  (which, in this case, is You, Emma, and Liam), and a characteristic function , where :

We can see how this is the same mapping we had in our table of profits by players:

Who was involved?Total profits
Only Emma$20
Only you$30
Only Liam$10
Emma and you$90
You and Liam$100
Liam and Emma$30
Emma, Liam, and you$280

We also define a synergy function labeled  where :

Similarly, the synergy function just corresponds to areas of the Venn diagram:

https://miro.medium.com/v2/resize:fit:4800/format:webp/1*QilJDt6fx0PcgpSnLks_fw.png

Thus, for a given player , the Shapley value is written as:

Which, in more compact notation, becomes:

The last is exactly the formula described on Wikipedia.


Concluding Notes

Shapley values are the ONLY way to guarantee:

  1. Efficiency — The sum of Shapley values adds up to the total payoff for the full group (in our case, $280).
  2. Symmetry — If two players interact identically with the rest of the group, their Shapley values are equal.
  3. Linearity — If the group runs a lemonade stand on two different days (with different team dynamics on each day), a player’s Shapley value is the sum of their payouts from each day.
  4. Null player — If a player contributes nothing on their own and never affects group dynamics, their Shapley value is 0.

Take a moment to justify these properties visually.

No matter what game you play and who you play with, Shapley values always preserve these natural properties of “fairness”.

Hopefully, you have gained some intuition for why Shapley values are “fair” and why they account for interactions among players. Proofs and more rigorous definitions can be found on Wikipedia.

Thanks for reading! :)

New Comment
8 comments, sorted by Click to highlight new comments since:

Shapley values are the ONLY way to guarantee: <Efficiency, Symmetry, Linearity, Null player properties>

Well it doesn't end at that: it turns out Shapley values for more than 2 players are not nicely behaved and instead violate Maximin Dominance, as demonstrated in https://www.lesswrong.com/posts/vJ7ggyjuP4u2yHNcP/threat-resistant-bargaining-megapost-introducing-the-rose#ROSE_Value__N_Player_Case__.

The article I link showed how this is fixed:

Shapley values are about adding everyone one-by-one to a team in a random order and everyone gets their marginal value they contributed to the team.

And that's kinda like giving everyone a random initiative ordering and giving everyone the surplus they can extract in the resulting initiative game.

If we're doing that, then maybe a player, regardless of their position, can ensure they get their maximin value? Maybe this sort of Random-Order Surplus Extraction can work. ROSE.

Curated. This was a quite nice introduction. I normally see Shapley values brought up in a context that's already moderately complicated, and having a nice simple explainer is helpful!

I'd like it if the post went into a bit more detail about when/how Shapley values tend to get used in real world contexts.

Do you know whether the person who wrote this would be OK with crossposting the complete content of the article to LW? I would be interested in curating it and sending it out in our 30,000 subscriber curation newsletter, if they were up for it.

Just asked him, will let you know!

This is amazingly clear!

Explaining the Shapley value in terms of the "synergies" (and the helpful split in the Venn diagram) makes much more intuitive sense than the more complex normal formula without synergies, which is usually just given without motivation. That being said, it requires first computing the synergies, which seems somewhat confusing for more than three players. The article itself doesn't mention the formula for the synergy function, but Wikipedia has it.

I was teaching myself bits of cooperative game theory and this is the clearest explanation I've found so far. I think it's a nice complement to this one.

Thanks, this is a beautiful explanation