The Undoing Project: A Friendship That Changed Our Minds
About this book
The book begins with people trying to improve decisions in fields such as sports and medicine, then turns to the Israeli psychologists whose experiments showed systematic ways intuition departs from statistical reasoning.
Kahneman and Tversky developed work on heuristics, biases, prospect theory, and the framing of choices through an unusually close collaboration. Lewis follows the intellectual exchange alongside the differences in temperament, ambition, and recognition that eventually strained it.
Their research changed economics, medicine, public policy, and the language used to discuss human error. The narrative also shows that identifying a bias does not make an individual observer automatically immune to it.
The Undoing Project is biography and history of ideas rather than a catalog of self-help rules. It is especially interested in how two minds worked together and how a scientific partnership can generate insights neither person would have reached alone.
Read a sample
5,079 words
Plain text
From the opening
Introduction
THE PROBLEM THAT NEVER GOES AWAY
Back in 2003 I published a book, called Moneyball, about the Oakland Athletics’ quest to find new and better ways to value baseball players and evaluate baseball strategies. The team had less money to spend on players than other teams, and so its management, out of necessity, set about rethinking the game. In both new and old baseball data—and in the work of people outside the game who had analyzed that data—the Oakland front office discovered what amounted to new baseball knowledge. That knowledge allowed them to run circles around the managements of other baseball teams. They found value in players who had been discarded or overlooked, and folly in much of what passed for baseball wisdom. When the book appeared, some baseball experts—entrenched management, talent scouts, journalists—were upset and dismissive, but a lot of readers found the story as interesting as I had. A lot of people saw in Oakland’s approach to building a baseball team a more general lesson: If the highly paid, publicly scrutinized employees of a business that had existed since the 1860s could be misunderstood by their market, who couldn’t be? If the market for baseball players was inefficient, what market couldn’t be? If a fresh analytical approach had led to the discovery of new knowledge in baseball, was there any sphere of human activity in which it might not do the same?
In the past decade or so, a lot of people have taken the Oakland A’s as their role model and set out to use better data, and better analysis of that data, to find market inefficiencies. I’ve read articles about Moneyball for Education, Moneyball for Movie Studios, Moneyball for Medicare, Moneyball for Golf, Moneyball for Farming, Moneyball for Book Publishing(!), Moneyball for Presidential Campaigns, Moneyball for Government, Moneyball for Bankers, and so on. “All of a sudden we’re ‘Moneyballing’ offensive linemen?” an offensive line coach for the New York Jets complained in 2012. After seeing the diabolically clever data-based approach taken by the North Carolina legislature in writing laws to make it more difficult for African Americans to vote, the comedian John Oliver congratulated the legislators for having “Money-balled racism.”
But the enthusiasm for replacing old-school expertise with new-school data analysis was often shallow. When the data-driven approach to high-stakes decision making did not lead to immediate success—and, occasionally, even when it did—it was open to attack in a way that the old approach to decision making was not. In 2004, after aping Oakland’s approach to baseball decision making, the Boston Red Sox won their first World Series in nearly a century. Using the same methods, they won it again in 2007 and 2013. But in 2016, after three disappointing seasons, they announced that they were moving away from the data-based approach and back to one where they relied upon the judgment of baseball experts. (“We have perhaps overly relied on numbers . . . ,” said owner John Henry.) The writer Nate Silver for several years enjoyed breathtaking success predicting election outcomes for the New York Times, using an approach to statistics he learned writing about baseball. For the first time in memory, a newspaper seemed to have an edge in calling elections. But then Silver left the Times, and failed to predict the rise of Donald Trump—and his data-driven approach to predicting elections was called into question . . . by the New York Times! “Nothing exceeds the value of shoe-leather reporting, given that politics is an essentially human endeavor and therefore can defy prediction and reason,” wrote a Times columnist, late in the spring of 2016. (Never mind that few shoe-leather reporters saw Trump coming, either, or that Silver later admitted that, because Trump seemed sui generis, he’d allowed an unusual amount of subjectivity to creep into his forecasts.)
I’m sure some of the criticism of people who claim to be using data to find knowledge, and to exploit inefficiencies in their industries, has some truth to it. But whatever it is in the human psyche that the Oakland A’s exploited for profit—this hunger for an expert who knows things with certainty, even when certainty is not possible—has a talent for hanging around. It’s like a movie monster that’s meant to have been killed but is somehow always alive for the final act.
And so, once the dust had settled on the responses to my book, one of them remained more alive and relevant than the others: a review by a pair of academics, then both at the University of Chicago—an economist named Richard Thaler and a law professor named Cass Sunstein. Thaler and Sunstein’s piece, which appeared on August 31, 2003, in the New Republic, managed to be at once both generous and damning. The reviewers agreed that it was interesting that any market for professional athletes might be so screwed-up that a poor team like the Oakland A’s could beat most rich teams simply by exploiting the inefficiencies. But—they went on to say—the author of Moneyball did not seem to realize the deeper reason for the inefficiencies in the market for baseball players: They sprang directly from the inner workings of the human mind. The ways in which some baseball expert might misjudge baseball players—the ways in which any expert’s judgments might be warped by the expert’s own mind—had been described, years ago, by a pair of Israeli psychologists, Daniel Kahneman and Amos Tversky. My book wasn’t original. It was simply an illustration of ideas that had been floating around for decades and had yet to be fully appreciated by, among others, me.
Continue reading the sampleClose the sample
That was an understatement. Until that moment I don’t believe I’d ever heard of either Kahneman or Tversky, even though one of them had somehow managed to win a Nobel Prize in economics. And I hadn’t actually thought much about the psychological aspects of the Moneyball story. The market for baseball players was rife with inefficiencies: why? The Oakland front office had talked about “biases” in the marketplace: Foot speed was overrated because it was so easy to see, for instance, and a hitter’s ability to draw walks was undervalued in part because walks were so forgettable—they seemed to require the hitter mainly to do nothing at all. Fat or misshapen players were more likely to be undervalued; handsome, fit players were more likely to be overvalued. All of these biases that the Oakland front office talked about I’d found interesting, but I hadn’t really pushed further and asked: Where do the biases come from? Why do people have them? I’d set out to tell a story about the way markets worked, or failed to work, especially when they were valuing people. But buried somewhere inside it was another story, one that I’d left unexplored and untold, about the way the human mind worked, or failed to work, when it was forming judgments and making decisions. When faced with uncertainty—about investments or people or anything else—how did it arrive at its conclusions? How did it process evidence—from a baseball game, an earnings report, a trial, a medical examination, or a speed date? What were people’s minds doing—even the minds of supposed experts—that led them to the misjudgments that could be exploited for profit by others, who ignored the experts and relied on data?
And how did a pair of Israeli psychologists come to have so much to say about these matters that they more or less anticipated a book about American baseball written decades in the future? What possessed two guys in the Middle East to sit down and figure out what the mind was doing when it tried to judge a baseball player, or an investment, or a presidential candidate? And how on earth does a psychologist win a Nobel Prize in economics? In the answers to those questions, it emerged, there was another story to tell. Here it is.
1
MAN BOOBS
You never knew what a kid in the interview room might say to jolt you out of your slumber and back to your senses and force you to pay attention. And once you were paying attention, you naturally placed far greater weight on whatever he had just said than you probably should: The most memorable moments in job interviews for the National Basketball Association were hard to consign to some appropriately sized compartment in the brain. In certain cases it was as if the players were trying to screw up your ability to judge them. For instance, when the Houston Rockets interviewer asked one player if he could pass a drug test, the guy had gone wide-eyed and grabbed the table and said, “You mean today!!!???” There was the college player who’d been arrested on charges (subsequently dropped) of domestic violence, and whose agent claimed it had been a simple misunderstanding. When they’d asked the player about it he’d explained, chillingly, that he’d grown weary of his girlfriend’s “bitching, so I just put my hands around her neck and I squeezed. ’Cause I needed her to shut up.” There was Kenneth Faried, the power forward out of Morehead State. When he showed up for his interview they’d asked him, “Do you prefer to be called Kenneth or Kenny?” “Manimal,” Faried said. He wanted to be called Manimal. What did you do with that? Roughly three out of every four of the black American players who came for NBA interviews—or at least came for interviews with the NBA’s Houston Rockets—had never really known their father. “It’s not uncommon, when you ask these guys who their biggest male influence was, for them to say, ‘My mom,’” said the Rockets’ director of player personnel, Jimmy Paulis. “One said, ‘Obama.’”
Then there was Sean Williams. Back in 2007 Sean Williams, six foot ten, was an off-the-charts player who had been suspended from his Boston College team the first two of his three seasons after being arrested for possession of marijuana (a charge that was later dropped). He’d played only fifteen games his sophomore year and still blocked 75 shots; the fans referred to his college games as The Sean Williams Block Party. Sean Williams looked like a big-time NBA player and was expected to be a first-round pick—in part because everyone assumed that his ability to get through his junior year without being suspended meant that he’d gotten his marijuana use under control. Before the 2007 NBA draft, he’d flown to Houston, at his agent’s request, to practice his interviewing skills. The agent cut the Rockets a deal: Williams would talk to the Rockets and the Rockets alone, and the Rockets would offer the agent tips about how to make Sean Williams more persuasive in a job interview. It actually went pretty well, until they got onto the topic of marijuana. “So you got caught smoking weed your freshman and sophomore years,” said the Rockets interviewer. “What happened your junior year?” Williams just shook his head and said, “They stopped testing me. And if you’re not going to test me, I’m gonna smoke!”
After that, Williams’s agent decided it was best for Sean Williams not to grant any more interviews. He still got himself drafted in the first round by the New Jersey Nets, and made brief appearances in 137 NBA games before leaving to play in Turkey.
Millions of dollars were at stake—NBA players were, on average, by far the highest-paid athletes in all of team sports. The future success of the Houston Rockets was on the line. These young people were hurling information about themselves at you that was meant to help you to make an employment decision. But a lot of times it was hard to know what to do with it.
Rockets interviewer: What do you know about the Houston Rockets?
Player: I know you are in Houston.
Rockets interviewer: Which foot did you hurt?
Player: I have been telling people my right foot.
Player: Coach and I did not see eye to eye.
Rockets interviewer: On what?
Player: Playing time.
Rockets interviewer: What else?
Player: He was shorter.
Ten years of grilling extremely tall people had reinforced in Daryl Morey, the general manager of the Houston Rockets, the sense that he should resist the power of any face-to-face interaction with some other person to influence his judgment. Job interviews were magic shows. He needed to fight whatever he felt during them—especially if he and everyone else in the room felt charmed. Extremely tall people had an unusual capacity to charm. “There’s a lot of charming bigs,” said Morey. “I don’t know if it’s like the fat kid on the playground or what.” The trouble wasn’t the charm but what the charm might mask: addictions, personality disorders, injuries, a deep disinterest in hard work. The bigs could bring you to tears with their story about their love of the game and the hardship they had overcome to play it. “They all have a story,” said Morey. “I could tell you a story about every guy.” And when the story was about perseverance in the face of incredible adversity, as it often was, it was hard not to grow attached to it. It was hard not to use it to create in your mind a clear picture of future NBA success.
But Daryl Morey believed—if he believed in anything—in taking a statistically based approach to decision making. And the most important decision he made was whom to allow onto his basketball team. “Your mind needs to be in a constant state of defense against all this crap that is trying to mislead you,” he said. “We’re always trying to figure out what’s a trick and what’s real. Are we seeing a hologram? Is this an illusion?” These interviews belonged on the list of the crap trying to mislead you. “Here’s the biggest reason I want to be in every interview,” said Morey. “If we pick him, and he has some horrible problem and the owner asks, ‘What did he say in the interview when you asked him that question?’ and I go, ‘I never actually spoke to him before we gave him one point five million dollars,’ I get fired.”
And so, in the winter of 2015, Morey, along with five members of his staff, sat in a conference room in Houston, Texas, waiting for another giant. The interview room contained nothing worth seeing. A conference table, some chairs, windows obscured by blinds. On the table rested a lone coffee mug, left by mistake, with a logo—National Sarcasm Society: Like We Need Your Support. The giant was . . . well, none of the men knew all that much about him except that he was still only nineteen years old, and that he was huge even by the standards of professional basketball. He’d been discovered five years earlier in a village in Punjab by some agent or talent scout—or so they’d been told. He was then fourteen years old, seven feet tall, and barefoot—or, at any rate, wearing shoes so tattered they revealed his feet.
They’d wondered about that. The kid’s family must have been so poor that they couldn’t afford to buy him shoes. Or maybe they’d decided it was pointless to buy shoes for feet that grew so rapidly. Or maybe the whole thing was a fiction invented by an agent. Either way, what lingered in the mind was the image: a seven-foot-tall, fourteen-year-old-boy, barefoot in the streets of India. They didn’t know how the boy had found his way out of the Indian village. Somebody, probably an agent, had arranged for him to travel to the United States to learn how to speak English and play basketball.
To the NBA he was a complete unknown. There was no video of the guy playing organized basketball. He hadn’t played, so far as the Rockets could determine. He hadn’t participated in the NBA Draft Combine, the formal audition for amateur players. It was only just that morning that the Rockets had been permitted to take his measurements. His feet were size 22, and his hands, from fingertip to wrist, were eleven and a half inches, the biggest hands the staff had ever measured. Shoeless, he stood seven foot two and weighed three hundred pounds, and his agent claimed he was still growing. He’d spent the past five years in southwest Florida learning basketball—most recently at IMG, a sports academy built to turn amateurs into professionals. Although no one they knew had seen him play, the few people who had laid eyes on him were still talking about it. Robert Upshaw, for instance. Upshaw was a thick seven-foot center who had been dismissed from his team at the University of Washington and was now auditioning for NBA teams. A few days earlier, in the Dallas Mavericks gym, he’d worked out with the Indian giant. Hearing from the Rockets scouts that he might be about to do it again, Upshaw’s eyes went wide and his face lit up and he said, “The dude is the biggest human being I’ve ever seen. And he can shoot the three-ball! It’s crazy.”
Back in 2006, when he was hired to run the Houston Rockets and figure out who should play pro basketball and who should not, Daryl Morey had been the first of his kind: the basketball nerd king. His job was to replace one form of decision making, which relied upon the intuition of basketball experts, with another, which relied mainly on the analysis of data. He had no serious basketball-playing experience and no interest in passing himself off as a jock or a basketball insider. He’d always been just the way he was, a person who was happier counting than feeling his way through life. As a kid he’d cultivated an interest in using data to make predictions until it became a ruling obsession. “That always seemed the coolest thing to me,” he said. “How do you use numbers to predict things? It was like a cool way to use numbers to be better than other people. And I really liked being better than other people.” He built forecasting models the way other kids built model airplanes. “It was always sports I was trying to predict. I didn’t know what else to apply it to—what, am I going to forecast my grades?”
His interest in sports and statistics had led him, at the age of sixteen, to pick up a book called The Bill James Historical Baseball Abstract. Bill James was then busy popularizing an approach, rooted in statistical reasoning, to thinking about baseball. With some help from the Oakland Athletics, that approach would trigger a revolution that ended with nerds running, or helping to run, virtually every team in Major League Baseball. In 1988, when he stumbled upon James’s book in a Barnes & Noble, Morey had no way of knowing that people with a gift for using numbers to predict things would overrun professional sports management and everyplace else high-stakes decisions were being made—or that basketball would be, in effect, waiting for him to grow up. He simply suspected that the established experts maybe didn’t know as much as everyone thought they did.
That particular suspicion had been born the year before, 1987, after Sports Illustrated splashed his favorite baseball team, the Cleveland Indians, on its cover and picked them to win the World Series. “I was like, ‘This Is It!!!! The Indians have sucked for years. Now we’re going to win the World Series!’” The Indians finished that season with the worst record in the major leagues: How did that happen? “The guys they had said were going to be so good were so bad,” recalled Morey. “And that was the moment when I thought: Maybe the experts don’t know what they’re talking about.”
Then he discovered Bill James and decided that, like Bill James, he might use numbers to make better predictions than the experts. If he could predict the future performance of professional athletes, he could build winning sports teams, and if he could build winning sports teams . . . well, that’s where Daryl Morey’s mind came to rest. All he wanted to do in life was to build winning sports teams. The question was: Who’d let him do it? In college he’d sent dozens of letters to professional sports franchises in the hope of being offered some menial job. He received not a single reply. “I didn’t have, like, any way to penetrate organized sports,” he said. “So I decided at that point that I had to be rich. If I was rich I could just buy a team and run it.”
His parents were middle-class midwesterners. He didn’t even know any rich people. He was also a distinctly unmotivated student at Northwestern University. He nevertheless set out to make enough money to buy a professional sports team, so that he might make the decisions about who would be on it. “Every week he’d take a sheet of paper and write on top, ‘My Goals,’” recalls his then-girlfriend, Ellen, now his wife. “The biggest life goal was, ‘I’m going to someday own a professional sports team.’” “I went to business school,” said Morey, “because I thought that’s where you had to go if you wanted to get rich.” Upon leaving business school, in 2000, he interviewed with consulting firms until he found one that got paid in the shares of the companies it advised. The firm was advising Internet companies during the Internet bubble: That sounded, at the time, like a way to get rich quick. Then the bubble burst and all the shares were worthless. “It turns out it was the worst decision ever,” said Morey.
From his stint as a consultant he learned something valuable, however. It seemed to him that a big part of a consultant’s job was to feign total certainty about uncertain things. In a job interview with McKinsey, they told him that he was not certain enough in his opinions. “And I said it was because I wasn’t certain. And they said, ‘We’re billing clients five hundred grand a year, so you have to be sure of what you are saying.’” The consulting firm that eventually hired him was forever asking him to exhibit confidence when, in his view, confidence was a sign of fraudulence. They’d asked him to forecast the price of oil for clients, for instance. “And then we would go to our clients and tell them we could predict the price of oil. No one can predict the price of oil. It was basically nonsense.”
A lot of what people did and said when they “predicted” things, Morey now realized, was phony: pretending to know things rather than actually knowing things. There were a great many interesting questions in the world to which the only honest answer was, “It’s impossible to know for sure.” “What will the price of oil be in ten years?” was such a question. That didn’t mean you gave up trying to find an answer; you just couched that answer in probabilistic terms.
Later, when basketball scouts came to him looking for jobs, the trait he looked for was some awareness that they were seeking answers to questions with no certain answers—that they were inherently fallible. “I always ask them, ‘Who did you miss?’” he said. Which future superstar had they written off, or which future bust had they fallen in love with? “If they don’t give me a good one, I’m like, ‘Fuck ’em.’”
By a stroke of luck, the consulting firm Morey worked for was asked to perform some analysis for a group trying to buy the Boston Red Sox. When that group failed in its bid to buy a professional baseball team, it went out and bought a professional basketball team, the Boston Celtics. In 2001 they asked Morey to quit his job consulting and come to work for the Celtics, where “they gave me the most difficult problems to figure out.” He helped hire new management, then helped to figure out how to price tickets, and, finally, inevitably, was asked to work on the problem of whom to select in the NBA draft. “How will that nineteen-year-old perform in the NBA?” was like “Where will the price of oil be in ten years?” A perfect answer didn’t exist, but statistics could get you to some answer that was at least a bit better than simply guessing.
Morey already had a crude statistical model to evaluate amateur players. He’d built it on his own, just for fun. In 2003 the Celtics had encouraged him to use it to pick a player at the tail end of the draft—the 56th pick, when the players seldom amount to anything. And thus Brandon Hunter, an obscure power forward out of Ohio University, became the first player picked by an equation.* Two years later Morey got a call from a headhunter who said that the Houston Rockets were looking for a new general manager. “She said they were looking for a Moneyball type,” recalled Morey.
The Rockets’ owner, Leslie Alexander, had grown frustrated with the gut instincts of his basketball experts. “The decision making wasn’t that good,” Alexander said. “It wasn’t precise. We now have all this data. And we have computers that can analyze that data. And I wanted to use that data in a progressive way. When I hired Daryl, it was because I wanted somebody that was doing more than just looking at players in the normal way. I mean, I’m not even sure we’re playing the game the right way.” The more the players got paid, the more costly to him the sloppy decisions. He thought that Morey’s analytical approach might give him an edge in the market for high-priced talent, and he was sufficiently indifferent to public opinion to give it a whirl. (“Who cares what other people think?” says Alexander. “It’s not their team.”) In his own job interview, Morey was reassured by Alexander’s social fearlessness, and the spirit in which he operated. “He asked me, ‘What religion are you?’ I remember thinking, I don’t think you’re supposed to ask me that. I answered it vaguely, and I think I was saying my family were Episcopalians and Lutherans when he stops me and says, ‘Just tell me you don’t believe any of that shit.’”
Alexander’s indifference to public opinion turned out to come in handy. Learning that a thirty-three-year-old geek had been hired to run the Houston Rockets, fans and basketball insiders were at best bemused and at worst hostile. The local Houston radio guys instantly gave him a nickname: Deep Blue. “There’s an intense feeling among basketball people that I don’t belong,” said Morey. “They remain silent during periods of success and pop up when they sense weakness.” In his decade in charge, the Rockets have had the third-best record of the thirty teams in the NBA, behind the San Antonio Spurs and the Dallas Mavericks, and have appeared in the playoffs more than all but four teams. They’ve never had a losing season. The people most upset by Morey’s presence had no choice at times but to go after him in moments of strength. In the spring of 2015, as the Rockets, with the second-best record in the NBA, headed into the Western Conference Finals against the Golden State Warriors, the former NBA All-Star and current TV analyst Charles Barkley went off on a four-minute tirade about Morey during what was meant to be a halftime analysis of a game. “. . . I’m not worried about Daryl Morey. He’s one of those idiots who believe in analytics. . . . I’ve always believed analytics was crap. . . . Listen, I wouldn’t know Daryl Morey if he walked in this room right now. . . . The NBA is about talent. All these guys who run these organizations who talk about analytics, they have one thing in common: They’re a bunch of guys who ain’t never played the game, and they never got the girls in high school and they just want to get in the game.”
There’d been a lot more stuff just like that. People who didn’t know Daryl Morey assumed that because he had set out to intellectualize basketball he must also be a know-it-all. In his approach to the world he was exactly the opposite. He had a diffidence about him—an understanding of how hard it is to know anything for sure. The closest he came to certainty was in his approach to making decisions. He never simply went with his first thought. He suggested a new definition of the nerd: a person who knows his own mind well enough to mistrust it.
One of the first things Morey did after he arrived in Houston—and, to him, the most important—was to install his statistical model for predicting the future performance of basketball players. The model was also a tool for the acquisition of basketball knowledge. “Knowledge is literally prediction,” said Morey. “Knowledge is anything that increases your ability to predict the outcome. Literally everything you do you’re trying to predict the right thing. Most people just do it subconsciously.” A model allowed you to explore the attributes in an amateur basketball player that led to professional success, and determine how much weight should be given to each. Once you had a database of thousands of former players, you could search for more general correlations between their performance in college and their professional careers. Obviously their performance statistics told you something about them. But which ones? You might believe—many then did—that the most important thing a basketball player did was to score points. That opinion could now be tested: Did an ability to score points in college predict NBA success? No, was the short answer. From early versions of his model, Morey knew that the traditional counting statistics—points, rebounds, and assists per game—could be wildly misleading. It was possible for a player to score a lot of points and hurt his team, just as it was possible for a player to score very little and be a huge asset. “Just having the model, without any human opinion at all, forces you to ask the right questions,” said Morey. “Why is someone ranked so high by scouts when the model has him ranked low? Why is someone ranked so low by scouts when the model has him ranked high?”