Tetlock, P.E., Kristel, O., Elson, B., Green M., &Lerner, J. Posing questions and letting the other person draw their own conclusions is more powerful than trying to give them your answer. Weak arguments dilute strong ones. Central to nearly all debates about politics, power, and justice is the tension between. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. 1996-2001 Harold Burtt Professor of Psychology and Political Science The Ohio State University. How do we know what we know, and how do we know if were right? The Dunning-Kruger effect: Identifies the disconnect between competence and confidence. Some smokejumpers held on to their equipment (as they were trained to do) despite the added weight (possibly) preventing them from surviving. Being persuaded is defeat. It looks like the CFO was in Prosecutor mode - calling out the flaws in your reasoning, marshalling arguments to prove you wrong and win her case. The test group outperformed the control group significantly and tended to pivot twice as often. Tetlock and Gardner (2015) also suggest that the public accountability of participants in the later IARPA tournament boosted performance. Required fields are marked *. Realistically, Grant could have turned each section into three separate bookshe covers a lot of ground that could benefit from greater depth. Preachers: We pontificate and promote our ideas. Since its original publication, Expert Political Judgment by New York Times bestselling author Philip Tetlock has established itself as a contemporary classic in the literature on evaluating expert opinion. Additionally, Good Judgment offers consulting services that are incredibly valuable for policymakers, who need to anticipate the global consequences of their decisions.7, Foresight isnt a mysterious gift bestowed at birth. I was most interested in the ideas from Part 1 and wish he focused on those more. Jason Zweig ofThe Wall Street Journalcalls it the most important book on decision making since Daniel KahnemansThinking, Fast and Slow, which, in the area of behavioral economics, is very high praise indeed. Tetlock has been interested in forecasting since the 1980s, he says during an interview at his home in Philadelphia. 29). That said, its hard to knock a book that preaches the importance of curiosity, open-mindedness, flexible thinking and empathy. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. Through consultations and workshops, Tetlock and his colleagues have been working to improve decision-making by promoting the qualities necessary to accurately foresee the outcomes of certain decisions. NASA took Lucas explanation at face value. How Can We Know? In the same study that yielded these somewhat sobering findings, however, Tetlock noticed that a few experts stood out from the crowd and demonstrated real foresight. They give examples of successful and unsuccessful decision-making processes, none more diametrically opposed as two US Army missions. Comparative politics is the study. Keeping your books Do preach to the people that already believe in your mission. In environments with psychological safety, teams will report more problems and errors (because they are comfortable doing so). We want to think of this idea when leading, when following, when making sales, when planning our marketing, and anywhere else we are dealing with the thoughts, opinions, and values of others. Lazaridis was brilliant and turned BlackBerry into a popular business tool. He has served on the faculty of the University of California, Berkeley (19791995, assistant professor), the Ohio State University (the Burtt Endowed Chair in Psychology and Political Science, 19962001) and again at the University of California Berkeley (the Mitchell Endowed Chair at the Haas School of Business, 20022010). One of the Latin roots of humility means from the earth. Its about being groundedrecognizing that were flawed and fallible.. Changing your mind is a sign of moral weakness. In P.E. He and his wife, Barbara Mellers, are the co-leaders of the Good Judgment Project, a multi-year forecasting study. the usefulness of hypothetical-society experiments in disentangling fact and value judgments of the impact of competing policy proposals. Tetlock, P.E., & Mitchell, G. (2009). Confident humility: An ideal wherein the individual has faith in their abilities but retains sufficient doubt and flexibility to recognize they could be wrong. The interviewer serves as a guide, not a leader or advisor. Deniers reject anything from the other side. They revert to preacher, prosecutor, and politician modes. It trades status seeking and prestige for our true calling. 5 Jun. When promoting your idea, you were being a Preacher - arguing your point of view based on a set of prior beliefs. How Can We Know? Study: Typically, researchers report new findings in scholarly journals and Tetlock (1998, 1999) has done so for of some part of the findings of his study. This is the mindset of the scientist. Political Psychology, 15, 567-577. In order to develop The Good Judgment Project, Tetlock worked alongside Barbara Mellers, a professor of psychology at the University of Pennsylvania. Psychological safety: The ability to take risks without fear of punishment or reprisal. Sunk costs are one explanation (an economic factor). Over the course of his career, Tetlock noticed that people spend a lot of time making judgments and decisions from three distinct 'mindsets': a preacher, a prosecutor, or a politician.. When he is pretty sure of how it is going to work, and he tells you, This is the way its going to work, Ill bet, he still is in some doubt. ", "From the commercial to the communal: Reframing taboo trade-offs in religious and pharmaceutical marketing", "Detecting and punishing unconscious bias", "Tetlock, P.E., Armor, D., & Peterson, R. (1994). the degree to which simple training exercises improved the accuracy of probabilistic judgments as measured by Brier scores; the degree to which the best forecasters could learn to distinguish many degrees of uncertainty along the zero to 1.0 probability scale (many more distinctions than the traditional 7-point verbal scale used by the National Intelligence Council); the consistency of the performance of the elite forecasters (superforecasters) across time and categories of questions; the power of a log-odds extremizing aggregation algorithm to out-perform competitors; the apparent ability of GJP to generate probability estimates that were "reportedly 30% better than intelligence officers with access to actual classified information. This approach to teaching is problematic as it involves passive transmission of ideas from expert to student. [24][25][26][27] Rather, humans prefer to believe that they have sacred values that provide firm foundations for their moral-political opinions. As if at some point you become something and thats the end., Kids might be better off learning about careers as actions to take rather than as identities to claim.. He covers a variety of topics, including the qualities he looks for in a good leader, whether it is becoming more difficult to make predictions about the world, and what we are able to infer from political speeches. or "How likely is the head of state of Venezuela to resign by a target date?" You wouldn't use a hammer to try to cut down a tree, and try to use an axe to drive nails and you're likely to lose a finger. Psychologist Peter T. Coleman experiments to learn how to reverse-engineer successful conversations between people about polarizing issues. Poking Counterfactual Holes in Covering Laws: Cognitive Styles and Historical Reasoning. Status is gained by holding the purest expression of these views. Opening story: Ursula Mercz, in the late 1800s, was diagnosed as blind but insisted she could see and was completely unaware of this fact. Dont try to politic a prosecutor, and be very careful if prosecuting a popular politician. Rethinking is fundamental to scientific thinking. Professor Tetlock, who's based at the University of Pennsylvania, famously did a 20-year study of political predictions involving more than 280 experts, and found that on balance their rate of . Our mini internal dictator. Skepticism is foundational to the scientific method, whereas denial is the a priori rejection of ideas without objective consideration.. How can we know? Critical Review. American Association for the Advancement of Science, International Society of Political Psychology, Intelligence Advanced Research Projects Activity, "Forecasting tournaments: Tools for increasing transparency and the quality of debate", "Identifying and Cultivating "Superforecasters" as a Method of Improving Probabilistic Predictions", "The Psychology of Intelligence Analysis: Drivers of Prediction Accuracy in World Politics", "Accounting for the effects of accountability", "Accountability and ideology: When left looks right and right looks left", "Cognitive biases and organizational correctives: Do both disease and cure depend on the ideological beholder? And if you absolutely mustand you better have a good reasondisobey them., The Government-funded research of the Good Judgment Project has manifested into a public platform called Good Judgment Open, where they recruit talented people to be trained to become a superforecaster.13They also have a global network of superforecasters who offer analytic services. A rivalry exists whenever we reserve special animosity for a group we see as competing with us for resources or threatening our identities.. (2000). Outrage goes viral and makes for better sound bites. Wilbur Wright: Honest argument is merely a process of mutually picking the beams and motes out of each others eyes so both can see clearly.. View being wrong as a good thing; an opportunity to learn something new. *These modes run throughout Adam Grants book, Think Again. For THE book on predictions and decisions in the face of uncertainty, see Philip Tetlocks Superforecasting., Your email address will not be published. Resisting the impulse to simplify is a step toward becoming more argument literate.. In this hour-long interview, Tetlock offers insight into what people look for in a forecaster everything from reassurance to entertainment and what makes a good forecaster it requires more than just intelligence. The Superforecasting book focused on shorter-range forecasts, the longest of which, about 12 months, being only as long as the shortest forecasts in the Expert Political Judgment project. philip tetlock preacher, prosecutor, politician. (2006) Ann Arbor, MI: University of Michigan Press. Administrative Science Quarterly 45 (2000), 293-326. Grit is essential for motivation (passion and perseverance), but it can also blind us to rethinking. He asked the man How can you hate me when you dont even know me? The men became friends and the KKK member eventually renounced his membership. The team was inserted into challenging conditions and the fire quickly overtook them. Listening well is more than a matter of talking less. One of the subjects was Ted Kaczynski (The Unabomber); he had one of the strongest negative responses to the study. We routinely fall into one or more of these roles when we engage with others and in our solitary conversations with ourselves. He is author of Expert Political Judgment: How Good Is It? When were in prosecution mode, we actively attack the ideas of others in an effort to win an argument. American Psychologist. Think about how this plays out in politics. Weve since come to rethink our approach to remote wildfires. Fuzzy thinking can never be proven wrong. If we want to gain alignment we have to understand where everyone is starting from. Binary bias: The human tendency to seek clarity by reducing a spectrum of categories to two opposites. He has written several non-fiction books at the intersection of psychology, political science and organizational behavior, including Superforecasting: The Art and Science of Prediction; Expert Political Judgment: How Good Is It? Although he too occasionally adopts this reductionist view of political psychology in his work, he has also raised the contrarian possibility in numerous articles and chapters that reductionism sometimes runs in reverseand that psychological research is often driven by ideological agenda (of which the psychologists often seem to be only partly conscious). Those who embraced flexible thinking did not. [10][11], In a 1985 essay, Tetlock proposed that accountability is a key concept for linking the individual levels of analysis to the social-system levels of analysis. Her research focuses on decision-making, specifically, the variables that influence the decisions we make that are often excluded from rational models of decision-making, such as emotions and the effects of context.15He has also collaborated with Dan Gardner, who works at the University of Ottawas Graduate School of Public Policy and International Affairs.16In addition to lecturing on risk, forecasting, and decision-making, Gardner offers consulting services to enable people to become better decision-makers, with one of his clients being none other than the Canadian Prime Minister, Justin Trudeau.17Gardner has also worked as a journalist and author18and he pennedSuperforecasting: The Art and Science of Predictionalong with Tetlock. philip tetlock preacher, prosecutor, politician2nd battalion, 4th field artillery regiment. Conventional vs. new views of intelligence: Psychologists find that test takers who second-guess their answers usually have better outcomes with their revised answers. The truth remains that for all our social science, the world manages to surprise us far more often than not. Cognitive Biases and Organizational Correctives: Do Both Disease and Cure Depend on the Ideological Beholder? When does accountability promote mindless conformity? (2004). When our 'sacred' beliefs are in jeopardy, we 'deliver sermons' to protect and promote our ideals. So Philip Tetlock reported in his 02005 book, Expert Political Judgement and in a January 02007 SALT talk. Prosecutors: We attack the ideas of others, often to win an argument. Journal of Experimental Social Psychology 43, 195-209. The interrogators would aggressively assault the subjects world-views (the goal was to mentally stress the participants). They argue that tournaments are ways of signaling that an organization is committed to playing a pure accuracy game and generating probability estimates that are as accurate as possible (and not tilting estimates to avoid the most recent "mistake"). The attack on Osama bin Ladens compound employed red teams and statistical risk assessments before the operation; whereas, the battle of the Bay of Pigs was undone by a failure to employ targeted questioning.5, When the scientist tells you he does not know the answer, he is an ignorant man. In the most comprehensive analysis of expert prediction ever conducted, Philip Tetlock assembled a group of some 280 anonymous volunteerseconomists, political scientists, intelligence analysts, journalistswhose work involved forecasting to some degree or other. A vaccine whisperer is called in. The very notion of applying group stereotypes to individuals is absurd., Chapter 7: Vaccine Whisperers and Mild-Mannered Interrogators. Search for truth through testing hypotheses, running experiments, and uncovering new truths. It consists of everything we choose to focus on. Logic bully: Someone who overwhelms others with rational arguments. Do recognize the ideas and the roles being applied and operate within them. Optimism and big-picture thinking will help you sell your business idea. Tetlock was born in 1954 in Toronto, Canada and completed his undergraduate work at the University of British Columbia and doctoral work at Yale University, obtaining his PhD in 1979. Still, Tetlock has gone beyond journal articles, turning to a If we want to get an idea across or attempt to change someones mind, our best bet is to first understand the lay of the land and the roles everyone is playing. Conflicts of interest and the case of auditor independence: Moral Seduction and Strategic Issue Cycling. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. By contrast, System 1 is largely a stranger to us. One finding: framing issues as binary (i.e. Synopsis. Western society views happiness at the individual level rather than the communal or societal level (interconnectedness). We would shut down., Philip Tetlock,Superforecasting: The Art and Science of Prediction. We will stand on any soapbox to sell it with tremendous enthusiasm. Tetlock, P. E. (1994). How Can We Know? Taboo Cognition and Sacred Values BACK TO TOP Defining and Assessing Good Judgment My 2005 book, Expert Political Judgment: How Good Is It? Opening story: Luca Parmitano, Italian astronaut who visited the International Space Station in 2013. "Hedgehogs" performed less well, especially on long-term forecasts within the domain of their expertise. This seems like an effective process until you realize that most of us are unable to accurately foresee the outcomes of our choices. American Political Science Review, 95, 829-843. Part IV: Conclusion 8 He went on to do his doctoral studies at Yale, where he obtained his Ph.D. in psychology in 1979. How Can We Know? What he found is that a person who is knowledgeable in a variety of areas is a better forecaster than a person who has an in-depth, but extremely narrow area of expertise. Whats the best way to find those out? When we dedicate ourselves to a plan and it isnt going as we hoped, our first instinct isnt usually to rethink it. How Can We Know? Contact: Philip Tetlock, (614) 292-1571; Tetlock.1@osu.edu Written by Jeff Grabmeier, (614) 292-8457; Grabmeier.1@osu.edu. Philip E. Tetlock (born 1954) is a Canadian-American political science writer, and is currently the Annenberg University Professor at the University of Pennsylvania, where he is cross-appointed at the Wharton School and the School of Arts and Sciences. By identifying the attributes shared by successful forecasters and the methodologies that allow for accurate forecasting, Tetlock and his team at Good Judgment are able to help companies promote these skills among their employees. Walk into Your Mind. Motivational interviewing: The best approach to changing someones mind is to help that person make the change on their own. We dont just hesitate to rethink our answers. It's also the question that Philip Tetlock, a psychologist at the University of Pennsylvania and a co-author of " Superforecasting: The Art and Science of Prediction ," has dedicated his career. It's also the question that Philip Tetlock, a psychologist at the University of Pennsylvania and a co-author of "Superforecasting: The Art and Science of Prediction," has dedicated his career to answering. Think Again is structured into three main parts.