Tetlock
<font size="4" face="Times New Roman">=</font><div><font size="4" face="Times New Roman">Cleese refs
</font><div><font size="4" face="Times New Roman">
</font></div><div><font size="4" face="Times New Roman">=</font></div><div><a href="https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/"><font size="4" face="Times New Roman" color="#000000">https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/</font></a></div><div><font size="4" face="Times New Roman">
</font></div><div><header class="content-header-component article-main-componentheader article-main-componentheader--default article-main-component__header--inner" style="box-sizing: border-box; border: 0px; font-variant-numeric: inherit; font-variant-east-asian: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 50px; padding: 0px; vertical-align: baseline; outline: 0px; background-color: rgb(255, 255, 255);"><h1 id="articleTitleFull" class="title" style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: 38px; margin: 0px 0px 50px; padding: 0px; vertical-align: baseline; transition: color 0.15s cubic-bezier(0.33, 0.66, 0.66, 1) 0s; word-break: break-word; overflow-wrap: break-word; letter-spacing: 0.0075em;"><font size="4" face="Times New Roman" style="">Do Political Experts Know What They're Talking About?</font></h1></header><article class="article-body-component article-body-component--null" style="box-sizing: border-box; border: 0px; font-variant-numeric: inherit; font-variant-east-asian: inherit; font-stretch: inherit; line-height: 28px; margin: 0px 0px 14px; padding: 0px 0px 0px 50px; vertical-align: baseline; outline: 0px; word-break: break-word;"><div style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><p style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><a href="http://www.psych.upenn.edu/node/20543" target="blank" style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline; transition: background 0.15s cubic-bezier(0.33, 0.66, 0.66, 1) 0s; word-break: break-word; overflow-wrap: break-word; background-color: transparent;">Philip Tetlock</a> is one of my favorite social scientists. I often joke that every cable news show should be forced to display a disclaimer, streaming in a loop at the bottom of the screen. The disclaimer would read: "These talking heads have been scientifically proven to not know what they are talking about. Their blather is for entertainment purposes only." The viewer would then be referred to Tetlock's most famous research project, which began in 1984. At the time, the cold war was flaring up again⎯Reagan was talking tough to the “Evil Empire”⎯and political pundits were sharply divided on the wisdom of American foreign policy. The “doves” thought Reagan was needlessly antagonizing the Soviets, while the “hawks” were convinced that the USSR needed to be aggressively contained. Tetlock was curious which group of pundits would turn out to be right, and so he began monitoring their predictions.</font></p><p style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><b style="background-color: rgb(255, 255, 255);">A few years later, after Reagan left office, Tetlock revisited the opinions of the pundits. His conclusion was sobering: everyone was wrong.</b><span style="font-weight: inherit; background-color: rgb(255, 255, 102);"> The doves assumed that Reagan’s bellicose stance would exacerbate Cold War tensions. They predicted a breakdown in diplomacy, as the USSR hardened its geopolitical stance. The reality, of course, was that the exact opposite happened. By 1985, Mikhail Gorbachev was in power. The Soviet Union began implementing a stunning series of internal reforms. The “evil empire” was undergoing glasnost.</span></font></p><p class="paywall" style="font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><span style="background-color: rgb(255, 255, 102);">But the hawks didn’t do much better. Even after Gorbachev began the liberalizing process, hawks tended to disparage changes to the Soviet system. They said the evil empire was still evil; Gorbachev was just a tool of the Politurbo. Hawks couldn’t imagine that a sincere reformer might actually emerge from a totalitarian state</span><span style="background-color: rgb(255, 255, 255);">.</span></font></p><p class="paywall" style="background-color: rgb(255, 255, 255); box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><b>The dismal performance of these pundits inspired Tetlock to turn his small case study into an epic experimental project. He picked a few hundred political experts - people who made their living “commenting or offering advice on political and economic trends” - and began asking them to make predictions about future events.</b><span style="font-weight: inherit;"> He had a long list of pertinent questions. Would George Bush be re-elected? Would there be a peaceful end to apartheid in South Africa? Would Quebec secede from Canada? Would the dot-com bubble burst? In each case, the pundits were asked to rate the probability of several possible outcomes. Tetlock then interrogated the pundits about their thought process, so that he could better understand how they made up their minds.</span></font></p><p class="paywall" style="font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style="background-color: rgb(255, 255, 102);">After Tetlock tallied up the data, the predictive failures of the pundits became obvious. Although they were paid for their keen insights into world affairs, they often performed worse than random chance. Most of Tetlock’s questions had three possible answers; the pundits, on average, selected the right answer less than 33 percent of the time. In other words, a dart-throwing chimp would have beaten the vast majority of professionals. These results are summarized in his excellent <span style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.amazon.com/Expert-Political-Judgment-Good-Know/dp/0691123020?tag=w050b-20" target="blank" rel="nofollow" style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline; transition: background 0.15s cubic-bezier(0.33, 0.66, 0.66, 1) 0s; word-break: break-word; overflow-wrap: break-word;">Expert Political Judgment</a></span>.</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman">Tetlock is currently embarking on an even more ambitious project. He was kind enough to answer a few questions about experts, hedgehogs and his future research.</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Lehrer</span>: You've been observing real world political experts for the last several decades. Could you tell me about some of the different styles you've observed among these experts?</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Tetlock</span>: My 2005 book, Expert Political Judgment, summarized a 20-year program of research in which we scored the accuracy of experts on a wide range of political and economic variables. We also explored experts' styles of thinking–and we found striking variation across individuals.</font></p><p class="paywall" style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><b style="background-color: rgb(255, 255, 255);">Some experts displayed a top-down style of reasoning: politics as a deductive art. They started with a big-idea premise about human nature, society, or economics and applied it to the specifics of the case. They tended to reach more confident conclusions about the future</b><span style="background-color: rgb(255, 255, 255); font-weight: inherit;">. And the positions they reached were easier to classify ideologically: that is the Keynesian prediction and that is the free-market fundamentalist prediction and that is the </span><b style="background-color: rgb(255, 255, 255);">worst-case environmentalist prediction </b><span style="background-color: rgb(255, 255, 255); font-weight: inherit;">and that is the </span><b style="background-color: rgb(255, 255, 255);">best case technology-driven growth prediction</b><span style="font-weight: inherit;"><span style="background-color: rgb(255, 255, 255);"> etc.</span><span style="background-color: rgb(255, 255, 102);"> Other experts displayed a bottom-up style of reasoning: politics as a much messier inductive art. They reached less confident conclusions and they are more likely to draw on a seemingly contradictory mix of ideas in reaching those conclusions (sometimes from the left, sometimes from the right).</span></span></font></p><p class="paywall" style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><b style="background-color: rgb(255, 255, 255);">We called the big-idea experts "hedgehogs"</b><span style="font-weight: inherit;"><span style="background-color: rgb(255, 255, 255);"> (they know one big thing) and the </span><span style="background-color: rgb(255, 255, 102);">more eclectic experts "foxes" (they know many, not so big things).</span></span></font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Lehrer</span>: Do these different styles correlate with levels of accuracy?</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><span style="font-weight: inherit; box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Tetlock</span><span style="font-weight: inherit;">: In assessing accuracy, it is crucial to make the "law of large numbers" work for you. Any fool can be lucky a few times. The key is consistency. So, in the first round of our studies, we assessed the accuracy of almost 30,000 predictions from almost 300 experts. We tested a lot of different hypotheses about the correlates of consistency and accuracy. Is ideology the key factor? Having a PhD? Having past access to classified information? And a lot of hypotheses bit the dust. </span><b>The most consistent predictor of consistently more accurate forecasts was "style of reasoning": experts with the more eclectic, self-critical, and modest cognitive styles tended to outperform the big-idea people (foxes tended to outperform hedgehogs).</b></font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Lehrer</span>: Your latest project aims to expand on this earlier research. Could you describe the project?</font></p><p class="paywall" style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><span style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Tetlock</span><span style="background-color: rgb(255, 255, 255); font-weight: inherit;">: The current project is supported by the </span><b style="background-color: rgb(255, 255, 255);">Intelligence Advanced Research Projects Agency </b><span style="font-weight: inherit;"><span style="background-color: rgb(255, 255, 255);">of the US government – and it is the most systematic effort, to date, at testing the effectiveness of competing approaches to improving forecasting accuracy in the domain of politics and national security.</span><span style="background-color: rgb(255, 255, 102);"> Multiple teams are participating – and we are recruiting people who, ideally, have three characteristics: a deep interest in thinking about how they think (and correcting blind spots and errors); a deep interest in world affairs; a willingness to translate those deep interests into hard work (it will take time and energy to master some of the tools that we will be giving various subgroups of forecasters).</span></span></font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Lehrer</span>: One of the things you've long been interested in is whether or not it's possible to improve the accuracy of expert predictions. (Given that our policy debates are heavily influenced by these experts, it's a rather pressing question.) Do you have a sense of whether or not improvement is possible? Or are we destined to always have experts that underperform random chance, if only because reality is so unpredictable?</font></p><p class="paywall" style="font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style="background-color: rgb(255, 255, 102);"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Tetlock</span>: I think it is useful to distinguish between radical and moderate skeptics on the question of how much room there is for improvement. Radical skeptics believe that there is zero predictability in the world – and that there is no such thing as good forecasting judgment (good judgment essentially equals good luck; that judgment equals bad luck).</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman">Warren Buffets are flukes: if you toss enough coins enough times, a few of them are bound to land "heads" 10 or 15 times in a row. Similarly, if you have enough forecasters making enough judgment calls, a few of them are bound to look spectacularly prescient.</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><b>As a moderate skeptic, I don't go that far. I don't rule out the possibility that there are individuals who can consistently outperform chance or simple extrapolation algorithms over long stretches of time (and these individuals are more likely to be foxes and hedgehogs). But these individuals are still hard to spot before the fact</b><span style="font-weight: inherit;">. And even these individuals often experience regression toward the mean in their performance (which means that their current performance is often already deteriorating toward "just average" by the time their previously superior performance comes to public attention).</span></font></p><p class="paywall" style="background-color: rgb(255, 255, 255); box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman" style=""><b>Balancing these arguments, my current gut instinct is that there is room for improvement but it may often be fairly small.</b><span style="font-weight: inherit;"> That said, even fairly small improvements may be enormously valuable to society. When you're talking about multi-trillion dollar decisions, you don't have to improve the accuracy of probabilistic forecasts by much to justify multi-million-dollar investments. A 10 or 20% improvement in accuracy could quite quickly translate into savings of many billions of dollars.</span></font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Lehrer</span>: Can non-experts do anything to encourage a more effective punditocracy? Should I feel bad about watching <span style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Meet the Press</span>?</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Tetlock</span>: Yes, non-experts can encourage more accountability in the punditocracy. Pundits are remarkably skillful at appearing to go out on a limb in their claims about the future, without actually going out on one. For instance, they often "predict" continued instability and turmoil in the Middle East (predicting the present) but they virtually never get around to telling you exactly what would have to happen to disconfirm their expectations. They are essentially impossible to pin down.</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman">If pundits felt that their public credibility hinged on participating in level playing field forecasting exercises in which they must pit their wits against an extremely difficult-to-predict world, I suspect they would be learn, quite quickly, to be more flexible and foxlike in their policy pronouncements.</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Lehrer</span>: What sort of subjects are you looking for in this study? And how can qualified people volunteer?</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span style="box-sizing: border-box; border: 0px; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Tetlock</span>: As I mentioned earlier, we are recruiting people who, ideally, have three characteristics: a deep interest in thinking about how they think (and correcting blind spots and errors); a deep interest in world affairs; a willingness to translate those deep interests into hard work (it will take time and energy to master some of the tools that we will be giving various subgroups of forecasters). I estimate that the total amount of work over the course of the year will be about 10 hours (we can pay a token honorary of $150 for participating for the entire year). We were required also to set a minimum educational requirement for participating (at least a baccalaureate degree – which is the minimum educational requirement for intelligence analysts).</font></p><p class="paywall" style="background-color: rgb(255, 255, 255); font-weight: inherit; box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-stretch: inherit; line-height: inherit; margin: 0px 0px 14px; padding: 0px; vertical-align: baseline; word-break: normal; overflow-wrap: break-word;"><font size="4" face="Times New Roman">The registration website is <a href="http://www.goodjudgment.info/" target="blank" style="box-sizing: border-box; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline; transition: background 0.15s cubic-bezier(0.33, 0.66, 0.66, 1) 0s; word-break: break-word; overflow-wrap: break-word; background-color: transparent;">www.goodjudgment.info</a></font></p></div></article></div><div><p><font size="4" face="Times New Roman">=</font></p><p><font size="4" face="Times New Roman"><span style="background-color: rgb(255, 255, 255);">aha rw pundits have an almost perfect record of being wrong [</span><span style="background-color: rgb(255, 255, 102);">cleese Tetlock 284?</span><span style="background-color: rgb(255, 255, 255);">], yet somehow paul Ehrlich gets to be poster boy punching bag for inaccurate prediction (did not help he lost long bet). Club of Rome? greens get tarred with that brush.</span></font></p><p><font size="4" face="Times New Roman"></font></p><h1 class="Uo8X3b" style="margin: 0px; padding: 0px; font-weight: normal; clip: rect(1px, 1px, 1px, 1px); height: 1px; overflow: hidden; position: absolute; white-space: nowrap; width: 1px; z-index: -1000; background-color: rgb(255, 255, 255);"><font size="4" face="Times New Roman">Search Results</font></h1><div eid="XKUpX7fhEIagtQWo5rCICA" data-async-context="query:tetlock%20pundit" id="rso" style="margin-top: 6px; background-color: rgb(255, 255, 255);"><div class="g" style="line-height: 1.2; width: 600px; margin-top: 0px; margin-bottom: 28px;"><h2 class="Uo8X3b" style="clip: rect(1px, 1px, 1px, 1px); height: 1px; margin: 0px; overflow: hidden; padding: 0px; position: absolute; white-space: nowrap; width: 1px; z-index: -1000;"><font size="4" face="Times New Roman">Web results</font></h2><div class="rc" data-hveid="CAQQAA" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFSgAMAB6BAgEEAA" style="position: relative;"><div class="r" style="margin: 0px; line-height: 1.58;"><a href="https://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert" ping="/url?sa=t&source=web&rct=j&url=https://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFjAAegQIBBAB" style="cursor: pointer; text-decoration-line: none;"><font face="Times New Roman" color="#000000" size="4">
</font></a><h3 class="LC20lb DKV0Md" style="font-weight: normal; margin: 0px 0px 3px; padding: 4px 0px 0px; display: inline-block; line-height: 1.3;"><a href="https://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert" ping="/url?sa=t&source=web&rct=j&url=https://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFjAAegQIBBAB" style="cursor: pointer; text-decoration-line: none;"><font face="Times New Roman" color="#000000" size="4">Everybody'S an Expert | The New Yorker</font></a></h3><div class="TbwUpd NJjxre" style="display: inline-block; line-height: 1.58; padding-bottom: 1px; padding-top: 1px; text-size-adjust: none; position: absolute; left: 0px; top: 0px;"><cite class="iUh30 bc tjvcx" style="font-style: normal; padding-top: 1px; line-height: 1.3;">www.newyorker.com<span class="eipWBe"> › everybodys-an-expert</span></cite></div><div class="B6fmyf" style="position: absolute; top: 0px; height: 0px; visibility: hidden; white-space: nowrap;"><font size="4" face="Times New Roman" /><div class="TbwUpd" style="display: inline-block; line-height: 1.58; padding-bottom: 1px; padding-top: 1px; text-size-adjust: none;"><font size="4" face="Times New Roman"><cite class="iUh30 bc tjvcx" style="font-style: normal; padding-top: 1px; line-height: 1.3;"><span class="eipWBe" /></cite></font></div><div class="eFM0qc" style="display: inline-block; padding-bottom: 1px; padding-top: 1px; padding-left: 2px; visibility: visible;"><div class="action-menu" style="display: inline; margin: 1px 3px 0px; position: relative; user-select: none; vertical-align: middle;"><a class="GHDvEf" href="https://www.google.com/search?q=tetlock+pundit&oq=tetlock+pundit&aqs=chrome..69i57.5535j0j7&sourceid=chrome&ie=UTF-8#" id="am-b0" aria-label="Result Options" aria-expanded="false" aria-haspopup="true" role="button" jsaction="m.tdd;keydown:m.hbke;keypress:m.mskpe" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQ7B0wAHoECAQQBA" style="display: inline-block; height: 12px; margin-top: 1px; user-select: none; width: 13px; cursor: pointer; text-decoration-line: none;"><span class="mn-dwn-arw" style="border-color: rgb(112, 117, 122) transparent; border-style: solid; border-width: 5px 4px 0px; width: 0px; height: 0px; margin-left: 3px; top: 7.5px; margin-top: -3px; position: absolute; left: 0px;" /></a><ol class="action-menu-panel" role="menu" tabindex="-1" jsaction="keydown:m.hdke;mouseover:m.hdhne;mouseout:m.hdhue" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQqR8wAHoECAQQBQ" style="margin: 0px; padding: 0px; border: 1px solid rgba(0, 0, 0, 0.2); position: absolute; left: 0px; top: 12px; visibility: hidden; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; z-index: 3; transition: opacity 0.218s ease 0s; box-shadow: rgba(0, 0, 0, 0.2) 0px 2px 4px;"><li class="action-menu-item" role="menuitem" style="margin: 0px; padding: 0px; border: 0px; list-style: none; cursor: pointer; user-select: none;"><a class="fl" href="https://webcache.googleusercontent.com/search?q=cache:ZYCWJb7fxz4J:https://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert+&cd=1&hl=en&ct=clnk&gl=us" ping="/url?sa=t&source=web&rct=j&url=https://webcache.googleusercontent.com/search%3Fq%3Dcache:ZYCWJb7fxz4J:https://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert%2B%26cd%3D1%26hl%3Den%26ct%3Dclnk%26gl%3Dus&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQIDAAegQIBBAG" style="text-decoration-line: none; cursor: pointer; display: block; padding: 7px 18px; outline: 0px;" /></li></ol></div></div></div></div><div class="s" style="max-width: 48em; line-height: 1.58;"><span class="st" style="line-height: 1.58; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span class="f" style="line-height: 1.58;">Dec 5, 2005 - </span>It is the somewhat gratifying lesson of Philip <span style="font-weight: bold;">Tetlock's</span> new book, ... and participate in <span style="font-weight: bold;">punditry</span> roundtables—are no better than the rest of us.</font></span></div><div jscontroller="m6a0l" id="eob_8" jsdata="fxg5tf;;Anf1pM" jsaction="rcuQ6b:npT2md" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQ2Z0BMAB6BAgEEAg" /></div></div><div class="g" style="line-height: 1.2; width: 600px; margin-top: 0px; margin-bottom: 28px;"><div class="rc" data-hveid="CAEQAA" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFSgAMAF6BAgBEAA" style="position: relative;"><div class="r" style="margin: 0px; line-height: 1.58;"><a href="https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/" ping="/url?sa=t&source=web&rct=j&url=https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFjABegQIARAB" style="cursor: pointer; text-decoration-line: none;"><font face="Times New Roman" color="#000000" size="4">
</font></a><h3 class="LC20lb DKV0Md" style="font-weight: normal; margin: 0px 0px 3px; padding: 4px 0px 0px; display: inline-block; line-height: 1.3;"><a href="https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/" ping="/url?sa=t&source=web&rct=j&url=https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFjABegQIARAB" style="cursor: pointer; text-decoration-line: none;"><font face="Times New Roman" color="#000000" size="4">Do Political Experts Know What They're Talking About? | WIRED</font></a></h3><div class="TbwUpd NJjxre" style="display: inline-block; line-height: 1.58; padding-bottom: 1px; padding-top: 1px; text-size-adjust: none; position: absolute; left: 0px; top: 0px;"><cite class="iUh30 bc tjvcx" style="font-style: normal; padding-top: 1px; line-height: 1.3;">www.wired.com<span class="eipWBe"> › 2011/08 › do-political-experts-know...</span></cite></div><div class="B6fmyf" style="position: absolute; top: 0px; height: 0px; visibility: hidden; white-space: nowrap;"><font size="4" face="Times New Roman" /><div class="TbwUpd" style="display: inline-block; line-height: 1.58; padding-bottom: 1px; padding-top: 1px; text-size-adjust: none;"><font size="4" face="Times New Roman"><cite class="iUh30 bc tjvcx" style="font-style: normal; padding-top: 1px; line-height: 1.3;"><span class="eipWBe" /></cite></font></div><div class="eFM0qc" style="display: inline-block; padding-bottom: 1px; padding-top: 1px; padding-left: 2px; visibility: visible;"><div class="action-menu" style="display: inline; margin: 1px 3px 0px; position: relative; user-select: none; vertical-align: middle;"><a class="GHDvEf" href="https://www.google.com/search?q=tetlock+pundit&oq=tetlock+pundit&aqs=chrome..69i57.5535j0j7&sourceid=chrome&ie=UTF-8#" id="am-b1" aria-label="Result Options" aria-expanded="false" aria-haspopup="true" role="button" jsaction="m.tdd;keydown:m.hbke;keypress:m.mskpe" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQ7B0wAXoECAEQBA" style="display: inline-block; height: 12px; margin-top: 1px; user-select: none; width: 13px; cursor: pointer; text-decoration-line: none;"><span class="mn-dwn-arw" style="border-color: rgb(112, 117, 122) transparent; border-style: solid; border-width: 5px 4px 0px; width: 0px; height: 0px; margin-left: 3px; top: 7.5px; margin-top: -3px; position: absolute; left: 0px;" /></a><ol class="action-menu-panel" role="menu" tabindex="-1" jsaction="keydown:m.hdke;mouseover:m.hdhne;mouseout:m.hdhue" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQqR8wAXoECAEQBQ" style="margin: 0px; padding: 0px; border: 1px solid rgba(0, 0, 0, 0.2); position: absolute; left: 0px; top: 12px; visibility: hidden; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; z-index: 3; transition: opacity 0.218s ease 0s; box-shadow: rgba(0, 0, 0, 0.2) 0px 2px 4px;"><li class="action-menu-item" role="menuitem" style="margin: 0px; padding: 0px; border: 0px; list-style: none; cursor: pointer; user-select: none;"><a class="fl" href="https://webcache.googleusercontent.com/search?q=cache:5byxr4RCO4kJ:https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/+&cd=2&hl=en&ct=clnk&gl=us" ping="/url?sa=t&source=web&rct=j&url=https://webcache.googleusercontent.com/search%3Fq%3Dcache:5byxr4RCO4kJ:https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/%2B%26cd%3D2%26hl%3Den%26ct%3Dclnk%26gl%3Dus&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQIDABegQIARAG" style="text-decoration-line: none; cursor: pointer; display: block; padding: 7px 18px; outline: 0px;" /></li><li class="action-menu-item" role="menuitem" style="margin: 0px; padding: 0px; border: 0px; list-style: none; cursor: pointer; user-select: none;"><a class="fl" href="https://www.google.com/search?q=related:https://www.wired.com/2011/08/do-political-experts-know-what-theyre-talking-about/+tetlock+pundit&tbo=1&sa=X&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQHzABegQIARAH" style="text-decoration-line: none; cursor: pointer; display: block; padding: 7px 18px; outline: 0px;" /></li></ol></div></div></div></div><div class="s" style="max-width: 48em; line-height: 1.58;"><span class="st" style="line-height: 1.58; overflow-wrap: break-word;"><font size="4" face="Times New Roman"><span class="f" style="line-height: 1.58;">Aug 4, 2011 - </span>Most of <span style="font-weight: bold;">Tetlock's</span> questions had three possible answers; the <span style="font-weight: bold;">pundits</span>, on average, selected the right answer less than 33 percent of the time. In ...</font></span></div></div></div><div class="g" style="line-height: 1.2; width: 600px; margin-top: 0px; margin-bottom: 28px;"><div class="rc" data-hveid="CAIQAA" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFSgAMAJ6BAgCEAA" style="position: relative;"><div class="r" style="margin: 0px; line-height: 1.58;"><a href="https://en.wikipedia.org/wiki/Philip_E.Tetlock" ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Philip_E.Tetlock&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFjACegQIAhAB" style="cursor: pointer; text-decoration-line: none;"><font face="Times New Roman" color="#000000" size="4">
</font></a><h3 class="LC20lb DKV0Md" style="font-weight: normal; margin: 0px 0px 3px; padding: 4px 0px 0px; display: inline-block; line-height: 1.3;"><a href="https://en.wikipedia.org/wiki/Philip_E.Tetlock" ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Philip_E.Tetlock&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQFjACegQIAhAB" style="cursor: pointer; text-decoration-line: none;"><font face="Times New Roman" color="#000000" size="4">Philip E. Tetlock - Wikipedia</font></a></h3><div class="TbwUpd NJjxre" style="display: inline-block; line-height: 1.58; padding-bottom: 1px; padding-top: 1px; text-size-adjust: none; position: absolute; left: 0px; top: 0px;"><cite class="iUh30 bc tjvcx" style="font-style: normal; padding-top: 1px; line-height: 1.3;">en.wikipedia.org<span class="eipWBe"> › wiki › Philip_E.Tetlock</span></cite></div><div class="B6fmyf" style="position: absolute; top: 0px; height: 0px; visibility: hidden; white-space: nowrap;"><font size="4" face="Times New Roman" /><div class="TbwUpd" style="display: inline-block; line-height: 1.58; padding-bottom: 1px; padding-top: 1px; text-size-adjust: none;"><font size="4" face="Times New Roman"><cite class="iUh30 bc tjvcx" style="font-style: normal; padding-top: 1px; line-height: 1.3;"><span class="eipWBe" /></cite></font></div><div class="eFM0qc" style="display: inline-block; padding-bottom: 1px; padding-top: 1px; padding-left: 2px; visibility: visible;"><div class="action-menu" style="display: inline; margin: 1px 3px 0px; position: relative; user-select: none; vertical-align: middle;"><a class="GHDvEf" href="https://www.google.com/search?q=tetlock+pundit&oq=tetlock+pundit&aqs=chrome..69i57.5535j0j7&sourceid=chrome&ie=UTF-8#" id="am-b2" aria-label="Result Options" aria-expanded="false" aria-haspopup="true" role="button" jsaction="m.tdd;keydown:m.hbke;keypress:m.mskpe" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQ7B0wAnoECAIQBA" style="display: inline-block; height: 12px; margin-top: 1px; user-select: none; width: 13px; cursor: pointer; text-decoration-line: none;"><span class="mn-dwn-arw" style="border-color: rgb(112, 117, 122) transparent; border-style: solid; border-width: 5px 4px 0px; width: 0px; height: 0px; margin-left: 3px; top: 7.5px; margin-top: -3px; position: absolute; left: 0px;" /></a><ol class="action-menu-panel" role="menu" tabindex="-1" jsaction="keydown:m.hdke;mouseover:m.hdhne;mouseout:m.hdhue" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQqR8wAnoECAIQBQ" style="margin: 0px; padding: 0px; border: 1px solid rgba(0, 0, 0, 0.2); position: absolute; left: 0px; top: 12px; visibility: hidden; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; z-index: 3; transition: opacity 0.218s ease 0s; box-shadow: rgba(0, 0, 0, 0.2) 0px 2px 4px;"><li class="action-menu-item" role="menuitem" style="margin: 0px; padding: 0px; border: 0px; list-style: none; cursor: pointer; user-select: none;"><a class="fl" href="https://webcache.googleusercontent.com/search?q=cache:PPsl4lqTsdIJ:https://en.wikipedia.org/wiki/Philip_E.Tetlock+&cd=3&hl=en&ct=clnk&gl=us" ping="/url?sa=t&source=web&rct=j&url=https://webcache.googleusercontent.com/search%3Fq%3Dcache:PPsl4lqTsdIJ:https://en.wikipedia.org/wiki/Philip_E.Tetlock%2B%26cd%3D3%26hl%3Den%26ct%3Dclnk%26gl%3Dus&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQIDACegQIAhAG" style="text-decoration-line: none; cursor: pointer; display: block; padding: 7px 18px; outline: 0px;" /></li><li class="action-menu-item" role="menuitem" style="margin: 0px; padding: 0px; border: 0px; list-style: none; cursor: pointer; user-select: none;"><a class="fl" href="https://www.google.com/search?q=related:https://en.wikipedia.org/wiki/Philip_E.Tetlock+tetlock+pundit&tbo=1&sa=X&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQHzACegQIAhAH" style="text-decoration-line: none; cursor: pointer; display: block; padding: 7px 18px; outline: 0px;" /></li></ol></div></div></div></div><div class="s" style="max-width: 48em; line-height: 1.58;"><span class="st" style="line-height: 1.58; overflow-wrap: break-word;"><font size="4" face="Times New Roman">Philip E. <span style="font-weight: bold;">Tetlock</span> (born 1954) is a Canadian-American political science writer, and is currently the Annenberg University Professor at the University of ...</font></span><div class="P1usbc rAWczb" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQrz0wAnoECAIQCA" style="display: table; margin: 5px 0px; line-height: 1.58; table-layout: fixed; width: 600px;"><div class="VNLkW" style="display: table-row; vertical-align: top;"><div class="i4vd5e" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQsz0oADACegQIAhAJ" style="display: table-cell; overflow: hidden; white-space: nowrap; width: 300px;"><div class="wrBvFf" style="overflow: hidden; text-overflow: ellipsis;"><font size="4" face="Times New Roman"><b>Nationality</b>: Canadian-American</font></div></div><div class="G1Rrjc" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQsz0oATACegQIAhAK" style="display: table-cell; padding-left: 15px; vertical-align: baseline; overflow: hidden; white-space: nowrap; width: 285px;"><div class="wrBvFf" style="overflow: hidden; text-overflow: ellipsis;"><font size="4" face="Times New Roman"><b>Born</b>: 1954 (age 65–66); <a class="fl" href="https://en.wikipedia.org/wiki/Toronto" ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Toronto&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQs2YoADACegQIAhAL" style="text-decoration-line: none; cursor: pointer;">Toronto, Ontario</a>, Canada</font></div></div></div><div class="k6DEPe" style="display: table-row; width: 600px;"><div class="i4vd5e" data-ved="2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQsz0oAjACegQIAhAM" style="display: table-cell; overflow: hidden; white-space: nowrap; width: 300px;"><div class="wrBvFf" style="overflow: hidden; text-overflow: ellipsis;"><font size="4" face="Times New Roman"><b>Alma mater</b>: <a class="fl" href="https://en.wikipedia.org/wiki/Yale_University" ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Yale_University&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQs2YoADACegQIAhAN" style="text-decoration-line: none; cursor: pointer;">Yale University</a> (<a class="fl" href="https://en.wikipedia.org/wiki/Ph.D." ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Ph.D.&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQs2YoATACegQIAhAO" style="text-decoration-line: none; cursor: pointer;">Ph.D.</a>) (1979)</font></div></div></div></div><div class="osl" style="margin-top: 0px;"><font size="4" face="Times New Roman"><a class="fl" href="https://en.wikipedia.org/wiki/Philip_E.Tetlock#Work" ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Philip_E.Tetlock%23Work&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQ0gIoADACegQIAhAQ" style="text-decoration-line: none; cursor: pointer; line-height: 1.58;">Work</a> · <a class="fl" href="https://en.wikipedia.org/wiki/Philip_E.Tetlock#The_Good_Judgment_Project" ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Philip_E.Tetlock%23The_Good_Judgment_Project&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQ0gIoATACegQIAhAR" style="text-decoration-line: none; cursor: pointer; line-height: 1.58;">The Good Judgment ...</a> · <a class="fl" href="https://en.wikipedia.org/wiki/Philip_E.Tetlock#Accountability_and_judgment_processes" ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Philip_E.Tetlock%23Accountability_and_judgment_processes&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQ0gIoAjACegQIAhAS" style="text-decoration-line: none; cursor: pointer; line-height: 1.58;">Accountability and ...</a> · <a class="fl" href="https://en.wikipedia.org/wiki/Philip_E.Tetlock#Political_versus_politicized_psychology" ping="/url?sa=t&source=web&rct=j&url=https://en.wikipedia.org/wiki/Philip_E._Tetlock%23Political_versus_politicized_psychology&ved=2ahUKEwi3_8y7k4LrAhUGUK0KHSgzDIEQ0gIoAzACegQIAhAT" style="text-decoration-line: none; cursor: pointer; line-height: 1.58;">Political versus ...</a></font></div><div class="osl" style="margin-top: 0px;"><font size="4" face="Times New Roman">...</font></div></div></div></div></div><p><font size="4" face="Times New Roman"></font></p></div><div><font size="4" face="Times New Roman">=</font></div></div>