Share this post

🔢 Key Takeaways

  1. From politicians seeking their services to proposed laws penalizing their predictions, the Romanian witch industry highlights the need to hold all professions accountable for their predictions and consider the impact of our own predictions in daily life.
  2. Be aware of the incentives behind predictions and consider the accuracy of sources before relying on them. False predictions have no accountability and can lead to outlandish but remembered claims.
  3. It's natural to want to make accurate predictions, but don't rely too heavily on them. Understanding the limitations of predictive abilities is important. However, prediction still has value in pursuing deterministic regularities in certain areas.
  4. Experts tend to use ambiguous language in making their predictions, making it difficult to assess their accuracy. We should approach their predictions with caution and not rely on them too heavily.
  5. Overconfidence and a reluctance to adjust predictions can lead to failed forecasts. Different areas of predicting require distinct approaches, as linear regularity and complex systems have a varying impact on accuracy. A flexible and unbiased attitude can enhance predictions.
  6. Forecasting is not always accurate due to ideological biases and unpredictability of extreme events. Consider multiple perspectives and exercise caution before making important decisions based on expert forecasts.
  7. Experts who make bold and accurate predictions about extreme events are often less accurate overall. This is because they prioritize attention-grabbing predictions over realistic ones. Therefore, relying on extreme event predictions can be unreliable.
  8. Prediction accuracy for football, just like in other fields, can be hindered by over-reliance on past data. Human-based predictions may not be much better than chance, but non-human-based predictions may be more successful.
  9. The USDA conducts surveys of around 85,000 farmers and ranchers to forecast America's agricultural output. They prioritize accuracy to meet industry standards, with monthly forecasts usually within 5% of the final prediction.
  10. While physical measurements can predict corn yield, weather patterns greatly affect it. The USDA uses averages instead of forecasts due to weather prediction difficulties. Accuracy is crucial to prevent market impacts and the USDA is committed to reporting.
  11. Despite our best efforts, the future is unpredictable. Beware of mistaking patterns where there are none and remain vigilant to avoid biases and illusions of certainty.
  12. Being a good predictor requires self-criticism, cognitive style, and being open to learning from mistakes. Adopting an opportunistic and pragmatic approach like foxes is helpful, admitting mistakes is key to success.
  13. Experts should track their accuracy and start small when predicting the future. Incentivizing pundits to have their predictions tested can improve accuracy. Successful prediction methods, like Pandora Radio, breakdown components to make recommendations.
  14. Pandora uses a million songs coded with 480 musical attributes to provide a personalized listening experience. Users can give feedback with the thumbs-up feature, improving recommendations.
  15. Prediction markets incentivize informed individuals to speak up, while discouraging uninformed noise. They provide more accurate forecasts, free from traditional biases, and are becoming increasingly popular in private organizations and for-profit markets.
  16. Prediction markets are useful for accurate predictions but require buy-in and ethical considerations. The outcomes are only as valuable as the information used to make them, but if done responsibly, they could save lives.
  17. Making predictions may seem easy and incentivized, but there are often no consequences for being wrong. This lack of accountability can lead to false information and fear, so it's important to take predictions with a grain of salt.

📝 Podcast Notes

The Witch Industry in Romania: Facing Opposition to Regulation

Witches in Romania hold a position of power and respect, with even politicians seeking their services. However, the Romanian witch industry has faced opposition, with proposed laws to regulate and penalize them for inaccurate predictions. This raises questions about holding other professions accountable for their predictions, such as politicians and money managers. Despite this, we must all recognize that we are constantly predicting the future in our daily lives, and should take a moment to consider the impact of our own predictions.

The Incentives Behind Society's Proliferation of Predictions

The reason why there are so many predictions made in our society is due to the incentives set up to encourage predictions. Those who make wildly unexpected predictions that come true have a strong incentive to remind others that they were right. However, those who make false predictions have no incentive to admit their mistakes. This lack of accountability leads to a proliferation of outlandish predictions that get remembered and talked about, even if they are often incorrect. In some cases, the goal of prediction is to be completely in line with everyone else, as there is safety in numbers. Therefore, it is important to be aware of the incentives behind predictions and to consider the accuracy of the sources before investing in or relying on them.

The Human Need for Prediction and the Pitfalls of Overvaluing It

Philip Tetlock, a research psychologist, has studied the hardwired human need for prediction and the prediction industry it has created. While accurate predictions are in high demand, there is a gap between demand and real supply which has led to the infusion of 'fake supply' such as the punditocracy that predicts ad nauseam, often on television. Tetlock warns against overvaluing prediction and emphasizes the importance of understanding the limitations of our predictive abilities. Despite this, prediction is valuable and has led to great collective successes in pursuit of deterministic regularities in messy phenomena such as agriculture and antibiotics.

Experts are not as accurate in their predictions as they think they are.

Experts tend to overestimate their ability to make accurate predictions. The words they use to make predictions, such as 'could,' have a wide range of interpretations, making it difficult to assess their accuracy. Political experts in particular are not very good at making predictions. In a study that tracked the accuracy of over 80,000 predictions by nearly 300 political experts over 20 years, Philip Tetlock found that there was a systematic gap between subjective probabilities that experts assigned to possible futures and the objective likelihoods of those futures actually occurring. Tetlock's study shows that we should approach expert predictions with caution and not rely on them too heavily.

The Pitfalls of Predicting Future Events

The experts who claim to have a better understanding of future events are not always correct. Overconfidence and an unwillingness to change their minds in the face of new evidence are the key reasons for their failure. The lack of distinction between different areas of predicting, such as political, economic, or sports predictions, is a common mistake. The linear regularity and complex systems of prediction in different realms have a significant impact on the accuracy of predictions. While Scandinavian politics is easier to predict, Middle Eastern politics are much harder to predict due to their complex systems. Experts should acknowledge their shortcomings and embrace a more flexible and unbiased approach to improve their predictions.

Expert Forecasting: Examining Political and Financial Predictions

Expert political forecasters are not always accurate in their predictions, with one major flaw being their rigid adherence to ideological beliefs. Meanwhile, even top Wall Street economists who make bold economic predictions based on hard data may struggle to predict extreme events accurately. A study by Christina Fang and Jerker Denrell highlights that those who correctly predict extreme outcomes tend to have lower overall forecasting accuracy. This suggests that while expertise and experience are important, there is also an element of luck involved in making accurate predictions. With this in mind, it may be wise to approach forecasts with caution and consider multiple perspectives before making important decisions.

The Limitations of Experts in Predicting Extreme Events

Experts who predict extreme events accurately often have a lower overall level of accuracy. They tend to make bold predictions to get attention instead of making more realistic predictions. Due to their apparent success, they are known for making the big one. However, our research suggests that these experts are unlikely to repeat their success very often, and their overall capability is not as impressive as it seems. This problem is not only limited to predicting the economy or political future but also applies to other fields such as football. Even the most experienced NFL experts had a 25% chance of making accurate predictions. Therefore, making bold predictions based on extreme events can be misleading and less reliable.

Experts Have Only Slightly Better NFL Prediction Accuracy than Animals

Experts in predicting NFL outcomes only have a slightly higher accuracy rate than an untrained animal, with only a three percentage point improvement over a casual fan. This is because they tend to rely too heavily on the previous year's standings. However, two NFL pundits, Pat Yasinskas and John Clayton, stand out for their excellent wild-card picks, although luck does play a role in their success. Despite the difficulty in predicting the future of human beings, accuracy rates for such predictions are barely better than a coin toss, whether it's football experts, economists or political pundits. Predicting outcomes with human beings as secondary to the main action, such as in agriculture, may yield more accurate results.

Forecasting America's Agricultural Output through Large Surveys

The USDA forecasts America's vast agricultural output by conducting large surveys of farmers and ranchers, sampling around 85,000 participants. They randomly select and mark off plots of crops, counting every plant and ear over time, estimating ear weight and fruit weight, and measuring harvest loss at the end. Joe Prusacki, the editor of publications like 'Acreage' and 'Prospective Plantings,' thinks their monthly forecasts are usually within five percent of the final prediction, with the majority of the time being within two to three percent. However, farmers expect more precision, making the USDA prioritize accuracy in their work to meet the industry's standards.

Predicting Corn Yield and the Impact of Weather

While predicting corn yield based on physical measurements of corn plants is relatively simple compared to predicting the weather, it is the weather factor that can greatly affect it. The USDA uses historic averages instead of forecasts because of the difficulty in predicting long-term weather patterns. Inaccurate forecasts can have significant impacts on the corn market and prices, as seen in the USDA's lower estimate of corn stockpiles last June and cut estimates of corn yield in October. However, the USDA is subject to confidentiality agreements and strict penalties to prevent the early release of information. Despite the potential for market manipulation, the USDA is committed to accurate and timely reporting.

The Challenge of Predicting the Future

Predicting the future is a daunting task and is often influenced by random events, making it challenging to maintain a sense of certainty. Despite our best efforts, the future will always hold unexpected surprises, and our need for prediction can often lead to biases and errors. While it may be costlier to not detect a pattern, mistaking the non-random for the random is what Nassim Taleb calls the 'one-way bias.' This bias can be harmful, as it creates illusions of certainty and makes us believe that things that have not exhibited risk, such as the stock market, are riskless. Therefore, it is important to keep in mind that the future is unpredictable and to avoid mistaking patterns where there are none, and always remain vigilant.

The Art of Prediction: Importance of Self-Criticism and Cognitive Style

To be a good predictor, one must have a capacity for constructive self-criticism and be open to changing their course of prediction. Philip Tetlock's study found that cognitive style was more important than any other factor in determining predictive ability. The foxes know many things and have an opportunistic and pragmatic approach towards forecasting, while the hedgehogs focus on one big thing and use a more focused approach. Predictions are often wrong, but admitting that and being willing to learn from mistakes is key to becoming a better predictor. Just like the turkey who assumes the butcher will always feed them, we must ask ourselves whether we are willing to be the butcher or the turkey in our predictive abilities.

The Hedgehog vs Fox Approach to Forecasting and Improving Prediction Accuracy

The hedgehog approach to forecasting is deductive and rigid, while the fox approach is flexible and less dogmatic. Foxes tend to be better predictors but hedgehogs are more likely to offer quotable sound bites in the media. To improve the accuracy of expert predictions, more effort should be put into tracking accuracy and incentivizing pundits to have their predictions tested. The first step toward predicting the future should be acknowledging our limitations and starting small. One example of successful prediction is Pandora Radio, which uses a breakdown of musical components to recommend similar music to listeners.

Pandora's Music Genome Project

Pandora's Music Genome Project has coded over a million songs with 480 musical attributes, allowing the website to make educated guesses about what users want to hear next. The thumbs-up feature allows users to give feedback, improving the music recommendations over time. While Pandora can't claim to map a user's emotional persona or predict the future hits, it does strive to provide a personalized and enjoyable listening experience. With the average American listening to 17 hours of music a week, Pandora's contribution to culture is in making that experience more meaningful and tailored to individual tastes.

Prediction Markets: A Reliable Tool for Understanding the Future

Prediction markets are emerging as a reliable tool for understanding the future. Unlike traditional forecasting methods, prediction markets incentivize individuals to speak up only when they have the best information and encourage everyone else to remain quiet, eliminating unnecessary noise. This allows for accurate forecasts as the few people with valuable knowledge speak up and the majority of uninformed individuals stay out of the conversation. Private organizations and for-profit prediction markets like InTrade, which allows individuals to place bets on events like terrorist attacks or the Euro’s fate, are flourishing. Prediction markets provide a path towards a more accurate understanding of the future, free from the biases of traditional experts.

Prediction markets can be valuable tools for forecasting a range of geopolitical and societal trends, but Buy-In from possible customers is necessary. Accurate forecasts and accountability excite the people who are willing to put the most resources in being right and punish those with bad predictions. The DARPA project to predict geopolitical trends in the Middle East could have provided essential early warnings about historical events. However, the market's untimely end shows the societal resistance to predictive marketing caused by ethical and cultural concerns. The outcomes of these markets are only as valuable as the information used to make them. If done responsibly, customers could make accurate, consequential predictions that could potentially save or help many lives.

The Consequences (or Lack Thereof) of Making Predictions

Predictions are encouraged because of the incentives, but the downside of being wrong is almost non-existent. If there was a consequence for incorrect predictions, such as a jail sentence, there would be few predictions made. Pundits often get away with bad calls by attributing them to timing. People often use excuses like a storm happening two states over to cover failed predictions. There are also doomsday predictions that turn out to be false, causing people to become fearful, especially children. Conclusively, people can say almost anything about the future, but there are often no consequences for incorrect predictions.