How to Make the World Add Up - critical summary review

Do you struggle with reading statistics? Or do you even doubt they can tell you anything valuable about the world? Economist Tim Harford is of the opinion that statistics can be incredibly powerful tools to help you understand the world. But in order to get the full use out of statistical data, you will need to know the 10 rules of how to read and approach statistics. So get ready to learn how to make sense of the world around you!

Rule 1: Search your feelings

Did you know that babies are actually delivered by storks? It can be proven using statistics. If you look at the number of babies born every year and the estimated stork population of countries across Europe, you will notice a strong similarity between the two numbers. That however does not mean that there is a causality – but if you were to draw a statistical graph, you would definitely have some strong “proof” that there is a connection between the number of storks and the number of babies in any given country.

It seems that it can be easy to lie with statistics, but that does not mean you should dismiss them altogether. Statistics can provide valuable information, especially in life-threatening circumstances such as the global pandemic of 2020. 

You only need to learn how to read statistics correctly, and how to separate valuable information from misinformation. The first lesson you should absorb is that your feelings can trump your expertise. As the author writes, “The more extreme the emotional reaction, the harder it is to think straight.” Don’t dismiss your emotional response entirely but be aware of it and take it into account when dealing with statistics.

Rule 2: Ponder your personal experience

Sometimes, statistics will provide a different picture than you might expect from your own experience, or they might even oppose your personal experience completely. Take the author as an example. On his daily commute to work using public transport in London, he regularly notices how overcrowded the Tube is. Yet, according to TfL (Transport for London), the “crush capacity” of a Tube train is more than a thousand people, while the average occupancy is 130. Yet, every time the author takes the Tube, it feels like it is filled to “crush capacity.”

How can this be? Obviously, personal experience is just as subjective as an emotional response. Still though, it should not be cast aside along with feelings – personal experience can often give important hints as to whether a statistic is trustworthy or not. 

In the TfL case, there is a simple explanation why the author’s personal experience differed so much from the statistics. While some trains run with very few passengers on them, most trains are filled to capacity. But having a few empty trains obviously reduces the number of average train occupants. If you were to measure the situation for the average passenger instead of the average train, you would end up with a number that would much more closely reflect the author’s personal experience (and that of most of his fellow travelers).

Rule 3: Avoid premature enumeration

In early 2020, when daily COVID-19 deaths were reported in the UK, the numbers given out were mostly incorrect. That’s because the statistics only represented hospital deaths, and not those people who had died of the virus at home. The second reason was because hospitals were stretched to capacity and would often report deaths with a three-day delay. During this initial period, death numbers were growing exponentially, so the numbers represented a much less severe situation than was actually the case.

The author says, “When we are trying to understand a statistical claim – any statistical claim – we need to start by asking ourselves what the claim actually means.” Always make sure you understand how and what is being measured or counted. 

For example, if you were to look at the number of nurses employed at a hospital, do you count two part-time nurses as one or as two? And looking at the wealth of the super-rich, how is their wealth measured? According to tax declarations, or including any hidden assets? When looking at statistics, we often fall for what the author calls “premature enumeration” – interpreting the numbers before having clarified what they actually mean.

Rule 4: Step back and enjoy the view 

In April 2018, many London newspapers claimed, “London’s Murder Rate Is Higher than New York’s for the First Time Ever!” Quite a scary proclamation at first glance, but upon reading the article it became clear that while New York had reported 14 murders in February 2018, London had recorded 15 murders. Objectively, these numbers do not tell you much about the actual risk of getting murdered. 

When you step back and put the numbers into perspective, you quickly learn that New York had more than 10 times the number of murders than London did in 1990. However, since then, murder rates decreased in both cities, a lot more drastically in New York than in London. New York still reports more murders than London yearly, but on a monthly basis, the numbers fluctuate. 

By looking at statistics with a wider temporal frame, it quickly becomes obvious that the headline was not big news at all, and that in fact, London was safer than ever before.

Rule 5: Get the back story

You may have heard of the study conducted by Sheena Iyengar and Mark Lepper, who, in an experiment, offered shoppers at a supermarket six varieties of jam, and another group of shoppers 24 varieties. They then noticed that people seemed more inclined to buy when they had fewer choices. And yet, companies such as Starbucks pride themselves on their endless choices and seem to be doing well. 

Indeed, for all its popularity, Iyengar and Lepper’s study was later disproved by several other studies, but most of these never reached publication. The stories that reach us are often biased: publishers decide what is worth sharing with us and what is not. That explains why you have probably heard of the “less choice is better” claim, but not of the countless other studies that proved the opposite was true, or at the very least showed a slightly different picture.

So when faced with statistics, always make sure to check the back story!

Rule 6: Ask who is missing

In the 1950s, psychologist Solomon Asch conducted a series of interesting studies, one of them proving that humans feel pressured to conform with a group, even when the group’s actions are obviously wrong. While Asch’s studies have been cited and used over and over again, it rarely happens that someone looks at the subjects Asch used for his studies: male American college students – hardly representative of the entire human population.

And indeed, many studies use so-called “WEIRD” demographics: Western, Educated and from Industrialized Rich Democracies. Women are especially underrepresented in statistical data, as Caroline Criado Perez demonstrated in her book “Invisible Women: Exposing Data Bias in a World Designed For Men.” Trials for new medical treatments are often conducted on a male demographic, for example, even though medicines often have different effects on people based on their gender. Data is only ever as neutral as the people collecting it.

Rule 7: Demand transparency when the computer says “no”

In recent years, algorithms and big data have become increasingly more popular. In 2009, Google developed the algorithm “Google Flu Trends,” which had taught itself how to recognize and predict the spread of flu through a population using data from Google searches.

For a while, “Google Flu Trends” was more accurate and quicker than the U.S. Centers for Disease Control and Prevention (CDC), but eventually, the algorithm failed. It was not able to predict the summer flu outbreak in 2009, because the algorithm had not looked at causality, but only at statistical patterns. It had learned to associate the flu with similar search trends that indicated winter.

This is not unusual for computer-taught algorithms: One algorithm developed to recognize skin cancer from pictures was fed pictures of both healthy and cancerous skin. It eventually decided that all those pictures with a ruler in it were pictures of skin cancer. Correlation is not causality, so be careful with computer-generated statistics!

Rule 8: Don’t take statistical bedrock for granted

As Rule 6 has already shown, data is not always neutral. This is exacerbated by the fact that countries all over the world produce their official statistics in different ways. In some countries, there is a lot of pressure from governments on national statistical institutes to report the “correct” data.

Take Argentina as an example: The populists presidents Néstor Kirchner and Cristina Fernandez Kirchner decided to tackle the problem of inflation by tweaking the inflation statistics. Official statisticians were instructed to round down all decimals in monthly inflation figures. This can make a big difference when looking at yearly numbers. When the monthly inflation rate is at 1% instead of 1.9%, you end up with a yearly inflation rate of 12.7% instead of 25.3%.

While the statistical bedrock is essential for understanding a country’s situation, you should always make sure you know how official statisticians are collecting and using their numbers.

Rule 9: Remember that misinformation can be beautiful, too

Charts and infographics are popular ways in which to present statistical insights. They usually reach a wider audience and leave a deeper impact. But when looking at infographics, we often overlook one important principle: just because it looks pretty does not mean that the underlying data is correct. Harford says, “Much depends on the intent of the chart’s creator, and the wisdom of the reader.”

There is also a lot that can go wrong when you turn numbers into pictures. For example, simply using a bold type for some numbers and not for others can focus your attention on selective data, and even make you oblivious to the fact that the numbers might not even be correct to start with.

Take “Debtris” as an example: this was an animation designed in the style of Tetris, showing falling blocks inscribed with the nation’s debts, such as “‘$3000bn: estimated total cost of Iraq war” or Walmart’s revenue and the UN budget. Even though the graphic is entertaining to look at, it obscures the fact that the numbers being compared are like comparing apples and oranges: this particular animation actually showed both stocks and flows, which is like comparing the annual rent price of a house with the total cost of buying it. 

Rule 10: Keep an open mind

Our brains are designed to make sense of the world around us, and that is why it often automatically fills in gaps in our perception or knowledge. That is why you can still hold a phone conversation on an unsteady line. But we are also keen to avoid uncomfortable truths. So when faced with statistics that do not conform with your worldview, keep an open mind, and assess them using the previous nine rules. 

The golden rule of statistical thinking is “be curious.” Always dig deeper and ask questions when faced with statistical information.

Final Notes

Statistics can help you make sense of the world around you, but only when you know how to read them correctly. First of all, be aware of your emotional response and worldview – you might not like what you see, but you should still keep an open mind and deal with the facts in front of you.  

Always be aware of the context, and the ways in which the data was gathered and measured, in order to reach a conclusive picture of what it is that the statistic is really telling you.

12min Tip

Next time you are faced with statistical findings, use the 10 rules to determine their factual value.