INTUITION vs. DATA

Our company – Things Solver, helps clients make better decisions by analyzing relevant data. During analysis of the data, we often notice significant differences between what our clients would intuitively decide and what their data tells them to do.

One striking example of a gap between our perception and reality is the state of the world around us. Hans Rosling, author of “Factfulness”, demonstrates in a playful, memorable manner, how only through better understanding of data can we have a better understanding of the world that we inhabit. Sounds simple? You think you already have a firm grasp of the world around us? We challenge you to take his simple test.

Chances are you did poorer than a chimpanzee. You are not alone. Furious with ignorance he encountered at such esteemed places as Harvard University and World Economic Forum, Hans set off on a journey to help the world get to know itself. First step in this journey was his data visualization project Gapminder. Second step is his masterpiece: Factfulness.

Factfulness is a book about the world around us and about how we can use widely available data and heuristics to aid our understanding of it. For a taste of the book, feel free to take a stroll down Dollarstreet or watch one of his TED talks.

Factfulness centers around ten instincts that make us perceive the world around us dramatically differently than if we were to zoom in on widely available data. Being mindful of our hard-wired biases lets us calibrate intuition with critical thinking, with an end goal of making better decisions. I’ll mention all the biases, and expand on my favourites in more details:

1. Gap instinct Humans have a basic urge to divide things into two distinct groups, with nothing but an empty gap in between. We often hear about developing and developed countries. This intuitively makes sense. But the data tells us a different story. Developing and developed countries were a roughly correct approximation of reality in the 1960s. The world has completely changed since. Newspaper headlines, often forming our view of the world more than they deserve.

People love to dichotomize. Dividing into two distinct sides is simple and intuitive, with some drama on the side. We dichotomize without thinking and do it all the time. In most cases there is no clear separation of the two groups. Even if averages are showing two distinct groups, the underlying reality is often vastly different. By digging deeper into the data we understand spreads and distributions. We often realize that what we perceived as distinct groups have very much in common.

2. The negativity instinct – Humans are hard-wired to perceive negative stimulus as much more important than positive stimulus. This is the reason why your brain reacts more intensely to a headline about terrorist attacks and ignores hundreds of thousands of lives saved daily by minor improvements in medicine.

3. The straight instinct – We often assume that a trend will continue to go along a straight line without knowing what are the underlying drivers of the trend. Many trends do not follow straight lines. They are S-bends, slides, humps, or doubling lines. “No child ever kept up the rate of growth it achieved in its first six months, and no parents would expect it to.”

4. The fear instinct – Our survival mechanism makes us overestimate risks around us.

5. The size instinct – A favourite of mine and the most useful in practice. A lonely number often seems impressive, but is almost always irrelevant. Ask: compared to what? We usually see absolute amounts thrown around because they are easier to find. Rates of change are often more meaningful.

6. The generalization instinct – Humans unconsciously categorize and generalize all the time. Categories are necessary for us to function. They give structure to our thought. Although very useful tool to explain the world around us, categories could be dangerous and misleading if not used correctly.

Wrong generalizations are mind-blockers for all kinds of understanding. If someone offers you a generalization, a single example and wants you to draw conclusions about a more general topic, you should ask for more examples. Are you happy to conclude that all chemicals are unsafe on the basis of one unsafe chemical? Would you be prepared to conclude that all chemicals are safe on the basis of one safe chemical? Hans gives us few tricks to control these instincts: look for differences within groups, look for similarities across groups, look for differences across groups, beware of “the majority” (could mean 51%, but also 99%), beware of vivid examples and assume people are not idiots.

7. The destiny instinct – relates to the idea that innate characteristics determine destiny of everything around us. This is because truly relevant change happens too slowly to notice. Control this instinct by constantly updating your data-based knowledge. You can also talk to your grandpa and remind yourself how small change could be a huge one over decades.

8. The single perspective instinct – is the idea that all problems have single cause and all problems have a single solution. This kind of thinking saves us a lot of time. We like to feel knowledgeable. We like to feel useful. We like to form instant opinions about any topic raised. We have no satisfaction from having an opinion only about a few topics that we know we are right about. Control this instinct by constantly testing your ideas for blind spots. Be curious about new information that doesn’t fit your view. Seek mental models from other fields. Rather than talking to people who agree with you, seek people that contradict you.

Through his first-hand experience with experts and consultants from a wide range of professions, Hans tells us that, unless they are operating within a very narrow circle of their competence, they are unlikely to be helpful. Experts from one domain rarely know their limits and often, with a remarkable degree of certainty, extrapolate their knowledge of one area into another. This is especially important when using data to make decisions. Expert knowledge, complemented and better informed with data, is the key to making right decisions.

9. The blame instinct – is giving clear, simple reason for why something has happened. It seems it comes naturally for us to decide that when things go wrong it must be because a bad individual, or a group wanted them to. We need to believe that individuals have power. Otherwise, the world feels unpredictable, confusing and frightening. It steals our focus and blocks our learning. Control system would be looking for different causes, not for individuals, but the same goes when an individual claims to have caused something good – you should ask yourself whether the outcome might have happened anyway.

10. The urgency instinct – “Now or never! Learn Factfulness now! Tomorrow may be too late!” This is what sales people and activist love and how they exploit us. Things are almost never that urgent. Yet, we have to admit that this instinct has served us well in the past. Those who stopped in front of the lion to carefully analyze the probabilities are not our ancestors. If something is urgent and important it should be measured. Beware of data that is relevant but inaccurate, or accurate but irrelevant. Only relevant and accurate data is useful and therefore it is crucial to protect its integrity and the credibility of those who produce it.  

 

Professor Rosling used to finish his lectures by swallowing a sword. It was a message for non-believers. They could see with their own eyes how their intuitive assumptions (70 year old man cannot swallow a sword!) crumble.

I would have been thrilled if only I had a chance to attend one of his classes. Please pass his wisdom along.

 

Big Data and Banking: How Our Data Protects Us

Some of the main tasks for the bankers are, among others, keeping, preserving, deriving more from less, taking the best out of what the clients deposit to them, with great confidence and trust. Almost the same definition could be applied to the Data Scientists dealing with the big data. They also deal with keeping, preserving, deriving more conclusions from a little information, getting the best out of the existing databases which they handle with great care and confidentiality. It is maybe because of this similarity in definitions, or maybe because of the fact that the databases in the banks are an ideal example on which the detailed analysis could be performed and the conclusions can be used to benefit both clients and banks that the financial industry is among the top users of Data Scientists services, according to the credible surveys.
One of the leading banks in the Serbian market, Banca Intesa has a client base of around 1.5 million. For Banca Intesa COO Aleksandar Stojadinovic, it is obvious that those databases should be treated as an important asset. When used properly, it could bring notable benefits for both clients and the companies.
As a part of the worldwide group whose headquarters is in Italy, Stojadinovic says that the use of big data in Serbia has never been closer to the world trends. “In the world of globalization, we have at our disposal all tools that are available to the most powerful companies in the world. The only thing that limits us is the lack of talents and specialists, as well as the size of the market and average living standards of the population.”
Banca Intesa decided to make a combination of their strong internal team and the experts of Things Solver in order to start a joint search for the solutions that can be applied right away.

Short Sprints Lead to Results

Stojadinovic describes the cooperation of data scientists and banking experts as an explosion of energy and ideas. “On one hand, we have Data Science experts. On the other, we have very experienced experts in banking, and they thoroughly know every issue we are trying to solve.” The last issue the team tackled was how to improve the user experience in the social networks. Findings on clients, their desires and needs in the communication with the bank, services they find important and the ways how they found them – it is all already in the big databases. The answers just need to be properly derived.

Intesa’s COO describes the process of looking for the answers with sports vocabulary. “We start from the long-term vision and we try to follow this vision through short sprints that last 2 to 4 weeks, providing solutions for parts of the problem or improving the existing solutions.”

Data as Our Shield

If we decide to stay within the sports vocabulary, it is important for the banks to remain loyal to the rules of fair-play, stay in the track of respecting rules, laws, and safety of the data. Intesa claims to be fully aware that the mutual trust is in the center of the profession. “There is an ethical, professional and legal limit – and that is good. On the other hand, it is clear that the limit is imposed only over the initiatives that can cause harm to the clients, while for everything else – the sky is the limit”, Stojadinovic concludes.

Because of the fact that there is no limit, the winner is the one that uses this ethically, professionally and legally limited space in a more creative and innovative way. “We use the data that we have to prevent frauds, to analyze and improve security, for risk assessment and analyzing the loan potential”, says Stojadinovic.
This is the way how the data that we willingly share, in the times when everybody is worried about how they are treated, is becoming our shield from frauds. This is how the data scientists and bankers get back to their original mission of preserving trust.