Read the original knowledge article in Spanish on Nada es Gratis Pedro Rey BielAssociate Professor, Department of Economics, Finance & Accounting A growing number of American companies are embracing experimentation in their decision-making processes. This evidence-based approach is the key to understanding the effectiveness of business decisions in terms of cause and effect. In recent months, and especially since the Nobel Prize in Economics was awarded to Richard Thaler, I have seen how this drive to use experimentation in decision-making is gaining momentum all over the world, including in Spain. Companies, public institutions, NGOs and financial institutions have started to create internal behavioral economics units. Those of us who work in this field have been surprised to receive frequent calls from organizations interested in adopting an experimental approach in their day-to-day business activities. Experimentation in decision-making is gaining momentum all over the world This interest in evidence-based decisions is also flourishing in the field of education. New courses on "experimentation in companies" have been cropping up in all sorts of master's degrees, which usually require a final project involving an experimental application. It used to be that we researchers had to proactively seek out "let's-do-an-experiment" collaborations. Therefore, we are pleased to be asked to run experiments with real application outcomes that benefit organizations' production processes. Now is a good time to reflect on both the potential and the difficulties of using the experimental method to make evidence-based decisions rather than relying on intuition or copying what has worked for others. What, exactly, is an experiment? It's is a small-scale test in which the result of a single measure applied to a randomly selected sample is compared with the result from a control group, which has also been selected randomly. In order to prove that the variable in question is actually the cause of the observed effect, it must be truly exogenous. The treatment group and the control group must be perfectly comparable - hence the randomization. In other words, one would expect a priori not to observe any differences between the two groups. The measured variable is only aspect that distinguishes one group from the other, so it must be the cause of the result. [Photo: ESADE Decision Lab] Our experiments are very similar to clinical trials, in which some patients are given a medication while others are given a placebo without their knowledge. The difference is that our experiments involve people, not pills. Here's a practical example from the business world. Imagine that your intuition tells you that a certain incentive program can increase your employees' productivity. Instead of applying the program to all employees at once - risking possible losses if your intuition is wrong - you can test its potential on a small group of employees. This group should have the same characteristics as the staff as a whole. Ideally, they should also be isolated from members of the "control group," in order to prevent interactions that could contaminate the results. If you observe that this group's productivity increases compared to other employees, you can conclude that the results are scalable and that the incentive program should be rolled out to other parts of the company. Similar experiments can be performed in many areas: price changes, special deals, communication policies, donations to charitable campaigns, and even organizational hierarchies. For this approach to succeed, both the experimentalist and the organization must understand the market very well and be willing to run tests based on informed intuition. Why are companies hesitant to adopt experimentation? Their reluctance may be due to a combination of factors. They may be wary of applying a new method. They may be concerned about the short-term loss of benefits during the experiment (even though the cost of an experiment is nothing compared to the losses associated with not taking optimal measures). Or it may be difficult for managers to admit that they don't know the answer to a problem and need to experiment in order to find it. Two of the greatest experts in the experimentation field, John List (co-author of The Why Axis) and Steve Levitt (co-author of Freakonomics), have written articles that give practical advice on how to perform useful experiments. These articles, Why economists should conduct field experiments and Field experiments in economics, draw on the authors' academic knowledge to explain how experimentation can be applied to real-life decisions. Business executives want to learn what works and what doesn't in order to make quick decisions These insights show that the objectives of academia and the corporate world are compatible, if not necessarily identical. Academics want to understand exactly why something happens, while business executives want to learn what works and what doesn't in order to make quick decisions. Whenever academics and business executives collaborate on an experiment, it is crucial to establish a solid agreement among the parties involved. In most cases, a confidentiality agreement should be reached to establish what results are publishable and under what conditions, with the aim of ensuring that competitors cannot benefit from the knowledge generated. It is also important to understand that the parties' incentives are multiple: behavioral economists are primarily interested in publishing their findings, but the inability to publish can be offset by access to a novel data source or economic compensation. By way of example, this article explains how a colleague and I studied ways of encouraging customers of a supermarket chain to answer questionnaires about consumer habits. I consider this to be one of my most interesting academic works, because it enabled us to understand a market and a type of incentive that were poorly understood at the time. I encourage you to perform an exercise of modesty and honesty: consider making decisions based on evidence from experimental tests rather than what your intuition tells you will work. Even if you are an expert in your field and your intuition is correct, there are still various degrees of freedom - method, sample size, expected effect quantification - that make experimentation interesting. Remember: experimentation does not involve changing many things at once. Making multiple changes only serves to obscure the real cause. With patience, you can learn much more by applying the scientific method to truly understand the reasons behind a phenomenon. About the author Pedro Rey Biel is an Associate Professor of Behavioral Economics at ESADE Business School. Before joining ESADE, he was a tenured faculty member at the Autonomous University of Barcelona. Since 2008, he has also served as a Visiting Professor at the Rady School of Management (University of California San Diego, UCSD), where he teaches classes on Managerial Economics and Strategy for the MBA program. Pedro is also the lead behavioral scientist for the "All of Us" Research Program, a key component of the United States' Precision Medicine Initiative launched by President Obama. Pedro's research combines theoretical and empirical methodologies, as well as laboratory and field experiments to understand economic decision-making and how it is affected by the design of proper incentive schemes which take into account the complex array of behavioral traits that motivate us. He has published numerous articles in leading academic economic journals such as Journal of European Economic Association, Journal of Economic Perspectives, Economic Journal and Journal of Public Economics. His work has also appeared regularly in specialized newspapers, including The Wall Street Journal and The Financial Times.

ESADE

Back to home

Business decisions: the benefits of using experimental evidence

10/2019

Read the original knowledge article in Spanish on Nada es Gratis




Pedro Rey Biel

Associate Professor, Department of Economics, Finance & Accounting





A growing number of American companies are embracing experimentation in their decision-making processes. This evidence-based approach is the key to understanding the effectiveness of business decisions in terms of cause and effect.


In recent months, and especially since the Nobel Prize in Economics was awarded to Richard Thaler, I have seen how this drive to use experimentation in decision-making is gaining momentum all over the world, including in Spain.


Companies, public institutions, NGOs and financial institutions have started to create internal behavioral economics units. Those of us who work in this field have been surprised to receive frequent calls from organizations interested in adopting an experimental approach in their day-to-day business activities.


Experimentation in decision-making is gaining momentum all over the world


This interest in evidence-based decisions is also flourishing in the field of education. New courses on "experimentation in companies" have been cropping up in all sorts of master's degrees, which usually require a final project involving an experimental application.


It used to be that we researchers had to proactively seek out "let's-do-an-experiment" collaborations. Therefore, we are pleased to be asked to run experiments with real application outcomes that benefit organizations' production processes.


Now is a good time to reflect on both the potential and the difficulties of using the experimental method to make evidence-based decisions rather than relying on intuition or copying what has worked for others.


What, exactly, is an experiment? It's is a small-scale test in which the result of a single measure applied to a randomly selected sample is compared with the result from a control group, which has also been selected randomly. In order to prove that the variable in question is actually the cause of the observed effect, it must be truly exogenous. The treatment group and the control group must be perfectly comparable - hence the randomization. In other words, one would expect a priori not to observe any differences between the two groups. The measured variable is only aspect that distinguishes one group from the other, so it must be the cause of the result.



[Photo: ESADE Decision Lab]


Our experiments are very similar to clinical trials, in which some patients are given a medication while others are given a placebo without their knowledge. The difference is that our experiments involve people, not pills.


Here's a practical example from the business world. Imagine that your intuition tells you that a certain incentive program can increase your employees' productivity. Instead of applying the program to all employees at once - risking possible losses if your intuition is wrong - you can test its potential on a small group of employees. This group should have the same characteristics as the staff as a whole. Ideally, they should also be isolated from members of the "control group," in order to prevent interactions that could contaminate the results.


If you observe that this group's productivity increases compared to other employees, you can conclude that the results are scalable and that the incentive program should be rolled out to other parts of the company.



Similar experiments can be performed in many areas: price changes, special deals, communication policies, donations to charitable campaigns, and even organizational hierarchies. For this approach to succeed, both the experimentalist and the organization must understand the market very well and be willing to run tests based on informed intuition.


Why are companies hesitant to adopt experimentation? Their reluctance may be due to a combination of factors. They may be wary of applying a new method. They may be concerned about the short-term loss of benefits during the experiment (even though the cost of an experiment is nothing compared to the losses associated with not taking optimal measures). Or it may be difficult for managers to admit that they don't know the answer to a problem and need to experiment in order to find it.


Two of the greatest experts in the experimentation field, John List (co-author of The Why Axis) and Steve Levitt (co-author of Freakonomics), have written articles that give practical advice on how to perform useful experiments. These articles, Why economists should conduct field experiments and Field experiments in economics, draw on the authors' academic knowledge to explain how experimentation can be applied to real-life decisions.


Business executives want to learn what works and what doesn't in order to make quick decisions


These insights show that the objectives of academia and the corporate world are compatible, if not necessarily identical. Academics want to understand exactly why something happens, while business executives want to learn what works and what doesn't in order to make quick decisions.


Whenever academics and business executives collaborate on an experiment, it is crucial to establish a solid agreement among the parties involved. In most cases, a confidentiality agreement should be reached to establish what results are publishable and under what conditions, with the aim of ensuring that competitors cannot benefit from the knowledge generated. It is also important to understand that the parties' incentives are multiple: behavioral economists are primarily interested in publishing their findings, but the inability to publish can be offset by access to a novel data source or economic compensation.


By way of example, this article explains how a colleague and I studied ways of encouraging customers of a supermarket chain to answer questionnaires about consumer habits. I consider this to be one of my most interesting academic works, because it enabled us to understand a market and a type of incentive that were poorly understood at the time.


I encourage you to perform an exercise of modesty and honesty: consider making decisions based on evidence from experimental tests rather than what your intuition tells you will work.


Even if you are an expert in your field and your intuition is correct, there are still various degrees of freedom - method, sample size, expected effect quantification - that make experimentation interesting.


Remember: experimentation does not involve changing many things at once. Making multiple changes only serves to obscure the real cause. With patience, you can learn much more by applying the scientific method to truly understand the reasons behind a phenomenon.


About the author


Pedro Rey Biel is an Associate Professor of Behavioral Economics at ESADE Business School. Before joining ESADE, he was a tenured faculty member at the Autonomous University of Barcelona. Since 2008, he has also served as a Visiting Professor at the Rady School of Management (University of California San Diego, UCSD), where he teaches classes on Managerial Economics and Strategy for the MBA program. Pedro is also the lead behavioral scientist for the "All of Us" Research Program, a key component of the United States' Precision Medicine Initiative launched by President Obama.


Pedro's research combines theoretical and empirical methodologies, as well as laboratory and field experiments to understand economic decision-making and how it is affected by the design of proper incentive schemes which take into account the complex array of behavioral traits that motivate us.


He has published numerous articles in leading academic economic journals such as Journal of European Economic Association, Journal of Economic Perspectives, Economic Journal and Journal of Public Economics. His work has also appeared regularly in specialized newspapers, including The Wall Street Journal and The Financial Times.

More Knowledge
Stereotypes are only a threat when beliefs are reinforced. On the sentitivity of gender differences in performance under competition to information provision
Iriberri , Nagore; Rey Biel, Pedro
Journal of Economic Behavior and Organization
Vol. 135, n -, 03/2017, p. 99 - 111
The cake looks yummy on the shelf up there: The interactive effect of retail shelf position and consumers' personal sense of power on indulgent choice
Wongkitrungrueng , Apiradee; Valenzuela Martnez, Ana; Sen , Sankar
Journal of Retailing
Vol. 94, n 3, 09/2018, p. 280 - 295
Back to home