Key aspects of algorithmic decision-making for the social good By Nuria OliverDirector of Research on Data Science at Vodafone and Chief Data Scientist at DataPop Alliance This article was originally published as part of Adoption and Impact of Big Data and Advanced Analytics in Spain, a study by the ESADE Institute for Data-Driven Decisions and the ESADE MBA The availability of huge amounts of data on human behavior is transforming the world. This data can be used to train algorithms, allowing researchers, companies, governments and other public-sector stakeholders to tackle complex problems. Decisions with an individual and collective impact that were once made by humans - including decisions regarding hiring, the granting of credits and loans, court verdicts and stock trades - are now made by algorithms. Algorithmic data-based decisions have the potential to improve the efficiency of governments and services by optimizing bureaucratic processes, providing real-time feedback and predicting results. History has shown that human decisions are subject to conflicts of interest, corruption and bias, resulting in unfair and/or inefficient processes and outcomes. Current interest in the use of data-based algorithms is a consequence of demands for greater objectivity in decision-making and deeper knowledge of our individual and collective behavior and needs. Human decisions are subject to conflicts of interest, corruption and bias Nevertheless, decision-making based on data-trained algorithms also has certain limitations. To ensure that this new form of decision-making has a positive impact, the following seven aspects must be taken into account: 1. Discrimination Algorithmic discrimination can come from various sources. First, the data used to train algorithms may have biases that lead to discriminatory decisions. Second, discrimination may arise from the use of a particular algorithm. Categorization, for example, can be considered a form of direct discrimination because it uses algorithms to determine different treatments of various classes. Third, algorithms can result in discrimination as a result of misuse of certain models in different contexts. Fourth, biased data can be used both as evidence for the training of algorithms and as evidence of their effectiveness. In addition, the use of algorithmic data-based decision processes may lead to people being denied opportunities not because of their own actions but because of the actions of others with whom they share certain characteristics. For example, some credit card companies have reduced credit limits not because of the customer's payment history but as a result of analyzing the behavior of other customers with poor payment histories who shopped at similar stores. Various solutions for preventing algorithmic discrimination and maximizing justice have been proposed in the literature. However, I would like to emphasize the urgent need for experts from various fields (including law, ethics, computer science, philosophy and political science) to invent, evaluate and validate various algorithmic justice metrics for real-world situations. In addition to this empirical research, a theoretical modeling framework - supported by empirical evidence - is needed to ensure that algorithmic decisions are made as fairly as possible. 2. Lack of transparency Transparency refers to the capacity to understand a computational model and therefore contribute to the attribution of responsibility for consequences derived from its use. A model is transparent if a person can easily observe it and understand it. It would therefore be desirable for models to have low computational complexity. Burrell describes three types of opacity (i.e. lack of transparency) in algorithmic decisions: - Intentional opacity. The objective of this type of opacity is to protect the algorithm inventors' intellectual property. This type of opacity could be mitigated through legislation that would require the use of open software systems. The European Union's new General Data Protection Regulation (GDPR), which protects the right to an explanation, is an example of this type of legislation. However, powerful commercial and governmental interests can make it difficult to eliminate this type of opacity. - Knowledge opacity. This type of opacity is due to the fact that the most people lack the technical skills to understand how algorithms and computational models are constructed. It could be reduced through educational programs that promote computational thinking and also by allowing people affected by algorithmic decision-making processes to seek the advice of independent experts. - Intrinsic opacity. This type of opacity arises from the nature of certain computer learning methods (e.g. deep learning models). It is well known in the computational learning research community and is also known as the interpretability problem. 3. Violation of privacy Reports and studies have focused on the misuse of users' personal data and on data aggregation by entities such as data brokers, which have direct implications for people's privacy. Often overlooked is the fact that advances in algorithms combined with the availability of new sources of data on human behavior (e.g. social media) make it possible to infer private information (e.g. sexual orientation, political inclinations, level of education, emotional stability) that has never been explicitly revealed. This element is the key to understanding the implications of algorithm use, as became evident in the recent Facebook-Cambridge Analytica scandal. 4. Digital literacy It is extremely important that we devote resources to digital and computer literacy programs for all citizens, from children to the elderly. If we do not, it will be very difficult (if not impossible) as a society to make decisions about technologies that we do not understand. The book Los nativos digitales no existen ("Digital natives do not exist") emphasizes the need to teach children and adolescents about computer thinking and proper use of technology. 5. Fuzzy responsibility As more and more decisions that affect millions of people are made automatically by algorithms, we must be clear about who is responsible for the consequences of these decisions. Transparency is often considered a fundamental factor in the clarity of attribution of responsibility. However, transparency and audits are not enough to guarantee clear responsibility. In a recent article, Kroll et al. proposed the use of computational methods to provide clarity regarding the attribution of responsibility, even when part of the information is hidden. 6. Lack of ethical frameworks Algorithmic data-based decision-making processes generate important ethical dilemmas regarding what actions are appropriate in light of the inferences made by algorithms. It is therefore essential that decisions be made in accordance with a clearly defined and accepted ethical framework. Several sets of ethical principles have been proposed in the literature for this purpose, for example by the Digital Ethics Lab at the University of Oxford and the AI Now Institute at New York University. However, this is still an active area of research and there is no single method for introducing ethical principles into algorithmic decision processes. It is crucial that all developers and professionals working in the development and use of decision-making algorithms behave in accordance with a clear code of conduct and ethics defined by their organization. 7. Lack of diversity Given the variety of cases in which algorithms can be applied for decision-making, it is important to reflect on the frequent lack of diversity in the teams that generate such algorithms. So far, data-based algorithms and artificial intelligence techniques for decision-making have been developed by homogeneous groups of IT professionals. In the future, we should make sure that teams are diverse in terms of areas of knowledge as well as demographic factors (particularly gender, given that women account for less than 20% of IT professionals at many technology companies). In conclusion, I would like to highlight three people-centered requirements that I consider vitally important to the positive disruption of algorithmic data-based decisions: (1) ownership and management of person-centered data; (2) transparency, accountability, respect for privacy, ethics, justice and diversity in algorithms; and (3) living labs for experimentation with data-centric policies. This third requirement entails developing laboratories for experimentation with decisions made through the management of information - that is, experimenting with and co-creating data-based policies and solutions that are in turn agreed with humans. At this unprecedented moment in the history of humanity, a large amount of data on human behavior is at our fingertips. Decision-making based on big data and intelligent algorithms offers great opportunities, as long as we take into account the risks and limitations that this entails. Only when we meet these three requirements will we be able to move from a possible tyranny of algorithms to a model of democratic governance based on data by and for the people. The potential to have a positive impact is immense. We must not waste this opportunity. To read more articles of the ESADE Institute for Data-Driven Decisions' world-class professors, researchers and collaborators visit its Blog.

ESADE

<< Back to home

7 challenges and opportunities in data-based decision-making

07/2018

Key aspects of algorithmic decision-making for the social good


By Nuria Oliver
Director of Research on Data Science at Vodafone and Chief Data Scientist at DataPop Alliance



This article was originally published as part of Adoption and Impact of Big Data and Advanced Analytics in Spain, a study by the ESADE Institute for Data-Driven Decisions and the ESADE MBA



The availability of huge amounts of data on human behavior is transforming the world. This data can be used to train algorithms, allowing researchers, companies, governments and other public-sector stakeholders to tackle complex problems.


Decisions with an individual and collective impact that were once made by humans - including decisions regarding hiring, the granting of credits and loans, court verdicts and stock trades - are now made by algorithms.


Algorithmic data-based decisions have the potential to improve the efficiency of governments and services by optimizing bureaucratic processes, providing real-time feedback and predicting results. History has shown that human decisions are subject to conflicts of interest, corruption and bias, resulting in unfair and/or inefficient processes and outcomes. Current interest in the use of data-based algorithms is a consequence of demands for greater objectivity in decision-making and deeper knowledge of our individual and collective behavior and needs.


Human decisions are subject to conflicts of interest, corruption and bias


Nevertheless, decision-making based on data-trained algorithms also has certain limitations. To ensure that this new form of decision-making has a positive impact, the following seven aspects must be taken into account:


1. Discrimination


Algorithmic discrimination can come from various sources. First, the data used to train algorithms may have biases that lead to discriminatory decisions. Second, discrimination may arise from the use of a particular algorithm. Categorization, for example, can be considered a form of direct discrimination because it uses algorithms to determine different treatments of various classes. Third, algorithms can result in discrimination as a result of misuse of certain models in different contexts. Fourth, biased data can be used both as evidence for the training of algorithms and as evidence of their effectiveness.


In addition, the use of algorithmic data-based decision processes may lead to people being denied opportunities not because of their own actions but because of the actions of others with whom they share certain characteristics. For example, some credit card companies have reduced credit limits not because of the customer's payment history but as a result of analyzing the behavior of other customers with poor payment histories who shopped at similar stores.



Various solutions for preventing algorithmic discrimination and maximizing justice have been proposed in the literature. However, I would like to emphasize the urgent need for experts from various fields (including law, ethics, computer science, philosophy and political science) to invent, evaluate and validate various algorithmic justice metrics for real-world situations. In addition to this empirical research, a theoretical modeling framework - supported by empirical evidence - is needed to ensure that algorithmic decisions are made as fairly as possible.



2. Lack of transparency


Transparency refers to the capacity to understand a computational model and therefore contribute to the attribution of responsibility for consequences derived from its use. A model is transparent if a person can easily observe it and understand it. It would therefore be desirable for models to have low computational complexity.


Burrell describes three types of opacity (i.e. lack of transparency) in algorithmic decisions:


- Intentional opacity. The objective of this type of opacity is to protect the algorithm inventors' intellectual property. This type of opacity could be mitigated through legislation that would require the use of open software systems. The European Union's new General Data Protection Regulation (GDPR), which protects the right to an explanation, is an example of this type of legislation. However, powerful commercial and governmental interests can make it difficult to eliminate this type of opacity.



- Knowledge opacity. This type of opacity is due to the fact that the most people lack the technical skills to understand how algorithms and computational models are constructed. It could be reduced through educational programs that promote computational thinking and also by allowing people affected by algorithmic decision-making processes to seek the advice of independent experts.


- Intrinsic opacity. This type of opacity arises from the nature of certain computer learning methods (e.g. deep learning models). It is well known in the computational learning research community and is also known as the interpretability problem.



3. Violation of privacy


Reports and studies have focused on the misuse of users' personal data and on data aggregation by entities such as data brokers, which have direct implications for people's privacy.


Often overlooked is the fact that advances in algorithms combined with the availability of new sources of data on human behavior (e.g. social media) make it possible to infer private information (e.g. sexual orientation, political inclinations, level of education, emotional stability) that has never been explicitly revealed. This element is the key to understanding the implications of algorithm use, as became evident in the recent Facebook-Cambridge Analytica scandal.



4. Digital literacy


It is extremely important that we devote resources to digital and computer literacy programs for all citizens, from children to the elderly. If we do not, it will be very difficult (if not impossible) as a society to make decisions about technologies that we do not understand. The book Los nativos digitales no existen ("Digital natives do not exist") emphasizes the need to teach children and adolescents about computer thinking and proper use of technology.



5. Fuzzy responsibility


As more and more decisions that affect millions of people are made automatically by algorithms, we must be clear about who is responsible for the consequences of these decisions. Transparency is often considered a fundamental factor in the clarity of attribution of responsibility. However, transparency and audits are not enough to guarantee clear responsibility. In a recent article, Kroll et al. proposed the use of computational methods to provide clarity regarding the attribution of responsibility, even when part of the information is hidden.



6. Lack of ethical frameworks


Algorithmic data-based decision-making processes generate important ethical dilemmas regarding what actions are appropriate in light of the inferences made by algorithms. It is therefore essential that decisions be made in accordance with a clearly defined and accepted ethical framework.


Several sets of ethical principles have been proposed in the literature for this purpose, for example by the Digital Ethics Lab at the University of Oxford and the AI Now Institute at New York University. However, this is still an active area of research and there is no single method for introducing ethical principles into algorithmic decision processes. It is crucial that all developers and professionals working in the development and use of decision-making algorithms behave in accordance with a clear code of conduct and ethics defined by their organization.



7. Lack of diversity


Given the variety of cases in which algorithms can be applied for decision-making, it is important to reflect on the frequent lack of diversity in the teams that generate such algorithms. So far, data-based algorithms and artificial intelligence techniques for decision-making have been developed by homogeneous groups of IT professionals. In the future, we should make sure that teams are diverse in terms of areas of knowledge as well as demographic factors (particularly gender, given that women account for less than 20% of IT professionals at many technology companies).



In conclusion, I would like to highlight three people-centered requirements that I consider vitally important to the positive disruption of algorithmic data-based decisions: (1) ownership and management of person-centered data; (2) transparency, accountability, respect for privacy, ethics, justice and diversity in algorithms; and (3) living labs for experimentation with data-centric policies. This third requirement entails developing laboratories for experimentation with decisions made through the management of information - that is, experimenting with and co-creating data-based policies and solutions that are in turn agreed with humans.


At this unprecedented moment in the history of humanity, a large amount of data on human behavior is at our fingertips. Decision-making based on big data and intelligent algorithms offers great opportunities, as long as we take into account the risks and limitations that this entails.


Only when we meet these three requirements will we be able to move from a possible tyranny of algorithms to a model of democratic governance based on data by and for the people. The potential to have a positive impact is immense. We must not waste this opportunity.


To read more articles of the ESADE Institute for Data-Driven Decisions' world-class professors, researchers and collaborators visit its Blog.

More Knowledge
Beyond posted prices: The past, present, and future of participative pricing mechanisms
Spann , Martin; Zeithammer , Robert; Bertini, Marco; Haruvy , Ernan; Jap , Sandy; Koenigsberg , Oded; Mak , Vincent; Popkowski Leszczyc , Peter; Skiera , Bernd; Thomas , Manoj
Customer Needs and Solutions
Vol. 5, n 1-2, 03/2018, p. 121 - 136
Adopcin e impacto del big data y advanced analytics en Espaa
ESADE. Institute for Data-Driven Decisions, 05/2018
60 p.
Consensus, dissension and precision in group decision making by means of an algebraic extension of hesitant fuzzy linguistic term sets
Montserrat Adell, Jordi; Agell Jan, Nria; Snchez Soler, Mnica; Ruiz Vegas, Francisco Javier
Information Fusion
N 42, 07/2018, p. 1 - 11
<< Back to home