I remember a small discussion with a fellow panelist of markedly neoliberal ideology who, when I remarked that it would be very important for ethics to govern economic affairs, replied, somewhat startled, whether I meant to turn the economy into a sharia.
I found it striking that something I took practically for granted was not obvious to everyone. And I was even more surprised that someone would link ethics to religion or even to some form of radical fundamentalism. Perhaps I was taking too many things for granted, so let me explain myself better.
Ethics is an academic discipline that, a priori, has absolutely nothing to do with religion. It is not something magical, mystical, pseudoscientific or irrational. Quite the opposite. The confusion of my dear fellow panelist lay in not distinguishing ethics from morality. The latter refers to the set of principles, norms or precepts that govern the coexistence of a community. Thus, Christianity or Islam propose a broad corpus of norms, so it is indeed possible to speak of Christian or Muslim morality. But ethics is not that. Ethics is reflection on morality. So when I say that ethics should govern economic affairs, I am not saying at all that some kind of religious precepts should govern economic development. I am maintaining the need for rational reflection on the decisions that are made. And that need concerns not only the economy but each and every element that makes up society, being most pressing in those with the greatest consequences for all of us. In our time, the subject on which to reflect is, without a doubt, artificial intelligence, the technology that will have (and is having) the greatest impact on our lives. So let us waste no more time on clarifications.
Algorithms have become key elements on which the services and infrastructures of so-called information societies are based. They are used not only to recommend series on Netflix or videos on YouTube: schools, hospitals, companies, financial, police and judicial institutions, and even governments use them to make decisions. As a rule, machine learning (ML) algorithms analyze massive amounts of data in search of patterns. Their heuristics do not operate causally but only look for correlations, which has the serious problem of easily falling into illusory correlations, or apophenias. For example, if we have an algorithm analyzing crime in a North American city and its data indicate that most crimes are committed by Black people, the machine may infer that being Black is a possible cause of criminality. It can also make mistakes because of poor data quality: its insufficiency, its lack of pertinence or relevance to the problem at hand or, most commonly, data that is already biased. Even if the system is ultimately controlled by a human, what is known as automation bias can occur: granting greater credibility to the decision made by the algorithm than to that of a human, even when the human appears to be right. A much-publicized example was when the COMPAS software, used by many American courts to predict prisoners’ likelihood of reoffending, besides having a marked racist bias and giving many false positives, was given more weight by judges than the agreements reached by prosecution and defense lawyers.
But the matter gets even thornier. ML algorithms have a quality that makes it very hard to understand how they work: they store knowledge in a distributed way. If we lift the hood of one of these machines and look at how it works, all that appears before us is an immense tangle of synaptic weights, a gigantic numerical table that seems to say absolutely nothing. There is no precise, well-located place that tells us why and how the decision was made. This is called algorithmic opacity or the black box problem. To alleviate this obscurity, a new discipline has been created: explainable artificial intelligence (XAI). The attempt is to convert this unintelligible amalgam of data into formulas that humans can understand, such as decision trees. However, ML architectures tend to be very large and complex, and it seems to be a rule that the bigger and more accurate a system is, the less explainable it becomes. Moreover, most analyses are done by the designers themselves and study only local parts of the networks in order to improve their performance, without worrying about explaining or understanding anything. Unfortunately, trying to explain is expensive and laborious, so it is not a priority for developers who focus solely on technological and economic criteria. There are even authors such as David Dansk, Axel J. London and Marion Oswald who argue that focusing excessively on transparency could be detrimental to research and divert resources intended to improve safety, performance and accuracy. Be that as it may, the truth is that ML algorithms tend to be quite opaque.
This is very important because it can greatly affect their robustness. If we have an ML algorithm and we ignore how it works, we can only know whether it does well by testing it. But it could be the case that our algorithm works wonderfully with one particular type of data while failing spectacularly with others. If we tested it only with the first type, we might think we have a good product, safe and ready to put on the market. Technology is running ahead of our knowledge, and of our slow legislation, and that sounds quite dangerous. In the 2017 article “Robust Physical-World Attacks on Deep Learning Models”, researchers from several universities showed how so-called adversarial attacks could generate small changes in images of traffic signs capable of completely confusing the driving software of autonomous vehicles. A terrorist who wanted to cause an accident only had to place a certain sticker on a type of sign and wait for an autonomous car to get confused. The important thing here is to understand that artificial intelligence makes mistakes that are completely different from the ones we humans make, so, if we do not understand well how it works, we cannot know either how or when it will fail.
Well, algorithms are not perfect, they can fail, and we do not understand well the decisions they make, but what does this have to do with ethics? Everything. Because to the extent that we are letting them make decisions, those decisions raise ethical problems. For example, if an algorithm working for a bank decides not to grant you a loan, it seems very reasonable to want to know the reason. The algorithm should be able to justify its decision so we can verify that it was based on ethically acceptable criteria. If it has denied me the loan for reasons of sex, race, religion, etc., it would be not only immoral but in violation of the law.
And here two major questions arise. First, only a few years ago, if a program malfunctioned and caused some harm, the obvious person responsible was the programmer. However, if algorithms are able to learn and to make decisions based on their learning, those decisions would not be made directly by the programmer, so he could be absolved of blame but… would the party responsible be a machine? Can a set of lines of code be legally responsible for anything? A need arises for new legal figures or, at the very least, for measures that address the problem. And second, b. Moreover, deep networks can be made artificially more opaque without reducing their effectiveness. A company could blur the evolution of its system’s learning so that it would be more complicated to trace the decision and escape any responsibility.
Some have argued that we are too demanding of algorithms. We demand more of them than of their human counterparts when carrying out their task. In the United Kingdom a study was carried out in 2001 to test the impartiality of judges. A sample of forty-one judges was used, who were presented with the same cases and their answers were compared. In none of the batches of cases did all the judges agree unanimously. What is more, many of the cases were repeated (only the names of the defendants were changed) and, even so, the judges did not agree on the sentence when they analyzed the case a second time. They did not even agree with themselves! Another famous 2011 study, carried out at Columbia Business School by Professor Jonathan Levay, held that the probability of obtaining parole decreased over the course of the morning, until lunchtime, after which the probabilities rose again. A judge who has just had lunch is more lenient than one who is hungry! There is much more evidence of how factors such as the day of the week (Monday is not the same as Friday), the order of the sentences (after a very harsh one, the next tends to be reduced), etc., bias judges’ rulings. If readers want to know more, I recommend the work of Jeffrey J. Rachlinski, professor of law at Cornell University. So, if judges also have biases, why do we demand that algorithms have none? Besides, it seems much harder to change a judge’s biases than an algorithm’s. Why not use them, then, if in many cases they also improve on human work? Researchers at Cambridge, analyzing arrest data from New York City for the years 2008 and 2013, designed an algorithm capable of jailing 41.8% fewer people while maintaining the same crime rate. What a saving!
I’m sorry, but we cannot. The main reason remains opacity. If we have an algorithm that recommends songs on Spotify and it fails 10% of the time, there seems to be no problem in its being unable to explain why it got it wrong. However, when it comes to passing a homicide sentence, it is strictly necessary for the machine to give us every detail of how and why it made its decision. It meets no ethical standard to send someone to prison without anyone knowing why it was done. Therefore, until the transparency of ML improves, we cannot use algorithms to make decisions that could have judicial or police consequences (this goes for autonomous cars too), other than as consultative tools.
But let us not lose heart. As David S. Watson argues, there is still a lot of room to improve the transparency of algorithms, and companies such as Google and IBM have stepped up their efforts to make their architectures more explainable and inclusive. There are publicly available tools such as Explainable AI, AI Explainability 360 or What-If that give developers visual interfaces that greatly improve legibility, case-based reasoning, or directly interpretable rules. They are even able to identify and mitigate unwanted biases in both datasets and algorithms. The fact that the ethical debate has become highly visible shows, at the very least, a growing concern on the part of public opinion which, we hope, will tend to crystallize into legal regulations and corporate policies. A more ethical artificial intelligence is still possible, and we optimistically hope that it will be. It would certainly be wonderful if the great technological achievement of our time were used well, and to do good.
Originally published in Spanish on IANética (2021). English version prepared for TecnoHumanismo.
Related articles
The Only Thing We Were Missing: Toward a Psychopathic Technology?
Could a machine be cruel, perverse, evil? Yes, as Norman shows, the world's first artificial intelligence trained to think like a psychopath.
Read article →3 min
Ethics of Advanced Robotics and Humanoids
As advanced robotics and humanoids take hold, an essential debate on ethics emerges: how to integrate AI into robots while protecting human dignity.
Read article →6 min
The Right to the Truth
The right to the truth means full knowledge of the facts, who took part and in what circumstances, especially gross human rights violations.
Read article →6 min