Collective Reflective Equilibrium in Practice (CREP) and controversial novel technologies

Abstract In this paper, we investigate how data about public preferences may be used to inform policy around the use of controversial novel technologies, using public preferences about autonomous vehicles (AVs) as a case study. We first summarize the recent ‘Moral Machine’ study, which generated preference data from millions of people regarding how they think AVs should respond to emergency situations. We argue that while such preferences cannot be used to directly inform policy, they should not be disregarded. We defend an approach that we call ‘Collective Reflective Equilibrium in Practice’ (CREP). In CREP, data on public attitudes function as an input into a deliberative process that looks for coherence between attitudes, behaviours and competing ethical principles. We argue that in cases of reasonable moral disagreement, data on public attitudes should play a much greater role in shaping policies than in areas of ethical consensus. We apply CREP to some of the global preferences about AVs uncovered by the Moral Machines study. We intend this discussion both as a substantive contribution to the debate about the programming of ethical AVs, and as an illustration of how CREP works. We argue that CREP provides a principled way of using some public preferences as an input for policy, while justifiably disregarding others.

thus entering an age in which we will have access to unprecedented amounts of data on public attitudes and preferences. How should this information inform policy-making for novel technologies?
One area where this issue is imminently important is the program-if a pedestrian walks in the path of the car, the car may have to either swerve into traffic and risk the life of the driver, or continue and risk to such situations is stimulating research into machine ethics. about public preferences may be used to inform ethical policy by looking closely at the specific question of how public preferences ---tudes should have no bearing on policy. We will argue that neither 1 CREP, data on public attitudes can have an important role to play in shaping policy but only as potential input into a deliberative process that looks for coherence between attitudes, behaviours, and ethical principles. We argue that in cases of reasonable moral disagreement, data on public attitudes should play a much greater role in shaping Moral Machines study. We intend this discussion both as a substantive contribution to the debate about the programming of ethical technologies. We argue that CREP provides a principled way of using some public preferences as an input for forming concrete policy that is practically implementable, politically legitimate, and ethically defensible, while justifiably disregarding other public preferences. | Several major car manufacturers have announced plans to release 2 3 reduced congestion. 4 One such chalwhere death is imminent. This challenge is notable for turning an abstract philosophical debate that has been ongoing for over five decades into a concrete ethical conundrum. trolleys that they describe far-fetched scenarios that are claimed to bear little resemblance to any real-life ethical decision.
will ample, if a woman pushing a baby in a stroller walks into the path of straight, risking the life of mother and child, or to swerve, risking the life of a single pedestrian or the occupant/s of the vehicle, if there is confront trolley dilemmas-as they will not be literally pulling a lever to rectly, via their prior programming.
counter in contexts where split-second decisions must be made about with a sort of 'moral status detector', which allows them to differentially respond to a backpack, an empty car, a dog, and a human-as well as, perhaps, even to different classes of humans (old vs. young, preg--1 | SAVULESCU ET AL.
points. 8 They identified nine factors that typically led participants to spare one group over another. These were, in decreasing order of strength, a preference for: • Sparing human lives over animal lives hold that we are required to divert a trolley that will kill five to a sidetrack where it will kill only one, as this will achieve the greatest good.
Yet some deontologists argue that it is wrong to divert the trolley. 21 the switch, as that will give each of the six people an equal chance of moral decision-making, then it is hard to see why the intuitions of the general public should be excluded from playing any such role, unless they can be shown to be uniquely untrustworthy.
We are inclined to say something even stronger. With many others, we will assume that moral intuitions, including intuitions about can confer justification on moral views and, by extension, on policies that implement those views-if these intuitions meet certain conditions that we will discuss below. While this assumption is rejected by Singer and others, it is widely accepted within moral philosophy. 24 the source of an intuition is exposed as prejudice or bias. 25 One factor, however, that is often taken to give further support to an intueven by members of different cultures. The decision procedure that we will propose draws on some key aspects of reflective equilibrium, but instead of focusing on the judg- Since the procedure we outline takes as its input data about public intuitions, these criteria obviously cannot be directly applied: few ordinary people answering a survey would approximate Rawls's ideal competent judges, and we cannot guarantee that the preferences we collect always reflect considered, informed judgments. What we can do, however, is to screen the initial public preferences to ensure that they are as robust as possible. This means excluding data that we have reason to think are unreliable or not genuinely representative. Here we can draw on our growing knowledge of problematic are the product of framing effects, or are contingent on a person's transient mood, will not produce reliable data. 42 We cannot entirely exclude the effects of such biases but we can ensure, for example, subject to order effects and that they are robust to irrelevant changes in the wording or presentation. We must similarly ensure that participants fully understand the relevant factors, and that they ensure that the data reflect the views of the population as a whole rather than of some arbitrary or privileged subset. 43 Thus, in CREP, competent judges are not using their own intuitions-they are applying the reflective equilibrium to the 'will or theory, principles, concepts and justification to the preferences of people affected by their policy.
may only indicate people's stated preferences, but not their revealed preferences. One example of this is survey data from preg- people in policy-making through the use of large-scale surveys, and CREP can also incorporate more in-depth deliberation involving of public intuitions as well as identify more general principles that shape public moral thinking.
and assemblies, when this is feasible. This then gets fed into the deliberation of ethically informed policymakers. One feature of CREP is that public input is upstream of expert deliberation, which is con- We will not discuss further design elements of individual deliberative activities, noting how these issues have been explored in depth elsewhere. The contribution of CREP is to clarify the role in this process of data about public preferences, on the one hand, and the constraints imposed by ethical theories on the other. Public preferences do not directly decide policy, but serve as a key input in a deliberative process that tests these attitudes against the best established current ethical theories. The judgments that end up shaping policy are not simply those of the majority, but those public attitudes that are widely held, that have been rigorously screened for bias, and that are supported by strong moral reasons from converging ethical theories.

|
We can now illustrate this approach by applying it to the global prefwill only consider the observed preferences to save the greatest number, to spare the young over the old, and to save females over the value of life. ences about who should be saved in an emergency context, it was even when the disability was described as mild. 58 will also consider this further preference.
conducted an online survey, with self-selected participants. This leads to serious concerns about the data being representative, as such a study design is likely to attract certain kinds of participants.
their 20s and 30s. This raises the prospect that the nine preferences identified merely reflect the preferences of young men, rather than the optional demographic survey on age, education, gender, income, and political and religious views. While most responses were from responses from other groups. This allowed the authors to investigate the role of demographics in the pattern of preferences they pact on any of the nine attributes'. The only statistically significant influences on preferences that was driven by demographic differfemales over males, and religiosity higher inclination to spare humans over animals (per standard deviaassociated with a change in the direction of the nine preferences.
We can therefore be reasonably confident that the nine preferences revealed by the study reflect genuine global preferences.
again this is somewhat mitigated by the study design. The authors | SAVULESCU ET AL.
people's robust intuitions about these cases-something close enough to their considered judgments-rather than a transient response to some irrelevant features of the images or wording.
pants' actual preferences, rather than just their stated preferences.
behave as if they value saving more lives rather than fewer lives, and in some contexts age is taken to be a relevant factor in saving lives, economic frameworks such as Quality Adjusted Life Years similarly give priority to younger people.
Moral Machine experiment is likely to accurately represent robust that they do not merely reflect framing effects-though, as we shall see below, they may nevertheless reflect other biases. must choose whether to collide into one of several groups of bystanders. Moreover, it is control-in most such cases, they will be as blameless as the bystanders outside the car.
a much higher chance of death by vehicle for everyone.
Three ethical approaches to programming autonomous vehicles SAVULESCU ET AL. | 4 | SAVULESCU ET AL. the non-responsible over the responsible, whereas contractualists can directly appeal to what different parties deserve.
The use of responsibility in the allocation of limited healthcare rereflecting consequentialist and desert-based approaches, which has cal reflective equilibrium promises to advance debate on the use of public preferences around responsibility in organ allocation and healthcare generally beyond the application of novel technologies.
There is a further way in which ethical input is critical in this conbe the result of an incredibly complex computational process that are programmed to make ethically loaded choices-whether via practical reflective equilibrium or otherwise-it is crucial that the real-life outputs of such algorithms be collected for ethical audit. The 'decialgorithms that generated these decisions are morally faulty. Thus, is to deciding what to put into it. | ties to obtain data about people's explicit and implicit preferences, and to derive their values from their behaviour. How should such preferences and intuitions play a role in shaping policy and law? existing laws and policies or to develop new ones.
nology to avoid difficult ethical issues and questions. We should not return to the 'tyranny of the majority' or, in contemporary terms, important. Ultimately, in a democracy, policymakers are accountable and responsible to the people. But nonetheless we should not fall victim to the 'naturalistic fallacy' of confusing facts with values.
The ethical enterprise requires a distinctive kind of reasoning. Our proposed procedure of CREP attempts to capture this.
Both public preferences and ethical theories, concepts and principles are necessary for moral progress. Where there is reasonable philosophical disagreement, public preferences that have been formed determining policy. But where there is robust philosophical agreement across multiple ethical perspectives, public preferences no matter how widespread should not rule the day. Preferences do not necessarily track value, and moral progress often requires reshaping preferences.
CREP offers one way of combining preference/intuition and philosophical theory in relation to policy proposals. Ultimately, though, in a democracy these will be subject to the will of the people.
WT104848/Z/14/Z. JS and CG, through their involvement with the