As algorithms have become ubiquitous in consequential domains, societal concerns about the potential for discriminatory outcomes have prompted urgent calls to address algorithmic bias. In response, a rich literature across computer science, law, and ethics is rapidly proliferating to advance approaches to designing fair algorithms. Yet computer scientists, legal scholars, and ethicists are often not speaking the same language when using the term ‘bias.’ Debates concerning whether society can or should tackle the problem of algorithmic bias are hampered (...) by conflations of various understandings of bias, ranging from neutral deviations from a standard to morally problematic instances of injustice due to prejudice, discrimination, and disparate treatment. This terminological confusion impedes efforts to address clear cases of discrimination. -/- In this paper, we examine the promises and challenges of different approaches to disambiguating bias and designing for justice. While both approaches aid in understanding and addressing clear algorithmic harms, we argue that they also risk being leveraged in ways that ultimately deflect accountability from those building and deploying these systems. Applying this analysis to recent examples of generative AI, our argument highlights unseen dangers in current methods of evaluating algorithmic bias and points to ways to redirect approaches to addressing bias in generative AI at its early stages in ways that can more robustly meet the demands of justice. (shrink)

Machine learning algorithms are increasingly used to shape high-stake allocations, sparking research efforts to orient algorithm design towards ideals of justice and fairness. In this research on algorithmic fairness, normative theorizing has primarily focused on identification of “ideally fair” target states. In this paper, we argue that this preoccupation with target states in abstraction from the situated dynamics of deployment is misguided. We propose a framework that takes dynamic trajectories as direct objects of moral appraisal, highlighting three respects (...) in which such trajectories can be subject to evaluation in relation to their temporal dynamics, robustness, and representation. (shrink)

Criminal justice systems have traditionally relied heavily on human decision-making, but new technologies are increasingly supplementing the human role in this sector. This paper considers what general limits need to be placed on the use of algorithms in sentencing decisions. It argues that, even once we can build algorithms that equal human decision-making capacities, strict constraints need to be placed on how they are designed and developed. The act of condemnation is a valuable element of criminal sentencing, and using (...) algorithms in sentencing – even in an advisory role – threatens to undermine this value. The paper argues that a principle of “meaningful public control” should be met in all sentencing decisions if they are to retain their condemnatory status. This principle requires that agents who have standing to act on behalf of the wider political community retain moral responsibility for all sentencing decisions. While this principle does not rule out the use of algorithms, it does require limits on how they are constructed. (shrink)

Artificial intelligence is increasingly permeating many types of high-stake societal decision-making such as the work at the criminal courts. Various types of algorithmic tools have already been introduced into sentencing. This article concerns the use of algorithms designed to deliver sentence recommendations. More precisely, it is considered how one should determine whether one type of sentencing algorithm (e.g., a model based on machine learning) would be ethically preferable to another type of sentencing algorithm (e.g., a model based on old-fashioned (...) programming). Whether the implementation of sentencing algorithms is ethically desirable obviously depends upon various questions. For instance, some of the traditional issues that have received considerable attention are algorithmic biases and lack of transparency. However, the purpose of this article is to direct attention to a further challenge that has not yet been considered in the discussion of sentencing algorithms. That is, even if is assumed that the traditional challenges concerning biases, transparency, and cost-efficiency have all been solved or proven insubstantial, there will be a further serious challenge associated with the comparison of sentencing algorithms; namely, that we do not yet possess an ethically plausible and applicable criterion for assessing how well sentencing algorithms are performing. (shrink)

Big Data evangelists often argue that algorithms make decision-making more informed and objective—a promise hotly contested by critics of these technologies. Yet, to date, most of the debate has focused on the instruments themselves, rather than on how they are used. This article addresses this lack by examining the actual practices surrounding algorithmic technologies. Specifically, drawing on multi-sited ethnographic data, I compare how algorithms are used and interpreted in two institutional contexts with markedly different characteristics: web journalism and criminal (...) justice. I find that there are surprising similarities in how web journalists and legal professionals use algorithms in their work. In both cases, I document a gap between the intended and actual effects of algorithms—a process I analyze as “decoupling.” Second, I identify a gamut of buffering strategies used by both web journalists and legal professionals to minimize the impact of algorithms in their daily work. Those include foot-dragging, gaming, and open critique. Of course, these similarities do not exhaust the differences between the two cases, which are explored in the discussion section. I conclude with a call for further ethnographic work on algorithms in practice as an important empirical check against the dominant rhetoric of algorithmic power. (shrink)

This article presents a conceptual investigation into the value impacts and relations of algorithms in the domain of justice and security. As a conceptual investigation, it represents one step in a value sensitive design based methodology. Here, we explicate and analyse the expression of values of accuracy, privacy, fairness and equality, property and ownership, and accountability and transparency in this context. We find that values are sensitive to disvalue if algorithms are designed, implemented or deployed inappropriately or without sufficient (...) consideration for their value impacts, potentially resulting in problems including discrimination and constrained autonomy. Furthermore, we outline a framework of conceptual relations of values indicated by our analysis, and potential value tensions in their implementation and deployment with a view towards supporting future research, and supporting the value sensitive design of algorithms in justice and security. (shrink)

Artificial Intelligence and algorithms are increasingly able to replace human workers in cognitively sophisticated tasks, including ones related to justice. Many governments and international organizations are discussing policies related to the application of algorithmic judges in courts. In this paper, we investigate the public perceptions of algorithmic judges. Across two experiments (N = 1,822), and an internal meta-analysis (N = 3,039), our results show that even though court users acknowledge several advantages of algorithms (i.e., cost and speed), (...) they trust human judges more and have greater intentions to go to the court when a human (vs. an algorithmic) judge adjudicates. Additionally, we demonstrate that the extent that individuals trust algorithmic and human judges depends on the nature of the case: trust for algorithmic judges is especially low when legal cases involve emotional complexities (vs. technically complex or uncomplicated cases). (shrink)

In the digital age, the use of advanced technology is becoming a new paradigm in police work, criminal justice, and the penal system. Algorithms promise to predict delinquent behaviour, identify potentially dangerous persons, and support crime investigation. Algorithm-based applications are often deployed in this context, laying the groundwork for a ‘smart criminal justice’. In this qualitative study based on 32 interviews with criminal justice and police officials, we explore the reasons why and extent to which such a (...) smart criminal justice system has already been established in Switzerland, and the benefits perceived by users. Drawing upon this research, we address the spread, application, technical background, institutional implementation, and psychological aspects of the use of algorithms in the criminal justice system. We find that the Swiss criminal justice system is already significantly shaped by algorithms, a change motivated by political expectations and demands for efficiency. Until now, algorithms have only been used at a low level of automation and technical complexity and the levels of benefit perceived vary. This study also identifies the need for critical evaluation and research-based optimization of the implementation of advanced technology. Societal implications, as well as the legal foundations of the use of algorithms, are often insufficiently taken into account. By discussing the main challenges to and issues with algorithm use in this field, this work lays the foundation for further research and debate regarding how to guarantee that ‘smart’ criminal justice is actually carried out smartly. (shrink)

This paper presents an intuitive explanation about why and how Rawlsian Theory of Justice (Rawls in A theory of justice, Harvard University Press, Harvard, 1971) provides the foundations to a solution for algorithmic bias. The contribution of the paper is to discuss and show why Rawlsian ideas in their original form (e.g. the veil of ignorance, original position, and allowing inequalities that serve the worst-off) are relevant to operationalize fairness for algorithmic decision making. The paper also (...) explains how this leads to a specific MinMaxfairness solution, which addresses the basic challenges of algorithmic justice. We combine substantive elements of Rawlsian perspective with an intuitive explanation in order to provide accessible and practical insights. The goal is to propose and motivate why and how the MinMaxfairness solution derived from Rawlsian principles overcomes some of the current challenges for algorithmic bias and highlight the benefits provided when compared to other approaches. The paper presents and discusses the solution by building a bridge between the qualitative theoretical aspects and the quantitative technical approach. (shrink)

A common criticism of the use of algorithms in criminal justice is that algorithms and their determinations are in some sense ‘opaque’—that is, difficult or impossible to understand, whether because of their complexity or because of intellectual property protections. Scholars have noted some key problems with opacity, including that opacity can mask unfair treatment and threaten public accountability. In this paper, we explore a different but related concern with algorithmic opacity, which centers on the role of public trust (...) in grounding the legitimacy of criminal justice institutions. We argue that algorithmic opacity threatens the trustworthiness of criminal justice institutions, which in turn threatens their legitimacy. We first offer an account of institutional trustworthiness before showing how opacity threatens to undermine an institution’s trustworthiness. We then explore how threats to trustworthiness affect institutional legitimacy. Finally, we offer some policy recommendations to mitigate the threat to trustworthiness posed by the opacity problem. (shrink)

La contribution part du constat qu’il existe actuellement peu d’exigences de transparence en ce qui concerne les algorithmes de justice prédictive qui seraient mis à la disposition des juges judiciaires et administratifs pour adopter des décisions de justice, en dépit des propositions formulées en ce sens. Elle recherche pourquoi de telles exigences devraient être imposées et comment elles pourraient être concrétisées. Elle expose dans un premier temps les raisons pour lesquelles de telles exigences seraient souhaitables, en rappelant les (...) nombreux risques inhérents aux algorithmes ( a fortiori dans l’optique de les mettre à disposition des juges dans un but de justice prédictive) et en démontrant que ces risques pourraient être contrebalancés par l’application d’exigences de transparence. Elle esquisse dans un second temps une proposition de concrétisation de ces exigences, en commençant par approfondir leurs fondements juridiques potentiels. Elle s’inspire des exigences de transparence administrative applicables aux administrations qui fondent des décisions individuelles sur des algorithmes pour proposer une modélisation d’exigences de transparence applicables aux magistrats pour ce qui concerne le degré d’utilisation d’algorithmes dans l’adoption de décisions de justice, dans le respect des prérogatives des magistrats et des garanties dont ils bénéficient. (shrink)

Algorithms influence every facet of modern life: criminal justice, education, housing, entertainment, elections, social media, news feeds, work… the list goes on. Delegating important decisions to machines, however, gives rise to deep moral concerns about responsibility, transparency, freedom, fairness, and democracy. Algorithms and Autonomy connects these concerns to the core human value of autonomy in the contexts of algorithmic teacher evaluation, risk assessment in criminal sentencing, predictive policing, background checks, news feeds, ride-sharing platforms, social media, and election interference. (...) Using these case studies, the authors provide a better understanding of machine fairness and algorithmic transparency. They explain why interventions in algorithmic systems are necessary to ensure that algorithms are not used to control citizens' participation in politics and undercut democracy. This title is also available as Open Access on Cambridge Core. (shrink)

L’utilisation de techniques prédictives dans le processus de la justice, avec le développement des techniques de fouille et d’analyse des données, est considérée non seulement comme une évolution nécessaire mais encore plus comme une révolution de nature à optimiser et à rationaliser la gestion du justiciable. Cet article se propose d’exposer en quoi cette évolution est porteuse de risques, quels sont les enjeux, certains importants et surtout pourquoi il est illusoire d’espérer quelque chose de positif pour les procédures judiciaires.

Data-driven predictive algorithms are widely used to automate and guide high-stake decision making such as bail and parole recommendation, medical resource distribution, and mortgage allocation. Nevertheless, harmful outcomes biased against vulnerable groups have been reported. The growing research field known as 'algorithmic fairness' aims to mitigate these harmful biases. Its primary methodology consists in proposing mathematical metrics to address the social harms resulting from an algorithm's biased outputs. The metrics are typically motivated by -- or substantively rooted in -- (...) ideals of distributive justice, as formulated by political and legal philosophers. The perspectives of feminist political philosophers on social justice, by contrast, have been largely neglected. Some feminist philosophers have criticized the local scope of the paradigm of distributive justice and have proposed corrective amendments to surmount its limitations. The present paper brings some key insights of feminist political philosophy to algorithmic fairness. The paper has three goals. First, I show that algorithmic fairness does not accommodate structural injustices in its current scope. Second, I defend the relevance of structural injustices -- as pioneered in the contemporary philosophical literature by Iris Marion Young -- to algorithmic fairness. Third, I take some steps in developing the paradigm of 'responsible algorithmic fairness' to correct for errors in the current scope and implementation of algorithmic fairness. I close by some reflections of directions for future research. (shrink)

In this article, I show why it is necessary to abolish the use of predictive algorithms in the US criminal justice system at sentencing. After presenting the functioning of these algorithms in their context of emergence, I offer three arguments to demonstrate why their abolition is imperative. First, I show that sentencing based on predictive algorithms induces a process of rewriting the temporality of the judged individual, flattening their life into a present inescapably doomed by its past. Second, I (...) demonstrate that recursive processes, comprising predictive algorithms and the decisions based on their predictions, systematically suppress outliers and progressively transform reality to match predictions. In my third and final argument, I show that decisions made on the basis of predictive algorithms actively perform a biopolitical understanding of justice as management and modulation of risks. In such a framework, justice becomes a means to maintain a perverse social homeostasis that systematically exposes disenfranchised Black and Brown populations to risk. (shrink)

Data‐driven algorithms are widely used to make or assist decisions in sensitive domains, including healthcare, social services, education, hiring, and criminal justice. In various cases, such algorithms have preserved or even exacerbated biases against vulnerable communities, sparking a vibrant field of research focused on so‐called algorithmic biases. This research includes work on identification, diagnosis, and response to biases in algorithm‐based decision‐making. This paper aims to facilitate the application of philosophical analysis to these contested issues by providing an overview (...) of three key topics: What is algorithmic bias? Why and how can it occur? What can and should be done about it? Throughout, we highlight connections—both actual and potential—with philosophical ideas and concerns. (shrink)

Inspired by recent breakthroughs in predictive modeling, practitioners in both industry and government have turned to machine learning with hopes of operationalizing predictions to drive automated decisions. Unfortunately, many social desiderata concerning consequential decisions, such as justice or fairness, have no natural formulation within a purely predictive framework. In efforts to mitigate these problems, researchers have proposed a variety of metrics for quantifying deviations from various statistical parities that we might expect to observe in a fair world and offered (...) a variety of algorithms in attempts to satisfy subsets of these parities or to trade o the degree to which they are satised against utility. In this paper, we connect this approach to fair machine learning to the literature on ideal and non-ideal methodological approaches in political philosophy. The ideal approach requires positing the principles according to which a just world would operate. In the most straightforward application of ideal theory, one supports a proposed policy by arguing that it closes a discrepancy between the real and the perfectly just world. However, by failing to account for the mechanisms by which our non-ideal world arose, the responsibilities of various decision-makers, and the impacts of proposed policies, naive applications of ideal thinking can lead to misguided interventions. In this paper, we demonstrate a connection between the fair machine learning literature and the ideal approach in political philosophy, and argue that the increasingly apparent shortcomings of proposed fair machine learning algorithms reflect broader troubles faced by the ideal approach. We conclude with a critical discussion of the harms of misguided solutions, a reinterpretation of impossibility results, and directions for future research. (shrink)

In this article, I show why it is necessary to abolish the use of predictive algorithms in the US criminal justice system at sentencing. After presenting the functioning of these algorithms in their context of emergence, I offer three arguments to demonstrate why their abolition is imperative. First, I show that sentencing based on predictive algorithms induces a process of rewriting the temporality of the judged individual, flattening their life into a present inescapably doomed by its past. Second, I (...) demonstrate that recursive processes, comprising predictive algorithms and the decisions based on their predictions, systematically suppress outliers and progressively transform reality to match predictions. In my third and final argument, I show that decisions made on the basis of predictive algorithms actively perform a biopolitical understanding of justice as management and modulation of risks. In such a framework, justice becomes a means to maintain a perverse social homeostasis that systematically exposes disenfranchised Black and Brown populations to risk. (shrink)

As government pressure on major technology companies builds, both firms and legislators are searching for technical solutions to difficult platform governance puzzles such as hate speech and misinformation. Automated hash-matching and predictive machine learning tools – what we define here as algorithmic moderation systems – are increasingly being deployed to conduct content moderation at scale by major platforms for user-generated content such as Facebook, YouTube and Twitter. This article provides an accessible technical primer on how algorithmic moderation works; (...) examines some of the existing automated tools used by major platforms to handle copyright infringement, terrorism and toxic speech; and identifies key political and ethical issues for these systems as the reliance on them grows. Recent events suggest that algorithmic moderation has become necessary to manage growing public expectations for increased platform responsibility, safety and security on the global stage; however, as we demonstrate, these systems remain opaque, unaccountable and poorly understood. Despite the potential promise of algorithms or ‘AI’, we show that even ‘well optimized’ moderation systems could exacerbate, rather than relieve, many existing problems with content policy as enacted by platforms for three main reasons: automated moderation threatens to further increase opacity, making a famously non-transparent set of practices even more difficult to understand or audit, further complicate outstanding issues of fairness and justice in large-scale sociotechnical systems and re-obscure the fundamentally political nature of speech decisions being executed at scale. (shrink)

Artificial intelligence (AI) systems are increasingly adopted to make decisions in domains such as business, education, health care, and criminal justice. However, such algorithmic decision systems can have prevalent biases against marginalized social groups and undermine social justice. Explainable artificial intelligence (XAI) is a recent development aiming to make an AI system’s decision processes less opaque and to expose its problematic biases. This paper argues against technical XAI, according to which the detection and interpretation of algorithmic (...) bias can be handled more or less independently by technical experts who specialize in XAI methods. Drawing on resources from feminist epistemology, we show why technical XAI is mistaken. Specifically, we demonstrate that the proper detection of algorithmic bias requires relevant interpretive resources, which can only be made available, in practice, by actively involving a diverse group of stakeholders. Finally, we suggest how feminist theories can help shape integrated XAI: an inclusive social-epistemic process that facilitates the amelioration of algorithmic bias. (shrink)

The application of machine-learning technologies to medical practice promises to enhance the capabilities of healthcare professionals in the assessment, diagnosis, and treatment, of medical conditions. However, there is growing concern that algorithmic bias may perpetuate or exacerbate existing health inequalities. Hence, it matters that we make precise the different respects in which algorithmic bias can arise in medicine, and also make clear the normative relevance of these different kinds of algorithmic bias for broader questions about justice (...) and fairness in healthcare. In this paper, we provide the building blocks for an account of algorithmic bias and its normative relevance in medicine. (shrink)

Since the Enlightenment, there have been advocates for the rationalizing efficiency of enlightened sovereigns, bureaucrats, and technocrats. Today these enthusiasms are joined by calls for replacing or augmenting government with algorithms and artificial intelligence, a process already substantially under way. Bureaucracies are in effect algorithms created by technocrats that systematize governance, and their automation simply removes bureaucrats and paper. The growth of algorithmic governance can already be seen in the automation of social services, regulatory oversight, policing, the justice (...) system, and the military. However, there have also always been justified democratic and economic criticisms of autocracy, bureaucracy, technocracy, and algorithmic governance. Bureaucrats, technocrats, and algorithms embody biases that tend to serve the interests of elites, and all require transparency and democratic accountability, oversight individual citizens are ill equipped to exercise. As state apparatuses are increasingly automated, mechanisms for collective action and democratic oversight also need to be automated. Algorithms and cyborg citizenry will enable a posthuman democracy. Democratically accountable algorithmic governance, enabled by artificial intelligence and human enhancement, can automate bottom-up citizen surveillance, inform debate, aggregate decision making, and ensure the efficient working of the gradually withering state. As paid work disappears and we transition to a postcapitalist economy with a universal basic income, market mechanisms can be replaced with democratic planning. Indeed, only algorithmic governance can secure our future against accelerating threats from technological innovation. (shrink)

As inductive decision-making procedures, the inferences made by machine learning programs are subject to underdetermination by evidence and bear inductive risk. One strategy for overcoming these challenges is guided by a presumption in philosophy of science that inductive inferences can and should be value-free. Applied to machine learning programs, the strategy assumes that the influence of values is restricted to data and decision outcomes, thereby omitting internal value-laden design choice points. In this paper, I apply arguments from feminist philosophy of (...) science to machine learning programs to make the case that the resources required to respond to these inductive challenges render critical aspects of their design constitutively value-laden. I demonstrate these points specifically in the case of recidivism algorithms, arguing that contemporary debates concerning fairness in criminal justice risk-assessment programs are best understood as iterations of traditional arguments from inductive risk and demarcation, and thereby establish the value-laden nature of automated decision-making programs. Finally, in light of these points, I address opportunities for relocating the value-free ideal in machine learning and the limitations that accompany them. (shrink)

In this position paper, we introduce a new epistemic lens for analyzing algorithmic harm. We argue that the epistemic lens we propose herein has two key contributions to help reframe and address some of the assumptions underlying inquiries into algorithmic fairness. First, we argue that using the framework of epistemic injustice helps to identify the root causes of harms currently framed as instances of representational harm. We suggest that the epistemic lens offers a theoretical foundation for expanding approaches (...) to algorithmic fairness in order to address a wider range of harms not recognized by existing technical or legal definitions. Second, we argue that the epistemic lens helps to identify the epistemic goals of inquiries into algorithmic fairness. There are two distinct contexts within which we examine algorithmic harm: at times, we seek to understand and describe the world as it is, and, at other times, we seek to build a more just future. The epistemic lens can serve to direct our attention to the epistemic frameworks that shape our interpretations of the world as it is and the ways we envision possible futures. Clarity with respect to which epistemic context is relevant in a given inquiry can further help inform choices among the different ways of measuring and addressing algorithmic harms. We introduce this framework with the goal of initiating new research directions bridging philosophical, legal, and technical approaches to understanding and mitigating algorithmic harms. (shrink)

As AI is linked to more and more aspects of our lives, the need for algorithms that can take decisions that are not only accurate but also fair becomes apparent. It can be seen both in discussions of future trends such as autonomous vehicles or the issue of superintelligence, as well as actual implementations of machine learning used to decide whether a person should be admitted in certain university or will be able to return a credit. In this paper, we (...) will use Dreyfus’ account on ethical expertise to show that, to give an AI some ability to make ethical judgements, a pure symbolic, conceptual approach is not enough. We also need the ability to make sense of the surroundings to reframe and define situations in a dynamic way, using multiple perspectives in a pre-reflective way. (shrink)

In this APA blog, I appeal to two 2020 cases of algorithms gone wrong to motivate philosophical attention to algorithmic oppression. I offer a simple definition, then describe a few of the ways it is engendered. References and extends work by Safiya Noble, Cathy O'Neil, Ruha Benjamin, Virginia Eubanks, Sara Wachter-Boettcher, Michael Kearns & Aaron Roth.

Philippe Baumard et Nadim Kobeissi explorent la relation entre liberté, transparence et sécurité. Dans le contexte de l’invalidation du 6 octobre 2015 de l’accord « Safe Harbour » par la Cour de justice de l’Union européenne, et revenant sur les implications de l’affaire Volkswagen, les auteurs dénouent les liens réputés inextricables entre les notions de souveraineté numérique, libertés individuelles et publiques, et de sécurité. Les deux auteurs considèrent ainsi la perspective qu’une meilleure transparence, gérée avec rigueur, peut battre aussi (...) bien l’efficience de marché, sans pour autant la délaisser, qu’une régulation excessive, ou l’opacité d’une économie numérique fondée sur le secret industriel et le secret des algorithmes ». (shrink)

La justice prédictive est ici sommairement définie comme une justice prédite par des algorithmes et, partant, envisagée comme une justice simplement virtuelle, c’est-à-dire seulement probable, non acquise dans son existence et dont l’actualisation est laissée à la création, à l’imagination et à l’intuition des utilisateurs qui feront que la prédiction restera « une » solution proposée ou deviendra « la » solution finalement adoptée. La justice prédictive apparaît ainsi comme un espace de prospective juridique pour les (...) justiciables, les professions juridiques et la science du droit. Cette introduction à une justice simplement virtuelle étudie, d’une part, le risque qui reste mesuré d’une domination du virtuel, lorsque celui-ci devient prescriptif pour les professionnels ou dissuasif pour les justiciables ; elle envisage, d’autre part, l’avenir espéré d’une responsabilisation face au virtuel, par une nécessaire compréhension technique de l’outil de justice prédictive et une essentielle respiration éthique de ses acteurs. (shrink)

Scholars have focused on algorithms used during sentencing, bail, and parole, but little work explores what we term “carceral algorithms” that are used during incarceration. This paper is focused on the Pennsylvania Additive Classification Tool used to classify prisoners’ custody levels while they are incarcerated. Algorithms that are used during incarceration warrant deeper attention by scholars because they have the power to enact the lived reality of the prisoner. The algorithm in this case determines the likelihood a person would endure (...) additional disciplinary actions, can complete required programming, and gain experiences that, among other things, are distilled into variables feeding into the parole algorithm. Given such power, examining algorithms used on people currently incarcerated offers a unique analytic view to think about the dialectic relationship between data and algorithms. Our examination of the PACT is two-fold and complementary. First, our qualitative overview of the historical context surrounding PACT reveals that it is designed to prioritize incapacitation and control over rehabilitation. While it closely informs prisoner rehabilitation plans and parole considerations, it is rooted in population management for prison securitization. Second, on analyzing data for 146,793 incarcerated people in PA, along with associated metadata related to the PACT, we find it is replete with racial bias as well as errors, omissions, and inaccuracies. Our findings to date further caution against data-driven criminal justice reforms that rely on pre-existing data infrastructures and expansive, uncritical, data-collection routines. (shrink)

The use of algorithms to support prediction-based decision-making is becoming commonplace in a range of domains including health, criminal justice, education, social services, lending, and hiring. An assumption governing such decisions is that there is a property Y such that individual a should be allocated resource R by decision-maker D if a is Y. When there is uncertainty about whether a is Y, algorithms may provide valuable decision support by accurately predicting whether a is Y on the basis of (...) known features of a. Based on recent work on statistical evidence in epistemology this article presents an argument against relying exclusively on algorithmic predictions to allocate resources when they provide purely statistical evidence that a is Y. The article then responds to the objection that any evidence that will increase the proportion of correct decisions should be accepted as the basis for allocations regardless of its epistemic deficiency. Finally, some important practical aspects of the conclusion are considered. (shrink)

With the increasing use of algorithms in high-stakes areas such as criminal justice and health has come a significant concern about the fairness of prediction-based decision procedures. In this article I argue that a prominent class of mathematically incompatible performance parity criteria can all be understood as applications of John Broome’s account of fairness as the proportional satisfaction of claims. On this interpretation these criteria do not disagree on what it means for an algorithm to be _fair_. Rather they (...) express different understandings of what grounds a claim to a good being allocated by an algorithmic decision procedure. I then argue that an important implication of the Broomean interpretation is that it strengthens the case for outcome-based criteria. Finally, I consider how a version of the levelling-down objection to performance parity criteria arises within the Broomean account. (shrink)

Through modern driver assistant systems, algorithmic decisions already have a significant impact on the behavior of vehicles in everyday traffic. This will become even more prominent in the near future considering the development of autonomous driving functionality. The need to consider ethical principles in the design of such systems is generally acknowledged. However, scope, principles and strategies for their implementations are not yet clear. Most of the current discussions concentrate on situations of unavoidable crashes in which the life of (...) human beings is existentially affected. In this paper, we argue that ethical considerations should be mandatory for any algorithmic decision of autonomous vehicles, instead of a limitation to hazard situations. Such an ethically aligned behavior is relevant because autonomous vehicles, like any other traffic participants, operate in a shared public space, where every behavioral decision impacts the operational possibilities of others. These possibilities concern the fulfillment of a road-user’s safety, utility and comfort needs. We propose that, to operate ethically in such space, an autonomous vehicle will have to take its behavior decisions according to a just distribution of operational possibilities among all traffic participants. Using an application on a partially-autonomous prototype vehicle, we describe how to apply and implement concepts of distributive justice to the driving environment and demonstrate the impact on its behavior in comparison to an advanced but egoistic decision maker. (shrink)

Algorithmic systems are increasingly used by state agencies to inform decisions about humans. They produce scores on risks of recidivism in criminal justice, indicate the probability for a job seeker to find a job in the labor market, or calculate whether an applicant should get access to a certain university program. In this contribution, we take an interdisciplinary perspective, provide a bird’s eye view of the different key decisions that are to be taken when state actors decide to (...) use an algorithmic system, and illustrate these decisions with empirical examples from case studies. Building on these insights, we discuss the main pitfalls and promises of the use of algorithmic system by the state and focus on four levels: The most basic question whether an algorithmic system should be used at all, the regulation and governance of the system, issues of algorithm design, and, finally, questions related to the implementation of the system on the ground and the human–machine-interaction that comes with it. Based on our assessment of the advantages and challenges that arise at each of these levels, we propose a set of crucial questions to be asked when such intricate matters are addressed. (shrink)

Everyday surveillance work is increasingly performed by non-human algorithms. These entities can be conceptualised as machinic flâneurs that engage in distanciated flânerie: subjecting urban flows to a dispassionate, calculative and expansive gaze. This paper provides some theoretical reflections on the nascent forms of algorithmic practice materialising in two Australian cities, and some of their implications for urban relations and social justice. It looks at the idealisation – and operational black boxing – of automated watching programs, before considering their (...) impacts on notions such as ‘the right to the city’ and ‘the right to the face’. It will argue that the turn to facial recognition software for the purposes of automating urban governance reconstitutes the meanings and phenomenology of the face. In particular, the fleshly and communicative physicality of the face is reduced to a measurable object that can be identified by a virtualised referent and then consequently tracked. Moreover, the asymmetrical and faceless nature of these machinic programs of recognition unsettles conventional notions of civil inattention and bodily sovereignty, and the prioritisation given to pattern recognition renders them amenable to ideas/ideals from phrenology and physiognomy. In this way, algorithmic governance may generate not only forms of facial vulnerability and estrangement, but also facial artifice, where individuals come to develop tacit and artful ways of de-facing and re-facing in order to subvert the processes of recognition which leverage these modes of biopower. Thus, the datafication of urban governance gives rise to a dynamic biopolitics of the face. (shrink)

While rapid advances in artificial intelligence hiring tools promise to transform the workplace, these algorithms risk exacerbating existing biases against marginalized groups. In light of these ethical issues, AI vendors have sought to translate normative concepts such as fairness into measurable, mathematical criteria that can be optimized for. However, questions of disability and access often are omitted from these ongoing discussions about algorithmic bias. In this paper, I argue that the multiplicity of different kinds and intensities of people’s disabilities (...) and the fluid, contextual ways in which they manifest point to the limits of algorithmic fairness initiatives. In particular, existing de-biasing measures tend to flatten variance within and among disabled people and abstract away information in ways that reinforce pathologization. While fair machine learning methods can help mitigate certain disparities, I argue that fairness alone is insufficient to secure accessible, inclusive AI. I then outline a disability justice approach, which provides a framework for centering disabled people’s experiences and attending to the structures and norms that underpin algorithmic bias. (shrink)

L’open data des décisions de justice a ouvert le marché des algorithmes de justice prédictive. Depuis, nombreuses sont les innovations alliant intelligence artificielle et monde du droit. La justice prédictive est un outil technologique au service de l’avocat mais peut-elle ne rester qu’un accessoire pertinent? Se substitue-t-elle au travail humain? Quel retour en tirer pour les professionnels du droit? Quels conseils d’utilisation donner pour intégrer de manière adéquate le changement à la pratique?

The problem of algorithmic fairness is typically framed as the problem of finding a unique formal criterion that guarantees that a given algorithmic decision-making procedure is morally permissible. In this paper, I argue that this is conceptually misguided and that we should replace the problem with two sub-problems. If we examine how most state-of-the-art machine learning systems work, we notice that there are two distinct stages in the decision-making process. First, a prediction of a relevant property is made. (...) Secondly, a decision is taken based (at least partly) on this prediction. These two stages have different aims: the prediction is aimed at accuracy, while the decision is aimed at allocating a given good in a way that maximizes some context-relative utility measure. Correspondingly, two different fairness issues can arise. First, predictions could be biased in discriminatory ways. This means that the predictions contain systematic errors for a specific group of individuals. Secondly, the system’s decisions could result in an allocation of goods that is in tension with the principles of distributive justice. These two fairness issues are distinct problems that require different types of solutions. I here provide a formal framework to address both issues and argue that this way of conceptualizing them resolves some of the paradoxes present in the discussion of algorithmic fairness. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:Reviewed by:Atonement, Justice, and Peace: The Message of the Cross and the Mission of the Church by Darrin W. Snyder Belousek, and: Restorative Justice: Theories and Practices of Moral Imagination by Amy LevadDana ScopatzReview of Atonement, Justice, and Peace: The Message of the Cross and the Mission of the Church DARRIN W. SNYDER BELOUSEK Grand Rapids, MI: Eerdmans, 2012. 668 pp. $55.00Review of Restorative Justice: (...) Theories and Practices of Moral Imagination AMY LEVAD El Paso, TX: LFB Scholarly Publishing, 2012. 300 pp. $80.00In Atonement, Justice, and Peace and Restorative Justice, the authors advocate for a restorative conception of justice, which they oppose to the “normal” retributive concept. Amy Levad draws on Aristotle and the virtue ethics tradition to write about “moral imagination” and restorative justice. Darrin Belousek uses Thomas Kuhn’s language to write about a “paradigm shift” in our understanding of theological themes relevant to restorative justice. Both contrast the restorative imagination or paradigm with a retributive one, although Levad also contrasts restoration with rehabilitation. Levad limits her study to restorative justice as [End Page 214] practiced in the criminal justice system, while Belousek expands his work to consider the Bible, theology, and multiple issues in society and in the church.Levad begins her book by discussing Aristotle’s idea of justice as equity, which she points out requires more discernment from judicial decision makers than it requires the algorithmic application of laws. In order for them to enact justice as equity, she argues, decision makers need a “vivid and expansive” moral imagination. Toward this end, she brings together two traditions: Aristotelian virtue ethics and modern literature on moral imagination. At the heart of her work is an ethnographic study intended to determine whether there are practices of restorative justice that can foster such a vivid and expansive moral imagination. After observing and interviewing participants in restorative justice programs in Colorado, Levad examined the narratives, metaphors, and symbols they used in order to determine whether and how they demonstrated a distinctively restorative moral imagination. Because the scope of her study was limited, and because she had difficulty getting a wide spectrum of roles involved in restorative justice to participate in the study, Levad’s results are more suggestive than conclusive. She acknowledges that we cannot generalize from them; although she draws on research methods from sociology and criminology, her findings do not meet their standards of evidence. Thus, the book’s greatest value lies in the recommendations it makes for further study. Empirical studies of the effectiveness of restorative justice practices have tended to focus on either reducing recidivism or the satisfaction of participants with the process. Levad brings to the fore two questions that have so far not been explored in previous research. First, how do restorative justice practices affect the moral imagination? And second, what effect might those practices have on ethical discernment? Her book takes some hopeful initial steps in addressing those questions.Although Levad writes as a Christian ethicist, she did not explicitly incorporate theological reflection into her study of state-sponsored programs. Belousek, by contrast, takes on an explicitly theological project of evaluating how interpretations of the significance of the cross of Jesus interact with such things as support for capital punishment, just war, and market economics. Since Belousek’s book is wide-ranging, this review will concentrate on the task it has in common with Levad’s book: that of reconceptualizing justice.In a fashion parallel to Levad’s use of Aristotle’s notion of justice as equity, which sometimes departs from strict legal justice, Belousek cites Paul’s vision of God’s justice “irrespective of” or “apart from” the law (372). Belousek begins by linking penal-substitutionary atonement theory to capital punishment, just war, and exchange economy in a single “paradigm,” arguing that they are all under-girded by what he calls the “Retributive Principle” (28). He then challenges this paradigm’s appropriateness for Christianity on biblical and doctrinal grounds. After examining arguments in favor of penal-substitutionary atonement in detail—drawing on the works of J. I. Packer and Thomas Schreiner, among [End Page 215] others—he turns to the scriptures cited in support... (shrink)

Most online platforms are becoming increasingly algorithmically personalized. The question is if these practices are simply satisfying users preferences or if something is lost in this process. This article focuses on how to reconcile the personalization with the importance of being able to share cultural objects - including fiction – with others. In analyzing two concrete personalization examples from the streaming giant Netflix, several tendencies are observed. One is to isolate users and sometimes entirely eliminate shared world aspects. Another tendency (...) is to blur the boundary between shared cultural objects and personalized content, which can be misleading and disorienting. A further tendency is for personalization algorithms to be optimized to deceptively prey on desires for content that mirrors one’s own lived experience. Some specific - often minority targeting -“clickbait” practices received public blowback. These practices show disregard both for honest labeling and for our desires to have access and representation in a shared world. The article concludes that personalization tendencies are moving towards increasingly isolating and disorienting interfaces, but that platforms could be redesigned to support better social world orientation. (shrink)

Efforts to promote equitable public policy with algorithms appear to be fundamentally constrained by the “impossibility of fairness” (an incompatibility between mathematical definitions of fairness). This technical limitation raises a central question about algorithmic fairness: How can computer scientists and policymakers support equitable policy reforms with algorithms? In this article, I argue that promoting justice with algorithms requires reforming the methodology of algorithmic fairness. First, I diagnose the problems of the current methodology for algorithmic fairness, which (...) I call “formal algorithmic fairness.” Because formal algorithmic fairness restricts analysis to isolated decision-making procedures, it leads to the impossibility of fairness and to models that exacerbate oppression despite appearing “fair.” Second, I draw on theories of substantive equality from law and philosophy to propose an alternative methodology, which I call “substantive algorithmic fairness.” Because substantive algorithmic fairness takes a more expansive scope of analysis, it enables an escape from the impossibility of fairness and provides a rigorous guide for alleviating injustice with algorithms. In sum, substantive algorithmic fairness presents a new direction for algorithmic fairness: away from formal mathematical models of “fair” decision-making and toward substantive evaluations of whether and how algorithms can promote justice in practice. (shrink)

Perhaps nowhere better than, "On the Names of God," can readers discern Laclau's appreciation of theology, specifically, negative theology, and the radical potencies of political theology. // It is Laclau's close attention to Eckhart and Dionysius in this essay that reveals a core theological strategy to be learned by populist reasons or social logics and applied in politics or democracies to come. // This mode of algorithmically informed negative political theology is not mathematically inert. It aspires to relate a fraction (...) or ratio to a series ... It strains to reduce the decided determinateness of such seriality ever condemned to the naive metaphysics of bad infinity. // It is worth considering that it is the specific 'number' of Dionysius in differential identification with an ineffable god (and, as such, a singular becoming between theology and numbers) that is floating in at least two dimensions [of signification] (be it political Demand on the horizontal dimension or theological Desire on [a] floating dimension) that cannot but *perform the link that relinks* names of god with any political life, populist reason, social justice, or radical democracy straining toward peace. (shrink)

A basic principle in sentencing offenders is proportionality. However, proportionality judgments are often left to the discretion of the judge, raising familiar concerns of arbitrariness and bias. This paper considers the case for systematizing judgments of proportionality in sentencing by means of an algorithm. The aim of such an algorithm would be to predict what a judge in that jurisdiction would regard as a proportionate sentence in a particular case. A predictive algorithm of this kind would not necessarily undermine (...) class='Hi'>justice in individual cases, is consistent with a particularistic account of moral judgment, and is attractive even in the face of uncertainty as to the legitimate purposes of punishment. (shrink)

This article presents a relatively straightforward theoretical framework about distributive justice with applications. It draws on a few key concepts of Sociological Game Theory. SGT is presented briefly in section 2. Section 3 provides a spectrum of distributive cases concerning principles of equality, differentiation among recipients according to performance or contribution, status or authority, or need. Two general types of social organization of distributive judgment are distinguished and judgment procedures or algorithms are modeled in each type of social organization. (...) Section 4 discusses briefly the larger moral landscapes of human judgment – how distribution may typically be combined with other value into consideration. The article suggests that Rawls, Elster, and Machado point in this direction. Finally, it is suggested that the SGT framework presented provides a useful point of departure to systematically link it and compare the Warsaw School of Fair Division, Rawls, and Elster, among others. (shrink)

Medical residency—specialty training after the completion of medical school—is an essential component of medical education and is required in order to be a licensed, independent medical practitioner in most jurisdictions. As things currently stand in the United States, the match between medical school graduates and residency programs is governed by a match between rank-order lists prepared by candidates and residencies alike. An applicant picks a number of residency programs and ranks them according to order of interest. The residency program prepares (...) a similar list, ranking the candidates it most wants in its program. A computer program compares the rankings and makes assignments according to a certain algorithm. Using these lists, the match system assigns approximately 24,000 applicants to approximately 21,000 training positions in pediatrics, obstetrics and gynecology, internal medicine, and the rest. These assignments are then announced to all parties on specific days. The system has been in place since 1952 and is overseen by the National Residency Match Program, a nonprofit organization. This system has several advantages. First of all, it standardizes the timetable for decisions, and applicants are in no position to tie up offers while waiting to hear from other institutions. Institutions are not held captive, either, in making assignments while waiting to hear from particular parties. (shrink)

This book provides a comprehensive examination of the police role from within a broader philosophical context. Contending that the police are in the midst of an identity crisis that exacerbates unjustified law enforcement tactics, Luke William Hunt examines various major conceptions of the police—those seeing them as heroes, warriors, and guardians. The book looks at the police role considering the overarching societal goal of justice and seeks to present a synthetic theory that draws upon history, law, society, psychology, and (...) philosophy. Each major conception of the police role is examined in light of how it affects the pursuit of justice, and how it may be contrary to seeking justice holistically and collectively. The book sets forth a conception of the police role that is consistent with the basic values of a constitutional democracy in the liberal tradition. Hunt’s intent is that clarifying the police role will likewise elucidate any constraints upon policing strategies, including algorithmic strategies such as predictive policing. This book is essential reading for thoughtful policing and legal scholars as well as those interested in political philosophy, political theory, psychology, and related areas. Now more than ever, the nature of the police role is a philosophical topic that is relevant not just to police officials and social scientists, but to everyone. (shrink)

Policing in many parts of the world—the United States in particular—has embraced an archetypal model: a conception of the police based on the tenets of individuated archetypes, such as the heroic police “warrior” or “guardian.” Such policing has in part motivated moves to (1) a reallocative model: reallocating societal resources such that the police are no longer needed in society (defunding and abolishing) because reform strategies cannot fix the way societal problems become manifest in (archetypal) policing; and (2) an (...) class='Hi'>algorithmic model: subsuming policing into technocratic judgements encoded in algorithms through strategies such as predictive policing (mitigating archetypal bias). This paper begins by considering the normative basis of the relationship between political community and policing. It then examines the justification of reallocative and algorithmic models in light of the relationship between political community and police. Given commitments to the depth and distribution of security—and proscriptions against dehumanizing strategies—the paper concludes that a nonideal-theory priority rule promoting respect for personhood (manifest in community and dignity-promoting policing strategies) is a necessary condition for the justification of the above models. (shrink)

Using artificial intelligence (AI) to make decisions in human resource management (HRM) raises questions of how fair employees perceive these decisions to be and whether they experience respectful treatment (i.e., interactional justice). In this experimental survey study with open-ended qualitative questions, we examine decision making in six HRM functions and manipulate the decision maker (AI or human) and decision valence (positive or negative) to determine their impact on individuals’ experiences of interactional justice, trust, dehumanization, and perceptions of decision-maker (...) role appropriate- ness. In terms of decision makers, the use of human decision makers over AIs generally resulted in better perceptions of respectful treatment. In terms of decision valence, people experiencing positive over negative decisions generally resulted in better perceptions of respectful treatment. In instances where these cases conflict, on some indicators people preferred positive AI decisions over negative human decisions. Qualitative responses show how people identify justice concerns with both AI and human decision making. We outline implications for theory, practice, and future research. (shrink)

Providing healthcare services frequently involves cognitively demanding tasks, including diagnoses and analyses as well as complex decisions about treatments and therapy. From a global perspective, ethically significant inequalities exist between regions where the expert knowledge required for these tasks is scarce or abundant. One possible strategy to diminish such inequalities and increase healthcare opportunities in expert-scarce settings is to provide healthcare solutions involving digital technologies that do not necessarily require the presence of a human expert, e.g., in the form of (...) artificial intelligent decision-support systems (AI-DSS). Such algorithmic decision-making, however, is mostly developed in resource- and expert-abundant settings to support healthcare experts in their work. As a practical consequence, the normative standards and requirements for such algorithmic decision-making in healthcare require the technology to be at least as explainable as the decisions made by the experts themselves. The goal of providing healthcare in settings where resources and expertise are scarce might come with a normative pull to lower the normative standards of using digital technologies in order to provide at least some healthcare in the first place. We scrutinize this tendency to lower standards in particular settings from a normative perspective, distinguish between different types of absolute and relative, local and global standards of explainability, and conclude by defending an ambitious and practicable standard of local relative explainability. (shrink)