The use of counterfactuals for considerations of algorithmic fairness and explainability is gaining prominence within the machine learning community and industry. This paper argues for more caution with the use of counterfactuals when the facts to be considered are social categories such as race or gender. We review a broad body of papers from philosophy and social sciences on social ontology and the semantics of counterfactuals, and we conclude that the counterfactual approach in machine learning fairness and social explainability can (...) require an incoherent theory of what social categories are. Our findings suggest that most often the social categories may not admit counterfactual manipulation, and hence may not appropriately satisfy the demands for evaluating the truth or falsity of counterfactuals. This is important because the wide- spread use of counterfactuals in machine learning can lead to misleading results when applied in high-stakes domains. Accordingly, we argue that even though counterfactuals play an essential part in some causal inferences, their use for questions of algorithmic fairness and social explanations can create more problems than they resolve. Our positive result is a set of tenets about using counterfactuals for fairness and explanations in machine learning. (shrink)

In response to growing concerns of bias, discrimination, and unfairness perpetuated by algorithmic systems, the datasets used to train and evaluate machine learning models have come under increased scrutiny. Many of these examinations have focused on the contents of machine learning datasets, finding glaring underrepresentation of minoritized groups. In contrast, relatively little work has been done to examine the norms, values, and assumptions embedded in these datasets. In this work, we conceptualize machine learning datasets as a type of informational infrastructure, (...) and motivate a genealogy as method in examining the histories and modes of constitution at play in their creation. We present a critical history of ImageNet as an exemplar, utilizing critical discourse analysis of major texts around ImageNet’s creation and impact. We find that assumptions around ImageNet and other large computer vision datasets more generally rely on three themes: the aggregation and accumulation of more data, the computational construction of meaning, and making certain types of data labor invisible. By tracing the discourses that surround this influential benchmark, we contribute to the ongoing development of the standards and norms around data development in machine learning and artificial intelligence research. (shrink)

ABSTRACT A key selling point of pharmacogenetics is the genetic stratification of either patients or diseases in order to target the prescribing of medicine. The hope is that genetically ‘tailored’ medicines will replace the current ‘one‐size‐fits‐all’ paradigm of drug development and usage. This paper is concerned with the relationship between difference and justice in the use of pharmacogenetics. This new technology, which facilitates the identification and use of difference, has, we shall argue, the potential to lead to injustice either by (...) the inappropriate use of difference or through the inappropriate failure to use difference. We build on empirical data from a detailed study of the range of options for the development of pharmacogenetics to present a consideration of the ethical issues that surround patient and disease stratification. In it we explore the ways in which the use of pharmacogenetics may lead to the creation of new, genetically stratified, forms of difference and new forms of injustice based on these divisions. We also examine the ways in which existing forms of difference and social stratification may interact with the use of pharmacogenetics. In conclusion, we suggest how an understanding of these ethical issues could usefully inform future policy discussions. (shrink)

Pharmacogenetics is an emerging biotechnology concerned with understanding the genetic basis of drug response, and promises to transform the development, marketing and prescription of medicines. This paper is concerned with analysing the move towards segmented drug markets, which is implicit in the commercial development of pharmacogenetics. It is claimed that in future who gets a particular drug will be determined by their genetic make up. Drawing on ideas from the sociology of expectations we examine how pharmaceutical and biotechnology companies are (...) constructing, responding to and realising particular ‘visions’ or expectations of pharmacogenetics and market stratification. We argue that the process of market segmentation remains uncertain, but that the outcome will be fashioned according to the convergence and divergence of the interests of key commercial actors. Qualitative data based both on interviews with industry executives and company documentation will be used to explore how different groups of companies are developing pharmacogenetics in distinct ways, and what consequences these different pathways might have for both clinical practice and health policy. In particular, the analysis will show a convergence of interests between biotechnology and pharmaceutical companies for creating segmented markets for new drugs, but a divergence of interest in segmenting established markets. Whilst biotechnology firms have a strong incentive to innovate, the pharmaceutical industry has no commercial interest in segmenting markets for existing products. This has important implications, as many of the claimed public health benefits of pharmacogenetics will derive from changing the prescribing of existing medicines. One significant implication of this is that biotechnology companies who wish to apply pharmacogenetics to existing medicines will have to explore an alternative convergence of interests with healthcare payers and providers . Healthcare providers may have a strong incentive to use pharmacogenetics to make the prescribing of existing medicine more cost-effective. However, we conclude by suggesting that a question mark hangs over their ability to provide the necessary economic and structural resources to bring such a vision to fruition. (shrink)

The ongoing debate about the FDA approval of BiDil in 2005 demonstrates how the first racially/ethnically licensed drug is entangled in both Utopian and dystopian future visions about the continued saliency of race/ethnicity in science and medicine. Drawing on the sociology of expectations, this paper analyzes how scientists in the field of pharmacogenetics are constructing certain visions of the future with respect to the use of social categories of race/ethnicity and the impact of high-throughput genotyping technologies that promise to transform (...) scientific practices. (shrink)

In a recent discussion about how scientific knowledge might potentially change our understanding of the nature and extent of human genetic, cultural, or biological variation, the sociologist David Skinner identified two competing visions of the future: one that was decidedly dystopian, which conjured up a “re-racialized” future, and an opposing utopian future in which the potential for racialized thinking might be finally overcome. We can situate the ongoing debates about the congestive heart failure drug BiDil, approved by the Food and (...) Drug Administration for use only by African Americans, in relation to these differing future prospects.When the FDA announced its approval of BiDil in June 2005, it located the drug, and perhaps the future of pharmaceutical development, within a particular vision of the future, heralding BiDil as “representing a step toward the promise of personalized medicine.” The discourse of “personalized medicine” can be characterized as part of a utopian future, one in which clinicians will be able to make increasingly individualized decisions based on each patient’s genetic makeup so that the drugs they take will be those that work best for them. (shrink)