What's sex got to do with machine learning?

The debate about fairness in machine learning has largely centered around competing substantive definitions of what fairness or nondiscrimination between groups requires. However, very little attention has been paid to what precisely a group is. Many recent approaches have abandoned observational, or purely statistical, definitions of fairness in favor of definitions that require one to specify a causal model of the data generating process. The implicit ontological assumption of these exercises is that a racial or sex group is a collection of individuals who share a trait or attribute, for example: the group "female" simply consists in grouping individuals who share female-coded sex features. We show this by exploring the formal assumption of modularity in causal models using directed acyclic graphs (DAGs), which hold that the dependencies captured by one causal pathway are invariant to interventions on any other causal pathways. Modeling sex, for example, as a node in a causal model aimed at elucidating fairness questions proposes two substantive claims: 1) There exists a feature, sex-on-its-own, that is an inherent trait of an individual that then (causally) brings about social phenomena external to it in the world; and 2) the relations between sex and its downstream effects can be modified in whichever ways and the former node would still retain the meaning that sex has in our world. Together, these claims suggest sex to be a category that could be different in its (causal) relations with other features of our social world via hypothetical interventions yet still mean what it means in our world. This fundamental stability of categories and causes (unless explicitly intervened on) is essential in the methodology of causal inference, because without it, causal operations can alter the meaning of a category, fundamentally change how it is situated within a causal diagram, and undermine the validity of any inferences drawn on the diagram as corresponding to any real phenomena in the world.

We argue that these methods' ontological assumptions about social groups such as sex are conceptual errors. Many of the "effects" that sex purportedly "causes" are in fact constitutive features of sex as a social status. They constitute what it means to be sexed. In other words, together, they give the social meaning of sex features. These social meanings are precisely, we argue, what makes sex discrimination a distinctively morally problematic type of act that differs from mere irrationality or meanness on the basis of a physical feature.

Correcting this conceptual error has a number of important implications for how analytical models can be used to detect discrimination. If what makes something discrimination on the basis of a particular social grouping is that the practice acts on what it means to be in that group in a way that we deem wrongful, then what we need from analytical diagrams is a model of what constitutes the social grouping. Such a model would allow us to explain the special moral (and legal) reasons we have to be concerned with the treatment of this category by reference to the empirical social relations and meanings that establish the category as what it is. Only then can we have the normative debate about what is fair or nondiscriminatory vis-à-vis that group. We suggest that formal diagrams of constitutive relations would present an entirely different path toward reasoning about discrimination (and relatedly, counterfactuals) because they proffer a model of how the meaning of a social group emerges from its constitutive features. Whereas the value of causal diagrams is to guide the construction and testing of sophisticated modular counterfactuals, the value of constitutive diagrams would be to identify a different kind of counterfactual as central to our inquiry into discrimination: one that asks how the social meaning of a group would be changed if its non-modular features were altered.