FAIRNESS IN THE MACHINE LEARNING PIPELINE: A HUMAN-IN-THE-LOOP PERSPECTIVE

Machine learning (ML) models are now ubiquitous in many fields, fitting into everyday routines and essential industrial processes. However, the human-led world is inherently marked by bias, and unfortunately, this bias can infiltrate ML models, leading them to act in ways that may not be fair or just. To better understand bias and fairness issues in ML models, adopting a human-in-the-loop perspective is essential, as humans play a key role in the development of the ML pipeline, appearing at its various stages. For instance, people can serve as annotators contributing their data for tasks like data labeling or influence the model through their daily decisions, such as responding to specific situations in life, which are then compiled into datasets to train these models. Subsequently, once these models are deployed in decision-making scenarios, such as determining loan approvals, they make crucial decisions on people, who then take the role of interacting with the models in more nuanced and strategic ways. If we examine the human roles in the ML pipeline more closely, we find that in each role they assume, humans significantly affect bias and fairness issues in the development of the ML pipeline. This is because bias and fairness concerns both originate from and directly impact humans themselves, and their strategic reactions to ML models can further influence the long-term fairness and biases of these models. Consequently, understanding and addressing human biases throughout the ML model development cycle, particularly at each stage of the pipeline itself, is essential. Careful monitoring of the models' statuses and tackling their stage-specific biases can allow us to identify and reduce the potential for unfair decisions, thereby ensuring the fair implementation of ML models. Therefore, this dissertation examines the ML pipeline through a human-in-the-loop lens to better understand and model how human roles contribute to fairness issues within ML models.

In this dissertation, I present my findings on the comprehension and modeling of human behavior and biases within various stages of the ML pipeline. First, in the data annotation/collection stage, my exploration reveals that human cognitive biases, such as confirmation bias, will significantly affect the quality of the training data. Building on this insight, I devised a bias-aware algorithm that directly models this bias. The algorithm has demonstrated effectiveness in inferring ground-truth labels for the training data, especially when subjects exhibit polarized values. Second, in the deployment stage, where ML models are implemented and begin making decisions on people, I investigate how decision subjects (i.e., the people who are subject to ML models' decisions) strategically react to an ML-based system's fairness across groups or favoritism towards certain groups, particularly within the context of loan lending. This research uncovers the ways in which decision subjects' perceptions of fairness and engagement with the system are influenced by the fairness properties of the ML models. Largely, the results unveil that decision subjects' behavior seems to be more influenced by the system's favoritism towards their own group rather than the system's fairness across groups. Finally, I investigate the feedback loop that emerges from the combination of multiple stages—specifically, the deployment stage, where ML models are implemented and make decisions about people, the behavioral data stage, where individuals react to these decisions, and the model update stage, where models are retrained using that behavioral data. In this loop, I examine how fairness evolves over time in ML models that are repeatedly updated using behavioral data that is generated as a reaction to models' previous decisions. Through a simulation study, I model interactions between decision subjects and ML models in a task assignment setting managed by the models themselves. The results reveal how initial fairness can degrade due to feedback loops formed by biased feedback and subject reactions—such as reduced effort or early disengagement in response to perceived unfairness. I further validate these findings through a human-subject experiment in a setting where subjects interact with a real ML model that assigns tasks. Notably, the results reveal that seemingly intuitive heuristics—such as compensating individuals with more tasks after they receive fewer due to unfair treatment—can slow the growth of disparities, but only to a limited extent if the model continues to rely on biased feedback, ultimately exacerbating disparities. Overall, this dissertation explores the human-in-the-loop perspective of the ML pipeline, identifying key areas where bias can be introduced or perpetuated. It emphasizes the importance of carefully examining and controlling each stage to ensure the development of fair ML models in their decision-making. These findings contribute valuable insights into the design of more responsible and equitable ML systems, highlighting the essential role of understanding and addressing human behavior and biases throughout the pipeline.