Bias in Team Member Evaluations: A Dual Perspective of Students and Mentors

Despite educators implementing bias-reducing strategies, accurately evaluating team members’ individual contributions remains a challenge. This study investigated the extent of bias in team member evaluations within a design thinking course. Self-, peer, and undergraduate teaching assistants (mentors) used the Comprehensive Assessment for Team Member Effectiveness (CATME) tool to rate team members based on their contributions to a capstone project.

To measure alignment between self-, peer, and mentor ratings, statistical analyses such as Kendall’s W Coefficient of Concordance and Spearman’s rank correlations were performed to assess interrater reliability. Other non-parametric tests were used to analyze the ratings’ correlations with grades and demographic data. The analyses revealed moderate alignment between peer and mentor ratings, which suggested that external perspectives were consistent and reliable measures of contributions. Self-ratings had weaker alignment with peer and mentor ratings, indicating a disconnect between students’ internal self-perception and how their teams and mentor saw them contribute. Although there were no statistically significant relationships between ratings and grades, a negative relationship between self-ratings and grades was found, which suggested that students who inflated self-evaluations performed slightly worse on assignments. Further analyses split by course section exposed how demographic factors influenced mentor ratings, illuminating hidden biases that were veiled in the pooled dataset.

The findings from this study highlight the importance of refining bias-reducing strategies. The objective was to enhance the fairness and accuracy of team member evaluations, thereby reducing frustration from unfair grading and improving the educational experience and outcomes for students engaged in collaborative projects.