AN EMPIRICAL STUDY OF TRUST & SAFETY ENGINEERING IN OPEN-SOURCE SOCIAL MEDIA PLATFORMS

Social Media Platforms (SMPs) are used by almost 60% of the global population. Along with the ubiquity of SMPs, there are increasing Trust & Safety (T&S) risks that expose users to spam, harassment, abuse, and other harmful content online. T&S Engineering is an emerging area of software engineering striving to mitigate these risks. This study provides the first step in understanding this form of software engineering.

This study examines how T&S Engineering is practiced by SMP engineers. I studied two open-source (OSS) SMPs, Mastodon and Diaspora, which comprise 89% of the 9.6 million OSS SMP accounts. I focused on the T&S design process by analyzing T&S discussions within 60 GitHub issues. I applied a T&S discussion model to taxonomize the T&S risks, T&S engineering patterns, and resolution rationales. I found that T&S issues persist throughout a platform’s lifetime, they are difficult to resolve, and engineers favor reactive treatments. To integrate findings, I mapped T&S engineering patterns onto a gen- eral model of SMPs. My findings give T&S engineers a systematic understanding of their T&S risk treatment options. I conclude with future directions to study and improve T&S Engineering, spanning software design, decision-making, and validation.