Evaluating the Forces Shaping the Trust & Safety Industry

Sujata Mukherjee heads the Trust & Safety research team at Google. Jan Eissfeldt serves as Director and Global Head of Trust & Safety at the Wikimedia Foundation, the non-profit hosting Wikidata, Wikipedia, and other free knowledge projects. Shutterstock Trust & Safety policymaking is becoming a multi-dimensional balancing act: between sweeping “universalization” of global policies and excessive “balkanization” of and within individual countries/locales; between scalable, federated moderation processes and distributed, community-reliant systems; between business interests and societal impact. Compounding this complexity is the issue of measurement: the impact of Trust & Safety policies is often indirect, can take time to materialize and unfolds across complex, entangled systems. In this commentary, we review the forces shaping the industry’s balancing act and the challenge of building agile, resilient systems that are future-ready. A new field emerges Trust & Safety (T&S) is an emerging field in the tech industry that has grown rapidly in recent years. T&S is used to refer to the teams and corporate function(s) at internet companies and service providers dedicated to ensuring that users are protected from unwanted or harmful experiences. Key functions include Policy, which is the team developing the company’s rules to govern its products, platforms and services, and Enforcement or Operations, the teams that work on ensuring that said rules are followed. Larger platform providers usually have whole ecosystems of support functions for these two foundational elements, including engineering and machine learning teams that build tools and models used for enforcement; data science professionals helping with metrics…Evaluating the Forces Shaping the Trust & Safety Industry