Last month, people shared several horrific videos on Facebook of Syrian children in the aftermath of a chemical weapons attack. The videos, which also appeared elsewhere on the internet, showed the children shaking, struggling to breathe and eventually dying.
The images were deeply shocking – so much so that we placed a warning screen in front of them and made sure they were only visible to adults. But the images also prompted international outrage and renewed attention on the plight of Syrians.
The Guardian’s reporting on how Facebook deals with difficult issues/images such as this gets a lot of things right. Reviewing online material on a global scale is complex, challenging and essential. The articles and the training materials published alongside this show just how hard it can be to identify what is harmful – and what is necessary to allow people ability to share freely. As the person in charge of doing this work for Facebook, I want to explain how and where we draw the line.
On an average day, more than a billion people will use Facebook. They will share posts in dozens of languages: everything from photos and status updates to live videos. A very small percentage of those will be reported to us and investigated by our moderation teams. The range of issues is broad – from bullying and hate speech to terrorism and war crimes – and complex. Designing policies that both keep people safe and enable them to share freely means understanding emerging social issues and the way they manifest themselves online, and being able to respond quickly to millions of reports a week from people all over the world.
For our reviewers, there is another hurdle: understanding context.
It’s hard to judge the intent behind one post, or the risk implied in another. Someone posts a graphic video of a terrorist attack. Will it inspire people to emulate the violence, or speak out against it?
Someone with a dark sense of humour posts a joke about suicide. Are they just being themselves, or is it a cry for help?
Cultural context is part of it too. In the UK, being critical of the monarchy might be acceptable. In some parts of the world it will get you a jail sentence. It’s easy to comply with a clear-cut law, but most of the time what’s acceptable is more about norms and expectations. Social attitudes are constantly evolving, and every society has its flash points. New ways to tell stories and share images can bring these tensions to the surface faster than ever.
Our approach is to try to set policies that keep people safe and enable them to share freely. We aim to remove any credible threat of violence, and we respect local laws. We don’t always share the details of our policies, because we don’t want to encourage people to find workarounds – but we do publish our Community Standards, which set out what is and isn’t allowed on Facebook, and why.
Our standards change over time as our community grows and social issues around the world evolve. We are in constant dialogue with experts and local organisations, on everything from child safety to terrorism to human rights.
Sometimes this means our policies can seem counter-intuitive. As the Guardian reported, experts in self-harm advised us that it can be better to leave live videos of self-harm running so that people can be alerted to help, but to take them down afterwards to prevent copycats.
Sometimes this is not enough to prevent tragedy, but sometimes it is.
When a girl in Georgia, USA, attempted suicide on Facebook Live two weeks ago, her friends were able to notify police, who managed to reach her in time. We are aware of at least another half-dozen cases like this from the past few months.
We also try hard to stay objective. The cases we review aren’t the easy ones: by definition something is reviewed when it falls within a grey area. Art and pornography aren’t always easily distinguished, but we’ve found that digitally generated images of nudity are more likely to be pornographic than handmade ones, so our policy reflects that.
There’s a big difference between general expressions of anger and specific calls for a named individual to be harmed, so we allow the former but don’t permit the latter.
These tensions – between raising awareness of violence and promoting it, between freedom of expression and freedom from fear, between bearing witness to something and gawking at it – are complicated, philosophical questions. Many organisations grapple with them, and there are rarely universal legal standards to provide clarity. Being as objective as we can is the only way we can be consistent across the world and in different contexts. But we still sometimes end up making the wrong call.
We appreciate the Guardian showing how tough it can be to draw the lines. The hypothetical situations we use to train reviewers are intentionally extreme. They’re designed to help the people who do this work deal with the most difficult cases, and they reflect the real problems they deal with every day. When we first created our content standards nearly a decade ago, much was left to the discretion of individual employees. But because no two people will have identical views of what defines hate speech or bullying – or any number of other issues – we soon evolved our standards to include clear definitions.
These definitions are deliberately precise, designed to achieve consistent decisions across the globe, whether the person reviewing the posts grew up in India or Brazil or the UK.
As the article noted, we face criticism from people who want more censorship and people who want less. We see that as a useful signal that we are not leaning too far in any one direction. The alternative – doing nothing and allowing anything to be posted – is not what our community want
We get things wrong, and we’re constantly working to make sure that happens less often. We put a lot of detailed thought into trying to find right answers, even when there aren’t any.
I hope that readers will understand that we take our role extremely seriously. For many of us on the team within Facebook, safety is a passion that predates our work at the company: I spent more than a decade as a criminal prosecutor in the United States and Thailand, investigating everything from child sexual exploitation to violent gangs to terrorism. Our team also includes a counter-extremist expert from the UK’s Institute for Strategic Dialogue, the former research director of West Point’s Combating Terrorism Center, a rape crisis centre worker, and a teacher.
‘No grey areas’: experts urge Facebook to change moderation policies
Labour’s Yvette Cooper is among those calling for more transparency from the company in wake of Guardian revelations
All of us know there is more we can do. Last month, we announced that we were hiring an extra 3,000 reviewers around the world to review what’s posted on Facebook. This is demanding work, and we will continue to do more to ensure we are giving them the right support, both by making it easier to escalate hard decisions quickly and by providing the psychological support and care they need.
We are making these investments because it’s the right thing to do and because we believe that the ability to share is worth protecting.
Technology has given more people more power to communicate more widely than ever before. We believe the benefits of sharing far outweigh the risks. But we also recognise that society is still figuring out what is acceptable and what is harmful, and that we, at Facebook, can play an important part of that conversation.