Facebook yesterday said it disabled 3.4 billion fake accounts between October last year and March this year. The company disclosed this in its biannual community standards report, which is a summary how much and what content it moderated. Of the 3.4 billion fake accounts removed, 1.2 billion were removed in Q4 2018, and 2.19 billion in Q1 2019.

Prevalence rate: Facebook said the prevalence rate of content that violated its adult nudity and sexual activity policy was 11-14 views out of 10,000 views. Prevalence rate measures violative content which Facebook hasn’t identified yet and people may still see. It’s measured by sampling content reviewed on Facebook and reviewing it to see the percentage that violates its community standards.

  • 11-14 views out of 10,000 views contained content which violated its adult nudity and sexual activity policy, which it said is below 1% of the content on Facebook is violative content
  • 25 out of 10,000 views contained content which violated Facebook’s violence and graphic content policy.
  • 5% of Facebook’s monthly active users are fake

Content acted upon means the how much content was violative and how well Facebook could identify such content. The actions include removing the content, applying a warning screen to it, or disabling accounts. Facebook disabled 1.2 billion accounts in Q4 2018 and 2.19 billion in Q1 2019.

Proactive Rate: This measures how accurate AI is in detecting violative content, and basically indicates how much of violative content was detected. Facebook claims that it proactively detected 95% of the content before it was reported. 65% of hate speech content was similarly detected, up from 24% about a year ago, it claimed. In Q1 2019, Facebook took down 4 million hate speech posts.  Facebook’s Guy Rosen explained that the proactive rate is for overall numbers across languages, including widespread languages like Spanish and English and smaller languages like Dutch. He said the company can proactively detect content in 40 languages and is working to increase this number.

What was taken down, how was much proactively detected, and more notes

Adult nudity and sexual activity

Facebook said nudity can be shared when it’s a form of protest, to raise awareness, and for educational or medical reasons. “We default to removing sexual images to prevent non-consensual or underage content from being shared. This section does not include violations on content that promotes sexual assault, violence and exploitation,” it explained.

  • Prevalence of adult nudity and sexual activity violations on Facebook fluctuated in the last six months, and declined in Q4 2018 before seeing an uptick again in Q1 2019.
  • Proactive rate: 96% of the content was detected automatically. Facebook claims that the prevalence of such content was no more than 0.15% at any time from October 2017 to March 2019.
  • Content acted upon: It took action on a roughly 20 million posts in Jan-March 2019, with the number peaking at 34.8 million in April-June 2018. It further claimed that at least 94% of the content was removed proactively between October 2017 and March 2019.

Bullying and Harassment

Facebook said it cannot estimate the prevalence of bullying and harassment content as the method for calculating this has to be different from that for other kinds of violative content. This, it said, it because such content is rooted in language and context, and depends on personal relationships. They require user reports, rather than proactive identification.

  • Content acted upon: Facebook acted on 2.6 million such pieces of content in Jan-March 2019.

Child Nudity and Sexual Exploitation of Children

Facebook ‘acts’ on content which includes nonsexual nude photos or exploitative content involving children, among other things.

  • Prevalence: Less than 0.03% or 3 out of every 10,000 views contained violative content of this kind.
  • Content taken action upon: 5.4 million pieces in Jan-March, 2019; 8.7 million in July-Sept 2018
  • Proactive rate: At least 99.2% from July 2018 to March 2019

Fake accounts

Facebook says it’s seen a “steep increase” in the creation of fake account in the last six months, and that it catches most accounts within minutes of registration. Automated attacks also increased in the period. On being asked who the bad actors behind fake accounts were, and whether it was election related, Guy Rosen said Facebook doesn’t have specific attribution, but that it detects such accounts by blocking ranges of IP addresses, which stopped bad actors from creating accounts.

  • Prevalence: 5% of Facebook’s worldwide MAUs during Q1 2019 and Q4 2018 were fake accounts
  • Content acted upon: More fake accounts were removed in Q1 2019 than in the quarter before that due to an increase in automated scripted attacks. “The majority of these accounts were caught within minutes of registration, before they became a part of our monthly active user (MAU) population.”
  • Proactive rate: Claimed to be at least 99% from October 2017 to March 2019

Hate speech

Facebook defines hate speech as violent or dehumanising statements of inferiority, or calls for exclusion or segregation based on protected characteristics. These characteristics include race, ethnicity, national origin, religious affiliation, sexual orientation, caste, sex, gender, gender identity, and serious disability or disease. This is another kind of violative content for which Facebook cannot estimate prevalence, since its team is “slowly expanding” to cover more languages and religions, and to account for cultural context and nuances in individual languages. “We are still developing a global metric, although our detection and enforcement of hate speech is very broad across the world.”

  • Content acted upon: Steadily increased to 4 million pieces of content in Jan-March 2019 from 1.6 million in Oct-Dec 2017
  • Proactive rate: One of the first areas in which Facebook doesn’t claim a staggering proactive. The rate was only 23.6% in Oct-Dec 2017, and has increased over the quarters to a high of 65% in Jan-March 2019

Regulated goods

Facebook said that in 2018 it began to use AI to detect violative content regarding regulated goods such as drugs and firearms. Again, Facebook claims that in Q1 2019, it “took action” on 900,000 pieces of drug sale content, and even detected 83% of this without any reports. In the same period, it took action on 670,000 pieces of firearm sale content, of which 69% was detected proactively.

On being asked how Facebook will deal with such content as it moves to a new privacy- and encryption-focused platform,  Rosen said, “There is a tradeoff between protecting privacy and protecting safety and that’s something that every society grapples with.” The company, he said, believes that encryption is “an incredibly powerful tool”. Rosen said Facebook has started to work on detecting bad actors through patterns and user reports, just as it works on WhatsApp. CEO Mark Zuckerberg also said he realises that privacy is going to make it harder to find harmful content: “We’ll be fighting that battle without one of the very important tools which is, of course, being able to look at the content itself” but he thinks it’s a tradeoff between privacy and security.

Spam

Spam can be automated via bots or scripts, or coordinated by actors using multiple accounts to spread disinformation.

  • Prevalence: Not estimated yet but Facebook, which is working on new methods, since existing methods rely heavily on manual reviewers.
  • Content acted upon: was the highest in the two quarters, Oct-Dec 2018 and Jan-Mar 2019, at 1.8 billion each quarter. Facebook said content acted upon had risen: “Spammers are adversarial and they routinely try new tactics to avoid our detection”.
  • Proactive rate: Facebook claims almost a 100% proactive rate from October 2017 to March 2019

“But a lot of the harmful content that we see, even including things like misinformation that are not as obviously commercially motivated are actually in fact commercially motivated. So one of the best tactics for removing and preventing that downstream is, if you can eliminate the fake accounts or remove the incentive to create fake accounts upstream then a lot of this harmful content doesn’t even get created in the first place.”
– Mark Zuckerberg

Terrorist propaganda

  • Prevalence: Facebook said this is very infrequent and hasn’t been calculated yet since a substantial sample size is necessary. Its estimates show a prevalence rate of 0.03% or 3 views in every 10,000 views in Q1 2019.
  • Content acted upon: 6.4 million pieces of content in Jan-Mar 2019; was the highest in Apr-Jun 2018 at 9.4 million pieces
  • Proactive rate: Claimed to be nearly 100% since January 2018 to March 2019

Violence and graphic content

  • Prevalence: The highest its been is 0.23% in Jul-Sep 2018 and in Jan-Mar 2019
  • Content acted upon: Increased from 18.7 million in Oct-Dec 2018 to 33.6 million in Jan-Mar 2019, primarily due to automated system detection in the period.
  • Proactive rate: Increased from roughly 70% in Oct-Dec 2017 to 98.9% in Jan-Mar 2019