How Facebook helped kill 6,700 people in one month
and other true stories from the dark side of Big Tech
One last trigger warning before we begin: This week’s letter features stories about attempted suicide, stochastic terrorism, and outright genocide enabled by big tech.
•
•
•
•
•
Bobbi Duncan was a young closeted lesbian. She meticulously adjusted various byzantine settings on Facebook to keep her sexual orientation private and shrouded from her conservative parents. When she was a freshman she joined The Queer Chorus at UT-Austin. The chorus director added her to their Facebook group. Unbeknownst to Bobbi, the director, or any of us at the time, Facebook had made a decision that group privacy settings should override personal privacy settings.
Bobbi’s parents received notifications about Bobbi’s affiliation with the group. Bobbi was disowned by her parents and attempted suicide. Mike Monteiro brought Bobbi’s story up to an audience at Facebook and engineers there insisted the chorus director was at fault not them.
Let’s talk about Facebook’s role in the genocides in Myanmar…
In 2014 less than 1% of Myanmar’s residents had Internet access.
By 2016 there were more FB users in Myanmar than any other South Asian country. It became the default news source for a nation riddled with religious and ethnic animosity.
By Aug-Sept 2017 hate speech on the platform reached critical levels.
In that same span of time more than 6,700 Rohingya Muslims were killed by Buddhist citizenry and military. More deaths followed as did looting, gang rapes, and other forms of sexual violence.
At least 730 young children were among the people shot, burned or beaten to death.
650,000 Rohingya refugees escaped to Bangladesh to avoid this genocide.
In 2015 there were only 2 outsourced content moderators that could speak Burmese to cover the whole region. As a result they counted on users to report problematic posts. The interface for doing so was in English.
They also counted on Facebook’s translation services. In Burmese, one post said: “Kill all the kalars that you see in Myanmar; none of them should be left alive.” Facebook’s translation into English read as: “I shouldn’t have a rainbow in Myanmar.”
Olivia Solon said “Facebook's failure in Myanmar is the work of a blundering toddler.” That’s obviously being kind. It’s the work of bloodthirsty assholes that are criminally negligent and callous in their indifference despite years of warnings. It brings to mind Grey’s Law. It states, “Any sufficiently advanced incompetence is indistinguishable from malice.”
James Bridle did a fantastic investigation into the torrent of nightmarish content aimed at kids on YouTube. “These videos, wherever they are made, however they come to be made, and whatever their conscious intention (i.e. to accumulate ad revenue) are feeding upon a system which was consciously intended to show videos to children for profit. The unconsciously-generated, emergent outcomes of that are all over the place. The system is complicit in the abuse.”
China is using drones disguised as pigeons and gait analysis to surveil their citizenry. Australia is already underway of their launch of “The Capability”, a powerful database that pools biometric information gleaned from driver’s licences, passports, visas and other sources — despite radical concerns about the effectiveness of the matching which some say are 93% wrong in some cases. The City of Baltimore has an on-again, off-again relationship with a company called Persistent Surveillance Systems that uses wide-area aerial drones in an almost pre-crime fashion.
Clayton Delery, in his book “Out for Queer Blood” defines stochastic terrorism as “The use of mass, public communication, usually against a particular individual or group, which incites or inspires acts of terrorism which are statistically probable but happen seemingly at random.” It’s a statistician’s way of saying what we all know to be true — when vile people like Donald Trump spew vile rhetoric people like the convicted Maga Bomber will take vile action.
I thought I’d let Barack Obama share my next example…
You thought fake news was bad? Wait until deepfakes are common.
You can’t have a rant about dystopia without some nod to either Huxley or Orwell right? Huxley said,
In regard to propaganda the early advocates of universal literacy and a free press envisaged only two possibilities: the propaganda might be true, or the propaganda might be false. They did not foresee what in fact has happened, above all in our Western capitalist democracies — the development of a vast mass communications industry, concerned in the main neither with the true nor the false, but with the unreal, the more or less totally irrelevant. In a word, they failed to take into account man’s almost infinite appetite for distractions.
Fun, amirite?
File under: #dystopia #surveillance #hatespeech #nightmarealgorithms #deepfakes
Next week: What can we do to fix things?