As the pandemic pushed more people to communicate and express themselves online, algorithmic content moderation systems have had an unprecedented impact on the words we choose, particularly on TikTok, and led to a new form of internet-driven Aesopian language.
Unlike other mainstream social platforms, the main way content is distributed on TikTok is through an algorithmically curated “For You” page; having followers is a secondary metric that does not guarantee that people will see your content. This shift has resulted in average users primarily targeting their content to the algorithm, rather than to a subsequent one, meaning abiding by content moderation rules is more important than ever.
When the pandemic broke out, people on TikTok and other apps started referring to it as the “Backstreet Boys reunion tour‘ or calling it the ‘panini’ or ‘panda express’ as platforms have downgraded videos mentioning the pandemic by name in an effort to combat misinformation. When young people started talking about struggling with mental health, they talked about “going alive” to have candid conversations about suicide without algorithmic punishment. Sex workers, long censored by moderation systems, call themselves “accountants” on TikTok and use the corn emoji as a replacement for the word “porn.”
As discussions about major events are filtered through algorithmic content delivery systems, more users are bending over to their language. Recently, when discussing the invasion of Ukraine, people on YouTube and TikTok have used the sunflower emoji to denote the country. When encouraging fans to follow them elsewhere, users say “blink in lio” for “link in bio”.
Euphemisms are especially common in radicalized or harmful communities. Pro-anorexia eating disorder communities have long adopted variations of moderated words to get around restrictions. A paper from the School of Interactive Computing, Georgia Institute of Technology found that the complexity of such variants actually increased over time. Last year, anti-vaccine groups on Facebook started changing their names to “dance party” or “dinner party,” and anti-vaccine influencers on Instagram used similar codewords, referring to vaccinated people as “swimmers.”
Changing language to avoid scrutiny predates the internet. Many religions have avoided pronouncing the devil’s name or they would summon him, while people living in repressive regimes developed code words to discuss taboo topics.
Early Internet users used alternate spellings or “leetspeak” to bypass word filters in chat rooms, picture boards, online games, and forums. But algorithmic content moderation systems are ubiquitous on the modern internet, ultimately silencing marginalized communities and important discussions.
During YouTube’s “adpocalypse” in 2017, when advertisers withdrew their dollars from the platform for fear of unsafe content, LGBTQ creators talked about demonizing videos for saying the word “gay.” Some started using the word less or replacing others to monetize their content. More recently, users on TikTok have started saying “horn of plenty” instead of “homophobia”, or saying that they are members of the “leg booty” community to indicate that they are LGBTQ.
“There’s a line we have to follow. It’s an endless battle to say something and get the message across without saying it directly,” said Sean Szolek-VanValkenburgh, a TikTok creator with more than 1.2 million followers. . “It has a disproportionate impact on the LGBTQIA community and the BIPOC community because we are the people who create that verbiage and come up with the colloquia.”
Conversations about women’s health, pregnancy and menstrual cycles on TikTok are also consistently ranked lower, said Kathryn Cross, a 23-year-old content creator and founder of Anja Health, a start-up that offers cord blood banking. She replaces the words for “sex”, “period” and “vagina” with other words or spells them with symbols in the captions. Many users say “nip nops” instead of “nipples”.
“It makes me feel like I need a disclaimer because I feel like you seem unprofessional to have these oddly spelled words in your captions,” she said, “especially for content that’s supposed to be serious and medically inclined. “
Because online algorithms often mark content that mentions certain words, without context, some users avoid pronouncing them altogether simply because they have an alternative meaning. “You have to say ‘saltines’ when you’re literally talking about crackers right now,” says Lodane Erisian, a community manager for Twitch creators (Twitch considers the word “cracker” a slur). Twitch and other platforms have even gone as far as removing certain emotes because people used them to convey certain words.
Black and trans users, and those from other marginalized communities, often use algospeak to discuss the oppression they face, swapping words for “white” or “racist.” Some are too nervous to pronounce the word “white” and simply hold their palms to the camera to signify white people.
“The reality is that tech companies have been using automated tools to moderate content for a long time and while it’s touted as this advanced machine learning, it’s often just a list of words they think are problematic,” said Ángel Díaz, lecturer at the UCLA School of Law that studies technology and racial discrimination.
In January, linguists Kendra Calhoun and Alexia Fawcett gave a TikTok presentation for an event hosted by the Linguistic Society of America on language. They outlined how, through self-censorship in TikToks’ captions, new algospeak code words emerged.
TikTok users now use the phrase “le dollar bean” instead of “lesbian” because it’s the way TikTok’s text-to-speech feature pronounces “Le$bian”, a censored way of writing “lesbian” that users think it will bypass content moderation.
Algorithms allow human language to be routed around them in real time. I listen to this youtuber say things like “the bad guy didn’t revive his followers” because words like “kill” are associated with demonetization
— badidea 🪐 (@ 0xabad1dea) December 15, 2021
Evan Greer, director of Fight for the Future, a digital rights nonprofit, said using specific words on platforms is a fool’s errand.
“First, it doesn’t really work,” she said. “The people who use platforms to orchestrate real damage are pretty good at figuring out how to get around these systems. And two, it leads to collateral damage to literal speech.” Attempting to regulate human speech on a scale of billions of people in dozens of different languages and trying to deal with things like humor, sarcasm, local context and slang cannot be made by simply tearing down certain words, Greer argues.
“I feel like this is a great example of why aggressive moderation will never really solve the damage we’re seeing from the business practices of big tech companies,” she said. “You can see how slippery this slope is. Over the years, we’ve seen more and more of the misguided public demand for platforms to remove more content quickly, regardless of cost.”
Major TikTok makers have created shared Google docs containing lists of hundreds of words that the app’s moderation systems say are problematic. Other users keep a running count of terms they believe have throttled certain videos in an attempt to reverse engineer the system.
“Zuck Got Me For”, a site created by a meme account manager who goes by Ana, is a place where creators can upload nonsensical content banned by Instagram’s moderation algorithms. In a manifesto about her project, she wrote, “Creative freedom is one of the few silver linings of this flaming online hell we all exist in… As the algorithms tighten up, it’s the independent creators who suffer.”
She also outlines how to speak online in a way to bypass filters. “If you’ve violated the terms of service, you may not be allowed to use swear words or negative words like ‘hate’, ‘kill’, ‘ugly’, ‘stupid’, etc.,” she said. “I often write: ‘I opposite of love xyz’ instead of ‘I hate xyz.’”
The Online Creators’ Association, a labor advocacy group, also released a list of public demands asking TikTok for greater transparency in how it moderates content. “People need to tone down their own language to avoid offending these all-seeing, all-knowing TikTok gods,” said Cecelia Gray, a TikTok creator and co-founder of the organization.
TikTok provides an online resource center for creators who want to learn more about its recommendation systems, and has also opened multiple transparency and accountability centers where guests can learn how the app’s algorithm works.
Vince Lynch, chief executive of IV.AI, an AI platform for understanding language, said that in some countries where moderation is heavier, people end up constructing new dialects to communicate. “They’re going to be real sub-languages,” he said.
But as algospeak becomes more popular and replacement words turn into common slang, users find themselves having to get more and more creative to get around the filters. “It turns into a game of whack-a-mole,” said Gretchen McCulloch, a linguist and author of “Because Internet,” a book about how the Internet shaped language. For example, when the platforms start to notice that people say “seggs” instead of “sex”, some users report that they think even replacement words are being flagged.
“Eventually we create new ways of speaking to avoid this kind of moderation,” said Díaz of the UCLA School of Law, “and eventually we embrace some of these words and they become common vernacular. It all grew out of this effort to to resist moderation.”
This does not mean that all efforts to eradicate bad behavior, harassment, abuse and misinformation are fruitless. But Greer argues that it is the core issues that should be prioritized. “Aggressive moderation will never really solve the damage we see in the business practices of large tech companies,” she said. “That’s a job for policy makers and for building better things, better tools, better protocols and better platforms.”
Finally, she added, “You will never be able to purge the internet.”