content moderation
Auto Added by WPeMatico
TikTok brings in outside experts to help it craft moderation and content policies
In October, TikTok href=”https://techcrunch.com/2019/10/15/tiktok-taps-corporate-law-firm-kl-gates-to-advise-on-its-u-s-content-moderation-policies/”> tapped corporate law firm K&L Gates to advise the company on its moderation policies and other topics afflicting social media platforms. As a part of those efforts, TikTok said it would form a new committee of experts to advise the business on topics like child safety, hate speech, misinformation, bullying and other potential problems. Today, TikTok is announcing the technology and safety experts who will be the company’s first committee members.
The committee, known as the TikTok Content Advisory Council, will be chaired by Dawn Nunziato, a professor at George Washington University Law School and co-director of the Global Internet Freedom Project. Nunziato specializes in free speech issues and content regulation — areas where TikTok has fallen short.
“A company willing to open its doors to outside experts to help shape upcoming policy shows organizational maturity and humility,” said Nunziato, of her joining. “I am working with TikTok because they’ve shown that they take content moderation seriously, are open to feedback and understand the importance of this area both for their community and for the future of healthy public discourse,” she added.
TikTok says it plans to grow the committee to around a dozen experts in time.
According to the company, other committee members include:
Rob Atkinson, Information Technology and Innovation Foundation, brings academic, private sector, and government experience as well as knowledge of technology policy that can advise our approach to innovation
Hany Farid, University of California, Berkeley Electrical Engineering & Computer Sciences and School of Information, is a renowned expert on digital image and video forensics, computer vision, deep fakes, and robust hashing
Mary Anne Franks, University of Miami Law School, focuses on the intersection of law and technology and will provide valuable insight into industry challenges including discrimination, safety, and online identity
Vicki Harrison, Stanford Psychiatry Center for Youth Mental Health and Wellbeing, is a social worker at the intersection of social media and mental health who understands child safety issues and holistic youth needs
Dawn Nunziato, chair, George Washington University Law School, is an internationally recognized expert in free speech and content regulation
David Ryan Polgar, All Tech Is Human, is a leading voice in tech ethics, digital citizenship, and navigating the complex challenge of aligning societal interests with technological priorities
Dan Schnur, USC Annenberg Center on Communication and UC Berkeley Institute of Governmental Studies, brings valuable experience and insight on political communications and voter information
Nunziato’s view of TikTok — of a company being open and willing to change — is a charitable one, it should be said.
The company is in dangerous territory here in the U.S., despite its popularity among Gen Z and millennial users. TikTok today is facing a national security review and a potential ban on all government workers’ phones. In addition, the Dept. of Defense suggested the app should be blocked on phones belonging to U.S. military personnel. Its 2017 acquisition of U.S.-based Musical.ly may even come under review.
Though known for its lighthearted content — like short videos of dances, comedy and various other creative endeavors — TikTok has also been accused of things like censoring the Hong Kong protests and more, which contributed to U.S. lawmakers’ fears that the Chinese-owned company may have to comply with “state intelligence work.”
TikTok has also been accused of having censored content from unattractive, poor or disabled persons, as well as videos from users identified as LGBTQ+. The company explained in December these guidelines are no longer used, as they were an early and misguided attempt to protect users from online bullying. TikTok had limited the reach of videos where such harassment could occur. But this suppression was done in the dark, unasked for by the “protected” parties — and it wasn’t until exposed by German site NetzPolitik that anyone knew these rules had existed.
In light of the increased scrutiny of its platform and its ties to China, TikTok has been taking a number of steps in an attempt to change its perception. The company released new Community Guidelines and published its first Transparency Report a few months ago. It also hired a global General Counsel and expanded its Trust & Safety hubs in the U.S., Ireland and Singapore. And it just announced a Transparency Center open to outside experts who want to review its moderation practices.
TikTok’s new Advisory Council will meet with the company’s U.S. leadership to focus on the key topics of importance starting at the end of the month, with an early focus on creating policies around misinformation and election interference.
Powered by WPeMatico
Teen hit Yolo raises $8M to let you Snapchat anonymously
It wasn’t a fad. Yolo became the country’s No. 1 app just a week after launch by letting teens ask for anonymous replies to questions they posted on Snapchat. But nine months later, Yolo is still in the top 100 iOS apps and has 10 million active users. Now it’s safeguarding the app from predators while revealing a smart new feature for spinning up anonymous group chats, powered by $8 million in fresh funding.
“What we are trying to build is a new kind of network where there’s a fluidity to identity,” Yolo co-founder Greg Henrion tells me. “We weren’t sure if Yolo was here to stay, but we’re still ranking well and there seems to be a real opportunity in anonymity starting with Snapchat Q&A.”
Yolo is the first big win for Snapchat’s Snap Kit platform that lets developers piggyback on its login, Bitmoji avatars, stickers and Stories. This lets tiny development teams build apps that hundreds of millions of people, teens in particular, can instantly sign up for in just a few taps. Another Snap Kit app for meeting new people called Hoop recently spiked to No. 2 on the charts
We haven’t seen this kind of social platform success since Zynga’s empire rose atop Facebook. Spawning more blockbusters like Yolo could ensure that a Snapchat account is a must-have utility for the next generation.
Sleepless nights atop the charts
“For two weeks we basically didn’t sleep,” Henrion recalls about the chaos he and co-founder Clément Raffenoux endured after Yolo shot to No. 1 last May. “You’re trying to stay afloat. It was very, very wild.”
The basic premise of Yolo is that you write a question like, “Who’s my celebrity look alike?”, “What do people really think of me?” or “How could I be nicer?” You’re then switched over to Snapchat, where you can post the question in your Story or messages with a link back to Yolo. There, people can anonymously leave a response; you can post that and your reply with another post on Snapchat.
Yolo co-founder and CEO Greg Henrion, in real life and Bitmoji
The result is that friends and followers feel comfortable giving you real talk. They don’t have to sugarcoat their answers. And that makes people race to open Yolo each time they get a message. Yolo has seen 26 million downloads across iOS and Android globally, with nearly 70% in the U.S, according to Sensor Tower.
Other anonymous apps like tbh (acquired by Facebook) and Sarahah (kicked off the app stores) quickly faded, and others eventually imploded due to bullying, like Secret and YikYak. Although tbh hit No. 1 in September 2017, it was out of the top 500 by November. It seems a combination of inherent virality via Snapchat, easy user acquisition via Snap Kit and sharp product design has given Yolo some staying power. It still managed 2.2 million downloads last month versus a peak of 5.5 million in its first month back in May 2019.
That June, Yolo quietly raised a $2 million seed round thanks to its sudden success. The team had been grinding since 2017 on a video reactions app called Popshow funded by a small pre-seed round from SV Angel, Shrug Capital and Product Hunt’s Ryan Hoover. They’d previously built music video-making app Mindie that eventually sold to influencer collective Shots Studios. Popshow never caught on, so the team began experimenting on Snap Kit, building a more official Q&A feature for Snapchat than predecessors like Sarahah and Polly. Then, boom. Days after launch, Yolo’s usage exploded.
But to keep users interested, Yolo needed to evolve. That would require more funding for the eight-person team split between Snapchat’s home of Los Angeles and Henrion’s home of Paris.
An honest way to chat
The concept of a social app where users could shift between full anonymity and representation via avatar attracted its $8 million Series A to invest in product and engineering. The round was led by Thrive Capital, Ron Conway’s A.Capital, former TechCrunch editor Alexia Tsotsis’ Dream Machine (also in the seed round), Shrug, Day One, Goodwater, Knight VC, ex-Facebooker Bobby Goodlatte, Twitter co-founder Biz Stone and SV Angel’s Brian Pokorny.
That cash fueled the release of Yolo’s new group chat feature. You can set up a chat room, give it a name and generate an invite URL or sticker you can post on Snapchat, just like its previous question feature. Friends or friends of friends that are already in can join the group chat, represented by their Bitmoji instead of their name. Yolo suggests people join the more open “party mode” chats where their friends are active.
What makes this special is that once an hour, users can tap the Yolo Superpowers button to send a totally anonymous message to the group. More Superpowers are coming, but there’s also an anonymous “Someone has a crush on [name]” message so you can secretly profess your affection to anyone or someone else in the chat.
“The limits of Q&A is that it doesn’t generate real conversation. It’s an ice breaker, but we also want conversations to happen,” Henrion stresses. “‘What do you think about this dress?’ The group chat is more about ‘let’s talk about the dress.’” The chats could be focused on people you actually know offline, or those you share interests with. The option to restrict group chats to either just your contacts or friends of friends “limits the amount of meeting strangers,” Henrion explains. “This is very different from the public communities like Reddit or the dating apps.”
Can “anonymous” be synonymous with “safe”?
Still, anonymous apps have consistently proven to be havens for cyberbullying and unsafe behavior. Without the accountability of having your name attached, people are free to say awful things. That can be even worse amongst teenagers who might get in trouble for being mean at school but not on an app.
Yolo first focused on messages blocking 10% of overall messages that contained offensive content. That meant blatant hate speech and trolling couldn’t spread through the app. “We’re strict on moderation. When looking at the reviews about bullying, it’s like nothing compared to any other anonymous app. I think we solved 90% of the problem.”
Now it’s working with Snapchat to safeguard the group chats feature. The goal is to ensure Yolo doesn’t actively recommend chat amongst adults to minors and vice-versa. Henrion says this update should roll out soon.
“It’s 2020 and we need to be very responsible” Henrion tells me. “Moderation and growth are the most difficult things to balance. It’s moderation first for sure. We don’t care about growth if it’s not healthy or sustainable.” The new funding also gives Yolo the luxury of pushing back monetization while it focuses on safely adding more users.
By making anonymity more private, Yolo has a chance to sidestep some of the worst elements of human behavior. Making fun of someone has less appeal if there’s no wider audience like trolls exploited in the feeds and comment reels of Secret and YikYak.
That could let the brighter side of anonymity shine through: vulnerability, honesty and deep connections that are enhanced by the absence of embarrassment. With all the change, uncertainty and anxiety that’s part of growing up, teens deserve a place where they can be open with each other and speak their minds. After all, you only live once.
Powered by WPeMatico
Facebook and YouTube’s moderation failure is an opportunity to deplatform the platforms
Facebook, YouTube, and Twitter have failed their task of monitoring and moderating the content that appears on their sites; what’s more, they failed to do so well before they knew it was a problem. But their incidental cultivation of fringe views is an opportunity to recast their role as the services they should be rather than the platforms they have tried so hard to become.
The struggles of these juggernauts should be a spur to innovation elsewhere: While the major platforms reap the bitter harvest of years of ignoring the issue, startups can pick up where they left off. There’s no better time to pass someone up as when they’re standing still.
Asymmetrical warfare: Is there a way forward?
At the heart of the content moderation issue is a simple cost imbalance that rewards aggression by bad actors while punishing the platforms themselves.
To begin with, there is the problem of defining bad actors in the first place. This is a cost that must be borne from the outset by the platform: With the exception of certain situations where they can punt (definitions of hate speech or groups for instance), they are responsible for setting the rules on their own turf.
That’s a reasonable enough expectation. But carrying it out is far from trivial; you can’t just say “here’s the line; don’t cross it or you’re out.” It is becoming increasingly clear that these platforms have put themselves in an uncomfortable lose-lose situation.
If they have simple rules, they spend all their time adjudicating borderline cases, exceptions, and misplaced outrage. If they have more granular ones, there is no upper limit on the complexity and they spend all their time defining it to fractal levels of detail.
Both solutions require constant attention and an enormous, highly-organized and informed moderation corps, working in every language and region. No company has shown any real intention to take this on — Facebook famously contracts the responsibility out to shabby operations that cut corners and produce mediocre results (at huge human and monetary cost); YouTube simply waits for disasters to happen and then quibbles unconvincingly.
Powered by WPeMatico
AntiToxin sells safetytech to clean up poisoned platforms
The big social networks and video games have failed to prioritize user well-being over their own growth. As a result, society is losing the battle against bullying, predators, hate speech, misinformation and scammers. Typically when a whole class of tech companies have a dire problem they can’t cost-effectively solve themselves, a software-as-a-service emerges to fill the gap in web hosting, payment processing, etc. So along comes AntiToxin Technologies, a new startup that wants to help web giants fix their abuse troubles with its safety-as-a-service.
It all started on Minecraft. AntiToxin co-founder Ron Porat is cybersecurity expert who’d started ad blocker Shine. Yet right under his nose, one of his kids was being mercilessly bullied on the hit children’s game. If even those most internet-savvy parents were being surprised by online abuse, Porat realized the issue was bigger than could be addressed by victims trying to protect themselves. The platforms had to do more, research confirmed.
A recent Ofcom study found almost 80% of children had a potentially harmful online experience in the past year. Indeed, 23% said they’d been cyberbullied, and 28% of 12 to 15-year-olds said they’d received unwelcome friend or follow requests from strangers. A Ditch The Label study found of 12 to 20-year-olds who’d been bullied online, 42% were bullied on Instagram.
Unfortunately, the massive scale of the threat combined with a late start on policing by top apps makes progress tough without tremendous spending. Facebook tripled the headcount of its content moderation and security team, taking a noticeable hit to its profits, yet toxicity persists. Other mainstays like YouTube and Twitter have yet to make concrete commitments to safety spending or staffing, and the result is non-stop scandals of child exploitation and targeted harassment. Smaller companies like Snap or Fortnite-maker Epic Games may not have the money to develop sufficient safeguards in-house.
“The tech giants have proven time and time again we can’t rely on them. They’ve abdicated their responsibility. Parents need to realize this problem won’t be solved by these companies” says AntiToxin co-founder and CEO Zohar Levkovitz, who previously sold his mobile ad company Amobee to Singtel for $321 million. “You need new players, new thinking, new technology. A company where ‘Safety’ is the product, not an after-thought. And that’s where we come-in.” The startup recently raised a multimillion-dollar seed round from Mangrove Capital Partners and is allegedly prepping for a double-digit millions Series A.
AntiToxin’s technology plugs into the backends of apps with social communities that either broadcast or message with each other and are thereby exposed to abuse. AntiToxin’s systems privately and securely crunch all the available signals regarding user behavior and policy violation reports, from text to videos to blocking. It then can flag a wide range of toxic actions and let the client decide whether to delete the activity, suspend the user responsible or how else to proceed based on their terms and local laws.
Through the use of artificial intelligence, including natural language processing, machine learning and computer vision, AntiToxin can identify the intent of behavior to determine if it’s malicious. For example, the company tells me it can distinguish between a married couple consensually exchanging nude photos on a messaging app versus an adult sending inappropriate imagery to a child. It also can determine if two teens are swearing at each other playfully as they compete in a video game or if one is verbally harassing the other. The company says that beats using static dictionary blacklists of forbidden words.
AntiToxin is under NDA, so it can’t reveal its client list, but claims recent media attention and looming regulation regarding online abuse has ramped up inbound interest. Eventually the company hopes to build better predictive software to identify users who’ve shown signs of increasingly worrisome behavior so their activity can be more closely moderated before they lash out. And it’s trying to build a “safety graph” that will help it identify bad actors across services so they can be broadly deplatformed similar to the way Facebook uses data on Instagram abuse to police connected WhatsApp accounts.
“We’re approaching this very human problem like a cybersecurity company, that is, everything is a Zero-Day for us” says Levkowitz, discussing how AntiToxin indexes new patterns of abuse it can then search for across its clients. “We’ve got intelligence unit alums, PhDs and data scientists creating anti-toxicity detection algorithms that the world is yearning for.” AntiToxin is already having an impact. TechCrunch commissioned it to investigate a tip about child sexual imagery on Microsoft’s Bing search engine. We discovered Bing was actually recommending child abuse image results to people who’d conducted innocent searches, leading Bing to make changes to clean up its act.
AntiToxin identified publicly listed WhatsApp Groups where child sexual abuse imagery was exchanged
One major threat to AntiToxin’s business is what’s often seen as boosting online safety: end-to-end encryption. AntiToxin claims that when companies like Facebook expand encryption, they’re purposefully hiding problematic content from themselves so they don’t have to police it.
Facebook claims it still can use metadata about connections on its already encrypted WhatApp network to suspend those who violate its policy. But AntiToxin provided research to TechCrunch for an investigation that found child sexual abuse imagery sharing groups were openly accessible and discoverable on WhatsApp — in part because encryption made them hard to hunt down for WhatsApp’s automated systems.
AntiToxin believes abuse would proliferate if encryption becomes a wider trend, and it claims the harm that it causes outweighs fears about companies or governments surveiling unencrypted transmissions. It’s a tough call. Political dissidents, whistleblowers and perhaps the whole concept of civil liberty rely on encryption. But parents may see sex offenders and bullies as a more dire concern that’s reinforced by platforms having no idea what people are saying inside chat threads.
What seems clear is that the status quo has got to go. Shaming, exclusion, sexism, grooming, impersonation and threats of violence have started to feel commonplace. A culture of cruelty breeds more cruelty. Tech’s success stories are being marred by horror stories from their users. Paying to pick up new weapons in the fight against toxicity seems like a reasonable investment to demand.
Powered by WPeMatico
Facebook cracks down on opioid dealers after years of neglect
Facebook’s role in the opioid crisis could become another scandal following yesterday’s release of harrowing new statistics from the Center for Disease Control. It estimated there were nearly 30,000 synthetic opioid overdose deaths in the U.S. in 2017, up from roughly 20,000 the year before. When recreational drugs like Xanax and OxyContin are adulterated with the more powerful synthetic opioid Fentanyl, the misdosage can prove fatal. Xanax, OxyContin and other pain killers are often bought online, with dealers promoting themselves on social media including Facebook.
Hours after the new stats were reported by The New York Times and others, a source spotted that Facebook’s internal search engine stopped returning posts, Pages and Groups for searches of “OxyContin,” “Xanax,” “Fentanyl” and other opioids, as well as other drugs like “LSD.” Only videos, often news reports deploring opiate abuse, and user profiles whose names match the searches, are now returned. This makes it significantly harder for potential buyers or addicts to connect with dealers through Facebook.
However, some dealers have taken to putting drug titles into their Facebook profile names, allowing accounts like “Fentanyl Kingpin Kilo” to continue showing up in search results. It’s not exactly clear when the search changes occurred.
On some search result pages for queries like “buy xanax,” Facebook is now showing a “Can we help?” box that says “If you or someone you know struggles with opioid misuse, we would like to help you find ways to get free and confidential treatment referrals, as well as information about substance use, prevention and recovery.” A “Get support” button opens the site of The Substance Abuse and Mental Health Services Administration, a branch of the U.S. department of health and human services that provides addiction resources. Facebook had promised back in June that this feature was coming.
Facebook search results for many drug names now only surface people and video news reports, and no longer show posts, Pages or Groups, which often offered access to dealers
When asked, Facebook confirmed that it’s recently made it harder to find content that facilitates the sale of opioids on the social network. Facebook tells me it’s constantly updating its approach to thwart bad actors who look for new ways to bypass its safeguards. The company confirms it’s now removing content violating its drug policies, and it’s blocked hundreds of terms associated with drug sales from showing results other than links to news about drug abuse awareness. It’s also removed thousands of terms from being suggested as searches in its typeahead.
Prior to recent changes, buyers could easily search for drugs and find posts from dealers with phone numbers to contact
Regarding the “Can we help?” box, Facebook tells me this resource will be available on Instagram in the coming weeks, and it provided this statement:
We recently launched the “Get Help Feature” in our Facebook search function that directs people looking for help or attempting to purchase illegal substances to the SAMHSA national helpline. When people search for help with opioid misuse or attempt to buy opioids, they will be prompted with content at the top of the search results page that will ask them if they would like help finding free and confidential treatment referrals. This will then direct them to the SAMHSA National Helpline. We’ve partnered with the Substance Abuse & Mental Health Services Administration to identify these search terms and will continue to review and update to ensure we are showing this information at the most relevant times.
Facebook’s new drug abuse resource feature
The new actions follow Facebook shutting down some hashtags like “#Fentanyl” on Instagram back in April that could let buyers connect with dealers. That only came after activists like Glassbreakers’ Eileen Carey aggressively criticized the company, demanding change. In some cases, when users would report Facebook Groups’ or Pages’ posts as violating its policy prohibiting the sale of regulated goods like drugs, the posts would be removed, but Facebook would leave up the Pages. This mirrors some of the problems it’s had with Infowars around determining the threshold of posts inciting violence or harassing other users necessary to trigger a Page or profile suspension or deletion.
Facebook in some cases deleted posts selling drugs, but not the Pages or Groups carrying them
Before all these changes, users could find tons of vendors illegally selling opioids through posts, photos and Pages on Facebook and Instagram. Facebook also introduced a new ads policy last week requiring addiction treatment centers that want to market to potential patients be certified first to ensure they’re not actually dealers preying on addicts.
Much of the recent criticism facing Facebook has focused on it failing to prevent election interference, privacy scandals and the spread of fake news, plus how hours of browsing its feeds can impact well-being. But its negligence regarding illegal opioid sales has likely contributed to some of the 72,000 drug overdose deaths in America last year. It serves as another example of how Facebook’s fixation on the positive benefits of social networking blinded it to the harsh realities of how its service can be misused.
Last November, Facebook CEO Mark Zuckerberg said that learning of the depths of the opioid crisis was the “biggest surprise” from his listening tour visiting states across the U.S, and that it was “really saddening to see.”
Zuckerberg meets with Opioid crisis caregivers and the families of victims in Ohio in April 2017
Five months later, Representative David B. McKinley (R-W.VA) grilled Zuckerberg about Facebook’s responsibility surrounding the crisis. “Your platform is still being used to circumvent the law and allow people to buy highly addictive drugs without a prescription” McKinley said during Zuckerberg’s congressional hearings in April. “With all due respect, Facebook is actually enabling an illegal activity, and in so doing, you are hurting people. Would you agree with that statement?” The CEO admitted “there are a number of areas of content that we need to do a better job policing on our service.”
Yet the fact that he called the crisis a “surprise” but failed to take stronger action when some of the drugs causing the epidemic were changing hands via his website is something Facebook hasn’t fully atoned for, nor done enough to stop. The new changes should be the start of a long road to recovery for Facebook itself.
Powered by WPeMatico