cyberbullying

Auto Added by WPeMatico

AntiToxin sells safetytech to clean up poisoned platforms

The big social networks and video games have failed to prioritize user well-being over their own growth. As a result, society is losing the battle against bullying, predators, hate speech, misinformation and scammers. Typically when a whole class of tech companies have a dire problem they can’t cost-effectively solve themselves, a software-as-a-service emerges to fill the gap in web hosting, payment processing, etc. So along comes AntiToxin Technologies, a new startup that wants to help web giants fix their abuse troubles with its safety-as-a-service.

It all started on Minecraft. AntiToxin co-founder Ron Porat is cybersecurity expert who’d started ad blocker Shine. Yet right under his nose, one of his kids was being mercilessly bullied on the hit children’s game. If even those most internet-savvy parents were being surprised by online abuse, Porat realized the issue was bigger than could be addressed by victims trying to protect themselves. The platforms had to do more, research confirmed.

A recent Ofcom study found almost 80% of children had a potentially harmful online experience in the past year. Indeed, 23% said they’d been cyberbullied, and 28% of 12 to 15-year-olds said they’d received unwelcome friend or follow requests from strangers. A Ditch The Label study found of 12 to 20-year-olds who’d been bullied online, 42% were bullied on Instagram.

Unfortunately, the massive scale of the threat combined with a late start on policing by top apps makes progress tough without tremendous spending. Facebook tripled the headcount of its content moderation and security team, taking a noticeable hit to its profits, yet toxicity persists. Other mainstays like YouTube and Twitter have yet to make concrete commitments to safety spending or staffing, and the result is non-stop scandals of child exploitation and targeted harassment. Smaller companies like Snap or Fortnite-maker Epic Games may not have the money to develop sufficient safeguards in-house.

“The tech giants have proven time and time again we can’t rely on them. They’ve abdicated their responsibility. Parents need to realize this problem won’t be solved by these companies” says AntiToxin co-founder and CEO Zohar Levkovitz, who previously sold his mobile ad company Amobee to Singtel for $321 million. “You need new players, new thinking, new technology. A company where ‘Safety’ is the product, not an after-thought. And that’s where we come-in.” The startup recently raised a multimillion-dollar seed round from Mangrove Capital Partners and is allegedly prepping for a double-digit millions Series A.

AntiToxin’s technology plugs into the backends of apps with social communities that either broadcast or message with each other and are thereby exposed to abuse. AntiToxin’s systems privately and securely crunch all the available signals regarding user behavior and policy violation reports, from text to videos to blocking. It then can flag a wide range of toxic actions and let the client decide whether to delete the activity, suspend the user responsible or how else to proceed based on their terms and local laws.

Through the use of artificial intelligence, including natural language processing, machine learning and computer vision, AntiToxin can identify the intent of behavior to determine if it’s malicious. For example, the company tells me it can distinguish between a married couple consensually exchanging nude photos on a messaging app versus an adult sending inappropriate imagery to a child. It also can determine if two teens are swearing at each other playfully as they compete in a video game or if one is verbally harassing the other. The company says that beats using static dictionary blacklists of forbidden words.

AntiToxin is under NDA, so it can’t reveal its client list, but claims recent media attention and looming regulation regarding online abuse has ramped up inbound interest. Eventually the company hopes to build better predictive software to identify users who’ve shown signs of increasingly worrisome behavior so their activity can be more closely moderated before they lash out. And it’s trying to build a “safety graph” that will help it identify bad actors across services so they can be broadly deplatformed similar to the way Facebook uses data on Instagram abuse to police connected WhatsApp accounts.

“We’re approaching this very human problem like a cybersecurity company, that is, everything is a Zero-Day for us” says Levkowitz, discussing how AntiToxin indexes new patterns of abuse it can then search for across its clients. “We’ve got intelligence unit alums, PhDs and data scientists creating anti-toxicity detection algorithms that the world is yearning for.” AntiToxin is already having an impact. TechCrunch commissioned it to investigate a tip about child sexual imagery on Microsoft’s Bing search engine. We discovered Bing was actually recommending child abuse image results to people who’d conducted innocent searches, leading Bing to make changes to clean up its act.

AntiToxin identified publicly listed WhatsApp Groups where child sexual abuse imagery was exchanged

One major threat to AntiToxin’s business is what’s often seen as boosting online safety: end-to-end encryption. AntiToxin claims that when companies like Facebook expand encryption, they’re purposefully hiding problematic content from themselves so they don’t have to police it.

Facebook claims it still can use metadata about connections on its already encrypted WhatApp network to suspend those who violate its policy. But AntiToxin provided research to TechCrunch for an investigation that found child sexual abuse imagery sharing groups were openly accessible and discoverable on WhatsApp — in part because encryption made them hard to hunt down for WhatsApp’s automated systems.

AntiToxin believes abuse would proliferate if encryption becomes a wider trend, and it claims the harm that it  causes outweighs fears about companies or governments surveiling unencrypted transmissions. It’s a tough call. Political dissidents, whistleblowers and perhaps the whole concept of civil liberty rely on encryption. But parents may see sex offenders and bullies as a more dire concern that’s reinforced by platforms having no idea what people are saying inside chat threads.

What seems clear is that the status quo has got to go. Shaming, exclusion, sexism, grooming, impersonation and threats of violence have started to feel commonplace. A culture of cruelty breeds more cruelty. Tech’s success stories are being marred by horror stories from their users. Paying to pick up new weapons in the fight against toxicity seems like a reasonable investment to demand.

Powered by WPeMatico

Tall Poppy aims to make online harassment protection an employee benefit

For the nearly 20 percent of Americans who experience severe online harassment, there’s a new company launching in the latest batch of Y Combinator called Tall Poppy that’s giving them the tools to fight back.

Co-founded by Leigh Honeywell and Logan Dean, Tall Poppy grew out of the work that Honeywell, a security specialist, had been doing to hunt down trolls in online communities since at least 2008.

That was the year that Honeywell first went after a particularly noxious specimen who spent his time sending death threats to women in various Linux communities. Honeywell cooperated with law enforcement to try and track down the troll and eventually pushed the commenter into hiding after he was visited by investigators.

That early success led Honeywell to assume a not-so-secret identity as a security expert by day for companies like Microsoft, Salesforce, and Slack, and a defender against online harassment when she wasn’t at work.

“It was an accidental thing that I got into this work,” says Honeywell. “It’s sort of an occupational hazard of being an internet feminist.”

Honeywell started working one-on-one with victims of online harassment that would be referred to her directly.

“As people were coming forward with #metoo… I was working with a number of high profile folks to essentially batten down the hatches,” says Honeywell. “It’s been satisfying work helping people get back a sense of safety when they feel like they have lost it.”

As those referrals began to climb (eventually numbering in the low hundreds of cases), Honeywell began to think about ways to systematize her approach so it could reach the widest number of people possible.

“The reason we’re doing it that way is to help scale up,” says Honeywell. “As with everything in computer security it’s an arms race… As you learn to combat abuse the abusive people adopt technologies and learn new tactics and ways to get around it.”

Primarily, Tall Poppy will provide an educational toolkit to help people lock down their own presence and do incident response properly, says Honeywell. The company will work with customers to gain an understanding of how to protect themselves, but also to be aware of the laws in each state that they can use to protect themselves and punish their attackers.

The scope of the problem

Based on research conducted by the Pew Foundation, there are millions of people in the U.S. alone, who could benefit from the type of service that Tall Poppy aims to provide.

According to a 2017 study, “nearly one-in-five Americans (18%) have been subjected to particularly severe forms of harassment online, such as physical threats, harassment over a sustained period, sexual harassment or stalking.”

The women and minorities that bear the brunt of these assaults (and, let’s be clear, it is primarily women and minorities who bear the brunt of these assaults), face very real consequences from these virtual assaults.

Take the case of the New York principal who lost her job when an ex-boyfriend sent stolen photographs of her to the New York Post and her boss. In a powerful piece for Jezebel she wrote about the consequences of her harassment.

As a result, city investigators escorted me out of my school pending an investigation. The subsequent investigation quickly showed that I was set up by my abuser. Still, Mayor Bill de Blasio’s administration demoted me from principal to teacher, slashed my pay in half, and sent me to a rubber room, the DOE’s notorious reassignment centers where hundreds of unwanted employees languish until they are fired or forgotten.

In 2016, I took a yearlong medical leave from the DOE to treat extreme post-traumatic stress and anxiety. Since the leave was almost entirely unpaid, I took loans against my pension to get by. I ran out of money in early 2017 and reported back to the department, where I was quickly sent to an administrative trial. There the city tried to terminate me. I was charged with eight counts of misconduct despite the conclusion by all parties that my ex-partner uploaded the photos to the computer and that there was no evidence to back up his salacious story. I was accused of bringing “widespread negative publicity, ridicule and notoriety” to the school system, as well as “failing to safeguard a Department of Education computer” from my abusive ex.

Her story isn’t unique. Victims of online harassment regularly face serious consequences from online harassment.

According to a  2013 Science Daily study, cyber stalking victims routinely need to take time off from work, or change or quit their job or school. And the stalking costs the victims $1200 on average to even attempt to address the harassment, the study said.

“It’s this widespread problem and the platforms have in many ways have dropped the ball on this,” Honeywell says.

Tall Poppy’s co-founders

Creating Tall Poppy

As Honeywell heard more and more stories of online intimidation and assault, she started laying the groundwork for the service that would eventually become Tall Poppy. Through a mutual friend she reached out to Dean, a talented coder who had been working at Ticketfly before its Eventbrite acquisition and was looking for a new opportunity.

That was in early 2015. But, afraid that striking out on her own would affect her citizenship status (Honeywell is Canadian), she and Dean waited before making the move to finally start the company.

What ultimately convinced them was the election of Donald Trump.

“After the election I had a heart-to-heart with myself… And I decided that I could move back to Canada, but I wanted to stay and fight,” Honeywell says.

Initially, Honeywell took on a year-long fellowship with the American Civil Liberties Union to pick up on work around privacy and security that had been handled by Chris Soghoian who had left to take a position with Senator Ron Wyden’s office.

But the idea for Tall Poppy remained, and once Honeywell received her green card, she was “chomping at the bit to start this company.”

A few months in the company already has businesses that have signed up for the services and tools it provides to help companies protect their employees.

Some platforms have taken small steps against online harassment. Facebook, for instance, launched an initiative to get people to upload their nude pictures  so that the social network can monitor when similar images are distributed online and contact a user to see if the distribution is consensual.

Meanwhile, Twitter has made a series of changes to its algorithm to combat online abuse.

“People were shocked and horrified that people were trying this,” Honeywell says. “[But] what is the way [harassers] can do the most damage? Sharing them to Facebook is one of the ways where they can do the most damage. It was a worthwhile experiment.”

To underscore how pervasive a problem online harassment is, out of the four companies where the company is doing business or could do business in the first month and a half there is already an issue that the company is addressing. 

“It is an important problem to work on,” says Honeywell. “My recurring realization is that the cavalry is not coming.”

Powered by WPeMatico

We’re looking at cyberbullying the wrong way

Bad egg A bully is a bully and a troll is a troll, no matter where you go online. For as long as online mass communication has existed, online bullying has existed. To effectively address the issue of cyberbullying, one must not only question the environments that yield such behaviors, but examine how and why the behavior exists in the first place. Read More

Powered by WPeMatico

Papa, what’s a shitpost?

whats-a-shitpost As we romp further into this silly and messy political season it’s come to light that Palmer Luckey, creator of the Oculus Rift and the boy who was last seen floating in a tropical paradise on the cover of time magazine, was the money man behind a group of Trump supporters calling themselves Nimble America. They dedicated the non-profit to proving “that shitposting is powerful and… Read More

Powered by WPeMatico

100% Fun

shutterstock_393282955 If 2016 taught us anything it’s that the internet isn’t fun anymore. It’s not that a soulless network of computers interconnected via TCP/IP was ever supposed to be fun. It’s that eventually fun overlaid itself on that network and created a world where nearly everyone could interact without fear. Kids grew up in a world where it was easier to talk to someone in… Read More

Powered by WPeMatico

Few Winners In Anonymous Social Networking, And Secret’s Not One Of Them

anonymity-apps Is anonymous social networking a flash-in-pan trend? A winner-take-all category? Which of the anonymous social networking apps around today are still thriving, and which are practically dead? These questions come to mind today as one of the leading companies in the “anonymous social” category, Secret, revamped its application, borrowing ideas from popular apps like Yik Yak and… Read More

Powered by WPeMatico