Speech Recognition

Auto Added by WPeMatico

Sanas aims to convert one accent to another in real time for smoother customer service calls

In the customer service industry, your accent dictates many aspects of your job. It shouldn’t be the case that there’s a “better” or “worse” accent, but in today’s global economy (though who knows about tomorrow’s) it’s valuable to sound American or British. While many undergo accent neutralization training, Sanas is a startup with another approach (and a $5.5M seed round): using speech recognition and synthesis to change the speaker’s accent in near real time.

The company has trained a machine learning algorithm to quickly and locally (that is, without using the cloud) recognize a person’s speech on one end and, on the other, output the same words with an accent chosen from a list or automatically detected from the other person’s speech.

Screenshot of the Sanas desktop application.

Image Credits: Sanas.ai

It slots right into the OS’s sound stack so it works out of the box with pretty much any audio or video calling tool. Right now the company is operating a pilot program with thousands of people in locations from the USA and UK to the Philippines, India, Latin America, and others. Accents supported will include American, Spanish, British, Indian, Filipino and Australian by the end of the year.

To tell the truth, the idea of Sanas kind of bothered me at first. It felt like a concession to bigoted people who consider their accent superior and think others below them. Tech will fix it… by accommodating the bigots. Great!

But while I still have a little bit of that feeling, I can see there’s more to it than this. Fundamentally speaking, it is easier to understand someone when they speak in an accent similar to your own. But customer service and tech support is a huge industry and one primarily performed by people outside the countries where the customers are. This basic disconnect can be remedied in a way that puts the onus of responsibility on the entry-level worker, or one that puts it on technology. Either way the difficulty of making oneself understood remains and must be addressed — an automated system just lets it be done more easily and allows more people to do their job.

It’s not magic — as you can tell in this clip, the character and cadence of the person’s voice is only partly retained and the result is considerably more artificial sounding:

But the technology is improving and like any speech engine, the more it’s used, the better it gets. And for someone not used to the original speaker’s accent, the American-accented version may very well be more easily understood. For the person in the support role, this likely means better outcomes for their calls — everyone wins. Sanas told me that the pilots are just starting so there are no numbers available from this deployment yet, but testing has suggested a considerable reduction of error rates and increase in call efficiency.

It’s good enough at any rate to attract a $5.5M seed round, with participation from Human Capital, General Catalyst, Quiet Capital, and DN Capital.

“Sanas is striving to make communication easy and free from friction, so people can speak confidently and understand each other, wherever they are and whoever they are trying to communicate with,” CEO Maxim Serebryakov said in the press release announcing the funding. It’s hard to disagree with that mission.

While the cultural and ethical questions of accents and power differentials are unlikely to ever go away, Sanas is trying something new that may be a powerful tool for the many people who must communicate professionally and find their speech patterns are an obstacle to that. It’s an approach worth exploring and discussing even if in a perfect world we would simply understand one another better.

Powered by WPeMatico

Microsoft brings transcriptions to Word

Microsoft today launched Transcribe in Word, its new transcription service for Microsoft 365 subscribers, into general availability. It’s now available in the online version of Word, with other platforms launching later. In addition, Word is also getting new dictation features, which now allow you to use your voice to format and edit your text, for example.

As the name implies, this new feature lets you transcribe conversations, both live and pre-recorded, and then edit those transcripts right inside of Word. With this, the company goes head-to-head with startups like Otter and Google’s Recorder app, though they all have their own pros and cons.

Image Credits: Microsoft

To get started with Transcribe in Word, you simply head for the Dictate button in the menu bar and click on “Transcribe.” From there, you can record a conversation as it happens — by recording it directly through a speakerphone and your laptop’s microphone, for example — or by recording it in some other way and then uploading that file. The service accepts .mp3, .wav, .m4a and .mp4 files.

As Dan Parish, Microsoft principal group PM manager for Natural User Interface & Incubation, noted in a press briefing ahead of today’s announcement, when you record a call live, the transcription actually runs in the background while you conduct your interview, for example. The team purposely decided not to show you the live transcript, though, because its user research showed that it was distracting. I admit that I like to see the live transcript in Otter and Recorder, but maybe I’m alone in that.

Like with other services, Transcribe in Word lets you click on individual paragraphs in the transcript and then listen to that at a variety of speeds. Because the automated transcript will inevitably have errors in it, that’s a must-have feature. Sadly, though, Transcribe doesn’t let you click on individual words.

One major limitation of the service right now is that if you like to record offline and then upload your files, you’ll be limited to 300 minutes, without the ability to extend this for an extra fee, for example. I know I often transcribe far more than five hours of interviews in any given month, so that limit seems low, especially given that Otter provides me with 6,000 minutes on its cheapest paid plan. The max length for a transcript on Otter is four hours while Microsoft’s only limit for is a 200MB file upload limit, with no limits on live recordings.

Another issue I noticed here is that if you mistakenly exit the tab with Word in it, the transcription process will stop and there doesn’t seem to be a way to restart it.

It also takes quite a while for the uploaded files to be transcribed. It takes roughly as long as the conversations I’ve tried to transcribe, but the results are very good — and often better than those of competing services. Transcribe for Word also does a nice job separating out the different speakers in a conversation. For privacy reasons, you must assign your own names to those — even when you regularly record the same people.

It’d be nice to get the same feature in something like OneNote, for example, and my guess is Microsoft may expand this to its note-taking app over time. To me, that’s the more natural place for it.

Image Credits: Microsoft

The new dictation features in Word now let you give commands like “bold the last sentence,” for example, and say “percentage sign” or “ampersand” if you need to add those symbols to a text (or “smiley face,” if those are the kinds of texts you write in Word).

Even if you don’t often need to transcribe text, this new feature shows how Microsoft is now using its subscription service to launch new premium features to convert free users to paying ones. I’d be surprised if tools like the Microsoft Editor (which offers more features for paying users), this transcription service, as well as some of the new AI features in the likes of Excel and PowerPoint, didn’t help to convert some users into paying ones, especially now that the company has combined into a single bundle Office 365 and Microsoft 365 for consumers. After all, just a subscription to something like Grammarly and Otter would be significantly more expensive than a Microsoft 365 subscription.

 

Powered by WPeMatico

Where is voice tech going?

Mark Persaud
Contributor

Mark Persaud is digital product manager and practice lead at Moonshot by Pactera, a digital innovation company that leads global clients through the next era of digital products with a heavy emphasis on artificial intelligence, data and continuous software delivery.

2020 has been all but normal. For businesses and brands. For innovation. For people.

The trajectory of business growth strategies, travel plans and lives have been drastically altered due to the COVID-19 pandemic, a global economic downturn with supply chain and market issues, and a fight for equality in the Black Lives Matter movement — amongst all that complicated lives and businesses already.

One of the biggest stories in emerging technology is the growth of different types of voice assistants:

  • Niche assistants such as Aider that provide back-office support.
  • Branded in-house assistants such as those offered by BBC and Snapchat.
  • White-label solutions such as Houndify that provide lots of capabilities and configurable tool sets.

With so many assistants proliferating globally, voice will become a commodity like a website or an app. And that’s not a bad thing — at least in the name of progress. It will soon (read: over the next couple years) become table stakes for a business to have voice as an interaction channel for a lovable experience that users expect. Consider that feeling you get when you realize a business doesn’t have a website: It makes you question its validity and reputation for quality. Voice isn’t quite there yet, but it’s moving in that direction.

Voice assistant adoption and usage are still on the rise

Adoption of any new technology is key. A key inhibitor of technology is often distribution, but this has not been the case with voice. Apple, Google, and Baidu have reported hundreds of millions of devices using voice, and Amazon has 200 million users. Amazon has a slightly more difficult job since they’re not in the smartphone market, which allows for greater voice assistant distribution for Apple and Google.

Image Credits: Mark Persaud

But are people using devices? Google said recently there are 500 million monthly active users of Google Assistant. Not far behind are active Apple users with 375 million. Large numbers of people are using voice assistants, not just owning them. That’s a sign of technology gaining momentum — the technology is at a price point and within digital and personal ecosystems that make it right for user adoption. The pandemic has only exacerbated the use as Edison reported between March and April — a peak time for sheltering in place across the U.S.

Powered by WPeMatico

Medallia acquires voice-to-text specialist Voci Technologies for $59M

M&A has largely slowed down in the current market, but there remain pockets of activity when the timing and price are right. Today, Medallia — a customer experience platform that scans online reviews, social media, and other sources to provide better insights into what a company is doing right and wrong and what needs to get addressed — announced that it would acquire Voci Technologies, a speech-to-text startup, for $59 million in cash.

Medallia plans to integrate the startup’s AI technology so that voice-based interactions — for example from calls into call centers — can be part of the data crunched by its analytics platform. Despite the rise of social media, messaging channels, and (currently) a shift for people to do a lot more online, voice still accounts for the majority of customer interactions for a business, so this is an important area for Medallia to tackle.

“Voci transcribes 100% of live and recorded calls into text that can be analyzed quickly to determine customer satisfaction, adding a powerful set of signals to the Medallia Experience Cloud,” said Leslie Stretch, president and CEO of Medallia, in a statement. “At the same time, Voci enables call analysis moments after each interaction has completed, optimizing every aspect of call center operations securely. Especially important as virtual and remote contact center operations take shape.”

While there are a lot of speech-to-text offerings in the market today, the key with Voci is that it is able to discern a number of other details in the call, including emotion, gender, sentiment, and voice biometric identity. It’s also able to filter out personal identifiable information to ensure more privacy around using the data for further analytics.

Voci started life as a spinout from Carnegie Mellon University (its three founders were all PhDs from the school), and it had raised a total of about $18 million from investors that included Grotech Ventures, Harbert Growth Parnters, and the university itself. It was last valued at $28 million in March 2018 (during a Series B raise), meaning that today’s acquisition was slightly more than double that value.

The company seems to have been on an upswing with its business. Voci has to date processed some 2 billion minutes of speech, and in January, the company published some momentum numbers that said bookings had grown some 63% in the last quarter, boosted by contact center customers.

In addition to contact centers, the company catered to companies in finance, healthcare, insurance and others areas of business process outsourcing, although it does not disclose names. As with all companies and organizations that have products that cater to offering services remotely, Voci has seen stronger demand for its business in recent weeks, at a time when many have curtailed physical contact due to COVID-19-related movement restrictions.

“Our whole company is delighted to be joining forces with experience management leader Medallia. We are thrilled that Voci’s powerful speech to text capabilities will become part of Medallia Experience Cloud,” said Mike Coney, CEO of Voci, in a statement. “The consolidation of all contact center signals with video, survey and other critical feedback is a game changer for the industry.”

It’s not clear whether Voci had been trying to raise money in the last few months, or if this was a proactive approach from Medallia. But more generally, M&A has found itself in a particularly key position in the world of tech: startups are finding it more challenging right now to raise money, and one big question has been whether that will lead to more hail-mary-style M&A plays, as one route for promising businesses and technologies to avoid shutting down altogether.

For its part, Medallia, which went public in July 2019 after raising money from the likes of Sequoia, has seen its stock hit like the rest of the market in recent weeks. Its current market cap is at around $2.8 billion, just $400 million more than its last private valuation.

The deal is expected to close in May 2020, Medallia said.

 

Powered by WPeMatico

Dasha AI is calling so you don’t have to

While you’d be hard pressed to find any startup not brimming with confidence over the disruptive idea they’re chasing, it’s not often you come across a young company as calmly convinced it’s engineering the future as Dasha AI.

The team is building a platform for designing human-like voice interactions to automate business processes. Put simply, it’s using AI to make machine voices a whole lot less robotic.

“What we definitely know is this will definitely happen,” says CEO and co-founder Vladislav Chernyshov. “Sooner or later the conversational AI/voice AI will replace people everywhere where the technology will allow. And it’s better for us to be the first mover than the last in this field.”

“In 2018 in the US alone there were 30 million people doing some kind of repetitive tasks over the phone. We can automate these jobs now or we are going to be able to automate it in two years,” he goes on. “If you multiple it with Europe and the massive call centers in India, Pakistan and the Philippines you will probably have something like close to 120M people worldwide… and they are all subject for disruption, potentially.”

The New York based startup has been operating in relative stealth up to now. But it’s breaking cover to talk to TechCrunch — announcing a $2M seed round, led by RTP Ventures and RTP Global: An early stage investor that’s backed the likes of Datadog and RingCentral. RTP’s venture arm, also based in NY, writes on its website that it prefers engineer-founded companies — that “solve big problems with technology”. “We like technology, not gimmicks,” the fund warns with added emphasis.

Dasha’s core tech right now includes what Chernyshov describes as “a human-level, voice-first conversation modelling engine”; a hybrid text-to-speech engine which he says enables it to model speech disfluencies (aka, the ums and ahs, pitch changes etc that characterize human chatter); plus “a fast and accurate” real-time voice activity detection algorithm which detects speech in under 100 milliseconds, meaning the AI can turn-take and handle interruptions in the conversation flow. The platform can also detect a caller’s gender — a feature that can be useful for healthcare use-cases, for example.

Another component Chernyshov flags is “an end-to-end pipeline for semi-supervised learning” — so it can retrain the models in real time “and fix mistakes as they go” — until Dasha hits the claimed “human-level” conversational capability for each business process niche. (To be clear, the AI cannot adapt its speech to an interlocutor in real-time — as human speakers naturally shift their accents closer to bridge any dialect gap — but Chernyshov suggests it’s on the roadmap.)

“For instance, we can start with 70% correct conversations and then gradually improve the model up to say 95% of correct conversations,” he says of the learning element, though he admits there are a lot of variables that can impact error rates — not least the call environment itself. Even cutting edge AI is going to struggle with a bad line.

The platform also has an open API so customers can plug the conversation AI into their existing systems — be it telephony, Salesforce software or a developer environment, such as Microsoft Visual Studio.

Currently they’re focused on English, though Chernyshov says the architecture is “basically language agnostic” — but does requires “a big amount of data”.

The next step will be to open up the dev platform to enterprise customers, beyond the initial 20 beta testers, which include companies in the banking, healthcare and insurance sectors — with a release slated for later this year or Q1 2020.

Test use-cases so far include banks using the conversation engine for brand loyalty management to run customer satisfaction surveys that can turnaround negative feedback by fast-tracking a response to a bad rating — by providing (human) customer support agents with an automated categorization of the complaint so they can follow up more quickly. “This usually leads to a wow effect,” says Chernyshov.

Ultimately, he believes there will be two or three major AI platforms globally providing businesses with an automated, customizable conversational layer — sweeping away the patchwork of chatbots currently filling in the gap. And of course Dasha intends their ‘Digital Assistant Super Human Alike’ to be one of those few.

“There is clearly no platform [yet],” he says. “Five years from now this will sound very weird that all companies now are trying to build something. Because in five years it will be obvious — why do you need all this stuff? Just take Dasha and build what you want.”

“This reminds me of the situation in the 1980s when it was obvious that the personal computers are here to stay because they give you an unfair competitive advantage,” he continues. “All large enterprise customers all over the world… were building their own operating systems, they were writing software from scratch, constantly reinventing the wheel just in order to be able to create this spreadsheet for their accountants.

“And then Microsoft with MS-DOS came in… and everything else is history.”

That’s not all they’re building, either. Dasha’s seed financing will be put towards launching a consumer-facing product atop its b2b platform to automate the screening of recorded message robocalls. So, basically, they’re building a robot assistant that can talk to — and put off — other machines on humans’ behalf.

Which does kind of suggest the AI-fuelled future will entail an awful lot of robots talking to each other… 🤖🤖🤖

Chernyshov says this b2c call screening app will most likely be free. But then if your core tech looks set to massively accelerate a non-human caller phenomenon that many consumers already see as a terrible plague on their time and mind then providing free relief — in the form of a counter AI — seems the very least you should do.

Not that Dasha can be accused of causing the robocaller plague, of course. Recorded messages hooked up to call systems have been spamming people with unsolicited calls for far longer than the startup has existed.

Dasha’s PR notes Americans were hit with 26.3BN robocalls in 2018 alone — up “a whopping” 46% on 2017.

Its conversation engine, meanwhile, has only made some 3M calls to date, clocking its first call with a human in January 2017. But the goal from here on in is to scale fast. “We plan to aggressively grow the company and the technology so we can continue to provide the best voice conversational AI to a market which we estimate to exceed $30BN worldwide,” runs a line from its PR.

After the developer platform launch, Chernyshov says the next step will be to open up access to business process owners by letting them automate existing call workflows without needing to be able to code (they’ll just need an analytic grasp of the process, he says).

Later — pegged for 2022 on the current roadmap — will be the launch of “the platform with zero learning curve”, as he puts it. “You will teach Dasha new models just like typing in a natural language and teaching it like you can teach any new team member on your team,” he explains. “Adding a new case will actually look like a word editor — when you’re just describing how you want this AI to work.”

His prediction is that a majority — circa 60% — of all major cases that business face — “like dispatching, like probably upsales, cross sales, some kind of support etc, all those cases” — will be able to be automated “just like typing in a natural language”.

So if Dasha’s AI-fuelled vision of voice-based business process automation come to fruition then humans getting orders of magnitude more calls from machines looks inevitable — as machine learning supercharges artificial speech by making it sound slicker, act smarter and seem, well, almost human.

But perhaps a savvier generation of voice AIs will also help manage the ‘robocaller’ plague by offering advanced call screening? And as non-human voice tech marches on from dumb recorded messages to chatbot-style AIs running on scripted rails to — as Dasha pitches it — fully responsive, emoting, even emotion-sensitive conversation engines that can slip right under the human radar maybe the robocaller problem will eat itself? I mean, if you didn’t even realize you were talking to a robot how are you going to get annoyed about it?

Dasha claims 96.3% of the people who talk to its AI “think it’s human”, though it’s not clear what sample size the claim is based on. (To my ear there are definite ‘tells’ in the current demos on its website. But in a cold-call scenario it’s not hard to imagine the AI passing, if someone’s not paying much attention.)

The alternative scenario, in a future infested with unsolicited machine calls, is that all smartphone OSes add kill switches, such as the one in iOS 13 — which lets people silence calls from unknown numbers.

And/or more humans simply never pick up phone calls unless they know who’s on the end of the line.

So it’s really doubly savvy of Dasha to create an AI capable of managing robot calls — meaning it’s building its own fallback — a piece of software willing to chat to its AI in future, even if actual humans refuse.

Dasha’s robocall screener app, which is slated for release in early 2020, will also be spammer-agnostic — in that it’ll be able to handle and divert human salespeople too, as well as robots. After all, a spammer is a spammer.

“Probably it is the time for somebody to step in and ‘don’t be evil’,” says Chernyshov, echoing Google’s old motto, albeit perhaps not entirely reassuringly given the phrase’s lapsed history — as we talk about the team’s approach to ecosystem development and how machine-to-machine chat might overtake human voice calls.

“At some point in the future we will be talking to various robots much more than we probably talk to each other — because you will have some kind of human-like robots at your house,” he predicts. “Your doctor, gardener, warehouse worker, they all will be robots at some point.”

The logic at work here is that if resistance to an AI-powered Cambrian Explosion of machine speech is futile, it’s better to be at the cutting edge, building the most human-like robots — and making the robots at least sound like they care.

Dasha’s conversational quirks certainly can’t be called a gimmick. Even if the team’s close attention to mimicking the vocal flourishes of human speech — the disfluencies, the ums and ahs, the pitch and tonal changes for emphasis and emotion — might seem so at first airing.

In one of the demos on its website you can hear a clip of a very chipper-sounding male voice, who identifies himself as “John from Acme Dental”, taking an appointment call from a female (human), and smoothly dealing with multiple interruptions and time/date changes as she changes her mind. Before, finally, dealing with a flat cancelation.

A human receptionist might well have got mad that the caller essentially just wasted their time. Not John, though. Oh no. He ends the call as cheerily as he began, signing off with an emphatic: “Thank you! And have a really nice day. Bye!”

If the ultimate goal is Turing Test levels of realism in artificial speech — i.e. a conversation engine so human-like it can pass as human to a human ear — you do have to be able to reproduce, with precision timing, the verbal baggage that’s wrapped around everything humans say to each other.

This tonal layer does essential emotional labor in the business of communication, shading and highlighting words in a way that can adapt or even entirely transform their meaning. It’s an integral part of how we communicate. And thus a common stumbling block for robots.

So if the mission is to power a revolution in artificial speech that humans won’t hate and reject then engineering full spectrum nuance is just as important a piece of work as having an amazing speech recognition engine. A chatbot that can’t do all that is really the gimmick.

Chernyshov claims Dasha’s conversation engine is “at least several times better and more complex than [Google] Dialogflow, [Amazon] Lex, [Microsoft] Luis or [IBM] Watson”, dropping a laundry list of rival speech engines into the conversation.

He argues none are on a par with what Dasha is being designed to do.

The difference is the “voice-first modelling engine”. “All those [rival engines] were built from scratch with a focus on chatbots — on text,” he says, couching modelling voice conversation “on a human level” as much more complex than the more limited chatbot-approach — and hence what makes Dasha special and superior.

“Imagination is the limit. What we are trying to build is an ultimate voice conversation AI platform so you can model any kind of voice interaction between two or more human beings.”

Google did demo its own stuttering voice AI — Duplex — last year, when it also took flak for a public demo in which it appeared not to have told restaurant staff up front they were going to be talking to a robot.

Chernyshov isn’t worried about Duplex, though, saying it’s a product, not a platform.

“Google recently tried to headhunt one of our developers,” he adds, pausing for effect. “But they failed.”

He says Dasha’s engineering staff make up more than half (28) its total headcount (48), and include two doctorates of science; three PhDs; five PhD students; and ten masters of science in computer science.

It has an R&D office in Russian which Chernyshov says helps makes the funding go further.

“More than 16 people, including myself, are ACM ICPC finalists or semi finalists,” he adds — likening the competition to “an Olympic game but for programmers”. A recent hire — chief research scientist, Dr Alexander Dyakonov — is both a doctor of science professor and former Kaggle No.1 GrandMaster in machine learning. So with in-house AI talent like that you can see why Google, uh, came calling…

Dasha

 

But why not have Dasha ID itself as a robot by default? On that Chernyshov says the platform is flexible — which means disclosure can be added. But in markets where it isn’t a legal requirement the door is being left open for ‘John’ to slip cheerily by. Bladerunner here we come.

The team’s driving conviction is that emphasis on modelling human-like speech will, down the line, allow their AI to deliver universally fluid and natural machine-human speech interactions which in turn open up all sorts of expansive and powerful possibilities for embeddable next-gen voice interfaces. Ones that are much more interesting than the current crop of gadget talkies.

This is where you could raid sci-fi/pop culture for inspiration. Such as Kitt, the dryly witty talking car from the 1980s TV series Knight Rider. Or, to throw in a British TV reference, Holly the self-depreciating yet sardonic human-faced computer in Red Dwarf. (Or indeed Kryten the guilt-ridden android butler.) Chernyshov’s suggestion is to imagine Dasha embedded in a Boston Dynamics robot. But surely no one wants to hear those crawling nightmares scream…

Dasha’s five-year+ roadmap includes the eyebrow-raising ambition to evolve the technology to achieve “a general conversational AI”. “This is a science fiction at this point. It’s a general conversational AI, and only at this point you will be able to pass the whole Turing Test,” he says of that aim.

“Because we have a human level speech recognition, we have human level speech synthesis, we have generative non-rule based behavior, and this is all the parts of this general conversational AI. And I think that we can we can — and scientific society — we can achieve this together in like 2024 or something like that.

“Then the next step, in 2025, this is like autonomous AI — embeddable in any device or a robot. And hopefully by 2025 these devices will be available on the market.”

Of course the team is still dreaming distance away from that AI wonderland/dystopia (depending on your perspective) — even if it’s date-stamped on the roadmap.

But if a conversational engine ends up in command of the full range of human speech — quirks, quibbles and all — then designing a voice AI may come to be thought of as akin to designing a TV character or cartoon personality. So very far from what we currently associate with the word ‘robotic’. (And wouldn’t it be funny if the term ‘robotic’ came to mean ‘hyper entertaining’ or even ‘especially empathetic’ thanks to advances in AI.)

Let’s not get carried away though.

In the meanwhile, there are ‘uncanny valley’ pitfalls of speech disconnect to navigate if the tone being (artificially) struck hits a false note. (And, on that front, if you didn’t know ‘John from Acme Dental’ was a robot you’d be forgiven for misreading his chipper sign off to a total time waster as pure sarcasm. But an AI can’t appreciate irony. Not yet anyway.)

Nor can robots appreciate the difference between ethical and unethical verbal communication they’re being instructed to carry out. Sales calls can easily cross the line into spam. And what about even more dystopic uses for a conversation engine that’s so slick it can convince the vast majority of people it’s human — like fraud, identity theft, even election interference… the potential misuses could be terrible and scale endlessly.

Although if you straight out ask Dasha whether it’s a robot Chernyshov says it has been programmed to confess to being artificial. So it won’t tell you a barefaced lie.

Dasha

How will the team prevent problematic uses of such a powerful technology?

“We have an ethics framework and when we will be releasing the platform we will implement a real-time monitoring system that will monitor potential abuse or scams, and also it will ensure people are not being called too often,” he says. “This is very important. That we understand that this kind of technology can be potentially probably dangerous.”

“At the first stage we are not going to release it to all the public. We are going to release it in a closed alpha or beta. And we will be curating the companies that are going in to explore all the possible problems and prevent them from being massive problems,” he adds. “Our machine learning team are developing those algorithms for detecting abuse, spam and other use cases that we would like to prevent.”

There’s also the issue of verbal ‘deepfakes’ to consider. Especially as Chernyshov suggests the platform will, in time, support cloning a voiceprint for use in the conversation — opening the door to making fake calls in someone else’s voice. Which sounds like a dream come true for scammers of all stripes. Or a way to really supercharge your top performing salesperson.

Safe to say, the counter technologies — and thoughtful regulation — are going to be very important.

There’s little doubt that AI will be regulated. In Europe policymakers have tasked themselves with coming up with a framework for ethical AI. And in the coming years policymakers in many countries will be trying to figure out how to put guardrails on a technology class that, in the consumer sphere, has already demonstrated its wrecking-ball potential — with the automated acceleration of spam, misinformation and political disinformation on social media platforms.

“We have to understand that at some point this kind of technologies will be definitely regulated by the state all over the world. And we as a platform we must comply with all of these requirements,” agrees Chernyshov, suggesting machine learning will also be able to identify whether a speaker is human or not — and that an official caller status could be baked into a telephony protocol so people aren’t left in the dark on the ‘bot or not’ question. 

“It should be human-friendly. Don’t be evil, right?”

Asked whether he considers what will happen to the people working in call centers whose jobs will be disrupted by AI, Chernyshov is quick with the stock answer — that new technologies create jobs too, saying that’s been true right throughout human history. Though he concedes there may be a lag — while the old world catches up to the new.

Time and tide wait for no human, even when the change sounds increasingly like we do.

Powered by WPeMatico

Google updates its speech tech for contact centers

Last July, Google announced its Contact Center AI product for helping businesses get more value out of their contact centers. Contact Center AI uses a mix of Google’s machine learning-powered tools to help build virtual agents and help human agents as they do their job. Today, the company is launching several updates to this product that will, among other things, bring improved speech recognition features to the product.

As Google notes, its automated speech recognition service gets to very high accuracy rates, even on the kind of noisy phone lines that many customers use to complain about their latest unplanned online purchase. To improve these numbers, Google is now launching a feature called “Auto Speech Adaptation in Dialogflow,” (with Dialogflow being Google’s tool for building conversational experiences). With this, the speech recognition tools are able to take into account the context of the conversation and hence improve their accuracy by about 40%, according to Google.

Speech Recognition Accuracy

In addition, Google is launching a new phone model for understanding short utterances, which is now about 15% more accurate for U.S. English, as well as a number of other updates that improve transcription accuracy, make the training process easier and allow for endless audio streaming to the Cloud Speech-to-Text API, which previously had a five-minute limit.

If you want to, you also can now natively download MP3s of the audio (and then burn them to CDs, I guess).

dialogflow virtual agent.max 1100x1100

Powered by WPeMatico

Apple’s Voice Control improves accessibility OS-wide on all its devices

Apple is known for fluid, intuitive user interfaces, but none of that matters if you can’t click, tap, or drag because you don’t have a finger to do so with. For users with disabilities the company is doubling down on voice-based accessibility with the powerful new Voice Control feature on Macs and iOS (and iPadOS) devices.

Many devices already support rich dictation, and of course Apple’s phones and computers have used voice-based commands for years (I remember talking to my Quadra). But this is a big step forward that makes voice controls close to universal — and it all works offline.

The basic idea of Voice Control is that the user has both set commands and context-specific ones. Set commands are things like “Open Garage Band” or “File menu” or “Tap send.” And of course some intelligence has gone into making sure you’re actually saying the command and not writing it, like in that last sentence.

But that doesn’t work when you have an interface that pops up with lots of different buttons, fields, and labels. And even if every button or menu item could be called by name, it might be difficult or time-consuming to speak everything out loud.

To fix this Apple simply attaches a number to every UI item in the foreground, which a user can show by saying “show numbers.” Then they can simply speak the number or modify it with another command, like “tap 22.” You can see a basic workflow below, though of course without the audio cues it loses a bit:

Remember that these numbers may be more easily referenced by someone with little or no vocal ability, and could in fact be selected from using a simpler input like a dial or blow tube. Gaze tracking is good but it has its limitations, and this is a good alternative.

For something like maps, where you could click anywhere, there’s a grid system for selecting where to zoom in or click. Just like Blade Runner! Other gestures like scrolling and dragging are likewise supported.

Dictation has been around for a bit but it’s been improved as well; You can select and replace entire phrases, like “Replace ‘be right back’ with ‘on my way.’ ” Other little improvements will be noted and appreciated by those who use the tool often.

All the voice processing is done offline, which makes it both quick and robust to things like signal problems or use in foreign countries where data might be hard to come by. And the intelligence built into Siri lets it recognize names and context-specific words that may not be part of the base vocabulary. Improved dictation means selecting emoji and adding dictionary items is a breeze.

Right now Voice Control is supported by all native apps, and third party apps that use Apple’s accessibility API should be able to take advantage of it easily. And even if they don’t do it specifically, numbers and grids should still work just fine, since all the OS needs to know are the locations of the UI items. These improvements should appear in accessibility options as soon as a device is updated to iOS 13 or Catalina.

Powered by WPeMatico

Live transcription and captioning in Android are a boon to the hearing-impaired

A set of new features for Android could alleviate some of the difficulties of living with hearing impairment and other conditions. Live transcription, captioning and relay use speech recognition and synthesis to make content on your phone more accessible — in real time.

Announced today at Google’s I/O event in a surprisingly long segment on accessibility, the features all rely on improved speech-to-text and text-to-speech algorithms, some of which now run on-device rather than sending audio to a data center to be decoded.

The first feature to be highlighted, live transcription, was already mentioned by Google. It’s a simple but very useful tool: open the app and the device will listen to its surroundings and simply display as text on the screen any speech it recognizes.

We’ve seen this in translator apps and devices, like the One Mini, and the meeting transcription highlighted yesterday at Microsoft Build. One would think that such a straightforward tool is long overdue, but, in fact, everyday circumstances like talking to a couple of friends at a cafe can be remarkably difficult for natural language systems trained on perfectly recorded single-speaker audio. Improving the system to the point where it can track multiple speakers and display accurate transcripts quickly has no doubt been a challenge.

Another feature enabled by this improved speech recognition ability is live captioning, which essentially does the same thing as above, but for video. Now when you watch a YouTube video, listen to a voice message or even take a video call, you’ll be able to see what the person in it is saying, in real time.

That should prove incredibly useful not just for the millions of people who can’t hear what’s being said, but also those who don’t speak the language well and could use text support, or anyone watching a show on mute when they’re supposed to be going to sleep, or any number of other circumstances where hearing and understanding speech just isn’t the best option.

Gif showing a phone conversation being captioned live.Captioning phone calls is something CEO Sundar Pichai said is still under development, but the “live relay” feature they demoed onstage showed how it might work. A person who is hearing-impaired or can’t speak will certainly find an ordinary phone call to be pretty worthless. But live relay turns the call immediately into text, and immediately turns text responses into speech the person on the line can hear.

Live captioning should be available on Android Q when it releases, with some device restrictions. Live transcribe is available now, but a warning states that it is currently in development. Live relay is yet to come, but showing it onstage in such a complete form suggests it won’t be long before it appears.

Powered by WPeMatico

Google’s new voice recognition system works instantly and offline (if you have a Pixel)

Voice recognition is a standard part of the smartphone package these days, and a corresponding part is the delay while you wait for Siri, Alexa or Google to return your query, either correctly interpreted or horribly mangled. Google’s latest speech recognition works entirely offline, eliminating that delay altogether — though of course mangling is still an option.

The delay occurs because your voice, or some data derived from it anyway, has to travel from your phone to the servers of whoever operates the service, where it is analyzed and sent back a short time later. This can take anywhere from a handful of milliseconds to multiple entire seconds (what a nightmare!), or longer if your packets get lost in the ether.

Why not just do the voice recognition on the device? There’s nothing these companies would like more, but turning voice into text on the order of milliseconds takes quite a bit of computing power. It’s not just about hearing a sound and writing a word — understanding what someone is saying word by word involves a whole lot of context about language and intention.

Your phone could do it, for sure, but it wouldn’t be much faster than sending it off to the cloud, and it would eat up your battery. But steady advancements in the field have made it plausible to do so, and Google’s latest product makes it available to anyone with a Pixel.

Google’s work on the topic, documented in a paper here, built on previous advances to create a model small and efficient enough to fit on a phone (it’s 80 megabytes, if you’re curious), but capable of hearing and transcribing speech as you say it. No need to wait until you’ve finished a sentence to think whether you meant “their” or “there” — it figures it out on the fly.

So what’s the catch? Well, it only works in Gboard, Google’s keyboard app, and it only works on Pixels, and it only works in American English. So in a way this is just kind of a stress test for the real thing.

“Given the trends in the industry, with the convergence of specialized hardware and algorithmic improvements, we are hopeful that the techniques presented here can soon be adopted in more languages and across broader domains of application,” writes Google, as if it is the trends that need to do the hard work of localization.

Making speech recognition more responsive, and to have it work offline, is a nice development. But it’s sort of funny considering hardly any of Google’s other products work offline. Are you going to dictate into a shared document while you’re offline? Write an email? Ask for a conversion between liters and cups? You’re going to need a connection for that! Of course this will also be better on slow and spotty connections, but you have to admit it’s a little ironic.

Powered by WPeMatico

Say ‘Aloha’: A closer look at Facebook’s voice ambitions

Facebook has been a bit slow to adopt the voice computing revolution. It has no voice assistant, its smart speaker is still in development, and some apps like Instagram aren’t fully equipped for audio communication. But much of that is set to change judging by experiments discovered in Facebook’s code, plus new patent filings.

Developing voice functionality could give people more ways to use Facebook in their home or on the go. Its forthcoming Portal smart speaker is reportedly designed for easy video chatting with distant family, including seniors and kids that might have trouble with phones. Improved transcription and speech-to-text-to-speech features could connect Messenger users across input mediums and keep them on the chat app rather than straying back to SMS.

But Facebook’s voice could be drowned out by the din of the crowd if it doesn’t get moving soon. All the major mobile hardware and operating system makers now have their own voice assistants like Siri, Alexa, Google Assistant and Samsung Bixby, as well as their own smart speakers. In Q2 2018, Canalys estimates that Google shipped 5.4 million Homes, and Amazon shipped 4.1 million Echoes. Apple’s HomePod is off to a slow start with less than 6 percent of the market, behind Alibaba’s smart speaker, according to Strategy Analytics. Facebook’s spotty record around privacy might deflect potential customers to its competitors.

Given Facebook is late to the game, it will need to arrive with powerful utility that solves real problems. Here’s a look at Facebook’s newest developments in the voice space, and how its past experiments lay the groundwork for its next big push.

Aloha voice

Facebook is developing its own speech recognition feature under the name Aloha for both the Facebook and Messenger apps, as well as external hardware — likely the video chat smart speaker it’s developing. Code inside the Facebook and Messenger Android apps dug up by frequent TechCrunch tipster and mobile researcher Jane Manchun Wong gives the first look at a prototype for the Aloha user interface.

Labeled “Aloha Voice Testing,” as a user speaks while in a message thread, a horizontal blue bar expands and contracts to visualize the volume of speech while recognizing and transcribing into text. The code describes the feature as having connections with external Wi-Fi or Bluetooth devices. It’s possible that the software will run on both Facebook’s hardware and software, similar to Google Assistant that runs both on phones and Google Home speakers. [Update: As seen below, the Aloha feature contains a “Your mobile device is now connected Portal” screen, confirming that name for the Facebook video chat smart speaker device.]

Facebook declined to comment on the video, with its spokesperson Ha Thai telling me, “We test stuff all the time — nothing to share today but my team will be in touch in a few weeks about hardware news coming from the AR/VR org.” It unclear if that hardware news will focus on voice and Aloha or Portal, or if it’s merely related to Facebook’s Oculus Connect 5 conference on September 25th.

A source previously told me that years ago, Facebook was interested in developing its own speech recognition software designed specifically to accurately transcribe how friends talk to each other. These speech patterns are often more casual, colloquial, rapid and full of slang than the way we formally address computerized assistants like Amazon Alexa or Google Home.

Wong also found the Aloha logo buried in Facebook’s code, which features volcano imagery. I can confirm that I’ve seen a Facebook Aloha Setup chatbot with a similar logo on the phones of Facebook employees.

If Facebook can figure this out, it could offer its own transcription features in Messenger and elsewhere on the site so users could communicate across mediums. It could potentially let you dictate comments or messages to friends while you have your hands full or can’t look at your screen. The recipient could then read the text instead of having to listen to it like a voice message. The feature also could be used to power voice navigation of Facebook’s apps for better hands-free usage.

Speaker and camera patents

Facebook awarded patent for speaker

Facebook’s video chat smart speaker was reportedly codenamed Aloha originally but later renamed Portal, Alex Heath of Business Insider and now Cheddar first reported in August 2017. The $499 competitor to the Amazon Echo Show was initially set to launch at Facebook’s F8 in May, but Bloomberg reported it was pushed back amid concerns that it would exacerbate the privacy scandal ignited by Cambridge Analytica.

A new patent filing reveals Facebook was considering building a smart speaker as early as December 26th, 2016 when it filed a patent for a cube-shaped device. The patent diagrams an “ornamental design for a speaker device” invented by Baback Elmieh, Alexandre Jais and John Proksch-Whaley. Facebook had acquired Elmieh’s startup Nascent Objects in September of that year and he’s now a technical project lead at Facebook’s secretive Building 8 hardware lab.

The startup had been building modular hardware, and earlier this year he was awarded patents for work at Facebook on several modular cameras. The speaker and camera technology Facebook has been developing could potentially evolve into what’s in its video chat speaker.

The fact that Facebook has been exploring speaker technology for so long and that the lead on these patents is still running a secret project in Building 8 strengthens the case that Facebook has big plans for the voice space.

Patents awarded to Facebook show designs for a camera (left) and video camera (right)

Instagram voice messaging

And finally, Instagram is getting deeper into the voice game, too. A screenshot generated from the code of Instagram’s Android app by Wong reveals the development of a voice clip messaging feature heading to Instagram Direct. This would allow you to speak into Instagram and send the audio clips similar to a walkie-talkie, or the voice messaging feature Facebook Messenger added back in 2013.

You can see the voice button in the message composer at the bottom of the screen, and the code explains that to “Voice message, press and hold to record.” The prototype follows the recent launch of video chat in Instagram Direct, another feature on which TechCrunch broke the news thanks to Wong’s research. An Instagram spokesperson declined to comment, as is typical when features are spotted in its code but aren’t publicly testing yet, saying, “Unfortunately nothing more to share on this right now.”

The long road to Voicebook

Facebook has long tinkered in the voice space. In 2015, it acquired natural language processing startup Wit.ai that ran a developer platform for building speech interfaces, though it later rolled Wit.ai into Messenger’s platform team to focus on chatbots. Facebook also began testing automatically transcribing Messenger voice clips into text in 2015 in what was likely the groundwork for the Aloha feature seen above. The company also revealed its M personal assistant that could accomplish tasks for users, but it was only rolled out to a very limited user base and later turned off.

The next year, Facebook’s head of Messenger David Marcus claimed at TechCrunch Disrupt that voice “is not something we’re actively working on right now,” but added that “at some point it’s pretty obvious that as we develop more and more capabilities and interactions inside of Messenger, we’ll start working on voice exchanges and interfaces.” However, a source had told me Facebook’s secretive Language Technology Group was already exploring voice opportunities. Facebook also began testing its Live Audio feature for users who want to just broadcast sound and not video.

By 2017, Facebook was offering automatic captioning for Pages’ videos, and was developing a voice search feature. And this year, Facebook began trying voice clips as status updates and Stories for users around the world who might have trouble typing in their native tongue. But executives haven’t spoken much about the voice initiatives.

The most detailed comments we have come from Facebook’s head of design Luke Woods at TechCrunch Disrupt 2017 where he described voice search saying it was, “very promising. There are lots of exciting things happening…. I love to be able to talk to the car to navigate to a particular place. That’s one of many potential use cases.” It’s also one that voice transcription could aid.

It’s still unclear exactly what Facebook’s Aloha will become. It could be a de facto operating system or voice interface and transcription feature for Facebook’s smart speaker and apps. It could become a more full-fledged voice assistant like M, but with audio. Or perhaps it could become Facebook’s bridge to other voice ecosystems, serving as Facebook’s Alexa Skill or Google Assistant Action.

When I asked Woods “How would Facebook on Alexa work?,” he said with a smile “That’s a very interesting question! No comment.”

Powered by WPeMatico