accessibility

Auto Added by WPeMatico

Google announces Action Blocks, a new accessibility tool for creating mobile shortcuts

Google today announced Action Blocks, a new accessibility tool that allows you to create shortcuts for common multi-step tasks with the help of the Google Assistant. In that respect, Action Blocks isn’t all that different from Shortcuts on iOS, for example, but Google is specifically looking at this as an accessibility feature for people with cognitive disabilities.

“If you’ve booked a rideshare using your phone recently, you’ve probably had to go through several steps: unlock your phone, find the right app, navigate through its screens, select appropriate options, and enter your address into the input box,” writes Google accessibility software engineer Ajit Narayanan. “At each step, the app assumes that you’re able to read and write, find things by trial-and-error, remember your selections, and focus for a sustained period of time.”

Google’s own research shows that 80% of people with severe cognitive disabilities, like advanced dementia, autism or Down syndrome, don’t use smartphones, in part because of these barriers.

BedtimeStory 1

Action Blocks are essentially a sequence of commands for the Google Assistant, so everything the Assistant can do can be scripted using this new tool, no matter whether that’s starting a call or playing a TV show. Once the Action Block is set up, you can create a shortcut with a custom image on your phone’s home screen.

For now, the only way to get access to Action Blocks is to join Google’s trusted tester program. It’s unclear when this will roll out to a wider audience. When it does, though, I’m sure a variety of users will want to use of this feature.

Powered by WPeMatico

Local governments are forcing the scooter industry to grow up fast

Gone are the days when tech companies can deploy their services in cities without any regard for rules and regulations. Before the rise of electric scooters, cities had already become hip to tech’s status quo (thanks to the likes of Uber and Lyft) and were ready to regulate. We explored some of this in “The uncertain future of shared scooters,” but since then, new challenges have emerged for scooter startups.

And for scooter startups, city regulations can make or break their businesses across nearly every aspect of operations, especially two major ones: ridership growth and ability to attract investor dollars. From issuing permits to determining how many scooters any one company can operate at any one time to enforcing low-income plans and impacting product roadmaps, the ball is really in the city’s court.

Powered by WPeMatico

Assistive technologies will be a $26 billion dollar market, and investors are only now addressing it

Rohan Silva is obsessed with social mobility and why certain groups are so under-represented in the technology industry.

He co-founded Second Home, a coworking space looking to bring together disparate civic-minded, cultural, creative and commercial entrepreneurs at sites in Lisbon, London and (now) Los Angeles, and he has spent years examining how gender, race and class impact access to technology as a now-reformed politician. Throughout that work though, one area that he says he overlooked was accessibility and entrepreneurship focused on people with disabilities.

“At Second Home, we pride ourselves on having a diverse community. I can count on one hand the number of founders with disabilities we have in our community, so there is definitely something going profoundly wrong,” Silva says.

Enlisting the help of the European venture capital fund Atomico, Silva has set up a micro-investment fund of £100,000 to tackle the problem.

“It’s a large amount compared to what I have and a small amount compared to most venture capital funds,” he explains. “The much bigger prize here is the ability to fund technologies that have the opportunities to improve the lives of people with disabilities.”

Silva isn’t alone. Organizations like Not Impossible Labs, a Los Angeles-based company, and startups like OrCam Technologies, eSight, B-Temia, Kinova Robotics, Open Bionics, Voiceitt and Whill are harnessing technology to bring solutions to people with disabilities across the world.

Powered by WPeMatico

Apple’s Voice Control improves accessibility OS-wide on all its devices

Apple is known for fluid, intuitive user interfaces, but none of that matters if you can’t click, tap, or drag because you don’t have a finger to do so with. For users with disabilities the company is doubling down on voice-based accessibility with the powerful new Voice Control feature on Macs and iOS (and iPadOS) devices.

Many devices already support rich dictation, and of course Apple’s phones and computers have used voice-based commands for years (I remember talking to my Quadra). But this is a big step forward that makes voice controls close to universal — and it all works offline.

The basic idea of Voice Control is that the user has both set commands and context-specific ones. Set commands are things like “Open Garage Band” or “File menu” or “Tap send.” And of course some intelligence has gone into making sure you’re actually saying the command and not writing it, like in that last sentence.

But that doesn’t work when you have an interface that pops up with lots of different buttons, fields, and labels. And even if every button or menu item could be called by name, it might be difficult or time-consuming to speak everything out loud.

To fix this Apple simply attaches a number to every UI item in the foreground, which a user can show by saying “show numbers.” Then they can simply speak the number or modify it with another command, like “tap 22.” You can see a basic workflow below, though of course without the audio cues it loses a bit:

Remember that these numbers may be more easily referenced by someone with little or no vocal ability, and could in fact be selected from using a simpler input like a dial or blow tube. Gaze tracking is good but it has its limitations, and this is a good alternative.

For something like maps, where you could click anywhere, there’s a grid system for selecting where to zoom in or click. Just like Blade Runner! Other gestures like scrolling and dragging are likewise supported.

Dictation has been around for a bit but it’s been improved as well; You can select and replace entire phrases, like “Replace ‘be right back’ with ‘on my way.’ ” Other little improvements will be noted and appreciated by those who use the tool often.

All the voice processing is done offline, which makes it both quick and robust to things like signal problems or use in foreign countries where data might be hard to come by. And the intelligence built into Siri lets it recognize names and context-specific words that may not be part of the base vocabulary. Improved dictation means selecting emoji and adding dictionary items is a breeze.

Right now Voice Control is supported by all native apps, and third party apps that use Apple’s accessibility API should be able to take advantage of it easily. And even if they don’t do it specifically, numbers and grids should still work just fine, since all the OS needs to know are the locations of the UI items. These improvements should appear in accessibility options as soon as a device is updated to iOS 13 or Catalina.

Powered by WPeMatico

ObjectiveEd is building a better digital curriculum for vision-impaired kids

Children with vision impairments struggle to get a solid K-12 education for a lot of reasons — so the more tools their teachers have to impart basic skills and concepts, the better. ObjectiveEd is a startup that aims to empower teachers and kids with a suite of learning games accessible to all vision levels, along with tools to track and promote progress.

Some of the reasons why vision-impaired kids don’t get the education they deserve are obvious, for example that reading and writing are slower and more difficult for them than for sighted kids. But other reasons are less obvious, for example that teachers have limited time and resources to dedicate to these special needs students when their overcrowded classrooms are already demanding more than they can provide.

Technology isn’t the solution, but it has to be part of the solution, because technology is so empowering and kids take to it naturally. There’s no reason a blind 8-year-old can’t also be a digital native like her peers, and that presents an opportunity for teachers and parents both.

This opportunity is being pursued by Marty Schultz, who has spent the last few years as head of a company that makes games targeted at the visually impaired audience, and in the process saw the potential for adapting that work for more directly educational purposes.

“Children don’t like studying and don’t like doing their homework,” he told me. “They just want to play video games.”

It’s hard to argue with that. True of many adults too, for that matter. But as Schultz points out, this is something educators have realized in recent years and turned to everyone’s benefit.

“Almost all regular education teachers use educational digital games in their classrooms and about 20% use it every day,” he explained. “Most teachers report an increase in student engagement when using educational video games. Gamification works because students own their learning. They have the freedom to fail, and try again, until they succeed. By doing this, students discover intrinsic motivation and learn without realizing it.”

Having learned to type, point and click, do geometry and identify countries via games, I’m a product of this same process, and many of you likely are as well. It’s a great way for kids to teach themselves. But how many of those games would be playable by a kid with vision impairment or blindness? Practically none.

Held back

It turns out that these kids, like others with disabilities, are frequently left behind as the rising technology tide lifts everyone else’s boats. The fact is it’s difficult and time-consuming to create accessible games that target things like Braille literacy and blind navigation of rooms and streets, so developers haven’t been able to do so profitably and teachers are left to themselves to figure out how to jury-rig existing resources or, more likely, fall back on tried and true methods like printed worksheets, in-person instruction and spoken testing.

And because teacher time is limited and instructors trained in vision-impaired learning are thin on the ground, these outdated methods are also difficult to cater to an individual student’s needs. For example a kid may be great at math but lack directionality skills. You need to draw up an “individual education plan” (IEP) explaining (among other things) this and what steps need to be taken to improve, then track those improvements. It’s time-consuming and hard! The idea behind ObjectiveEd is to create both games that teach these basic skills and a platform to track and document progress as well as adjust the lessons to the individual.

How this might work can be seen in a game like Barnyard, which like all of ObjectiveEd’s games has been designed to be playable by blind, low-vision or fully sighted kids. The game has the student finding an animal in a big pen, then dragging it in a specified direction. The easiest levels might be left and right, then move on to cardinal directions, then up to clock directions or even degrees.

“If the IEP objective is ‘Child will understand left versus right and succeed at performing this task 90% of the time,’ the teacher will first introduce these concepts and work with the child during their weekly session,” Schultz said. That’s the kind of hands-on instruction they already get. “The child plays Barnyard in school and at home, swiping left and right, winning points and getting encouragement, all week long. The dashboard shows how much time each child is playing, how often, and their level of success.”

That’s great for documentation for the mandated IEP paperwork, and difficulty can be changed on the fly as well:

“The teacher can set the game to get harder or faster automatically, or move onto the next level of complexity automatically (such as never repeating the prompt when the child hesitates). Or the teacher can maintain the child at the current level and advance the child when she thinks it’s appropriate.”

This isn’t meant to be a full-on K-12 education in a tablet app. But it helps close the gap between kids who can play Mavis Beacon or whatever on school computers and vision-impaired kids who can’t.

Practical measures

Importantly, the platform is not being developed without expert help — or, as is actually very important, without a business plan.

“We’ve developed relationships with several schools for the blind as well as leaders in the community to build educational games that tackle important skills,” Schultz said. “We work with both university researchers and experienced Teachers of Visually Impaired students, and Certified Orientation and Mobility specialists. We were surprised at how many different skills and curriculum subjects that teachers really need.”

Based on their suggestions, for instance, the company has built two games to teach iPhone gestures and the accessibility VoiceOver rotor. This may be a proprietary technology from Apple, but it’s something these kids need to know how to use, just like they need to know how to run a Google search, use a mouse without being able to see the screen, and other common computing tasks. Why not learn it in a game like the other stuff?

Making technological advances is all well and good, but doing so while building a sustainable business is another thing many education startups have failed to address. Fortunately, public school systems actually have significant money set aside specifically for students with special needs, and products that improve education outcomes are actively sought and paid for. These state and federal funds can’t be siphoned off to use on the rest of the class, so if there’s nothing to spend them on, they go unused.

ObjectiveEd has the benefit of being easily deployed without much specialty hardware or software. It runs on iPads, which are fairly common in schools and homes, and the dashboard is a simple web one. Although it may eventually interface with specialty hardware like Braille readers, it’s not necessary for many of the games and lessons, so that lowers the deployment bar as well.

The plan for now is to finalize and test the interface and build out the games library — ObjectiveEd isn’t quite ready to launch, but it’s important to build it with constant feedback from students, teachers and experts. With luck, in a year or two the visually-impaired youngsters at a school near you might have a fun new platform to learn and play with.

“ObjectiveEd exists to help teachers, parents and schools adapt to this new era of gamified learning for students with disabilities, starting with blind and visually impaired students,” Schultz said. “We firmly believe that well-designed software combined with ‘off-the-shelf’ technology makes all this possible. The low cost of technology has truly revolutionized the possibilities for improving education.”

Powered by WPeMatico

Live transcription and captioning in Android are a boon to the hearing-impaired

A set of new features for Android could alleviate some of the difficulties of living with hearing impairment and other conditions. Live transcription, captioning and relay use speech recognition and synthesis to make content on your phone more accessible — in real time.

Announced today at Google’s I/O event in a surprisingly long segment on accessibility, the features all rely on improved speech-to-text and text-to-speech algorithms, some of which now run on-device rather than sending audio to a data center to be decoded.

The first feature to be highlighted, live transcription, was already mentioned by Google. It’s a simple but very useful tool: open the app and the device will listen to its surroundings and simply display as text on the screen any speech it recognizes.

We’ve seen this in translator apps and devices, like the One Mini, and the meeting transcription highlighted yesterday at Microsoft Build. One would think that such a straightforward tool is long overdue, but, in fact, everyday circumstances like talking to a couple of friends at a cafe can be remarkably difficult for natural language systems trained on perfectly recorded single-speaker audio. Improving the system to the point where it can track multiple speakers and display accurate transcripts quickly has no doubt been a challenge.

Another feature enabled by this improved speech recognition ability is live captioning, which essentially does the same thing as above, but for video. Now when you watch a YouTube video, listen to a voice message or even take a video call, you’ll be able to see what the person in it is saying, in real time.

That should prove incredibly useful not just for the millions of people who can’t hear what’s being said, but also those who don’t speak the language well and could use text support, or anyone watching a show on mute when they’re supposed to be going to sleep, or any number of other circumstances where hearing and understanding speech just isn’t the best option.

Gif showing a phone conversation being captioned live.Captioning phone calls is something CEO Sundar Pichai said is still under development, but the “live relay” feature they demoed onstage showed how it might work. A person who is hearing-impaired or can’t speak will certainly find an ordinary phone call to be pretty worthless. But live relay turns the call immediately into text, and immediately turns text responses into speech the person on the line can hear.

Live captioning should be available on Android Q when it releases, with some device restrictions. Live transcribe is available now, but a warning states that it is currently in development. Live relay is yet to come, but showing it onstage in such a complete form suggests it won’t be long before it appears.

Powered by WPeMatico

An Xbox controller with a built-in Braille display is Microsoft’s latest gaming accessibility play

Microsoft has been leaning into accessibility in gaming lately, most visibly with its amazing Adaptive Controller, and a new patent suggests another way the company may be accommodating disabled gamers: an Xbox controller with a built-in Braille display.

As you might expect, it’s already quite hard for a visually impaired gamer to play some games, and although that difficulty can’t be entirely alleviated, there are definitely things worth doing. For instance: the text on screen that sighted people take for granted, documenting player status, items, onscreen dialog or directions — how could these be read by a low-vision gamer who might be able to otherwise navigate the game world?

In many circumstances a screen reader is what a visually impaired person would use to interact with this kind of data, but often that text is relayed to them in audio form, which is far less appealing an option when you’re in-game. Who wants to have a computer voice reading off your armor levels and inventory burden while you’re trying to take in the ambient environment?

There are already some Braille display accessories for this kind of thing, but there’s nothing like having support direct from your console’s designer, and that’s what Microsoft has demonstrated with its patent for a Braille-enabled controller.

The patent was filed last year and just recently became public, and was soon spotted by German tech site Let’s Go Digital; there have been no official announcements, though the timing is favorable for an E3 reveal. That said, patents don’t necessarily represent real products in development, though in this case I think it’s worth highlighting regardless.

The Braille Controller, as it’s referred to in the patent, is very much like an ordinary Xbox One gamepad, except on the back there appears to be a sort of robotic insect sticking out of it. This is the Braille display, consisting of both a dot matrix that mechanically reproduces the bumps which players can run their fingers over, and a set of swappable paddles allowing for both input and output.

The six paddles correspond to the six dot positions on a Braille-coded character, and a user may use them to chord or input text that way, or to receive text communications without moving their fingers off the paddles. Of course the mechanisms also could be used to send haptic feedback of other types, like directional indicators or environmental effects like screen shake. I wouldn’t mind having something like this on my controller, in fact.

Naturally this means games will need (and increasingly are including) a metadata layer for this kind of conversion of visual cue to auditory one, and vice versa, among many other considerations for gamers with disabilities. It’s on everyone’s minds, but Microsoft and Xbox seem to be taking more concrete steps than the rest, so kudos to them for that. Hopefully their leadership in this space will help convince other developers and manufacturers to join up.

We’ll be sure to ask the Xbox team about their plans for this controller design and other accessibility improvements when we talk with them at E3 in June.

Powered by WPeMatico

Blind users can now explore photos by touch with Microsoft’s Seeing AI

Microsoft’s Seeing AI is an app that lets blind and limited-vision folks convert visual data into audio feedback, and it just got a useful new feature. Users can now use touch to explore the objects and people in photos.

It’s powered by machine learning, of course, specifically object and scene recognition. All you need to do is take a photo or open one up in the viewer and tap anywhere on it.

“This new feature enables users to tap their finger to an image on a touch-screen to hear a description of objects within an image and the spatial relationship between them,” wrote Seeing AI lead Saqib Shaikh in a blog post. “The app can even describe the physical appearance of people and predict their mood.”

Because there’s facial recognition built in as well, you could very well take a picture of your friends and hear who’s doing what and where, and whether there’s a dog in the picture (important) and so on. This was possible on an image-wide scale already, as you can see in this image:

But the app now lets users tap around to find where objects are — obviously important to understanding the picture or recognizing it from before. Other details that may not have made it into the overall description may also appear on closer inspection, such as flowers in the foreground or a movie poster in the background.

In addition to this, the app now natively supports the iPad, which is certainly going to be nice for the many people who use Apple’s tablets as their primary interface for media and interactions. Lastly, there are a few improvements to the interface so users can order things in the app to their preference.

Seeing AI is free — you can download it for iOS devices here.

Powered by WPeMatico

Official emoji debut for disabled folks, service dogs, waffles and more

A gaggle of new emoji have just been approved by the Unicode Consortium, meaning they’ll be standard across any platforms that choose to support them. This batch includes some much-needed representation for people with various disabilities, new animals from guide dogs to otters, food and many more objects.

Folks with disabilities get a nice variety of new emoji, though of course these aren’t exhaustive (for example, how do you represent a learning disability or mental illness?). Still, Apple’s proposal for the new emoji points out the necessity of, for example, having both mechanical and manual wheelchairs:

The type of assistive technology that is used by individuals is very personal and mandated by their own disability need. For someone who cannot self-propel and therefore uses an electric wheelchair, it would not be realistic to only show a manual chair. For those who can use a manual version, it would not be realistic to insinuate that they have less mobility than they do. Therefore, these should be seen as two totally separate forms of assistive device.

These images, as usual, are only samples; the final emoji that will be used depend on your device or service. However, since Apple proposed these ones and they are of course a popular platform for emoji use, you can probably expect these to be very like the final ones.

There are lots of other useful things added as well. Guide and service dogs; otters and flamingos; some tasty food like waffles and butter (my breakfast can now finally be represented accurately); and some items particularly relevant to Indian users — a sari, diya lamp and tuk-tuk.

Adding support for people of different colors and genders, including non-gendered imagery, has been an ongoing process for the last few years. The latest addition is a pair of non-gendered people holding hands, with the full set of color variations. Expect more along these lines; other proposals have been made but haven’t yet been finalized.

You can browse the full list of new emoji here; expect them to be added to your favorite messaging app after a handful of months once art and code updates are final.

Powered by WPeMatico

Google intros a pair of Android accessibility features for people with hearing loss

Google this morning unveiled a pair of new Android features for people who are deaf or hard of hearing. As the company notes in a blog post this morning, the WHO estimates that 900 million people will be living with hearing loss by 2055. The ubiquity of mobile devices — Android in particular — offers a promising potential to help open the lines of communication.

Live Transcribe is, perhaps, the more compelling of the two offerings. As its name implies, the feature transcribes audio in real time, so users with hearing loss can read text, in order to enable a live, two-way conversation. It defaults to white text on a black background, making it easier to read, and also can connect to external microphones for better results.

The feature leverages much of the company’s work in speech to text and translation. It starts rolling out today in limited beta for Pixel 3 users. It will be available in more than 70 languages and dialects.

Announced back at last year’s Google I/O, Sound Amplifier is designed to filter out ambient and unwanted noise, without boosting the volume on already loud sounds. The feature works with headphones, letting users manually adjust the settings for the right fit. That one is available now via the Play Store.

Powered by WPeMatico