augmented reality
Auto Added by WPeMatico
Octi launches a social network built around augmented reality
Octi has created a new social network that uses augmented reality to connect the act of seeing your friends in real life with viewing digital content, like their favorite YouTube videos and Spotify songs.
When I wrote about the startup in 2018, it was building AR technology that could do a better job of recognizing the human body and movement. Last week, co-founder and CEO Justin Fuisz (pictured above) told me that this was “a really cool feature,” but that Octi’s investors pushed him “to do more, go deeper.”
Speaking of those investors, the startup says it has now raised $12 million in funding (including a previously announced seed round of $7.5 million) from Live Nation, Anheuser-Busch InBev, Peter Diamandis’ Bold Capital Partners, Human Ventures, I2BF, Day One Ventures, Tom Conrad, Scott Belsky and Josh Kushner.
Last week, Fuisz demonstrated what he now sees as Octi’s “mic drop” moment — opening the new app and pointing his iPhone camera at a colleague. The app quickly recognized her, allowing Fuisz to send her a friend request. And once the request was accepted, Fuisz could look at her through the camera again, where she was surrounded by a floating “belt” of virtual items that she’d created with videos, songs and photos.
Octi also allows you to include fun effects and stickers. Your friends can change your profile too, making you wear a funny hat or giving you a rousing theme song for the day.
To create a facial recognition experience that’s fast and simple, Fuisz said that Octi’s powered by a “neural network on the edge,” allowing the app to process images on the device (rather than uploading them to the cloud) in a privacy-friendly way.
He said the company has taken other steps to optimize the process, like prioritizing friends-of-friends rather than searching through the faces of everyone in the network, resulting in an app that can identify a friend in as little as 20 milliseconds.
While Octi allows you to view friends’ profiles remotely, it’s worth emphasizing that the core experience is meant to be in-person. In fact, the company provided a statement from analyst Rich Greenfield in which he described the app as “an impressive technology that gives teens a compelling reason to be present and communicate with their phones, while gathered with their closest friends.”
I wondered whether a new social dynamic also provides new opportunities for harassment and bullying, but Fuisz noted that for now, Octi profiles and belts are only visible to friends that you’ve approved. So if one of your connections is doing something you don’t like, “You just say goodbye. That’s it. That’s a simple way of dealing with it.”
Fuisz added that this initial version provides a foundation for many more experiences: “There’s endless opportunity for games and other fun things you can do.”
Ultimately, he’s hoping to turn this into a WeChat-style platform for outside developers to build social tools and content. And because Octi works on iPhone 7 and above (with plans for an Android version later this year), it can potentially reach an enormous audience out of the gate, rather than facing the scale issues of a more specialized AR or VR hardware platform.
Powered by WPeMatico
Zuckerberg ditches annual challenges, but needs cynics to fix 2030
Mark Zuckerberg won’t be spending 2020 focused on wearing ties, learning Mandarin or just fixing Facebook. “Rather than having year-to-year challenges, I’ve tried to think about what I hope the world and my life will look in 2030,” he wrote today on Facebook. As you might have guessed, though, Zuckerberg’s vision for an improved planet involves a lot more of Facebook’s family of apps.
His biggest proclamations in today’s notes include that:
- AR – Phones will remain the primary computing platform for most of the decade but augmented reality could get devices out from between us so we can be present together — Facebook is building AR glasses
- VR – Better virtual reality technology could address the housing crisis by letting people work from anywhere — Facebook is building Oculus
- Privacy – The internet has created a global community where people find it hard to establish themselves as unique, so smaller online groups could make people feel special again — Facebook is building more private groups and messaging options
- Regulation – The big questions facing technology are too thorny for private companies to address by themselves, and governments must step in around elections, content moderation, data portability and privacy — Facebook is trying to self-regulate on these and everywhere else to deter overly onerous lawmaking
These are all reasonable predictions and suggestions. However, Zuckerberg’s post does little to address how the broadening of Facebook’s services in the 2010s also contributed to a lot of the problems he presents:
- Isolation – Constant passive feed scrolling on Facebook and Instagram has created a way to seem like you’re being social without having true back-and-forth interaction with friends
- Gentrification – Facebook’s shuttled employees have driven up rents in cities around the world, especially the Bay Area
- Envy – Facebook’s algorithms can make anyone without a glamorous, Instagram-worthy life look less important, while hackers can steal accounts and its moderation systems can accidentally suspend profiles with little recourse for most users
- Negligence – The growth-first mentality led Facebook’s policies and safety to lag behind its impact, creating the kind of democracy, content, anti-competition and privacy questions it’s now asking the government to answer for it
Noticeably absent from Zuckerberg’s post are explicit mentions of some of Facebook’s more controversial products and initiatives. He writes about “decentralizing opportunity” by giving small businesses commerce tools, but never mentions cryptocurrency, blockchain or Libra directly. Instead he seems to suggest that Instagram store fronts, Messenger customer support and WhatsApp remittance might be sufficient. He also largely leaves out Portal, Facebook’s smart screen that could help distant families stay closer, but that some see as a surveillance and data collection tool.
I’m glad Zuckerberg is taking his role as a public figure and the steward of one of humanity’s fundamental utilities more seriously. His willingness to even think about some of these long-term issues instead of just quarterly profits is important. Optimism is necessary to create what doesn’t exist.
Still, if Zuckerberg wants 2030 to look better for the world, and for the world to look more kindly on Facebook, he may need to hire more skeptics and cynics that see a dystopic future instead — people who understand human impulses toward greed and vanity. Their foresight on where societal problems could arise from Facebook’s products could help temper Zuckerberg’s team of idealists to create a company that balances the potential of the future with the risks to the present.
Every new year of the last decade I set a personal challenge. My goal was to grow in new ways outside my day-to-day work…
Posted by Mark Zuckerberg on Thursday, January 9, 2020
For more on why Facebook can’t succeed on idealism alone, read:
Powered by WPeMatico
Facebook is building an operating system so it can ditch Android
Facebook doesn’t want its hardware like Oculus or its augmented reality glasses to be at the mercy of Google because they rely on its Android operating system. That’s why Facebook has tasked Mark Lucovsky, a co-author of Microsoft’s Windows NT, with building the social network an operating system from scratch, according to The Information’s Alex Heath. To be clear, Facebook’s smartphone apps will remain available on Android.
“We really want to make sure the next generation has space for us,” says Facebook’s VP of Hardware, Andrew ‘Boz’ Bosworth. “We don’t think we can trust the marketplace or competitors to ensure that’s the case. And so we’re gonna do it ourselves.”
Eye OS
By moving to its own OS, Facebook could have more freedom to bake social interaction — and hopefully privacy — deeper into its devices. It could also prevent a disagreement between Google and Facebook from derailing the roadmaps of its gadgets. Facebook tells TechCrunch the focus of this work is on what’s needed for AR glasses. It’s exploring all the options right now, including potentially partnering with other companies or building a custom OS specifically for augmented reality.
One added bonus of moving to a Facebook-owned operating system? It could make it tougher to force Facebook to spin out some of its acquisitions, especially if Facebook goes with Instagram branding for its future augmented reality glasses.
Facebook has always been sore about not owning an operating system and having to depend on the courtesy of some of its biggest rivals. Those include Apple, whose CEO Tim Cook has repeatedly thrown jabs at Facebook and its chief Mark Zuckerberg over privacy and data collection. In a previous hedge against the power of the mobile operating systems, Facebook worked on a secret project codenamed Oxygen circa 2013 that would help it distribute Android apps from outside the Google Play store if necessary, Vox’s Kurt Wagner reported.
That said, its last attempt to wrestle more control of mobile away from the OS giants in 2013 went down in flames. The Facebook phone, built with HTC hardware, ran a forked version of Android and the Facebook Home user interface. But drowning the experience in friends’ photos and Messenger chat bubbles proved wildly unpopular, and both the HTC First and Facebook Home were shelved.
Investing in tomorrow tech
Now Facebook is hoping to learn from past mistakes as it ramps up its hardware efforts with a new office for the AR/VR team in Burlingame, 15 miles north of the company’s headquarters. The 770,000-square-foot space is designed to house roughly 4,000 employees. Facebook tells TechCrunch the team will move there in the second half of 2020 to make use of its labs, prototype space and testing areas. The AR/VR team will still have members at other offices across California, Washington, New York and abroad.
TechCrunch asked for more info about the space, and Facebook revealed that it’s planning to open a public-facing, experiential space — possibly the first Facebook-branded permanent location that anyone can visit. There, people will be able to come play with its augmented reality and virtual reality products. Those could range from the Oculus Quest headsets and Facebook Portal smart displays it currently sells to potential future products like the camera glasses it’s reportedly building with Ray Ban-maker Luxottica and eventually its full-fledged AR eyewear.
A rendering of Facebook’s under-construction new space in Burlingame, Calif.
Facebook says it’s considering building true retail space into the Burlingame office to let people try and then buy its hardware products. This would be a significant first step toward self-branded Facebook retail spaces in the vein of Apple and Microsoft’s stores.
Interested in potentially controlling more of the hardware stack, Facebook held acquisition talks with $4.5 billion market cap semiconductor company Cirrus Logic, which makes audio chips for Apple and more, The Information reports. That deal never happened, and it’s unclear how far the talks went given tech giants constantly keep their M&A teams open to discussions. But it shows how serious Facebook is taking hardware, even if Portal and Oculus sales have been slow to date. Facebook declined to comment on the matter.
That could start to change next year, though, as flagship virtual reality experiences hit the market. I got a press preview of the upcoming Medal of Honor first-person shooter that will launch on the Oculus Quest in 2020. An hour of playing the World War II game flew by, and it was one of the first VR games that felt like you could enjoy it week after week rather than being just a tech demo. Medal of Honor could prove to be the killer app that convinces gamers they have to get a Quest.
Social hardware
Facebook has also been working on hardware experiences for the enterprise. Facebook Workplace video calls can now run on Portal, with its smart camera auto-zooming to keep everyone in the board room in frame or focused on the action. The Information reports Facebook is also prototyping a VR videoconferencing system that Boz has been testing with his team. Facebook tells TechCrunch that Boz hosted two internal events where he videoconferenced through VR to about 100 of his team leaders using virtual Q&A software Facebook is prototyping internally. It’s hoping to learn what would be necessary to consistently hold meetings in VR.
The hardware initiatives, meanwhile, feed back into Facebook’s core ad business. It’s now using some data about what people do on their Oculus or Portal to target them with ads. From playing certain games to accessing kid-focused experiences to virtually teleporting to vacation destinations, there’s plenty of lucrative data for Facebook to potentially mine.
Facebook tells TechCrunch that Portal currently takes data — like if you log in, make calls or use certain features — to inform ad targeting. For example, it could show you ads related to video calling if you do that a lot. With Oculus, if you connect your Facebook account, then data about apps you use or events you join could be used to tune its algorithms or target ads.
Facebook even wants to know what’s on our mind before we act on it. The Information reports that Facebook’s brain-computer interface hardware for controlling interfaces by employing sensors to recognize a word a user is thinking has been shrunk down. It’s gone from the size of a refrigerator to something hand-held, but is still far from ready for integration into a phone. Facebook tells TechCrunch it’s making progress, improving the word error rate significantly up the state-of-the-art research and expanding the dictionary of words that can be recognized. Facebook can now decode brain activity in real time, and it’s working on an intermediary system for identifying single words as it pushes toward 100 words-per-minute brain typing.
Selling Oculus headsets, Portal screens and mind-readers might never generate the billions in profits Facebook earns from its efficient ads business, but they could ensure the social network isn’t locked out of the next waves of computing. Whether those are fully immersive like virtual reality, convenient complements to our phones like smart displays or minimally invasive sensors, Facebook wants them to be social. If it can bring your friends along to your new gadgets, Facebook will find some way to squeeze out revenue while keeping these devices from making us more isolated and less human.
Powered by WPeMatico
Minecraft Earth is live, so get tapping
Microsoft’s big experiment in real-world augmented reality gaming, Minecraft Earth, is live now for players in North America, the U.K. and a number of other areas. The pocket-size AR game lets you collect blocks and critters wherever you go, undertake little adventures with friends and, of course, build sweet castles.
I played an early version of Minecraft Earth earlier this year, and found it entertaining and the AR aspect surprisingly seamless. The gameplay many were first introduced to in Pokémon GO is adapted here in a more creative and collaborative way.
You still walk around your neighborhood, rendered in this case charmingly like a Minecraft world, and tap little icons that pop up around your character. These may be blocks you can use to build, animals you can collect or events like combat encounters that you can do alone or with friends for rewards.
Ultimately all this is in service of building stuff, which you do on “build plates” of various sizes. These you place in AR mode on a flat surface, which they lock onto, letting you move around freely to edit and play with them. This sounded like it could be fussy or buggy when I first heard about it, but actually doing it was smooth and easy. It’s easy to “zoom in” to edit a structure by just moving your phone closer, and multiple people can play with the same blocks and plate at the same time.
Once you’ve put together something fun, you can take it to an outdoor location and have it represented at essentially “real” size, so you can walk around the interior of your castle or dungeon. Of course, you can’t climb steps, because they’re not real, but the other aspects work as expected: you can manipulate doors and other items, breed cave chickens and generally enjoy yourself.
The game is definitely more open-ended than the collection-focused Pokémon GO and Harry Potter: Wizards Unite. Whether that proves to be to its benefit or detriment when it comes to appeal and lasting power remains to be seen — but one thing is for sure: People love Minecraft and they’re going to want to at least try this out.
And now they can, if they’re in one of the following countries — with others coming throughout the holiday season:
- United States
- United Kingdom
- Canada
- South Korea
- Philippines
- Sweden
- Mexico
- Australia
- New Zealand
- Iceland
You can download Minecraft Earth for iOS here and for Android here.
Powered by WPeMatico
8th Wall’s new Cloud Editor helps customers quickly build mobile AR experiences
The world of phone-based AR has involved a lot of promises, but the future that’s developed has so far been more iterative and less platform shift-y. For startups exclusively focused on mobile AR, there’s been some soul-searching to find ways to bring more lightweight experiences to life that don’t require as much friction or commitment from users.
8th Wall is a team focused on building developer tools for mobile AR experiences. The startup has raised more than $10 million to usher developers into the augmented world.
The company announced this week that they’ve built a one-stop shop authoring platform that will help its customers create and ship AR experiences that will be hosted by 8th Wall . It’s a step forward in what they’ve been trying to build and a further sign that marketing activations are probably the most buoyant money-makers in the rather flat phone-based AR space at the moment.
The editor supports popular immersive web frameworks like A-Frame, three.js and Babylon.js. It’s a development platform, but while game engine tools like Unity have features focused on heavy rendering, 8th Wall is more interested in “very fast, lightweight projects that can be built up to any scale,” the startup’s CEO Erik Murphy tells TechCrunch.
8th Wall’s initial sell was an augmented reality platform akin to ARKit and ARCore that allowed developers to build content that supported a wider breadth of smartphones. Today, 8th Wall’s team of 14 is focused on a technology called WebAR that allows mobile phones to call up web experiences inside the browser.
The main sell of WebAR is the same appeal of web apps; users don’t need to download anything and they can access the experience with just a link. This is great for branded marketing interactions, where expecting users to download an app is pretty laughable; moving this process to the web with a link or a QR code makes life much easier.
The startup’s cloud-based authoring and hosting platform is available now for its agency and
business users.
Powered by WPeMatico
HTC’s new CEO discusses the phonemaker’s future
On September 17, HTC announced that cofounder Cher Wang would be stepping down as CEO. In her place, Yves Maitre stepped into the role of Chief Executive, after more than a decade at French telecom giant, Orange.
It’s a tough job at an even tougher time. The move comes on the tail of five consecutive quarterly losses and major layoffs, including a quarter of the company’s staff, which were let go in July of last year.
It’s a far fall for a company that comprised roughly 11 percent of global smartphone sales, some eight years ago. These days, HTC is routinely relegated to the “other” column when these figures are published.
All of this is not to say that the company doesn’t have some interesting irons in the fire. With Vive, HTC has demonstrated its ability to offer a cutting edge VR platform, while Exodus has tapped into an interest in exploring the use of blockchain technologies for mobile devices.
Of course, neither of these examples show any sign of displacing HTC’s once-booming mobile device sales. And this January’s $1.1 billion sale of a significant portion of its hardware division to Google has left many wondering whether it has much gas left in the mobile tank.
With Wang initially scheduled to appear on stage at Disrupt this week, the company ultimately opted to have Maitre sit in on the panel instead. In preparation for the conversation, we sat down with the executive to discuss his new role and future of the struggling Taiwanese hardware company.
5G, XR and the future of the HTC brand
Powered by WPeMatico
Amid AR/VR experiments, Apple buys UK visual effects studio
Apple has reportedly acquired U.K. special effects studio IKinema, a startup that may be useful in Apple’s quest to bolster its mobile devices with AR special effects and in its more far-flung attempts to enter the AR/VR headset market.
The company issued its standard confirmation for the deal to TechCrunch “Apple buys smaller companies from time to time, and we generally don’t discuss our purpose or plans.” The news was first reported by FT after rumors were floated by MacRumors. .
IKenema has used motion-tracking work to live-animate the bodily movements of digital characters, but the team has also stockpiled this information to create realistic models of movements of digital characters in digital environments, specifically in the context of games and virtual reality titles.
These models were highlighted in the startup’s RunTime product that integrated into game engines like Epic Games’ Unreal Engine. RunTime was powering avatar interactions in experiences like The Void’s “Star Wars: Secrets of the Empire,” a virtual reality experience at Disney resorts, as well as in works by studios like Capcom Linden Lab, Microsoft Studios, Nvidia, Respawn and Square Enix.
RunTime being used in Impulse Gears’ PSVR game, Farpoint
The company’s Orion product allowed motion capture with lower-cost input, essentially syncing limited input like head and hand movement with motion models that could allow for a hybrid solution that still looked realistic. The technology was being used for visualizations by teams at NASA, Tencent and others.
What does Apple want with this company?
There are a handful of spots that this tech could prove helpful, the most obvious of which would be bringing special AR effects to the iOS camera, juxtaposing the spatial data that the camera can gather from a real-world space with digital AR models. This could theoretically allow something like an AR figure to walk up your stairs or sit on a chair. The issue iKenema doesn’t solve in these scenarios is computer vision segmentation to be able to tell what surface is a table versus a floor versus a couch cushion, but enabling digital models to interact with these spaces is a big advance.
Where else?
Well, a little bit more of a reach for Apple would be using this tech in the context of a VR or AR avatar system. And although iKenema worked a lot with motion capture, they did so with the explicit purpose of designing models for digital humans to interact with digital environments in real time. Their solution was already being used by virtual reality developers to give VR gamers a way to visualize how their own bodies moved in VR given limited inputs.
Facebook Horizon’s legless avatars
Virtual reality systems typically only know the location of your head and hands, given trackers are on the controllers and headset, but IKinema’s solution allowed developers to make the rest of users’ in-game bodies look more natural inside games. This is a pretty difficult challenge, and it’s the reason plenty of VR titles have made avatar systems that are missing legs, necks, arms and shoulders, because everything looks awful if the movements are off.
Apple’s computer vision needs are only heightening as they bowl forward on AR and VR devices while also aiming to bolster the camera on the iPhone as a point of differentiation from Google and Samsung devices.
Powered by WPeMatico
Not all is predictable on Facebook’s social Horizon
Most of the people I spoke with at Facebook’s Oculus Connect see the proliferation of virtual reality as a foregone conclusion, one that’s just a matter of timing at this point. For Facebook, the conference’s “The Time is Now” catchphrase showcased that they feel their hardware is ready for everyone.
But despite the success they feel like they’ve tapped into when it comes to hardware iterations, the company’s bread and butter social networking prowess feels like it’s barely improved in-headset in the past several years of VR experimentations.
“On the social side, looking back, it’s kind of embarrassing all of the stages we’ve gone through at Oculus,” Oculus CTO and veteran programmer John Carmack conceded onstage during his signature rambling annual keynote, noting that his own social APK was followed by Oculus Rooms, Oculus Venues, Facebook Spaces and now the company’s latest shiny pearl Facebook Horizon.
Horizon’s debut this year included a flashy trailer for what quickly seemed to be the company’s biggest gamble and first potential social hit, a massive multi-player online world. In introducing the software, Zuckerberg talked about people-centric software as Facebook’s “bread-and-butter,” noting, “We build a lot of the best social experiences for phones and computers, and we want to do this for virtual reality as well.”
But Facebook does not actually appear to hold that much of an advantage over much smaller game studios in terms of understanding how to make social virtual reality experience take off.
Powered by WPeMatico
Facebook expands its playable and AR ad formats
Ahead of Advertising Week, Facebook is announcing the expansion of three interactive ad formats.
First, it says that poll ads (which you may already have seen in Instagram Stories) are moving to the main feed of the Facebook mobile app. Second, the augmented reality ads that Facebook has already been testing are moving into open beta this fall. Third, Facebook is making playable ads available to all advertisers, not just gaming companies.
The company showed off each format at a press event yesterday in New York City.
E!, for example, says it ran ads with interactive polls to promote one of its TV shows, leading to a 1.6x increase in brand awareness. Meanwhile, Vans created a playable ad where players could guide skateboarder Steve Van Doren down a mountain, resulting in a 4.4% lift in ad recall. And WeMakeUp ran an AR ad campaign allowing users to virtually try on new shades of makeup, leading to a 27.6% lift in purchases.
Mark D’Arcy, Facebook’s chief creative officer and vice president of global business marketing, said that while the initial playable ad examples had “very literal gaming mechanics, doing brands in a game,” there could be “a whole range” of different interactions over time.
D’Arcy also acknowledged that including polls, games and AR in ads aren’t exactly new ideas, but he suggested that in the past, they’ve generally been “heavy” experiences, requiring things like a separate microsite. By bringing them front-and-center on Facebook, the company is making them “super lightweight, fun and super scalable.”
As result, he suggested that each of these formats will evolve as more advertisers get to experiment with them: “In 12 months, even six months, we’re going to look at these examples and they’ll be fundamentally different.”
And if you’re wondering how these new formats will handle user data, the Facebook team said that only the aggregate results of polls — not individual user data — will be shared with advertisers. Similarly, any images created by users through an AR ad can be saved to their camera roll, but won’t be shared with advertisers.
Powered by WPeMatico
North now offers Focals smart glasses fittings and purchases via app
North’s Focals smart glasses are the first in the category to even approach mainstream appeal, but to date, the only way to get a pair has been to go into a physical North showroom and get a custom fitting, then return once they’re ready for a pickup and final adjustment. Now, North has released its Showroom app, which makes Focals available across the U.S. and Canada without an in-person appointment.
This approach reduces considerable friction, and it’s able to do so thanks to technology available on board the iPhone X or later — essentially the same tech that makes Face ID possible. People can go through the sizing and fitting process using these later model iPhones (and you can borrow a friend’s if you’re on Android or an older iOS device) and then North takes those measurements and can produce either prescription or non-prescription Focals, shipped directly to your door after a few weeks.
The Showroom app also includes an AR-powered virtual try-on feature for making sure you like the look of the frames, and for picking out your favorite color. Once the Focals show up at your door, the final fitting process is also something you can do at home, guided by the app’s directions for getting the fit just right.
Should you still want to hit an actual physical showroom, North’s still going to be operating its Brooklyn and Toronto storefronts, and will be operating pop-ups across North America as well.
Focals began shipping earlier this year, bringing practical smart notification, guidance and other software experiences to your field of view via a tiny projector and in-lens transparent display. North, which previously existed as Thalmic Labs and created the Myo gesture control armband, recognized that they were building control devices optimized for exactly this kind of application, but also found that no one was yet getting wearable tech like smart glasses right. Last year, Thalmic Labs pivoted to become North and focus on Focals as a result.
Since launching its smart glasses to consumers, it’s been iterating the software to consistently add new features, and making them more accessible to customers. An early price drop significantly lessened sticker shock, and now removing the requirement to actually visit a location in person to both order and collect the glasses should help expand their customer base further still.
Powered by WPeMatico