Computer Vision

Auto Added by WPeMatico

Voxel51 raises $2 million for its video-native identification of people, cars and more

Many companies and municipalities are saddled with hundreds or thousands of hours of video and limited ways to turn it into usable data. Voxel51 offers a machine learning-based option that chews through video and labels it, not just with simple image recognition but with an understanding of motions and objects over time.

Annotating video is an important task for a lot of industries, the most well-known of which is certainly autonomous driving. But it’s also important in robotics, the service and retail industries, for police encounters (now that body cams are becoming commonplace) and so on.

It’s done in a variety of ways, from humans literally drawing boxes around objects every frame and writing what’s in it to more advanced approaches that automate much of the process, even running in real time. But the general rule with these is that they’re done frame by frame.

A single frame is great if you want to tell how many cars are in an image, or whether there’s a stop sign, or what a license plate reads. But what if you need to tell whether someone is walking or stepping out of the way? What about whether someone is waving or throwing a rock? Are people in a crowd going to the right or left, generally? This kind of thing is difficult to infer from a single frame, but looking at just two or three in succession makes it clear.

That fact is what startup Voxel51 is leveraging to take on the established competition in this space. Video-native algorithms can do some things that single-frame ones can’t, and where they do overlap, the former often does it better.

Voxel51 emerged from computer vision work done by its co-founders, CEO Jason Corso and CTO Brian Moore, at the University of Michigan. The latter took the former’s computer vision class and eventually the two found they shared a desire to take ideas out of the lab.

“I started the company because I had this vast swath of research,” Corso said, “and the vast majority of services that were available were focused on image-based understanding rather than video-based understanding. And in almost all instances we’ve seen, when we use a video-based model we see accuracy improvements.”

While any old off-the-shelf algorithm can recognize a car or person in an image, it takes much more savvy to make something that can, for example, identify merging behaviors at an intersection, or tell whether someone has slipped between cars to jaywalk. In each of those situations the context is important and multiple frames of video are needed to characterize the action.

“When we process data we look at the spacio-temporal volume as a whole,” said Corso. “Five, 10, 30 frames… our models figure out how far behind and forward it should look to find a robust inference.”

In other, more normal words, the AI model isn’t just looking at an image, but at relationships between many images over time. If it’s not quite sure whether a person in a given frame is crouching or landing from a jump, it knows that it can scrub a little forward or backward to find the information that will make that clear.

And even for more ordinary inference tasks like counting the cars in the street, that data can be double-checked or updated by looking back or skipping ahead. If you can only see five cars because one’s big and blocks a sixth, that doesn’t change the fact that there are six cars. Even if every frame doesn’t show every car, it still matters for, say, a traffic monitoring system.

The natural objection to this is that processing 10 frames to find out what a person is doing is more expensive, computationally speaking, than processing a single frame. That’s certainly true if you are treating it like a series of still images, but that’s not how Voxel51 does it.

scoop voxel51

“We get away with it by processing fewer pixels per frame,” Corso explained. “The total amount of pixels we process might be the same or less as a single frame, depending on what we want it to do.”

For example, on video that needs to be closely examined but speed isn’t a concern (like a backlog of traffic cam data), it can expend all the time it needs on each frame. But for a case where the turnaround needs to be quicker, it can do a fast, real-time pass to identify major objects and motions, then go back through and focus on the parts that are the most important — not the unmoving sky or parked cars, but people and other known objects.

The platform is highly parameterized and naturally doesn’t share the limitations of human-driven annotation (though the latter is still the main option for highly novel applications where you’d have to build a model from scratch).

“You don’t have to worry about, is it annotator A or annotator B, and our platform is a compute platform, so it scales on demand,” said Corso.

They’ve packed everything into a drag-and-drop interface they call Scoop. You drop in your data — videos, GPS, things like that — and let the system power through it. Then you have a browsable map that lets you enumerate or track any number of things: types of signs, blue BMWs, red Toyotas, right turn only lanes, people walking on the sidewalk, people bunching up at a crosswalk, etc. And you can combine categories, in case you’re looking for scenes where that blue BMW was in a right turn only lane.

dogwalk2

Each sighting is attached to the source video, with bounding boxes laid over it indicating the locations of what you’re looking for. You can then export the related videos, with or without annotations. There’s a demo site that shows how it all works.

It’s a little like Nexar’s recently announced Live Maps, though obviously also quite different. That two companies can pursue AI-powered processing of massive amounts of street-level video data and still be distinct business propositions indicates how large the potential market for this type of service is.

Despite its street-feature smarts, Voxel51 isn’t going after self-driving cars to start. Companies in that space, like Waymo and Toyota, are pursuing fairly narrow, vertically oriented systems that are highly focused on identifying objects and behaviors specific to autonomous navigation. The priorities and needs are different from, say, a security firm or police force that monitors hundreds of cameras at once — and that’s where the company is headed right now. That’s consistent with the company’s pre-seed funding, which came from a NIST grant in the public safety sector.

map1

Built with no human intervention from 250 hours of video, a sign/signal map like this would be helpful to many a municipality

“The first phase of go to market is focusing on smart cities and public safety,” Corso said. “We’re working with police departments that are focused on citizen safety. So the officers want to know, is there a fire breaking out, or is a crowd gathering where it shouldn’t be gathering?”

“Right now it’s an experimental pilot — our system runs alongside Baltimore’s CitiWatch,” he continued, referring to a crime-monitoring surveillance system in the city. “They have 800 cameras, and five or six retired cops that sit in a basement watching those — so we help them watch the right feed at the right time. Feedback has been exciting: When [CitiWatch overseer Major Hood] saw the output of our model, not just the person but the behavior, arguing or fighting, his eyes lit up.”

Now, let’s be honest — it sounds a bit dystopian, doesn’t it? But Corso was careful to note that they are not in the business of tracking individuals.

“We’re primarily privacy-preserving video analytics; we have no ability or interest in running face identification. We don’t focus on any kind of identity,” he said.

It’s good that the priority isn’t on identity, but it’s still a bit of a scary capability to be making available. And yet, as anyone can see, the capability is there — it’s just a matter of making it useful and helpful rather than simply creepy. While one can imagine unethical uses like cracking down on protestors, it’s also easy to imagine how useful this could be in an Amber or Silver alert situation. Bad guy in a beige Lexus? Boom, last seen here.

At any rate, the platform is impressive and the computer vision work that went into it even more so. It’s no surprise that the company has raised a bit of cash to move forward. The $2 million seed round was led by eLab Ventures, a Palo Alto and Ann Arbor-based VC firm, and the company earlier attracted the $1.25 million grant from NIST mentioned earlier.

The money will be used for the expected purposes, establishing the product, building out support and the non-technical side of the company and so on. The flexible pricing and near-instant (in video processing terms) results seem like something that will drive adoption fairly quick, given the huge volumes of untapped video out there. Expect to see more companies like Corso and Moore’s as the value of that video becomes clear.

Powered by WPeMatico

Reality Check: The marvel of computer vision technology in today’s camera-based AR systems

Alex Chuang
Contributor

Alex Chuang is the Managing Partner of Shape Immersive, a boutique studio that helps enterprise and brands transform their businesses by incorporating VR/AR solutions into their strategies.

British science fiction writer, Sir Arther C. Clark, once said, “Any sufficiently advanced technology is indistinguishable from magic.”

Augmented reality has the potential to instill awe and wonder in us just as magic would. For the very first time in the history of computing, we now have the ability to blur the line between the physical world and the virtual world. AR promises to bring forth the dawn of a new creative economy, where digital media can be brought to life and given the ability to interact with the real world.

AR experiences can seem magical but what exactly is happening behind the curtain? To answer this, we must look at the three basic foundations of a camera-based AR system like our smartphone.

  1. How do computers know where it is in the world? (Localization + Mapping)
  2. How do computers understand what the world looks like? (Geometry)
  3. How do computers understand the world as we do? (Semantics)

Part 1: How do computers know where it is in the world? (Localization)

Mars Rover Curiosity taking a selfie on Mars. Source: https://www.nasa.gov/jpl/msl/pia19808/looking-up-at-mars-rover-curiosity-in-buckskin-selfie/

When NASA scientists put the rover onto Mars, they needed a way for the robot to navigate itself on a different planet without the use of a global positioning system (GPS). They came up with a technique called Visual Inertial Odometry (VIO) to track the rover’s movement over time without GPS. This is the same technique that our smartphones use to track their spatial position and orientation.

A VIO system is made out of two parts.

Powered by WPeMatico

Edgybees’s new developer platform brings situational awareness to live video feeds

San Diego-based Edgybees today announced the launch of Argus, its API-based developer platform that makes it easy to add augmented reality features to live video feeds.

The service has long used this capability to run its own drone platform for first responders and enterprise customers, which allows its users to tag and track objects and people in emergency situations, for example, to create better situational awareness for first responders.

I first saw a demo of the service a year ago, when the team walked a group of journalists through a simulated emergency, with live drone footage and an overlay of a street map and the location of ambulances and other emergency personnel. It’s clear how these features could be used in other situations as well, given that few companies have the expertise to combine the video footage, GPS data and other information, including geographic information systems, for their own custom projects.

Indeed, that’s what inspired the team to open up its platform. As the Edgybees team told me during an interview at the Ourcrowd Summit last month, it’s impossible for the company to build a new solution for every vertical that could make use of it. So instead of even trying (though it’ll keep refining its existing products), it’s now opening up its platform.

“The potential for augmented reality beyond the entertainment sector is endless, especially as video becomes an essential medium for organizations relying on drone footage or CCTV,” said Adam Kaplan, CEO and co-founder of Edgybees. “As forward-thinking industries look to make sense of all the data at their fingertips, we’re giving developers a way to tailor our offering and set them up for success.”

In the run-up to today’s launch, the company has already worked with organizations like the PGA to use its software to enhance the live coverage of its golf tournaments.

Powered by WPeMatico

Blind users can now explore photos by touch with Microsoft’s Seeing AI

Microsoft’s Seeing AI is an app that lets blind and limited-vision folks convert visual data into audio feedback, and it just got a useful new feature. Users can now use touch to explore the objects and people in photos.

It’s powered by machine learning, of course, specifically object and scene recognition. All you need to do is take a photo or open one up in the viewer and tap anywhere on it.

“This new feature enables users to tap their finger to an image on a touch-screen to hear a description of objects within an image and the spatial relationship between them,” wrote Seeing AI lead Saqib Shaikh in a blog post. “The app can even describe the physical appearance of people and predict their mood.”

Because there’s facial recognition built in as well, you could very well take a picture of your friends and hear who’s doing what and where, and whether there’s a dog in the picture (important) and so on. This was possible on an image-wide scale already, as you can see in this image:

But the app now lets users tap around to find where objects are — obviously important to understanding the picture or recognizing it from before. Other details that may not have made it into the overall description may also appear on closer inspection, such as flowers in the foreground or a movie poster in the background.

In addition to this, the app now natively supports the iPad, which is certainly going to be nice for the many people who use Apple’s tablets as their primary interface for media and interactions. Lastly, there are a few improvements to the interface so users can order things in the app to their preference.

Seeing AI is free — you can download it for iOS devices here.

Powered by WPeMatico

Sam’s Club to test new Scan & Go system that uses computer vision instead of barcodes

In October, Walmart-owned Sam’s Club opened a test store in Dallas where it planned to trial new technology, including mobile checkout, an Amazon Go-like camera system, in-store navigation, electronic shelf labels and more. This morning, the retailer announced it will now begin testing a revamped Scan & Go service as well, which leverages computer vision and machine learning to make mobile scanning easier and faster.

The current Scan & Go system, launched two years ago, requires Sam’s Club shoppers to locate the barcode on the item they’re buying and scan it using the Sam’s Club mobile app. The app allows shoppers to account for items they’re buying as they place them in their shopping cart, then pay in the app instead of standing in line at checkout.

However convenient, the system itself can still be frustrating at times because you’ll need to actually find the barcode on the item — often turning the item over from one side to the other to find the sticker or tag. This process can be difficult for heavier items, and frustrating when the barcoded label or tag has fallen off.

It also can end up taking several seconds to complete — which adds up when you’re filling a cart with groceries during a big stocking-up trip.

The new scanning technology will instead use computer vision and ML (machine learning) to recognize products without scanning the barcode, cutting the time it takes for the app to identify the product in question, the retailer explains.

In a video demo, Sam’s Club showed how it might take a typical shopper 9.3 seconds to scan a pack of water using the old system, versus 3.4 seconds using the newer technology.

Of course, the times will vary based on the shopper’s skill, the item being scanned and how well the technology performs, among other factors. A large package of water is a more extreme example, but one that demonstrates well the potential of the system… if it works.

The idea with the newly opened Dallas test store is to put new technology into practice quickly in a real-world environment, to see what performs well and what doesn’t, while also gathering customer feedback. Dallas was chosen as the location for the store because of the tech talent and recruiting potential in the area, and because it’s a short trip from Walmart’s Bentonville, Arkansas headquarters, the company said earlier.

Sam’s Club says it has filed a patent related to the new scanning technology, and will begin testing it this spring at the Dallas area “Sam’s Club Now” store. It will later expand the technology to the tools used by employees, too.

Powered by WPeMatico

Medivis has launched its augmented reality platform for surgical planning

After two years of development, Medivis, a New York-based company developing augmented reality data integration and visualization tools for surgeons, is bringing its first product to market.

The company was founded by Osamah Choudhry and Christopher Morley who met as senior residents at NYU Medical Center.

Initially a side-project, the two residents roped in some engineers to help develop their first prototypes and after a stint in NYU’s Summer Launchpad program the two decided to launch the company.

Now, with $2.3 million in financing led by Initialized Capital and partnerships with Dell and Microsoft to supply hardware, the company is launching its first product, called SurgicalAR.

In fact, it was the launch of the HoloLens that really gave Medivis its boost, according to Morley. That technology pointed a way toward what Morley said was one of the dreams for technology in the medical industry.

“The Holy Grail is to be able to holographically render a patient,” he said.

For now, Medivis is able to access patient data and represent it visually in a three-dimensional model for doctors to refer to as they plan surgeries. That model is mapped back to the patient to give surgeons a plan for how best to approach an operation.

“The interface between medical imaging and surgical utility from it is really where we see a lot of innovation being possible,” says Morley.

So far, Medivis has worked with the University of Pennsylvania and New York University to bring their prototypes into a surgical setting.

The company is integrating some machine learning capabilities to be able to identify the most relevant information from patients’ medical records and diagnostics as they begin to plan the surgical process.

“What we’ve been working on over this time is developing this really disruptive 3D pipeline,” says Morley. “What we have seen is that there is a distinct lack of 3D pipelines to allow people to directly interface… very quickly try to automate the entire rendering process.”

For now, Medivis is selling a touchscreen monitor, display and a headset. The device plugs into a hospital network and extracts medical imaging to display from their servers in about 30 seconds, according to Choudhry.

“That’s where we see this immediately being useful in that pre-surgical planning stage,” Choudhry says. “The use in surgical planning and being able to extend this through surgical navigation… Streamline the process that requires a large amount of pieces and components and setups so you only need an AR headset to localize pathology and make decisions off of that.”

Already the company has performed 15 surgeries in consultation with the company’s technology.

“When we first met Osamah and Chris, we immediately understood the magnitude of the problem they were out to solve. Medical imaging as it relates to surgical procedures has largely been neglected, leaving patients open to all sorts of complications and general safety issues,” said Eric Woersching, general partner, Initialized Capital, in a statement. “We took one look at the Medivis platform and knew they were poised to transform the operating room. Not only was their hands-free approach to visualization meeting a real need for greater surgical accuracy, but the team has the passion and expertise in the medical field to bring it all to fruition. We couldn’t be more thrilled to welcome Medivis to the Initialized family.”

Powered by WPeMatico

Facebook picks up retail computer vision outfit GrokStyle

If you’ve ever seen a lamp or chair that you liked and wished you could just take a picture and find it online, well, GrokStyle let you do that — and now the company has been snatched up by Facebook to augment its own growing computer vision department.

GrokStyle started as a paper — as AI companies often do these days — at 2015’s SIGGRAPH. A National Science Foundation grant got the ball rolling on the actual company, and in 2017 founders Kavita Bala and Sean Bell raised $2 million to grow it.

The basic idea is simple: matching a piece of furniture (or a light fixture, or any of a variety of product types) in an image to visually similar ones in stock at stores. Of course, sometimes the simplest ideas are the most difficult to execute. But Bala and Bell made it work, and it was impressive enough in action that Ikea on first sight demanded it be in the next release of its app. I saw it in action and it’s pretty impressive.

Facebook’s acquisition of the company (no terms disclosed) makes sense on a couple of fronts: First, the company is investing heavily in computer vision and AI, so GrokStyle and its founders are naturally potential targets. Second, Facebook is also trying to invest in its marketplace, and using the camera as an interface for it fits right into the company’s philosophy.

One can imagine how useful it would be to be able to pull up the Facebook camera app, point it at a lamp you like at a hotel and see who’s selling it or something like it on the site.

Facebook did not answer my questions regarding how GrokStyle’s tech and team would be used, but offered the following statement: “We are excited to welcome GrokStyle to Facebook. Their team and technology will contribute to our AI capabilities.” Well!

There’s an “exciting journey” message on GrokStyle’s webpage, so the old site and service is gone for good. But one assumes that it will reappear in some form in the future. I’ve asked the founders for comment and will update the post if I hear back.

Powered by WPeMatico

Computer vision startup AnyVision pulls in new funding from Lightspeed

While there have been a few massive surveillance startups in China that have raised funds on the back of computer vision advances, there’s seemed to be less fervor outside of that market. Tel Aviv-based AnyVision is aiming to leverage its computer vision chops in tracking people and objects to create some pretty clear utility for the enterprise world.

After announcing a $28 million Series A in mid-2018, the computer vision startup is bringing Lightspeed Venture Partners into the raise, closing out the round at $43 million.

“When you have a company with the technology AnyVision has, and the market need that I’m hearing from across industries, what you need to do is push the gas pedal and build an organization which can monetize and take on this opportunity to grow massively,” Lightspeed partner Raviraj Jain told TechCrunch.

Right now the 200-person company has its eyes on the security and identity markets as it aims to bring its computer vision technology into more industry-tailored solutions.

The company’s “Better Tomorrow” product delivers camera-agnostic surveillance insights from its object and human-tracking tech. “Sesame” is the company’s consumer-facing play for bringing mobile banking authentication to hundreds of millions of phones. The company is still looking to release a retail analytics platform to customers, as well.

Powered by WPeMatico

Agtech startup Imago AI is using computer vision to boost crop yields

Presenting onstage today in the 2018 TC Disrupt Berlin Battlefield is Indian agtech startup Imago AI, which is applying AI to help feed the world’s growing population by increasing crop yields and reducing food waste. As startup missions go, it’s an impressively ambitious one.

The team, which is based out of Gurgaon near New Delhi, is using computer vision and machine learning technology to fully automate the laborious task of measuring crop output and quality — speeding up what can be a very manual and time-consuming process to quantify plant traits, often involving tools like calipers and weighing scales, toward the goal of developing higher-yielding, more disease-resistant crop varieties.

Currently they say it can take seed companies between six and eight years to develop a new seed variety. So anything that increases efficiency stands to be a major boon.

And they claim their technology can reduce the time it takes to measure crop traits by up to 75 percent.

In the case of one pilot, they say a client had previously been taking two days to manually measure the grades of their crops using traditional methods like scales. “Now using this image-based AI system they’re able to do it in just 30 to 40 minutes,” says co-founder Abhishek Goyal.

Using AI-based image processing technology, they can also crucially capture more data points than the human eye can (or easily can), because their algorithms can measure and asses finer-grained phenotypic differences than a person might pick up on or be easily able to quantify just judging by eye alone.

“Some of the phenotypic traits they are not possible to identify manually,” says co-founder Shweta Gupta. “Maybe very tedious or for whatever all these laborious reasons. So now with this AI-enabled [process] we are now able to capture more phenotypic traits.

“So more coverage of phenotypic traits… and with this more coverage we are having more scope to select the next cycle of this seed. So this further improves the seed quality in the longer run.”

The wordy phrase they use to describe what their technology delivers is: “High throughput precision phenotyping.”

Or, put another way, they’re using AI to data-mine the quality parameters of crops.

“These quality parameters are very critical to these seed companies,” says Gupta. “Plant breeding is a very costly and very complex process… in terms of human resource and time these seed companies need to deploy.

“The research [on the kind of rice you are eating now] has been done in the previous seven to eight years. It’s a complete cycle… chain of continuous development to finally come up with a variety which is appropriate to launch in the market.”

But there’s more. The overarching vision is not only that AI will help seed companies make key decisions to select for higher-quality seed that can deliver higher-yielding crops, while also speeding up that (slow) process. Ultimately their hope is that the data generated by applying AI to automate phenotypic measurements of crops will also be able to yield highly valuable predictive insights.

Here, if they can establish a correlation between geotagged phenotypic measurements and the plants’ genotypic data (data which the seed giants they’re targeting would already hold), the AI-enabled data-capture method could also steer farmers toward the best crop variety to use in a particular location and climate condition — purely based on insights triangulated and unlocked from the data they’re capturing.

One current approach in agriculture to selecting the best crop for a particular location/environment can involve using genetic engineering. Though the technology has attracted major controversy when applied to foodstuffs.

Imago AI hopes to arrive at a similar outcome via an entirely different technology route, based on data and seed selection. And, well, AI’s uniform eye informing key agriculture decisions.

“Once we are able to establish this sort of relation this is very helpful for these companies and this can further reduce their total seed production time from six to eight years to very less number of years,” says Goyal. “So this sort of correlation we are trying to establish. But for that initially we need to complete very accurate phenotypic data.”

“Once we have enough data we will establish the correlation between phenotypic data and genotypic data and what will happen after establishing this correlation we’ll be able to predict for these companies that, with your genomics data, and with the environmental conditions, and we’ll predict phenotypic data for you,” adds Gupta.

“That will be highly, highly valuable to them because this will help them in reducing their time resources in terms of this breeding and phenotyping process.”

“Maybe then they won’t really have to actually do a field trial,” suggests Goyal. “For some of the traits they don’t really need to do a field trial and then check what is going to be that particular trait if we are able to predict with a very high accuracy if this is the genomics and this is the environment, then this is going to be the phenotype.”

So — in plainer language — the technology could suggest the best seed variety for a particular place and climate, based on a finer-grained understanding of the underlying traits.

In the case of disease-resistant plant strains it could potentially even help reduce the amount of pesticides farmers use, say, if the the selected crops are naturally more resilient to disease.

While, on the seed generation front, Gupta suggests their approach could shrink the production time frame — from up to eight years to “maybe three or four.”

“That’s the amount of time-saving we are talking about,” she adds, emphasizing the really big promise of AI-enabled phenotyping is a higher amount of food production in significantly less time.

As well as measuring crop traits, they’re also using computer vision and machine learning algorithms to identify crop diseases and measure with greater precision how extensively a particular plant has been affected.

This is another key data point if your goal is to help select for phenotypic traits associated with better natural resistance to disease, with the founders noting that around 40 percent of the world’s crop load is lost (and so wasted) as a result of disease.

And, again, measuring how diseased a plant is can be a judgement call for the human eye — resulting in data of varying accuracy. So by automating disease capture using AI-based image analysis the recorded data becomes more uniformly consistent, thereby allowing for better quality benchmarking to feed into seed selection decisions, boosting the entire hybrid production cycle.

Sample image processed by Imago AI showing the proportion of a crop affected by disease

In terms of where they are now, the bootstrapping, nearly year-old startup is working off data from a number of trials with seed companies — including a recurring paying client they can name (DuPont Pioneer); and several paid trials with other seed firms they can’t (because they remain under NDA).

Trials have taken place in India and the U.S. so far, they tell TechCrunch.

“We don’t really need to pilot our tech everywhere. And these are global [seed] companies, present in 30, 40 countries,” adds Goyal, arguing their approach naturally scales. “They test our technology at a single country and then it’s very easy to implement it at other locations.”

Their imaging software does not depend on any proprietary camera hardware. Data can be captured with tablets or smartphones, or even from a camera on a drone or using satellite imagery, depending on the sought for application.

Although for measuring crop traits like length they do need some reference point to be associated with the image.

“That can be achieved by either fixing the distance of object from the camera or by placing a reference object in the image. We use both the methods, as per convenience of the user,” they note on that.

While some current phenotyping methods are very manual, there are also other image-processing applications in the market targeting the agriculture sector.

But Imago AI’s founders argue these rival software products are only partially automated — “so a lot of manual input is required,” whereas they couch their approach as fully automated, with just one initial manual step of selecting the crop to be quantified by their AI’s eye.

Another advantage they flag up versus other players is that their approach is entirely non-destructive. This means crop samples do not need to be plucked and taken away to be photographed in a lab, for example. Rather, pictures of crops can be snapped in situ in the field, with measurements and assessments still — they claim — accurately extracted by algorithms which intelligently filter out background noise.

“In the pilots that we have done with companies, they compared our results with the manual measuring results and we have achieved more than 99 percent accuracy,” is Goyal’s claim.

While, for quantifying disease spread, he points out it’s just not manually possible to make exact measurements. “In manual measurement, an expert is only able to provide a certain percentage range of disease severity for an image example; (25-40 percent) but using our software they can accurately pin point the exact percentage (e.g. 32.23 percent),” he adds.

They are also providing additional support for seed researchers — by offering a range of mathematical tools with their software to support analysis of the phenotypic data, with results that can be easily exported as an Excel file.

“Initially we also didn’t have this much knowledge about phenotyping, so we interviewed around 50 researchers from technical universities, from these seed input companies and interacted with farmers — then we understood what exactly is the pain-point and from there these use cases came up,” they add, noting that they used WhatsApp groups to gather intel from local farmers.

While seed companies are the initial target customers, they see applications for their visual approach for optimizing quality assessment in the food industry too — saying they are looking into using computer vision and hyper-spectral imaging data to do things like identify foreign material or adulteration in production line foodstuffs.

“Because in food companies a lot of food is wasted on their production lines,” explains Gupta. “So that is where we see our technology really helps — reducing that sort of wastage.”

“Basically any visual parameter which needs to be measured that can be done through our technology,” adds Goyal.

They plan to explore potential applications in the food industry over the next 12 months, while focusing on building out their trials and implementations with seed giants. Their target is to have between 40 to 50 companies using their AI system globally within a year’s time, they add.

While the business is revenue-generating now — and “fully self-enabled” as they put it — they are also looking to take in some strategic investment.

“Right now we are in touch with a few investors,” confirms Goyal. “We are looking for strategic investors who have access to agriculture industry or maybe food industry… but at present haven’t raised any amount.”


Powered by WPeMatico

That night, a forest flew: DroneSeed is planting trees from the air

Wildfires are consuming our forests and grasslands faster than we can replace them. It’s a vicious cycle of destruction and inadequate restoration rooted, so to speak, in decades of neglect of the institutions and technologies needed to keep these environments healthy.

DroneSeed is a Seattle-based startup that aims to combat this growing problem with a modern toolkit that scales: drones, artificial intelligence and biological engineering. And it’s even more complicated than it sounds.

Trees in decline

A bit of background first. The problem of disappearing forests is a complex one, but it boils down to a few major factors: climate change, outdated methods and shrinking budgets (and as you can imagine, all three are related).

Forest fires are a natural occurrence, of course. And they’re necessary, as you’ve likely read, to sort of clear the deck for new growth to take hold. But climate change, monoculture growth, population increases, lack of control burns and other factors have led to these events taking place not just more often, but more extensively and to more permanent effect.

On average, the U.S. is losing 7 million acres a year. That’s not easy to replace to begin with — and as budgets for the likes of national and state forest upkeep have shrunk continually over the last half century, there have been fewer and fewer resources with which to combat this trend.

The most effective and common reforestation technique for a recently burned woodland is human planters carrying sacks of seedlings and manually selecting and placing them across miles of landscapes. This back-breaking work is rarely done by anyone for more than a year or two, so labor is scarce and turnover is intense.

Even if the labor was available on tap, the trees might not be. Seedlings take time to grow in nurseries and a major wildfire might necessitate the purchase and planting of millions of new trees. It’s impossible for nurseries to anticipate this demand, and the risk associated with growing such numbers on speculation is more than many can afford. One missed guess could put the whole operation underwater.

Meanwhile, if nothing gets planted, invasive weeds move in with a vengeance, claiming huge areas that were once old growth forests. Lacking the labor and tree inventory to stem this possibility, forest keepers resort to a stopgap measure: use helicopters to drench the area in herbicides to kill weeds, then saturate it with fast-growing cheatgrass or the like. (The alternative to spraying is, again, the manual approach: machetes.)

At least then, in a year, instead of a weedy wasteland, you have a grassy monoculture — not a forest, but it’ll do until the forest gets here.

One final complication: helicopter spraying is a horrendously dangerous profession. These pilots are flying at sub-100-foot elevations, performing high-speed maneuvers so that their sprays reach the very edge of burn zones but they don’t crash head-on into the trees. This is an extremely dangerous occupation: 80 to 100 crashes occur every year in the U.S. alone.

In short, there are more and worse fires and we have fewer resources — and dated ones at that — with which to restore forests after them.

These are facts anyone in forest ecology and logging are familiar with, but perhaps not as well known among technologists. We do tend to stay in areas with cell coverage. But it turns out that a boost from the cloistered knowledge workers of the tech world — specifically those in the Emerald City — may be exactly what the industry and ecosystem require.

Simple idea, complex solution

So what’s the solution to all this? Automation, right?

Automation, especially via robotics, is proverbially suited for jobs that are “dull, dirty, and dangerous.” Restoring a forest is dirty and dangerous to be sure. But dull isn’t quite right. It turns out that the process requires far more intelligence than anyone was willing, it seems, to apply to the problem — with the exception of those planters. That’s changing.

Earlier this year, DroneSeed was awarded the first multi-craft, over-55-pounds unmanned aerial vehicle license ever issued by the FAA. Its custom UAV platforms, equipped with multispectral camera arrays, high-end lidar, six-gallon tanks of herbicide and proprietary seed dispersal mechanisms have been hired by several major forest management companies, with government entities eyeing the service as well.

These drones scout a burned area, mapping it down to as high as centimeter accuracy, including objects and plant species, fumigate it efficiently and autonomously, identify where trees would grow best, then deploy painstakingly designed seed-nutrient packages to those locations. It’s cheaper than people, less wasteful and dangerous than helicopters and smart enough to scale to national forests currently at risk of permanent damage.

I met with the company’s team at their headquarters near Ballard, where complete and half-finished drones sat on top of their cases and the air was thick with capsaicin (we’ll get to that).

The idea for the company began when founder and CEO Grant Canary burned through a few sustainable startup ideas after his last company was acquired, and was told, in his despondency, that he might have to just go plant trees. Canary took his friend’s suggestion literally.

“I started looking into how it’s done today,” he told me. “It’s incredibly outdated. Even at the most sophisticated companies in the world, planters are superheroes that use bags and a shovel to plant trees. They’re being paid to move material over mountainous terrain and be a simple AI and determine where to plant trees where they will grow — microsites. We are now able to do both these functions with drones. This allows those same workers to address much larger areas faster without the caloric wear and tear.”

It may not surprise you to hear that investors are not especially hot on forest restoration (I joked that it was a “growth industry” but really because of the reasons above it’s in dire straits).

But investors are interested in automation, machine learning, drones and especially government contracts. So the pitch took that form. With the money DroneSeed secured, it has built its modestly sized but highly accomplished team and produced the prototype drones with which is has captured several significant contracts before even announcing that it exists.

“We definitely don’t fit the mold or metrics most startups are judged on. The nice thing about not fitting the mold is people double take and then get curious,” Canary said. “Once they see we can actually execute and have been with 3 of the 5 largest timber companies in the U.S. for years, they get excited and really start advocating hard for us.”

The company went through Techstars, and Social Capital helped them get on their feet, with Spero Ventures joining up after the company got some groundwork done.

If things go as DroneSeed hopes, these drones could be deployed all over the world by trained teams, allowing spraying and planting efforts in nurseries and natural forests to take place exponentially faster and more efficiently than they are today. It’s genuine change-the-world-from-your-garage stuff, which is why this article is so long.

Hunter (weed) killers

The job at hand isn’t simple or even straightforward. Every landscape differs from every other, not just in the shape and size of the area to be treated but the ecology, native species, soil type and acidity, type of fire or logging that cleared it and so on. So the first and most important task is to gather information.

For this, DroneSeed has a special craft equipped with a sophisticated imaging stack. This first pass is done using waypoints set on satellite imagery.

The information collected at this point is really far more detailed than what’s actually needed. The lidar, for instance, collects spatial information at a resolution much beyond what’s needed to understand the shape of the terrain and major obstacles. It produces a 3D map of the vegetation as well as the terrain, allowing the system to identify stumps, roots, bushes, new trees, erosion and other important features.

This works hand in hand with the multispectral camera, which collects imagery not just in the visible bands — useful for identifying things — but also in those outside the human range, which allows for in-depth analysis of the soil and plant life.

The resulting map of the area is not just useful for drone navigation, but for the surgical strikes that are necessary to make this kind of drone-based operation worth doing in the first place. No doubt there are researchers who would love to have this data as well.

Now, spraying and planting are very different tasks. The first tends to be done indiscriminately using helicopters, and the second by laborers who burn out after a couple of years — as mentioned above, it’s incredibly difficult work. The challenge in the first case is to improve efficiency and efficacy, while in the second case is to automate something that requires considerable intelligence.

Spraying is in many ways simpler. Identifying invasive plants isn’t easy, exactly, but it can be done with imagery like that the drones are collecting. Having identified patches of a plant to be eliminated, the drones can calculate a path and expend only as much herbicide is necessary to kill them, instead of dumping hundreds of gallons indiscriminately on the entire area. It’s cheaper and more environmentally friendly. Naturally, the opposite approach could be used for distributing fertilizer or some other agent.

I’m making it sound easy again. This isn’t a plug and play situation — you can’t buy a DJI drone and hit the “weedkiller” option in its control software. A big part of this operation was the creation not only of the drones themselves, but the infrastructure with which to deploy them.

Conservation convoy

The drones themselves are unique, but not alarmingly so. They’re heavy-duty craft, capable of lifting well over the 57 pounds of payload they carry (the FAA limits them to 115 pounds).

“We buy and gut aircraft, then retrofit them,” Canary explained simply. Their head of hardware, would probably like to think there’s a bit more to it than that, but really the problem they’re solving isn’t “make a drone” but “make drones plant trees.” To that end, Canary explained, “the most unique engineering challenge was building a planting module for the drone that functions with the software.” We’ll get to that later.

DroneSeed deploys drones in swarms, which means as many as five drones in the air at once — which in turn means they need two trucks and trailers with their boxes, power supplies, ground stations and so on. The company’s VP of operations comes from a military background where managing multiple aircraft onsite was part of the job, and she’s brought her rigorous command of multi-aircraft environments to the company.

The drones take off and fly autonomously, but always under direct observation by the crew. If anything goes wrong, they’re there to take over, though of course there are plenty of autonomous behaviors for what to do in case of, say, a lost positioning signal or bird strike.

They fly in patterns calculated ahead of time to be the most efficient, spraying at problem areas when they’re over them, and returning to the ground stations to have power supplies swapped out before returning to the pattern. It’s key to get this process down pat, since efficiency is a major selling point. If a helicopter does it in a day, why shouldn’t a drone swarm? It would be sad if they had to truck the craft back to a hangar and recharge them every hour or two. It also increases logistics costs like gas and lodging if it takes more time and driving.

This means the team involves several people, as well as several drones. Qualified pilots and observers are needed, as well as people familiar with the hardware and software that can maintain and troubleshoot on site — usually with no cell signal or other support. Like many other forms of automation, this one brings its own new job opportunities to the table.

AI plays Mother Nature

The actual planting process is deceptively complex.

The idea of loading up a drone with seeds and setting it free on a blasted landscape is easy enough to picture. Hell, it’s been done. There are efforts going back decades to essentially load seeds or seedlings into guns and fire them out into the landscape at speeds high enough to bury them in the dirt: in theory this combines the benefits of manual planting with the scale of carpeting the place with seeds.

But whether it was slapdash placement or the shock of being fired out of a seed gun, this approach never seemed to work.

Forestry researchers have shown the effectiveness of finding the right “microsite” for a seed or seedling; in fact, it’s why manual planting works as well as it does. Trained humans find perfect spots to put seedlings: in the lee of a log; near but not too near the edge of a stream; on the flattest part of a slope, and so on. If you really want a forest to grow, you need optimal placement, perfect conditions and preventative surgical strikes with pesticides.

Although it’s difficult, it’s also the kind of thing that a machine learning model can become good at. Sorting through messy, complex imagery and finding local minima and maxima is a specialty of today’s ML systems, and the aerial imagery from the drones is rich in relevant data.

The company’s CTO led the creation of an ML model that determines the best locations to put trees at a site — though this task can be highly variable depending on the needs of the forest. A logging company might want a tree every couple of feet, even if that means putting them in sub-optimal conditions — but a few inches to the left or right may make all the difference. On the other hand, national forests may want more sparse deployments or specific species in certain locations to curb erosion or establish sustainable firebreaks.

Once the data has been crunched, the map is loaded into the drones’ hive mind and the convoy goes to the location, where the craft are loaded with seeds instead of herbicides.

But not just any old seeds! You see, that’s one more wrinkle. If you just throw a sagebrush seed on the ground, even if it’s in the best spot in the world, it could easily be snatched up by an animal, roll or wash down to a nearby crevasse, or simply fail to find the right nutrients in time despite the planter’s best efforts.

That’s why DroneSeed’s head of Planting and his team have been working on a proprietary seed packet that they were unbelievably reticent to detail.

From what I could gather, they’ve put a ton of work into packaging the seeds into nutrient-packed little pucks held together with a biodegradable fiber. The outside is dusted with capsaicin, the chemical that makes spicy food spicy (and also what makes bear spray do what it does). If they hadn’t told me, I might have guessed, since the workshop area was hazy with it, leading us all to cough and tear up a little. If I were a marmot, I’d learn to avoid these things real fast.

The pucks, or “seed vessels,” can and must be customized for the location and purpose — you have to match the content and acidity of the soil, things like that. DroneSeed will have to make millions of these things, but it doesn’t plan to be the manufacturer.

Finally these pucks are loaded in a special puck-dispenser which, closely coordinating with the drone, spits one out at the exact moment and speed needed to put it within a few centimeters of the microsite.

All these factors should improve the survival rate of seedlings substantially. That means that the company’s methods will not only be more efficient, but more effective. Reforestation is a numbers game played at scale, and even slight improvements — and DroneSeed is promising more than that — are measured in square miles and millions of tons of biomass.

Proof of life

DroneSeed has already signed several big contracts for spraying, and planting is next. Unfortunately, the timing on their side meant they missed this year’s planting season, though by doing a few small sites and showing off the results, they’ll be in pole position for next year.

After demonstrating the effectiveness of the planting technique, the company expects to expand its business substantially. That’s the scaling part — again, not easy, but easier than hiring another couple thousand planters every year.

Ideally the hardware can be assigned to local teams that do the on-site work, producing loci of activity around major forests from which jobs can be deployed at large or small scales. A set of five or six drones does the work of one helicopter, roughly speaking, so depending on the volume requested by a company or forestry organization, you may need dozens on demand.

That’s all yet to be explored, but DroneSeed is confident that the industry will see the writing on the wall when it comes to the old methods, and identify them as a solution that fits the future.

If it sounds like I’m cheerleading for this company, that’s because I am. It’s not often in the world of tech startups that you find a group of people not just attempting to solve a serious problem — it’s common enough to find companies hitting this or that issue — but who have spent the time, gathered the expertise and really done the dirty, boots-on-the-ground work that needs to happen so it goes from great idea to real company.

That’s what I felt was the case with DroneSeed, and here’s hoping their work pays off — for their sake, sure, but mainly for ours.

Powered by WPeMatico