At long last, pet portraits with background blur are possible on the iPhone XR

Published October 29, 2018

The new iPhones have some great new photography features, but the XR lacks a couple, for instance portrait mode for non-people subjects, owing to its sadly having only the one camera. So last year! Fortunately third-party camera app Halide is here to help you get that professional-looking bokeh in your doggo shots.

There’s more to this than simply the lack of a second camera. As you know, because you read my article, The future of photography is code — and the present too, really. What’s great about this is that features that might otherwise rely on specific hardware, a chip or sensor, can often be added in software. Not always, but sometimes.

The future of photography is code

In the case of the iPhone XR, the lack of a second camera means depth data is very limited, meaning the slack has to be taken up with code. The problem was that Apple’s machine learning systems on there are only trained to recognize and create high-quality depth maps of people. Not dogs, cats, plants or toy robots.

People would be frustrated if the artificial background blur inexplicably got way worse when it was pointed at something that wasn’t a person, so the effect just doesn’t trigger unless someone’s in the shot.

The Halide team, not bound by Apple’s qualms, added the capability back in by essentially taking the raw depth data produced by the XR’s “focus pixels” and applying their own processing and blur effect to make sure it doesn’t do weird things. It works on anything that can realistically be separated from the background — pets, toy robots, etc. — because it isn’t a system specific to human faces.

As they write in a blog post explaining some of this at length, the effect isn’t perfect, and because of how depth data is sent from the camera to the OS, you can’t preview the function. But it’s better than nothing at all, and maybe people on Instagram will think you shelled out for the XS instead of the XR (though you probably made the right choice).

The update (1.11) is awaiting Apple approval and should be available soon. If you don’t already own Halide, it costs $6. Small price to pay for a velvety background blur in your chinchilla pics.

Spectral Edge’s image enhancing tech pulls in $5.3M

Published April 10, 2018

Cambridge, U.K.-based startup Spectral Edge has closed a $5.3M Series A funding round from existing investors Parkwalk Advisors and IQ Capital.

The team, which in 2014 spun the business out of academic research at the University of East Anglia, has developed a mathematical technique for improving photographic imagery in real-time, also using machine learning technology.

As we’ve reported previously, their technology — which can be embedded in software or in silicon — is designed to enhance pictures and videos on mass-market devices. Mooted use cases include for enhancing low light smartphone images, improving security camera footage or even for drone cameras.

This month Spectral Edge announced its first customer, IT services provider NTT data, which said it would be incorporating the technology into its broadcast infrastructure offering — to offer its customers an “HDR-like experience”, via improved image quality, without the need for them to upgrade their hardware.

“We are in advanced trials with a number of global tech companies — household names — and hope to be able to announce more deals later this year,” CEO Rhodri Thomas tells us, adding that he expects 2-3 more deals in the broadcast space to follow “soon”, and enhance viewing experiences “in a variety of ways”.

On the smartphone front, Thomas says the company is waiting for consumer hardware to catch up — noting that RGB-IR sensors “haven’t yet begun to deploy on smartphones on a great scale”.

Once the smartphone hardware is there he reckons its technology will be able to help with various issues such as white balancing and bokeh processing.

“Right now there is no real solution for white balancing across the whole image [on smartphones] — so you’ll get areas of the image with excessive blues or yellows, perhaps, because the balance is out — but our tech allows this to be solved elegantly and with great results,” he suggests. “We also can support bokeh processing by eliminating artifacts that are common in these images.”

The new funding is going towards ramping up Spectral Edge’s efforts to commercialize its tech, including by growing the R&D team to 12 — with hires planned for specialists in image processing, machine learning and embedded software development.

The startup will also focus on developing real-world apps for smartphones, webcams and security applications alongside its existing products for the TV & display industries.

“The company is already very IP strong, with 10 patent families in the world (some granted, some filed and a couple about to be filed),” says Thomas. “The focus now is productizing and commercializing.”

“In a year, I expect our technology to be launched or launching on major flagship [smartphone] devices,” he adds. “We also believe that by then our CVD (color vision deficiency) product, Eyeteq, is helping millions of people suffering from color blindness to enjoy significantly better video experiences.”

A system to tell good fake bokeh from bad

Published February 21, 2018

The pixel-peepers at DxOMark have shared some of the interesting metrics and techniques they use to judge the quality of a smartphone’s artificial bokeh, or background blur in photos. Not only is it difficult to do in the first place, but they have to systematize it! Their guide should provide even seasoned shooters with some insight into the many ways computational bokeh varies in quality. Read More