One of the biggest problems when talking about advances in AI is that data scientists and engineers are problem solvers. They're not always looking at the commercial possibilities of their research.
Earlier this year, Facebook announced pioneering computer vision technology that can extract a recipe from a picture of a plate of food. This is a challenge that has stumped researchers for a number of years.
In the case of the recipe problem, previous efforts focused on matching photographs of food with thousands of images in a database. Once you had a decent match, all you had to do was call up the corresponding recipe.
Simple, but not very subtle. No match, no recipe. The Facebook team took a different approach using two neural networks. The first extracts ingredients from an image.
The second takes this list to make a smart guess about the dish in question, and assemble a recipe. It can even fill in the gaps for substances that aren't visible in the photograph (the salt in hamburger, for example).
Now, there are limitations to this approach. Sometimes important ingredients are overlooked. Esoteric dishes are more challenging to read. But recreating recipes isn't really the main goal.
Instead, Facebook is looking for ways to 'squeeze' information from an image and turn it into something more valuable than just a list of tags. And you can bet they'll find some use for it in the not too distant future.
As for the rest of us, don't forget to read beneath the AI headlines - there's always something exciting going on even if it isn't obvious at first glance. And for the time being don't throw away your recipe books. The computer vision chef isn't coming to your kitchen any time soon.
Naturally, the success of this method depends on the size and quality of the cookbook, the handiwork of both the photographer and the chef, and some pot luck. “It is hard to match if a recipe isn’t in the data set and the image or dish appearance are different to the data set” says Drozdzal. In other words, the retrieval approach is like finding a needle in a haystack when the system doesn’t know what a needle looks like.