When you’re on a mobile device, the image search part of Microsoft’s Bing search engine on the web now lets you highlight a specific portion of an image and find images that are similar to that particular part. To access the feature, you simply tap the magnifying glass in the top left corner of the image. It came first to Bing’s native iOS app earlier this year, and it arrived on Bing’s mobile site in May, a Microsoft spokesperson told VentureBeat in an email.
But before Bing got this nifty functionality, it was available on Pinterest, that app for pinning photos and other content to boards that you can share. In September 2014 Pinterest started experimenting with “visually similar results” before rolling it out to all users on web and mobile in November.
It wouldn’t be fair to say that Bing is catching up to Pinterest in the domain of visual search, though, because Bing actually launched visual searching using the camera on your mobile device before Pinterest introduced that feature.
The general point, though, is that technology companies have been developing more efficient ways for people to find exactly what they’re looking for.
There are no armies of part-time employees sitting around and picking out recommendations for images that might be relevant to end users — it’s happening automatically, using a type of artificial intelligence called deep learning, which involves training artificial neural networks on lots of data, such as images, and having them make inferences about new data. Microsoft was exploring deep learning in the context of Bing image search as far back as 2013. Two months after Microsoft published a blog post documenting that work, Pinterest announced its acquisition of VisualGraph, the team that later built the visually similar search into Pinterest.
Google, which has done extensive research in image recognition and deep learning, has not yet deployed visually similar search on mobile devices, although it has taken steps in that direction — last month it introduced object recognition in the Google Now on Tap feature of Android, and earlier this week that feature became capable of reading QR codes and barcodes. (Bing’s native Android and iOS apps can read those, too.) Reverse image search has long been available from Google — and Bing — but it doesn’t let you select a specific part of a given image and ask Google to find images that are similar to that part.
Meanwhile, image search from both Google and Bing have previously taken inspiration from Pinterest’s carousel of keyword suggestions to refine searches.
Microsoft did not immediately respond to a request for comment. Pinterest had no comment.