Simon’s Blog

Well, you’re almost bound to have checked out Riya’s new site - Like.com by now. So, I won’t bore you with a long introduction… essentially, Like.com is a new type of search engine. It lets you search images of consumer products, based on what they look like (photos), not text descriptions. So… what do I think of what they’ve achieved?

Well, first let me say that content-based searching of images is a genuinely hard problem. One of my companies does state-of-the-art content-based searching of video, as well as static images. It’s a huge amount of fun - but also challenging when it comes to algorithm development. Why? Well there are two big problems.

First, different people see different things in images. It this respect it’s like natural language. There is no “one true interpretation” of an image, or a piece of written language. For example, if I write, “Britney Spears is so talented” - what do I mean? I could mean that I think she is genuinely talented, or I could be being sarcastic and meaning that she is completely untalented: you have no way of knowing. And neither does a computer. So, whatever the computer “sees” in a photograph, it might not be anything like what you see.

Secondly, computers aren’t fast enough. If you want to push the boundaries of computer vision, you need a lot of computing power. At the state-of-the-art, content-based searching of large volumes of image data is a true super-computing problem.

So with that said, back to Riya, and Like.com. From my quick look-see, in order to make the analyses fast enough, it looks to me that they’re actually operating rather far away from the state-of-the-art in computer vision. Many of the results look like they could be achieved by using pretty simple, classical image analysis techniques and algorithms. What do I mean by this? Well, I mean that the algorithms often appear to have no real “concepts” built in for what they’re “looking at”. That’s what a state-of-the-art approach to this problem would have. For example, when looking at a part of an earring, Like.com appears to have no concepts of what those parts might be. Let’s take a concrete example. Let’s say I want to find earrings with a particular type of bar design (the part that goes through the ear). When I do a likeness search for that part of the earring, the algorithm is clearly looking at the overall shape of the earring, not the design of the bar. Check it out for yourself. Notice that the design of the bar is a sort of “cross shape”, and see how in the seventh best match, the ornamental design of the earring is a cross. Thus, the lack of an underlying model means that matches simply don’t end up being “like” what I was looking for.

Another giveaway to the simple underlying algorithms at Like.com is the fact that they’re often not “rotation-independent”. What does that mean? It means that it matters much more “which way up” the object is, than it matters what the object actually “looks like”. Staying with the previous example, notice how the ear-rings in the results of the search tend to be oriented vertically. Now, lets search on an almost identical earring - the only difference being that in the search photo, the earring is shown horizontally. You can see immediately that in these search results, the earrings returned from the search tend to be oriented horizontally. That means that when I’m searching for products, I will miss loads of products that are just like what I’m looking for, simply because of which way up the product was when it was photographed.

It’s easy to understand why the Riya guys have ended up where they have. Making their systems work fast will have been a major challenge for them. And they’ve clearly had to go with simple algorithms to get where they needed to be in terms of performance. Whether they’ve done the best possible job (used the most efficient algorithms to get any given quality of results), given where we are today with compute power, I’m not sure. Understanding that would require a much deeper look at what they’re doing.

The big question, however, is - does all this matter?

It might matter from the point of view of competition. If you want to have the highest-quality state-of-the-art computer vision algorithms, you need a “hard-to-assemble” team of top-talent PhDs. If you want to do what Like.com does, however, you don’t need that kind of top talent. In other words, it wouldn’t hard to compete from a technical point of view - it appears, from my superficial analysis, to be broadly equivalent to simple image processing.

However, where the quality of Like.com’s work will matter most will be in how well the search engine works from the perspective of the average consumer. In other words - how easy and pleasurable is it to find stuff that is not only in some senses “like” what they were looking for, but is also “stuff that they like and want to buy”. If the limitations of the Like.com algorithms mean that people spend forever fiddling with parameters and clicking to find stuff that they like, then it will fail. If, on the other hand, people can find stuff they really like quickly and frequently, then it may succeed. “Quickly and frequently” is important because then the site will catch people’s attention - if they tend to see products they are interested in from their searches, they’ll spend hours browsing (and buying).

I wish all involved with Like.com success - it’s great to see people pioneering content-based searching of images in the search engine space. There’s no doubt in my mind that content-based searching of unstructured data is a big part of the future. Whether that future is here now (or not), followers of Like.com are about to find out. Ladies and Gentlemen, place your bets…

[Update: I have done a non-statistical consumer test on the site. Women seem to really love the shoe browsing and searching (not sure if this is because women that love shoes tend to love lots of shoes!). Not so impressed by jewelry search - more difficult to find stuff they like. And, they wish they could search for belts - which, apparently, are this season's "must have"]