It’s election fever in the UK: in dramatic results, the incumbent party is being pummelled at the polls. So too for search engines? Microsoft employee Michael Kordahi set up a blind search test. Perform a search, select your favourite from three columns of results. It started well for Bing, but market leader Google soon asserted a lead:
Blind search engine test at http://blindsearch.fejus.com Right now: "Google: 45%, Bing: 33%, Yahoo: 21% | 8,518 votes"
said Mr Google Matt Cutts.
Still, that’s not bad for Bing, considering that its market share is tiny in comparison to Google. 5.5% vs 81.5% according to stats I dug up for this Register piece. The real loser is Yahoo, whose second place in search is now under threat from the Microsoft juggernaut.
But can you trust the results? At some point last night Yahoo started an unlikely surge:
Internet search blind test: Google: 34%, Bing: 26%, Yahoo: 40% Try it out! http://blindsearch.fejus.com/
tweeted Bill Hamilton a few hours after Cutts. Someone was gaming the system:
not surprisingly, #blindsearch has been compromised you can still play, but i’m not currently showing results
said Kordahi, as Yahoo hit 57%.
Will Kordahi be able to insulate his test from fraudsters? Who knows; but it is still an interesting experiment.
I tried the test and found the results generally close, with a small edge to Google in my searches. Still, it would be interesting to measure not only which results are best, but also the margin of difference. In the past I’ve found Live Search almost useless, so Bing has made a substantial improvement from my perspective. The UI changes are important too. I’m a minimalist at heart, which again favours Google, but I like some of Bing’s features, especially the site and video previews.
Google’s Wave is of course more interesting from a technical perspective; but it would be a mistake to downplay the business significance of Microsoft improving its search market share. Search drives advertising income.
It’s also worth noting that in search, quantity drives quality. Program Manager Nathan Buggia explained to me how Bing’s categorisation feature works:
For the categorised results those are driven more off the search behaviour we see on our web site, not actually the semantic information that we infer from their web site. What we’ve done is to take all the queries that come into live search and analysed them to see what user intent those queries have. We take a look at the other search terms that they use to figure out where they go, we aggregate that information and use that to define categories, and we are able to draw on that.
Currently Bing only displays category tabs for around the top 10-20% of searches. The reason it is limited to that, according to Buggia, is insufficient volume of data. Using the Xbox as an example, he told me:
If we have a high enough volume of XBox data and we’ve seen that there are a specific set of intents that people are looking for, then we feel confident enough to show the quick tabs.
In other words, Bing could improve its results simply by more people using it.
What happens next? The easy prediction is that Bing will make at least small gains in market share, and that Yahoo will likely decline, perhaps to third place. For Microsoft, that would be no small achievement, but would do little to dislodge the big G. Further, if it sees significant traffic moving to Bing, Google will be quick to counter it with its own improvements. Personally I would like to see more competition in search, which for many users forms a portal that controls which sites they see and which they do not see, but a good launch for Bing is not enough to effect real change.
It could be the beginning of a change though, and that possibility makes Bing worth watching.