X

We're all guinea pigs in Google's search experiment

Google tests slightly different versions of its search pages and carefully watches how users respond. Also: the minimalist search page began as an accident, not by design.

Stephen Shankland Former Principal Writer
Stephen Shankland worked at CNET from 1998 to 2024 and wrote about processors, digital photography, AI, quantum computing, computer science, materials science, supercomputers, drones, browsers, 3D printing, USB, and new computing technology in general. He has a soft spot in his heart for standards groups and I/O interfaces. His first big scoop was about radioactive cat poop.
Expertise Processors, semiconductors, web browsers, quantum computing, supercomputers, AI, 3D printing, drones, computer science, physics, programming, materials science, USB, UWB, Android, digital photography, science. Credentials
  • Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
Stephen Shankland
4 min read

SAN FRANCISCO--When it comes to search quality, Google has a split personality.

Google uses a method called split A/B testing to measure exactly what changes it should make to its main search Web site--both to its famously Spartan search box and to the results it produces. With the approach, Google shows different versions of the pages to users and measures how they respond, said Marissa Mayer, vice president of search products and user experience, in a speech at the Google I/O conference here Thursday.

Marissa Mayer, vice president of search products and user experience at Google, speaks at the Google I/O conference.
Marissa Mayer, vice president of search products and user experience at Google, speaks at the Google I/O conference. Stephen Shankland/CNET News.com

For example, Mayer said, the company wanted to find out how many search results to show users--the customary 10, or 20, 25, or 30? When asked directly, users said they'd like more results on a page, but testing showed otherwise.

Specifically, Google found that when the results increased to 30 per page, people searched 20 percent less overall, Mayer said. After much analysis of server logs, the company found it was because it took about twice as long to display the longer results list for the user, and speed matters.

"As Google gets faster, people search more, and as it gets slower, people search less," she said.

The same effect happened with Google Maps. When the company trimmed the 120KB page size down by about 30 percent, the company started getting about 30 percent more map requests. "It was almost proportional. If you make a product faster, you get that back in terms of increased usage," she said.

Split A/B testing also led Google to refine exactly how much white space to pad around its logo and other elements on the search results page. And it changed from the industry practice of a pale blue background behind ads to a pale yellow background. People not only clicked on ads more, they also searched more in general, she said.

The subject clearly is close to Mayer's heart. She's an engineer who also has an interest in the more aesthetic realm of design.

"On the Web in general, (creating sites) is much more a design than an art," she said. "You can find small differences and mathematically learn which is right."

A history of Google's search page
Google's search page, with its abundance of empty white space and its almost boastful "I'm feeling lucky" button, looks downright ordinary today. But it wasn't always the case.

Mayer said that back when Google was a relatively unknown 80-person start-up, the company tested Stanford students on how well they could use Google to find which country won the most gold medals in the 1994 Olympics. The result: students would sit in front of the Google screen for 15 seconds, 30 seconds, 45 seconds, a minute...Google was perplexed.

So Mayer would eventually intervene and ask what was holding up the searchers. "I'm waiting for the rest of it," they'd say. Clearly they expected more of the flashy ads and busy text of other search pages of the 1990s.

"The very first home page was that misunderstood. People didn't resonate with it," Mayer said. One woman even thought the Web site was a fake construction that was part of a psychology experiment.

Marissa Mayer, vice president of search products and user experience at Google, shows three slightly different versions of Google's search results page that the company tested with users. The top, with the least white space, was more popular as measured by how much users searched.
Marissa Mayer, vice president of search products and user experience at Google, shows three slightly different versions of Google's search results page that the company tested with users. The top, with the least white space, was more popular as measured by how much users searched. Stephen Shankland/CNET News.com

As a result, the company put a copyright notice at the bottom of the page. "It's not there for legal reasons," Mayer said. "It's there as punctuation. That's it. (It tells the searcher) 'Nothing else is coming; please start searching now.'"

Mayer oversaw much of Google's design, but the sparse start page wasn't her doing and wasn't even part of a plan, she said. Instead, it was the design of co-founder Sergey Brin.

Why so minimalist, she wondered? Sergey's response: "We didn't have a Webmaster, and I don't do HTML."

Google also decided against presenting newbie and expert versions of its search page, Mayer said. People figure it out quickly, so the company aims its product at the experts.

"The learning curve on search is really fast," she said. "People go from 'Where can I get spaghetti and meatballs in Silicon Valley' to 'italian food san jose' really fast," she said.

The complexity of search
Google tries to look simple from the outside, but its search process is, as no one will be surprised to hear, quite complicated.

A typical search will require actions from between 700 to 1,000 machines today, Mayer said.

That's grown more complicated as Google moved to what it calls universal search, in which the regular search results are mixed with results from its other search areas such as books, news, blogs, images, and maps.

With those other, narrower search services, Google lost sight of the simplicity users need in its haste to bring the services to market, Mayer said.

"The urgent can drown out the important," she said. "It's great we did these urgent, expedient search indices, but what we really need to do is put them on the same page."

Indeed, in the longer run, she envisions universal search growing far more sophisticated, with a page filled with "images, videos, and graphs--not a list of 10 URLs but as a holistic answer to your query."

Search also will become more personal, with results tailored for individuals. (Google has begun offering personal results for those who sign up.) One reason personalization is important, she said: a very useful factor Google can weigh in its search results is what a person just tried searching for previously, she said. Knowing that, "We know what you discarded or are refining from," she said.

"We know 10 years out search will probably be a lot more personalized," she said. "And there will be a lot more content to index. When we think how to build search, it's important to think about the 10-year case."