[MLUG - DISCUSSION] A Search Engine That's Becoming an Inventor

[Mike Miller <EMAIL:PROTECTED>]

Maybe they were trying to be fair and balanced when they interviewed Bill 
Gates for this article, but to me it came off as silly and a waste of 
space.  Not surprisingly, Gates pretends that Microsoft is similar to 
Google.  The reality is that Microsoft does not innovate and Google is all 
about innovation.  This is a pretty good article about Google though. 
--Mike

--------------------------------------------------------------------------


http://www.nytimes.com/2006/07/03/technology/03google.html

N.Y. Times
July 3, 2006

A Search Engine That's Becoming an Inventor

By SAUL HANSELL and JOHN MARKOFF

When Google was a graduate-school project being run from a Silicon Valley 
garage, its founders, Larry Page and Sergey Brin, built their own 
computers out of cheap parts meant for personal computers. They wanted to 
save money, and they felt that they could design a network of computers 
that would search the Web more efficiently than those available from 
traditional manufacturers.

Google no longer needs to pinch pennies. It is a solid member of the 
Fortune 500 with $9 billion in cash. Still, it is stubbornly sticking to 
its do-it-yourself approach to technology. Even as it spends more than 
$1.5 billion this year on operations centers and technology, most of the 
hundreds of thousands of servers it will deploy are being custom-made 
based on Google's own eccentric designs.

To be closer to its users and speed response time, it is building a 
worldwide string of data centers, including a huge site in The Dalles, 
Ore., with technologies it designed to reduce its ravenous need for 
electricity. These computers in turn use software developed with advanced 
tools that Google also designed itself. There are signs that Google is 
even preparing to create its own custom microchips.

"Google is as much about infrastructure as it is about the search engine," 
said Martin Reynolds, an analyst with the Gartner Group. "They are 
building an enormous computing resource on a scale that is almost 
unimaginable." He said he believed that Google was the world's 
fourth-largest maker of computer servers, after Dell, Hewlett-Packard and 
I.B.M.

Google's biggest rivals, Microsoft and Yahoo, certainly write much of 
their own software, and they work to configure their computers and data 
centers to their own needs. But they largely buy machines from existing 
manufactures like Dell, Sun Microsystems and Rackable Systems.

"At some point you have to ask yourself what is your core business," said 
Kevin Timmons, Yahoo's vice president for operations. "Are you going to 
design your own router, or are you going to build the world's most popular 
Web site? It is very difficult to do both."

Google, in fact, has decided it will do both. In many ways, it still has 
the head of a graduate-school project grafted onto the body of a 
multinational corporation. The central tenet of its strategy is that its 
growing cadre of world-class computer scientists can design a network of 
machines that can store and process more information more efficiently than 
anyone else.

Mr. Reynolds estimated that Google's computing costs are half those of 
other large Internet companies and a tenth those of traditional corporate 
technology users.

Google will not comment on its costs, but it does claim an advantage. "We 
don't think our competitors can deploy systems cheaper, faster or at 
scale," Alan Eustace, Google's vice president for research and systems 
engineering, told analysts in March. "That will give us a two-, three-, 
five-year lead."

Despite those boasts, some argue that Google's home-brew approach is 
unnecessary and inefficient, a headstrong indulgence masked for now by the 
growth and profitability of its advertising business. And Google's rivals 
say their networks are plenty efficient and powerful.

"Google doesn't have anything magic here," Bill Gates, the Microsoft 
chairman, said in an interview. "We spend a little bit more per machine. 
But to do the same tasks, we have less machines."

Google is notoriously secretive about its technology. Yet it also has 
published papers on some of its developments and been granted patents on 
others. These, along with the public statements of Google executives and 
interviews with current and former employees, vendors and other technology 
executives, paint a picture of a company devoted to pushing the boundaries 
of modern computer science, and applying those concepts on a vast scale.

"Google took the best ideas from the supercomputer research community and 
wove them into a working system," said Stephen E. Arnold, a technology 
consultant to investors and the author of "The Google Legacy" 
(Infonortics, 2005), a book on Google's technology.

Some of its innovations are designed to wring pennies from its growing 
spending on technology. Last year, it was granted a patent (06906920) on a 
"drive-cooling baffle," meant to funnel air into a rack of computers held 
together with Velcro, a Google design signature.

But some innovations are bolder, like a series of software tools that 
simplify the way it can divide a problem to be handled by thousands of 
processors simultaneously, an approach called parallel processing.

One such program, called MapReduce, is based on ideas discussed in 
computer science literature for decades, according to Urs Hölzle, Google's 
senior vice president for operations. "What surprised us was how useful it 
turned out to be in our environment," he said.

MapReduce, he said, "allows Joe Schmo software engineer to process large 
amounts of data and take advantage of our infrastructure."

Mr. Arnold, the consultant, said these tools created a significant cost 
advantage. "If you talk to guys who work in massively parallel computing 
operations, as much as 30 percent of their coding time is spent trying to 
figure out how to get the thing to run," he said.  Google "has figured out 
how they can reduce a lot of the hassle and work of creating parallel 
applications."

Mr. Gates acknowledged that MapReduce was a significant technology, but he 
asserted that Microsoft was building its own parallel processing software, 
opening another front in the technological war between the two companies.

"They did MapReduce; we have this thing called Dryad that's better," Mr. 
Gates said. "But they'll do one that's better."

Moreover, Google's focus on building general purpose tools and systems is 
different from that of most companies, which develop systems tailored to 
specific applications. And it is building these systems rapidly, with the 
billions of dollars in cash it generates and the thousands of engineers it 
hires each year. It hopes that it can build a lead that will allow it to 
create products that do more, for less money, than its rivals.

"If they can get a 30 percent cost advantage, in operating a service on 
the Internet that is a huge difference," said John M. Lervik, the chief 
executive of Fast Search & Transfer, a Norwegian search company.

Google's academic approach can be traced not only to its founders' 
graduate work in computer science, but even to their early home life, Mr. 
Arnold said, noting that Mr. Page and Mr. Brin had come from families with 
expertise in computer science and mathematics.

"The stuff they did in 1996 to 1998 was not as immature as it should have 
been," he said of the Google founders. He said that told him the two men 
learned a lot "when their parents were talking at the kitchen table."

By the time Mr. Page and Mr. Brin were designing Google, parallel 
processing was more than an academic dream; it was enabled on a large 
scale by the low prices of processors, memory and disk drives used to make 
personal computers. These components were hardly of the highest quality 
and could be counted on to fail often.

Mr. Page designed the initial Google servers, with the assumption that 
parts would fail on a regular basis. At first he tried to simplify 
assembly -- and reduce the presumed repair time -- by not fastening 
components to the servers at all but simply laying them on a bed of cork. 
This proved to be unstable, and so parts were connected with Velcro.

"Nobody builds servers as unreliably as we do," Mr. Hölzle said in a 
speech last year at CERN, the Swiss particle physics institute. Google is 
reducing cost while maintaining performance by shifting the burden of 
reliability from hardware to software -- individual hardware components 
can fail, but software automatically shifts the local task and the data to 
other machines.

For example, Google designed a software system it calls the Google File 
System that keeps copies of data in several places so Google does not have 
to worry when one of its cheap servers fails. This approach also means 
that it does not have to make regular backup copies of its data as other 
companies do.

Another system, called the Google Work Queue, allows a big pool of servers 
to be assigned to various tasks as needed and reassigned to other projects 
later. This concept, called "virtualization," has become a trend among 
large data center operators, which also want to reduce the expense of 
having separate servers dedicated to each system. But most companies buy 
commercial software to track which computers are doing what, a complex 
process.

While Google's servers are built on inexpensive parts, the designs it uses 
have been modified every year or so, to improve their efficiency and 
increasingly to customize them to Google's applications. The current 
generation uses the powerful Opteron chip from Advanced Micro Devices, 
which uses less power than the Intel chips Google had used.

Google is among Advanced Micro's five largest clients, and the largest 
that does not make computers to resell, according to a semiconductor 
industry executive with knowledge of Advanced Micro's business.

Google is increasingly doing business with Sun Microsystems as well. Sun, 
known for systems that are both reliable and expensive, would not seem a 
natural match for a company that emphasizes economy and self-sufficiency. 
But Eric E. Schmidt, Google's chief executive, is a former Sun executive, 
and Sun has developed a new microchip that is especially efficient in 
electricity use.

Moreover, Google increasingly needs systems that are less likely to fail 
than those it uses for its search engine in order to handle important 
information, like e-mail and payments in its new Google Checkout service.

Beyond servers, there are signs that Google is now designing its own 
microchips. The company has hired many of the engineers responsible for 
the Digital Equipment Corporation's well-regarded Alpha chip.

"Google's next step is to build high-performance silicon," said Mark 
Stahlman, an independent technology analyst.

Mr. Hölzle said Google had considered custom semiconductor design, but he 
declined to say if the company had built any. He said that, in general, 
Google did not want to build anything from scratch if it could buy 
something that was just as good.

But he added that Google continued to believe that its approach to 
designing its own cheap and fast computer networks gave it an edge.

"Having lots of relatively unreliable machines and turning them into a 
reliable service is a hard problem," Mr. Hölzle said. "That is what we 
have been doing for a while."


Copyright 2006 The New York Times Company

_______________________________________________
discussion mailing list
EMAIL:PROTECTED
http://mlug.missouri.edu/mailman/listinfo/discussion