home > book reviews > Steven Levy
Google > Steven Levy


Books

Books : reviews

Steven Levy.
Hackers.
Penguin. 1984

Steven Levy.
Artificial Life.
Jonathan Cape. 1992

rating : 2.5 : great stuff

The story of the pioneers of artificial life

Steven Levy.
Crypto.
Penguin. 2000

Steven Levy.
In the Plex: how Google thinks, works, and shapes our lives.
Simon & Schuster. 2011

rating : 3 : worth reading
review : 26 December 2012

I've known for a while that Google offers spelling corrections (my spelling is none too hot, and my typing a little erratic, so I often see the "Showing results for <correctly spelled words>; search instead for <what I actually typed>" result. What I did not know, until I read this history of Google, was how the corrections are implemented. If I had thought about it, I might have assumed some sort of dictionary checker, although on reflection, the suggestions are usually so much better than anything any other spell checker comes up with. What Google actually uses is user data, gathered from analysing user corrections.

Google uses this "insane amounts of user data plus extremely clever algorithms to exploit that data in novel ways, in order to perform the task at hand" to the wide range of things it does. I loved learning about all these different applications of the approach. Levy details this technique in Google's approach to language translation:

p65. … Google's system created a "language model" for each tongue Och's team examined. The next step was to work with texts in different languages that had already been translated and let the machines figure out the implicit algorithms that dictate how one language converts to another. ...
     The most important data were pairs of documents that were skill-fully translated from one language to another. Before the Internet, the main source material for these translations had been corpuses such as UN documents that had been translated into multiple languages. But the web had produced an unbelievable treasure trove-and Google's indexes made it easy for its engineers to mine billions of documents, unearthing even the most obscure efforts at translating one document or blog post from one language to another. Even an amateurish translation could provide some degree of knowledge, but Google's algorithms could figure out which translations were the best by using the same principles that Google used to identify important websites. …
      ... "One of the things we did was to build very, very, very large language models, much larger than anyone has ever built in the history of mankind." Then they began to train the system. To measure progress, they used a statistical model that, given a series of words, would predict the word that came next. Each time they doubled the amount of training data, they got a .5 percent boost in the metrics that measured success in the results. "So we just doubled it a bunch of times."

Back in the day, AI was mainly about clever algorithms. Then it got rebadged as "Machine Learning", using similar algorithms but now applied to messy real world, rather than artificial or sanitised data. Now Google are taking the next step in AI: qualitative improvements in functionality from quantitative changes in data volumes, plus the clever algorithms:

p100. compressing data was equivalent in many ways to understanding it. That concept ... could be a key to algorithmically squeezing meaning from web pages.

Going from data to meaning via compression algorithms!

This insistence of being data driven, of backing up arguments with statistics and evidence, permeates everything they do, including performance management.

p163. OKRs [Objectives and Key Results] became an essential component of Google culture. Every employee had to set, and then get approval for, quarterly OKRs and annual OKRs. There were OKRs at the team level, the department level, and even the company level. ...
     … [Googlers] saw the OKRs as data, a means of putting a number on the traditionally gooey means of assessing performance. It was essential that OKRs be measurable. ... Even worse than failing to make an OKR was exceeding the standard by a large measure; it implied that an employee had sandbagged it, played it safe, thought small. Google had no place for an audacity-challenged person whose grasp exceeded his reach.
     The sweet spot was making about .7 or .8 of your OKR.

They are not always driven by data, however. They do have some prejudices:

p140. Google persisted in asking for [SAT scores and GPAs] even after its own evidence showed that the criteria weren't relevant to how well people actually performed at Google.

One of the key messages from this story is how old intuitions about what is expensive in computer systems no longer hold. An appreciation of current constraints leads to new kinds of solutions. This applies to Gmail, for example, where you no long need to throw anything away, rather than more traditional email applications that require constant scrimping and archiving:

p179. Gates's implicit criticism of Gmail was that it was wasteful in its means of storing each email. Despite his currency with cutting-edge technologies, his mentality was anchored in the old paradigm of storage being a commodity that must be conserved. He had written his first programs under a brutal imperative for brevity. And Microsoft's web-based email service reflected that parsimony.
     The young people at Google had no such mental barriers.

I first became consciously aware of how storage abundance and faster algorithms, particularly search, can change things qualitatively when reading Cochrane's Tips for Time Travellers a decade ago, than had it reinforced by Weinberger's Everything is Miscellaneous. Here the concept is made explicit in relation to Chrome:

p200. When you run a program faster by an order of magnitude, you haven't made something better---you've made something new.

More storage, more speed, more data: more is different.

The other place where this different mind-set is clear is in Google's approach to server farms. Rather than requiring high accuracy and good quality of all their systems, they recognise that maybe 10% of their servers will fail, and have infrastructure in place to work around this. This makes their systems scalable, and, boy, have they scaled! While the rest of the world has been agonising over how to do parallelism "properly", these kids have just ploughed ahead and done it. As a result, we now have sophisticated software support for highly parallel cloud computing.

p198. "Suddenly, you have a program that doesn't run on anything smaller than a thousand machines," he says. "So you can't look at that as a thousand machines, but instead look at those thousand machines as a single computer. I look at a data center as a computer."

This combination of scale, sophisticated infrastructure, and appreciation of where the real costs lie allows some startling solutions:

p197. One of the most power-intensive components of the operation is the huge chillers that refrigerate water to keep the temperature in the building no higher than around 80 degrees F. Google augmented these chillers with much more efficient systems that take in fresh air when outside temperatures are cool. The data center in Saint-Ghislain, completed in 2008, actually eliminated chillers entirely. The average summer temperature in Brussels is around 70 degrees, but during the occasional scorcher, Google would shift the information load to other data centers.

Power consumption is the expensive part; when you are big enough to have server farms located in different parts of the planet, just don't use those servers where it's hot today!

When facing technological problems, Google is supreme at solving them. When facing people issues, maybe not so great. Their problems in China, with copyright, with social networking applications, with public perception of their actions -- these all seem to stem from the fact that not only don't they seem to get people who think differently from them, they don't seem to get that there are people who think differently. They think that all they need is the evidence, and people will fall in line. All the evidence and reasoning in the world doesn't help if people are starting from different axioms, however.

However, as I too am more interested in technology than in people, I get them. This also meant I wasn't as enamoured of those parts of the book that are about the people, rather than the technology, but, hey. It does have rather less than usual of that irritating pop science style of introducing characters with little illustrative snippets. However, it suffers from the occasional impenetrably parochial cultural references (here provided by that very introductory style I dislike):

p73. The head was John Doerr, a bony blond man with oversize spectacles who looked a bit like Sherman in the Mr. Peabody cartoons but loomed over Silicon Valley like Bill Russell in the Boston Celtics' glory years.

I read this as "who looked a bit like <a character I've never heard of> in <a show I've never heard of> but loomed over Silicon Valley like <a person I've never heard of> in <a sports team (I assume!) I've never heard of>'s glory years". I assume it means something like: "John Doerr looked like a nerd but nevertheless had a lot of influence", which is a bit insulting (to nerds), really.

Despite these relatively minor niggles, I enjoyed reading this, and learned a lot, about Google, about big data, and about a possible future of computing.