Anyone familiar with James Gleick’s earlier book Chaos needs no introduction to one of the finest science writers of our generation. In his new book, The Information: A History, A Theory, A Flood, Gleick tackles an even slipperier subject.
The depth of historical detail is impressive, and includes some surprising topics like how African talking drums work. It is amusing to see quotes of people complaining of information overload in the 1600s (due to the printing press).
Following Walter Ong, he explores how reading and writing changed the way we think; the reasoning of preliterate oral peoples is substantially different from what we now think of as normal:
A typical question:
In the Far North, where there is snow, all bears are white.
Novaya Zembla is in the Far North, and there is always snow there.
What color are the bears?
Typical response: "I don't know. I've seen a black bear. I've never seen any others. ... Each locality has its own animals."
By contrast, a man who has just learned to read and write responds, "To go by your words, they should all be white."
Only with writing does information become detached from specific things and experiences, so that logic and reasoning become possible.
We delve into cuneiform, including Don Knuth’s amazing 1972 discovery that a broken Old Babylonian tablet, laying part in the British Museum and part in Berlin (and part missing), held an algorithm for taking square roots. It ends “This is the procedure.”
Gleick spends several chapters on organizing information. The first attempt at an alphabetical dictionary was published in 1604; before that, most educated intelligent people had never seen any kind of alphabetized or numerically sorted list in their entire lives. Library catalogs were in shelf order. Street addresses did not exist; people had to say things like “at London by Thomas Vautroullier dwelling in the blak-friers by Lud-gate” to specify a location.
Charles Babbage and Ada Lovelace get a thorough treatment, moving through a crowd which included Boole and De Morgan. Ada solves Peg Solitaire by hand and wonders (to Babbage)
... if the problem admits of being put into a mathematical formula, & solved in this manner. ... There must be a definite principle, a compound I imagine of numerical & geometrical properties, on which the solution depends, & which can be put into symbolic language.
Not long after, she was writing recursive algorithms to solve Taylor series like
e^x = 1 + x + x^2/2 + x^3/6 + … + x^n/n! + …
on a computer that existed only on paper, and speculating about how it might be programmed to play chess or compose music.
Telegraphs (including pre-electric mechanical systems like Napoleon’s) get detailed coverage, and drove the first frenzy of data compression: when each word costs you money, how much can you say in how few? Codes and codebooks abounded both for compression and for secrecy; cryptography became a public fad. Mathematician John Wilkins published a book of codes in 1641, one of which used 2 letters in groups of 5 to encode a 32-character alphabet, probably the first such use of binary, and the last for several centuries. Babbage, Poe, Verne and Balzac were all amateur cryptographers.
After a chapter on telephones, we start hitting the technical meat with Shannon, Godel, and Turing. There won’t be too much surprising here for people who have already been over that ground, but the coverage includes Russel and Whitehead, Berry’s Paradox, Shannon’s thesis on relay circuits, and so on. It covers the conventional topics well, but fails to push into less conventional areas nearby. There is no mention of paraconsistent logics, within some of which the Godel incompleteness proof fails (so while any fully consistent theory of arithmetic must be incomplete, a paraconsistent one might not have to be). There is no mention of Fisher Information, which predates Shannon Information by two decades and has powerful physical implications (more on that in my previous note). Gleick is also a bit muddy on the confusion between “information = entropy” (Shannon’s original way of putting it) and “information = negative entropy” (the viewpoint of Wiener and Schrodinger, which I think is correct and clearer). Shannon himself knew what he was doing and never let this lead him into error, but several generations of subsequent physicists have been sloppier and not so fortunate. Gleick uses both viewpoints somewhat interchangeably, which is unhelpful for beginners and annoying for experts. The minus sign matters.
The coverage of Maxwell’s Demon is extensive, but again, it does not embody the fastest route to true understanding. Most physicists (including Von Neumann) thought Brillouin has exorcised the demon by showing that measurement takes work, but we now know this is incorrect. Measurement can be done reversibly; even quantum systems can do something like measurement by entanglement (“measurement” has in QM a very specific meaning, different from the general one), which does not collapse the wavefunction and is reversible. What does generate entropy is any irreversible action, like erasing a bit of memory, so the real reason the demon fails is that it can’t keep recording results reversibly forever into a finite memory, and eventually has to start erasing memory to make room for a new computation. Pressing the reset button – going from one of a collection of unknown states to a single known state – must generate entropy. Gleick covers Brillouin in chapter 9 as though he was right, and doesn’t correct the misconception until chapter 13 (on Bennett and Landauer and quantum computing).
The coverage of biological information is brief, given the vast scale of the subject, and mostly focuses on the discovery of DNA and the working out of the triplet code. The former part is covered better in The Double Helix, and the latter is too vague to understand any of the details. There is almost no reference to any modern (post-1960) topic. (We now know, for example, that there are over 13 different genetic codes operating on the planet (including two inside your own body, nuclear and mitochondrial), and that they form a nested family tree. We know, in other words, that the genetic code evolved over time, and can even begin to guess which amino acids were missing in earlier versions.)
A chapter on memes covers chain letters and hula hoops before Dawkins and internet viruses.
The chapter on Chaitin-Kolmogorov complexity is very nice, more lucid than Chaitin’s own popular writing which tends to get bogged down in technical details. It manages to relate computability, compressibility, decidability, incompleteness, randomness, and inductive inference without a single equation. This is Gleick in top form.
Elsewhere in the book we peer into the inner workings of the Oxford English Dictionary and of Wikipedia, consider the historical impact of printing, and follow the origins of words like network and ba-da-bing. All in all, a fine read. Because of omissions noted above, this is nothing like “the last book you’ll ever need” on information theory, but it’s a great place for a beginner to start.
Energy and Time 