Being a physicist by training, I understand reductionism -- it's a natural way for me to think. I've been reading about complexity and self-organisation for a while now. It's certainly saying something very important, but I've been struggling to fit its emergent, holistic approach into a scientific world with 'real' laws. After reading At Home in the Universe, I think I see that it might be possible.
I find the prose a bit purple in places, especially the introductory and concluding chapters. But in between there is an excellent discussion of the inevitability of spontaneous emergent self-organisation. The exposition is structured around several simple models used to 'train your intuition' about optimisation and evolution.
I have been reading this book slowly over the past few weeks. Slowly, because it is so densely full of new and wonderful insights that I needed time to absorb them, and slowly, because I simply didn't want it to finish. And now I have finished it, I have to review it. But how to review a book like this? It is so marvelously original and stunningly imaginative, and so full of profound ideas, that a short review cannot possibly hope to do it justice. Go and read it yourself.
In his previous book, At Home in the Universe, Kaufmann discusses spontaneous emergent self-organisation. Here he takes his ideas, I was going to say a step, but actually it is a giant leap, further forward, covering life, evolution, and the universe itself. And fortunately his prose style, although still slightly over-poetic in places for my personal taste, is a lot less purple than before.
He starts small, with a definition of life suitably broad enough for a general biology (that is, one not constrained merely to the contingent developments of life on earth).
The quoted definition may look rather bald, but he discusses each of the aspects -- self reproducing autocatalytic systems, work, and work cycles, in fascinating detail. [But I do wonder if self-reproduction is a necessary condition for life -- maybe it is simply one possible mechanism?]
Chapter 2 comes as a bit of a shock after the readily readable chapter 1 -- it assumes rather more detailed biochemistry than I have -- but keep courage and keep reading -- there's lots of explanation, with lots of different examples.
There is a lot of discussion on how life is necessarily a non-equilibrium system, and how non-equilibrium systems can (self-)organise to do the work required for life.
We get discussions of systems building themselves recursively. To survive, systems like cells need to be close to, but on the stable side of, the edge of chaos, lest they lose their identity under the onslaught of novelty. They operate recursively, reaching some kind of 'fixed point', or attractor. Kauffman wonders if Category Theory, with the elements recursively constructing themselves, might be a suitable mathematical basis for these systems. On the other hand, evolving things like ecosystems are 'supercritical', constantly generating new kinds of members, with a generative, non-convergent, open-ended, forever growing recursion. Current computer models seem to be able to generate at most a few levels of novelty, and have not yet demonstrated ever increasing novelty.
But simply getting a good definition of life isn't enough for Kauffman. He goes on to describe how life evolves and coevolves, how complexity necessarily flows into the "adjacent possible" (the states a single step away from the system's current state), and why the science of this kind of evolution is qualitatively different from "classical" science: because it is impossible even in principle [I am not sure I agree with that "in principle", but I do with the sufficient "in practice"] to finitely prespecify the state space [Cohen and Stewart also make this point -- their favourite examples are oxygen and grass], and because the evolution of the universe is so vastly non-ergodic, so vastly contingent. He discusses that the contingent evolution of the world is confined not just to the species level, but is present in the molecules that go to make up life on earth, and maybe even the fundamental laws of nature themselves.
Having covered life, and how it evolves, he next goes on to asking why evolution works. Macready and Wolpert's "no-free-lunch" theorem tells us that there is no one search algorithm better than any other when its performance is averaged over all search landscapes. Random search is as good as any other on average. So how come the search algorithms evolution uses just happen to be ones that work well on the evolutionary fitness landscape? Coincidence? No. The creatures and the fitness landscapes co-construct each other:
Kauffman doesn't like anything to be coincidence. In his final chapters he leaves biology, and ventures into economics and fundamental physics.
In an evolving economic world, where new goods and services are forever coming into being, where the future goods available cannot be prespecified, the old ideas of equilibrium economics and clearing markets cannot hold. He shows that economic agents on coevolving fitness landscapes must necessarily have bounded rationality -- they can use only recent data in building predictive models of other agents, data from when the landscapes were roughly as they are now, and they can devise only simple theories, to avoid overfitting the sparse data. Ironically, in periods of relative stability, the available data increase and the theories can get more sophisticated, but hence more fragile, until a small change in reality can cause catastrophic failure of the now fragile models, and result in an avalanche of new behaviours and consequently requirements for new models: hence cycles of stability and rapid change. [Here, for Kauffman's argument to work, the economic fitness landscape must be of a sort readily searchable by the search procedures the economic agents have to hand -- but he doesn't say what these procedures are -- mutation and selection, clearly -- but what is the economic analogue to biology's sexual recombination?]
Finally, Kauffman considers the universe itself, and has a dislike for even the weak anthropic principle -- he wants to explain why the universe has to be the way it is, or at least, close to the way it is. He uses quantum decoherence (covered in some detail in The Quark and the Jaguar) to explain why the universe is getting more complex. Bigger systems, more complex systems, decohere more rapidly, and hence preferentially come into classical existence. He then applies Feynman's idea of sums over histories and constructive quantum interference to explain why the universe has the laws and geometry it does. Nearby histories constructively interfere when they are similar -- when small changes to whatever is being measured lead to small changes in the outcomes -- which might help explain our smooth and flat space-time: the irregular ones have destructively interfered themselves out of existence. [I admit to having missed the point here -- Kauffman seems to be saying that the universe gets ever more complex because the complex co-observing systems decohere -- become classical -- fastest, whilst we get flat space-time because that is the geometry that decoheres slowest. But maybe I can be excused this lapse in understanding: the writing is at its densest here, with whole new cosmologies packed into a few spare paragraphs. I would love to see this chapter expanded into a whole book of its own. Some of it can be found in The Life of the Cosmos, but most goes beyond that.]
In all, this is a glorious, mind-expanding book. My attempts at summarising Kauffman's many rich, diverse arguments cannot possibly do them justice. I have just skimmed the surface of his thesis here. There are many more fascinating details and arguments in the book. Read it. And if its ideas are not yet in your own adjacent possible, go and do some background reading (not helped by the rather sparse bibliography herein), then read it.