Talk for USENIX'05, April 2005
At the moment, I have these html "slides" available at:
Defending against Futurology Foolishness
- Don't buy a trend extrapolation, however sophisticated
- Don't buy a single grand projection
- Work out a variety of credible scenarios
- Scenario-based planning
- For example the future of oil prices and the founders of GBN
- How to use good scenarios
- Confess: this approach does not cover the space of all surprises!
(But the hope is that it gives us greater flexibility than
betting the farm on one vision of the future.)
- Watching for symptoms
- Taking advantage of opportunities
- Maybe, preventing the future
- My strategy in this talk
- Four scenarios, marking out some extremes of the probability
- I base 3 of the 4 scenarios on the future of Moore's Law
Hardware Trends and Software Futures
We have enjoyed a long period of exponential growth in hardware
performance. Without the hardware, we don't have practical software,
but what effect can improving hardware have on software?
In real life, what sort of things can happen to exponential growth?
I'll take each of these as a future for hardware trends, imagine
how that development might come to pass and what the resulting
software environment might be like.
Scenario 1: The Machines Stop
Or, no more computers, so no more software (and no more programmers).
For the hardware trends, this corresponds to
exponential growth with catastrophic collapse.
How could this scenario happen?
- Through the mother of all software failures?
Maybe. Certainly, enormous service disruptions are out there
in our future -- caused by software and deliberate attacks
on software. My intuition (my scenario, after all :-) is that
as long as we have heterogeneous, multiple software solutions,
that software will not be the cause of a complete failure.
- Semiconductor technology as the single failure point.
- The homogeneity of certain semiconductor parts. (This
threat is a matter of principle, not just the obvious
and credible threat of EM Pulse attacks.)
- The ubiquity of semiconductor parts
- The trend that almost all of us in this room are pushing
hard every day.
- Most of us have a treacherous psychological blindspot: If a
behavior runs a catastrophic but unquantifiable risk, but the
behavior yields progressively more magnificent rewards, then
we are happy to ignore the possibility of disaster. That is
our current situation with computing, in particular with
embedded microcontrollers (and networks of such devices).
The use of programmatic solutions implemented in embedded microcontrollers is a
mind-bogglingly powerful economic miracle. It harnesses Moore's Law to make most
classical goods as cheap as dirt. It opens the way to a standard
of living that towers above the past. It also puts us way out on a limb. Is
this a fact we can ignore? If not, what can we do about it?
No doubt, I'm going to continue be peddling the wondrous upside of
computation in my fiction, but more fiction and scenarios about this
risk should be undertaken, and perhaps suggestions for solutions. Some
- We should encourage old-time (no semi-conductor) hobbyists.
This won't save an absolutely large number of lives, but
it might save civilization.
- Known threats to this single failure point should be much
more intensively discussed (and without the foregone conclusion
that since we really really need this tech, the threat must
Science Fiction Related to Scenario 1
(I think these are good examples, but in some cases the connection is tenuous
or with a very different point of view:)
- Dies the Fire, by S. M. Stirling, (ROC).
(The background is very different from what is imagined in this
talk, but the actual crash is similar and vividly described.)
- The End of Man, by Hannes Alfvén, (Coward-McAnn).
(The crash is just a small part of the book, but vividly
described, even though it happens at a distance.)
Scenario 2: Legacy Software Forever
In terms of hardware trends, this one comes with exponential growth with saturation
For at least ten years, people have been saying that Moore's Law
is soon to run out of steam, so there should be lot's of people
who buy into this scenario.
- Maybe the hardware people will run out of inventiveness.
- Maybe the software people find that they can't manage
complexity above a certain limit.
It would be interesting to observe this leveling off. Some possible symptoms
(depending on the underlying cause from above):
- More and more software bloat with less and less payoff
- Edelson's law (due to Steve Edelson, but possibly muddled here):
If knowledge is increasing exponentially, then "The number of
inventions that could be made -- but aren't -- increases
- Enormous software project failures
- Murphy's counterpoint to Moore's Law
"The maximum possible effectiveness of a software system
increases in direct proportion to the log of the effectiveness (ie, speed,
bandwidth, memory capacity) of the underlying hardware."
- Ultimately, the above problems would mean physics and materials
science research would flounder -- and we'd see a leveling out of the
- Very likely, some types of hardware progress would continue
forward, even as others leveled off.
- Simple, large structures (RAM and DSP logic)
- Related consumer electronics
After equilibrium was achieved, some (naive?) programmers might give a deep
sigh of relief: "Finally, we'll have time to do things right!"
Even more than now, programming would come down to a choice between software
archeology and re-inventing the wheel.
Science Fiction Related to Scenario 2
- A Deepness in the Sky, by Vernor Vinge (Tor Books).
This story takes place thousands of years after hardware/software
equilibrium is achieved.
Scenario 3: The Technological Singularity
In terms of hardware trends, this one comes from exponential growth continuing, at least
long enough to create our post-human successors.
This is the scenario I've been peddling for a long time:
- From 1983: "First Word" OpEd piece in Omni magazine, January 1983, page 10.
- From 1993: "The Technological Singularity", NASA Lewis VISION-21 Symposium, 30-31 March 1993.
It's a little extreme, but in the wider view of future scenarios, I
think it's the most likely of the non-disasters. (For numerous deadly
disasters, see Martin Rees, Our Final Hour (nonfiction)
Why call it the "Technological Singularity"?
- By analogy with the use of "singularity" in Math
- Not necessarily a place where anything becomes infinite
- A qualitatively different step of progress than in our
- A place where the rules profoundly change
- By analogy with the use of "singularity" in Physics
- What comes after is intrinsically less knowable/predictable than
previous technological advances.
- The apocalyptic endpoint of radical optimism
The Singularity as the dual of Fermi's Paradox
- What if: AI (Artificial intelligence) research succeeds?
- What if: The internet itself attains unquestioned life and
- Gregory Stock, Metaman (nonfiction)
- What if: IA (Intelligence Amplification) occurs as the radical
endpoint of Human/computer interface research?
- Poul Anderson, "Kings Who Die"
- Vernor Vinge, "Bookworm, Run!"
- What if: Fine-grained distributed systems are aggressively
- Communicating, embedded micros
- The physical world awakes
- The end of software engineering. More precisely, the partition of
software projects into complexity categories:
- Programs that can be mathematically proof-verified. Wherever
this can be done, it should be; however, my guess is the domain
for such certainty will remain with extremely-constrained applications.
- Projects that are susceptible to engineering approaches. There
should still be many projects like this, ones where
quantitative analysis and a step-by-step implementation of an
overall vision produces effective results. These are projects
in which we can at least fool ourselves that there is a
well-defined product specification.
- Mongo Projects. These are things people really really want, but
the desired product is usually very large, involving vague or
incomplete specification (even though the customers are sure they
will recognize success when they see it).
- An organizing principle for Very Large Data Systems:
- Reality should be its own database.
"Every object knows what it is, where it is, and can talk
to nearby peers (and thus in principle talk across the world)."
(Cf the term "thalience" in Ventus by Karl Schroeder (Tor Books).)
- And so cyberspace leaks into the real-world. There is a heuristic for
invention in the new age: In the 20th century, we often used real world
metaphors in our programming. Now turn that around: almost every
gimmick/insight that we've been using in computation will have some real
world analog (Google on 'Accenture "virtual collaborating double"').
- The rise of biological models for problem solving. People have been
working on this for a long time ("genetic
programming", "neural nets", ...). Now we are entering an era where
the hardware may be up to the promise.
- Precedent: Consider the surprises that numerical analysts
(for example, the differential equations people) suffered when
automatic computation arrived.
- Programming as an exercise in evolution
- Programming as an exercise in embryology
- Programming as an exercise in training/teaching
- Programming as an exercise in sociology
- "Maneki Neko", by Bruce Sterling, F&SF, May 1998
- Software with levels of "resource abuse" that make the most
wasteful 2005 xml/java programmers look like
Scrooge. Worse, the exact source of the waste will not be as
obvious as in our time.
- What about trusting life-critical applications to such "software"?
(A tadpole speculates on wetlands management. Some of the more extreme
bullets in the preceding slide might be moved into this one, but
post-Singularity speculation naturally goes beyond futurology into
- Nightmare analogies -- our treatment of animals
- Vernor Vinge, "The Cookie Monster" currently at
It is very likely that plenty of unpleasant jobs
will remain for critters of about human intelligence.
- Hard takeoff: transition into Singularity takes only a few
days (or a few hours!). This version sounds tailor made for nightmares.
- Soft takeoff: transition into Singularity takes a years. Imagine
the distributed scenario discussed above. There might be time to plan,
time to create good feeling with the new entities
- Other analogies
Science Fiction Related to Scenario 3
I have some refs in the slides above. Also:
In my 1993 "Technological Singularity" essay
I have a pretty good bibliography of related fiction (and non-fiction).
This is slightly elaborated upon at
As of 2005, there are lots of stories in this category. Many people writing
hard science fiction feel a necessity for addressing this possibility -- if
only to explain why they can keep it at least partially offstage! About half of
modern interstellar adventure has a background that involves post-human or
transhuman players (including almost all my stuff).
In the last few years, there have been more stories that look at the near
future (when, after all, the Singularity is likely to happen) and try to do the
nearly impossible -- that is, write a fun story with characters who are
slipping beyond human ken. Charles Stross's Accelerando, to
appear (Ace Books) is one of the most interesting of this genre.
Scenario 4: Ubiquitous Law Enforcement
Unlike the previous scenarios, this one does not correspond to one of the
hardware curves. It does assume that hardware improvements continue for at least a few
years more. A workable version of this scenario also assumes some
spectacular (and malignant) improvements in software, but overall the
situation in this scenario is not at all mysterious.
- Vernor Vinge, presentation at Computers, Freedom, and Privacy
conference, 1996. Here is a redrawing of my main graphic.
- John Walker, "The Digital Imprimatur"
Science Fiction Related to Scenario 4
Examples solicited. I don't have any clearcut fiction examples of this exact
idea. I waved my hands at ULE in A Deepness in the Sky
(Tor Books). Also, Ubiquitous Law Enforcement is a major background element
in the novel I am currently finishing up (working title: Rainbows
End) -- where I use ULE to make plausible the success of the secret
police against the hackers. :-)