Could a Computer Ever
be Conscious?
Steven Pinker
Steven Pinker is Professor and Director of
the Center for Cognitive
Neuroscience
of the Massachusetts Institute of Technology and author of [The
Language
Instinct]. This article is adapted from his forthcoming book [How the
Mind Works]
(Norton, October).
In one of the first episodes of the
[Twilight Zone], a man named James Corry
is serving a
fifty-year sentence in solitary confinement on a barren asteroid.
Allenby, the
captain of a supply ship takes pity on him and leaves behind a
crate
containing "Alicia," a robot that looks and acts like a woman. Corry, of
course, soon
falls deeply in love. A year later Allenby returns with the news
that Corry
has been pardoned and that he has come to get him and a maximum of
fifteen
pounds of gear. Alicia, unfortunately, weighs more than that. When
Corry
refuses to leave, Allenby shoots Alicia in the face, exposing a tangle of
smoking
wires. He tells a devastated Corry,
"All you're leaving behind is
loneliness."(Note
1)
The horrifying climax raises two vexing
questions. Could a mechanical device
ever
duplicate human intelligence, the ultimate test being whether it could
cause a real
human to fall in love with it? And if a humanlike machine could be
built, would
it actually be [conscious]? Would dismantling it be the snuffing
out of a
sentient being that we felt we had witnessed on the small screen?
Pose the first question to experts in
Artificial Intelligence, and you'll get
one of two
answers: lifelike robots are just around the corner, or it will
never
happen.(Note 2) Don't believe either of
them. These are the kinds of
"experts"
who a few decades ago predicted that nuclear-powered vacuum cleaners
were in our
future or that man will never reach the moon.(Note 3) Certainly
computers
will continue to get smarter, as the recent defeat of the world chess
champion,
Gary Kasparov, by IBM's Deep Blue reminds us. Today's computers can
converse in
English on restricted topics, control mechanical arms that weld and
spray-paint,
and duplicate human expertise in dozens of areas, from prescribing
drugs to
diagnosing equipment breakdowns. {And artificial intelligence has
jumped from
the laboratory to everyday life. Most people today have had their
speech
recognized by telephone directory assistance systems, and many have used
intelligent
search engines on the World Wide Web, appliances controlled by
fuzzy logic
chips, or mutual fund portfolios selected by artificial neural
networks.}(Note
4)
{Still, today's computers are not even close
to a four-year-old human in
their
ability to see, talk, move, or use common sense. One reason is sheer
computing
power. It has been estimated that the information processing capacity
of even the
most powerful supercomputer is equal to the nervous system of a
snail -- a
tiny fraction of the power available to the supercomputer inside the
bloated
human skull.(Note 5) But the kinds of processing are different, too.
Computers
find it easy to remember a twenty-five digit number, but find it hard
to summarize
the gist of [Little Red Riding Hood]; humans find it hard to
remember the
number but easy to summarize the story. One reason for the
difference
is that computers have a single, reliable processor (or a small
number of
them) working very, very fast; the brain's processors are slower and
noisier, but
there are hundreds of billions of them, each connected to
thousands of
others. That allows the human brain to
recognize complicated
patterns in
an instant, whereas computers have to reason out every niggling
detail one
step at a time. Human brains also have
the advantage of sitting
inside human
beings, and can soak up terabytes of information over the years as
the humans
interact with with other humans and with the environment. And brains
have the
benefit of a billion-year R&D effort in which evolution equipped them
with cheat
sheets for figuring out how to outmaneuver objects, plants, animals,
and other
humans.}
So how well will tomorrow's machines do?
Technological progress is
notoriously
unpredictable. When it comes to
replacement parts for the body,
who knew
that artificial hips would become commonplace and artificial hearts
elusive? When it comes to the performance of
duplicates of the mind, the most
reasonable
answer is that computers will probably do a lot better than they do
now, for
some kinds of thinking, and they will probably not do as well as a
human being,
for other kinds.
But let's return to science fiction and
assume that someday we really will
have
Alicia-class robots. Will they be "conscious"? It all depends on what
you
mean by the
word. Woody Allen once wrote a
hypothetical course catalogue with
a listing
for Introductory Psychology that read, "Special consideration is
given to a
study of consciousness as opposed to unconsciousness, with many
helpful
hints on how to remain conscious."(Note 6) We laugh because we realize
that the
word "consciousness" has at least two meanings.(Note 7)
One of them is Freud's famous distinction
between the conscious and
unconsious
mind. I ask, "A penny for your thoughts?" You reply by telling me
the content
of your daydreams, your plans for the day, your aches and itches,
and the
colors, shapes, and sounds in front of you. But you cannot tell me
about the
enzymes secreted by your stomach, the current settings of your heart
and
breathing rate, the projections on your retinas, the rules of syntax that
order words
as you speak, or the sequence of muscle contractions that allow you
to pick up a
glass. This shows that information processing in the nervous
system falls
into two pools. One pool can be accessed by the brain modules
behind
verbal reports, rational thought, and deliberate decision-making. The
other pool,
which includes gut responses, the brain's calculations for vision,
language,
and movement, and repressed desires or memories (if there are any),
cannot be
accessed by those modules. Sometimes information can pass from one
pool to the
other. When we first learn how to use a stick shift, every motion
has to be
thought out, but with practice the skill becomes automatic (conscious
processes
becomes unconscious). With intense
concentration and biofeedback, we
can focus on
a hidden sensation like our heartbeat (unconscious processes
become
conscious).
Will computers ever become conscious, in
this sense of access to a subset of
the
information in the whole system? In a
way, they already are. The operating
system of
your computer is designed so that certain kinds of information are
available to
the programmer or user -- opening and saving files, sending
messages to
the printer, displaying directories -- and others are not -- such
as the
movements of the disk drive head or the codes sent by the keyboard.
That's
because any information system, computer or brain, has to work in real
time. A
device in which every morsel of information had to be easily available
at all times
to every process would be perpetually lost in thought. It would
have to
calculate whether the price of tea in China was relevant to which foot
should be
put in front of the other one next.
Only some kinds of information
are relevant
to what the system is doing at a given time, and only that
information
should be routed in to the system's main processors. {Even robots
of the
future, with their thousands of processors, will need some kind of
control
system that limits what goes into and out of the individual processors.
Otherwise
the whole robot would lurch and zigzag as the processors fight for
control,
like Steve Martin in [All of Me] when his right side was controlled by
the ghost of
Lily Tomlin.} So in that sense, computers, now and in the future,
are built
with a distinction between "conscious" and "unconscious"
processing.(Note
8)
But it's a very different sense of the word
"consciousness" that people find
particularly
fascinating. That sense is [sentience]: pure being, subjective
experience,
raw feels, first-person present tense, "what it is like" to see red
or feel pain
or taste salt. When asked to define "consciousness" in this sense,
we have no
better answer than Louis Armstrong's when a reporter asked him to
define jazz:
"Lady, if you have to ask, you'll never know."(Note 9)
How can we ever know whether Alicia is
conscious in this sense -- whether
there's
"anyone home" seeing the world through her camera-eyes and feeling
the
signals from
her pressure sensors? No matter how
smart she acts, no matter how
responsive,
no matter how vehemently she says she is conscious, an Allenby can
always
insist that she's just a very fancy stimulus-response machine programmed
to act [as
if] she were sentient. Try as hard as you like, but you will not
come up with
an experimental test that will refute him.
Perhaps it is some consolation to know that
our befuddlement here is not just
a
technological puzzle but is a piece with some of the deepest problems in
philosophy. If I can't know whether Alicia is sentient,
how can I know whether
[you] are
sentient? I [think] you are, and I'm not so sure about Alicia, but
maybe I'm
just chauvinistic about creatures that are made out of meat rather
than
metal. How can I be so confident that
consciousness is secreted by the
brain tissue
in my skull, rather than lurking in the software that my brain is
running --
software that Alicia's computer could run just as well?(Note 10)
Lest you think that the answer is obvious
one way or another, ponder these
thought
experiments. Suppose surgeons replaced
one of your hundred billion
neurons with
a microchip. Presumably you would feel and behave exactly as
before. Then they replace a second one, and a third
one, and so on, until more
and more of
your brain becomes silicon. The chips do what the neurons did, so
your
behavior and memory never change. Do
you even notice the difference? Does
it feel like
dying? Is some [other] conscious entity
moving in with you?
Suppose that
the transporter in [Star Trek] works as follows. It scans in a
blueprint of
Kirk's body, destroying it in the process, and assembles an exact
duplicate
out of new molecules on the planet below.
When Kirk is beamed down,
is he taking
a nap or committing suicide?
The head spins in confusion; it's hard to
imagine what a satisfying answer to
these
questions would even look like. But
they are not just brain-teasers for
late-night college
dorm-room bull sessions. The imponderables also drive our
intuitions
about right and wrong. Was Allenby guilty of destuction of property,
or of
murder? Does a newborn boy feel pain when he is circumcised, or is his
crying just
a reflex? What about a lobster boiled
alive, or a worm impaled on
a fishhook?
These problems won't be solved any time
soon, so don't expect someone to tell
you with
certainty whether a computer will ever be sentient. Perhaps it is a
meaningless
question, and we have been deluded by misleading verbiage into
taking it
seriously. Perhaps some unborn genius will have a thunderbolt of
insight and
we will slap our foreheads and wonder why the problem took so long
to be
solved. But perhaps the problem never
will be solved. Perhaps the human
mind, a mere
product of evolution of one species on this planet, is
biologically
incapable of understanding the solution. If so, our invention the
computer
would present us with the ultimate tease. Never mind whether a
computer can
be conscious. Our [own] consciousness,
the most obvious thing
there is,
may be forever beyond our conceptual grasp.(Note 11)
Notes
(1)Zicree, M. S. 1989. [The Twilight Zone
companion]. 2d ed. Hollywood:
Silman-James
Press.
(2)Crevier, D. 1993. [AI: The tumultuous history of the search for
artificial
intelligence].
New York: Basic Books.
(3)Cerf, C., & Navasky, V. 1984. [The
experts speak]. New York: Pantheon.
(4)Hendler, J. 1994. High-performance artificial intelligence. [Science,
265],
891--892. Crevier, D. 1993. [AI: The
tumultuous history of the search
for
artificial intelligence].
(5)Crevier, op cit.
(6)Allen, W. 1983. [Without feathers]. New
York: Ballantine.
(7)Block, N., & commentators. 1995. On a confusion about a function of
consciousness.
[Behavioral and Brain Sciences, 18], 227--287.
Jackendoff,
R. 1987.
[Consciousness and the computational mind].
Cambridge, Mass.: MIT
Press.
(8)Baars, B. 1988. [A cognitive theory of
consciousness]. New York: Cambridge
University
Press.
(9)Block, N. 1978. Troubles with
functionalism. In C. W. Savage (Ed.),
[Perception
and cognition: Issues in the foundations of psychology. Minnesota
Studies in
the Philosophy of Science], Vol. 9. Minneapolis: University of
Minnesota.
(10)Dennett, D. C. 1991. [Consciousness
explained]. Boston: Little, Brown.
(11)McGinn, C. 1993. [Problems in philosophy: The limits of
inquiry].
Cambridge,
Mass.: Blackwell.