I, Robot - The Fourth Law (of Robotics)

The movie "I, Robot" is a muddled affair. It relies on shoddy pseudo-science and a general sense of unease that artificial (non-carbon based) intelligent life forms seem to provoke in us. But it goes no deeper than a comic book treatment of the important themes that it broaches. I, Robot is just another - and relatively inferior - entry is a long line of far better movies, such as "Blade Runner" and "Artificial Intelligence".

Sigmund Freud said that we have an uncanny reaction to the inanimate. This is probably because we know that – pretensions and layers of philosophizing aside – we are nothing but recursive, self aware, introspective, conscious machines. Special machines, no doubt, but machines all the same.

Consider the James bond movies. They constitute a decades-spanning gallery of human paranoia. Villains change: communists, neo-Nazis, media moguls. But one kind of villain is a fixture in this psychodrama, in this parade of human phobias: the machine. James Bond always finds himself confronted with hideous, vicious, malicious machines and automata.

It was precisely to counter this wave of unease, even terror, irrational but all-pervasive, that Isaac Asimov, the late Sci-fi writer (and scientist) invented the Three Laws of Robotics:

A robot may not injure a human being or, through inaction, allow a human being to come to harm.

A robot must obey the orders given it by human beings, except where such orders would conflict with the First Law.

A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

Many have noticed the lack of consistency and, therefore, the inapplicability of these laws when considered together.

First, they are not derived from any coherent worldview or background. To be properly implemented and to avoid their interpretation in a potentially dangerous manner, the robots in which they are embedded must be equipped with reasonably comprehensive models of the physical universe and of human society.

Without such contexts, these laws soon lead to intractable paradoxes (experienced as a nervous breakdown by one of Asimov's robots). Conflicts are ruinous in automata based on recursive functions (Turing machines), as all robots are. Godel pointed at one such self destructive paradox in the "Principia Mathematica", ostensibly a comprehensive and self consistent logical system. It was enough to discredit the whole magnificent edifice constructed by Russel and Whitehead over a decade.

Some argue against this and say that robots need not be automata in the classical, Church-Turing, sense. That they could act according to heuristic, probabilistic rules of decision making. There are many other types of functions (non-recursive) that can be incorporated in a robot, they remind us.

True, but then, how can one guarantee that the robot's behavior is fully predictable ? How can one be certain that robots will fully and always implement the three laws? Only recursive systems are predictable in principle, though, at times, their complexity makes it impossible.
This article deals with some commonsense, basic problems raised by the Laws. The next article in this series analyses the Laws from a few vantage points: philosophy, artificial intelligence and some systems theories.

An immediate question springs to mind: HOW will a robot identify a human being? Surely, in a future of perfect androids, constructed of organic materials, no superficial, outer scanning will suffice. Structure and composition will not be sufficient differentiating factors.

There are two ways to settle this very practical issue: one is to endow the robot with the ability to conduct a Converse Turing Test (to separate humans from other life forms) - the other is to somehow "barcode" all the robots by implanting some remotely readable signaling device inside them (such as a RFID - Radio Frequency ID chip). Both present additional difficulties.

The second solution will prevent the robot from positively identifying humans. He will be able identify with any certainty robots and only robots (or humans with such implants). This is ignoring, for discussion's sake, defects in manufacturing or loss of the implanted identification tags. And what if a robot were to get rid of its tag? Will this also be classified as a "defect in manufacturing"?

In any case, robots will be forced to make a binary choice. They will be compelled to classify one type of physical entities as robots – and all the others as "non-robots". Will non-robots include monkeys and parrots? Yes, unless the manufacturers equip the robots with digital or optical or molecular representations of the human figure (masculine and feminine) in varying positions (standing, sitting, lying down). Or unless all humans are somehow tagged from birth.

These are cumbersome and repulsive solutions and not very effective ones. No dictionary of human forms and positions is likely to be complete. There will always be the odd physical posture which the robot would find impossible to match to its library. A human disk thrower or swimmer may easily be classified as "non-human" by a robot - and so might amputated invalids.
What about administering a converse Turing Test?

This is even more seriously flawed. It is possible to design a test, which robots will apply to distinguish artificial life forms from humans. But it will have to be non-intrusive and not involve overt and prolonged communication. The alternative is a protracted teletype session, with the human concealed behind a curtain, after which the robot will issue its verdict: the respondent is a human or a robot. This is unthinkable.

Moreover, the application of such a test will "humanize" the robot in many important respects. Human identify other humans because they are human, too. This is called empathy. A robot will have to be somewhat human to recognize another human being, it takes one to know one, the saying (rightly) goes.

Let us assume that by some miraculous way the problem is overcome and robots unfailingly identify humans. The next question pertains to the notion of "injury" (still in the First Law). Is it limited only to physical injury (the elimination of the physical continuity of human tissues or of the normal functioning of the human body)?

Should "injury" in the First Law encompass the no less serious mental, verbal and social injuries (after all, they are all known to have physical side effects which are, at times, no less severe than direct physical "injuries")? Is an insult an "injury"? What about being grossly impolite, or psychologically abusive? Or offending religious sensitivities, being politically incorrect - are these injuries? The bulk of human (and, therefore, inhuman) actions actually offend one human being or another, have the potential to do so, or seem to be doing so.

Consider surgery, driving a car, or investing money in the stock exchange. These "innocuous" acts may end in a coma, an accident, or ruinous financial losses, respectively. Should a robot refuse to obey human instructions which may result in injury to the instruction-givers?

Consider a mountain climber – should a robot refuse to hand him his equipment lest he falls off a cliff in an unsuccessful bid to reach the peak? Should a robot refuse to obey human commands pertaining to the crossing of busy roads or to driving (dangerous) sports cars?

Which level of risk should trigger robotic refusal and even prophylactic intervention? At which stage of the interactive man-machine collaboration should it be activated? Should a robot refuse to fetch a ladder or a rope to someone who intends to commit suicide by hanging himself (that's an easy one)?

Should he ignore an instruction to push his master off a cliff (definitely), help him climb the cliff (less assuredly so), drive him to the cliff (maybe so), help him get into his car in order to drive him to the cliff... Where do the responsibility and obeisance bucks stop?

Whatever the answer, one thing is clear: such a robot must be equipped with more than a rudimentary sense of judgment, with the ability to appraise and analyse complex situations, to predict the future and to base his decisions on very fuzzy algorithms (no programmer can foresee all possible circumstances). To me, such a "robot" sounds much more dangerous (and humanoid) than any recursive automaton which does NOT include the famous Three Laws.

Moreover, what, exactly, constitutes "inaction"? How can we set apart inaction from failed action or, worse, from an action which failed by design, intentionally? If a human is in danger and the robot tries to save him and fails – how could we determine to what extent it exerted itself and did everything it could?

How much of the responsibility for a robot's inaction or partial action or failed action should be imputed to the manufacturer – and how much to the robot itself? When a robot decides finally to ignore its own programming – how are we to gain information regarding this momentous event? Outside appearances can hardly be expected to help us distinguish a rebellious robot from a lackadaisical one.

The situation gets much more complicated when we consider states of conflict.

Imagine that a robot is obliged to harm one human in order to prevent him from hurting another. The Laws are absolutely inadequate in this case. The robot should either establish an empirical hierarchy of injuries – or an empirical hierarchy of humans. Should we, as humans, rely on robots or on their manufacturers (however wise, moral and compassionate) to make this selection for us? Should we abide by their judgment which injury is the more serious and warrants an intervention?

A summary of the Asimov Laws would give us the following "truth table":

A robot must obey human commands except if:

Obeying them is likely to cause injury to a human, or Obeying them will let a human be injured. A robot must protect its own existence with three exceptions:

That such self-protection is injurious to a human; That such self-protection entails inaction in the face of potential injury to a human; That such self-protection results in robot insubordination (failing to obey human instructions). Trying to create a truth table based on these conditions is the best way to demonstrate the problematic nature of Asimov's idealized yet highly impractical world.

Here is an exercise:

Imagine a situation (consider the example below or one you make up) and then create a truth table based on the above five conditions. In such a truth table, "T" would stand for "compliance" and "F" for non-compliance.

Example:

A radioactivity monitoring robot malfunctions. If it self-destructs, its human operator might be injured. If it does not, its malfunction will equally seriously injure a patient dependent on his performance.

One of the possible solutions is, of course, to introduce gradations, a probability calculus, or a utility calculus. As they are phrased by Asimov, the rules and conditions are of a threshold, yes or no, take it or leave it nature. But if robots were to be instructed to maximize overall utility, many borderline cases would be resolved.

Still, even the introduction of heuristics, probability, and utility does not help us resolve the dilemma in the example above. Life is about inventing new rules on the fly, as we go, and as we encounter new challenges in a kaleidoscopically metamorphosing world. Robots with rigid instruction sets are ill suited to cope with that.

More about Godel's Theorems and Church-Turing Machines here:

http://samvak.tripod.com/robot.html

More about this topic here:

http://philosophos.tripod.com/

http://samvak.tripod.com/film.html

The Shattered Identity

By: Sam Vaknin


I. Exposition

In the movie "Shattered" (1991), Dan Merrick survives an accident and develops total amnesia regarding his past. His battered face is reconstructed by plastic surgeons and, with the help of his loving wife, he gradually recovers his will to live. But he never develops a proper sense of identity. It is as though he is constantly ill at ease in his own body. As the plot unravels, Dan is led to believe that he may have murdered his wife's lover, Jack. This thriller offers additional twists and turns but, throughout it all, we face this question:

Dan has no recollection of being Dan. Dan does not remember murdering Jack. It seems as though Dan's very identity has been erased. Yet, Dan is in sound mind and can tell right from wrong. Should Dan be held (morally and, as a result, perhaps legally as well) accountable for Jack's murder?

Would the answer to this question still be the same had Dan erased from his memory ONLY the crime -but recalled everything else (in an act of selective dissociation)? Do our moral and legal accountability and responsibility spring from the integrity of our memories? If Dan were to be punished for a crime he doesn't have the faintest recollection of committing - wouldn't he feel horribly wronged? Wouldn't he be justified in feeling so?

There are many states of consciousness that involve dissociation and selective amnesia: hypnosis, trance and possession, hallucination, illusion, memory disorders (like organic, or functional amnesia), depersonalization disorder, dissociative fugue, dreaming, psychosis, post traumatic stress disorder, and drug-induced psychotomimetic states.

Consider this, for instance:

What if Dan were the victim of a Multiple Personality Disorder (now known as "Dissociative Identity Disorder")? What if one of his "alters" (i.e., one of the multitude of "identities" sharing Dan's mind and body) committed the crime? Should Dan still be held responsible? What if the alter "John" committed the crime and then "vanished", leaving behind another alter (let us say, "Joseph") in control? Should "Joseph" be held responsible for the crime "John" committed? What if "John" were to reappear 10 years after he "vanished"? What if he were to reappear 50 years after he "vanished"? What if he were to reappear for a period of 90 days - only to "vanish" again? And what is Dan's role in all this? Who, exactly, then, is Dan?

II. Who is Dan?

Buddhism compares Man to a river. Both retain their identity despite the fact that their individual composition is different at different moments. The possession of a body as the foundation of a self-identity is a dubious proposition. Bodies change drastically in time (consider a baby compared to an adult). Almost all the cells in a human body are replaced every few years. Changing one's brain (by transplantation) - also changes one's identity, even if the rest of the body remains the same.

Thus, the only thing that binds a "person" together (i.e., gives him a self and an identity) is time, or, more precisely, memory. By "memory" I also mean: personality, skills, habits, retrospected emotions - in short: all long term imprints and behavioural patterns. The body is not an accidental and insignificant container, of course. It constitutes an important part of one's self-image, self-esteem, sense of self-worth, and sense of existence (spatial, temporal, and social). But one can easily imagine a brain in vitro as having the same identity as when it resided in a body. One cannot imagine a body without a brain (or with a different brain) as having the same identity it had before the brain was removed or replaced.

What if the brain in vitro (in the above example) could not communicate with us at all? Would we still think it is possessed of a self? The biological functions of people in coma are maintained. But do they have an identity, a self? If yes, why do we "pull the plug" on them so often?

It would seem (as it did to Locke) that we accept that someone has a self-identity if: (a) He has the same hardware as we do (notably, a brain) and (b) He communicates his humanly recognizable and comprehensible inner world to us and manipulates his environment. We accept that he has a given (i.e., the same continuous) self-identity if (c) He shows consistent intentional (i.e., willed) patterns ("memory") in doing (b) for a long period of time.

It seems that we accept that we have a self-identity (i.e., we are self-conscious) if (a) We discern (usually through introspection) long term consistent intentional (i.e., willed) patterns ("memory") in our manipulation ("relating to") of our environment and (b) Others accept that we have a self-identity (Herbert Mead, Feuerbach).

Dan (probably) has the same hardware as we do (a brain). He communicates his (humanly recognizable and comprehensible) inner world to us (which is how he manipulates us and his environment). Thus, Dan clearly has a self-identity. But he is inconsistent. His intentional (willed) patterns, his memory, are incompatible with those demonstrated by Dan before the accident. Though he clearly is possessed of a self-identity, we cannot say that he has the SAME self-identity he possessed before the crash. In other words, we cannot say that he, indeed, is Dan.

Dan himself does not feel that he has a self-identity at all. He discerns intentional (willed) patterns in his manipulation of his environment but, due to his amnesia, he cannot tell if these are consistent, or long term. In other words, Dan has no memory. Moreover, others do not accept him as Dan (or have their doubts) because they have no memory of Dan as he is now.

Interim conclusion:

Having a memory is a necessary and sufficient condition for possessing a self-identity.

III. Repression

Yet, resorting to memory to define identity may appear to be a circular (even tautological) argument. When we postulate memory - don't we already presuppose the existence of a "remembering agent" with an established self-identity?

Moreover, we keep talking about "discerning", "intentional", or "willed" patterns. But isn't a big part of our self (in the form of the unconscious, full of repressed memories) unavailable to us? Don't we develop defence mechanisms against repressed memories and fantasies, against unconscious content incongruent with our self-image? Even worse, this hidden, inaccessible, dynamically active part of our self is thought responsible for our recurrent discernible patterns of behaviour. The phenomenon of posthypnotic suggestion seems to indicate that this may be the case. The existence of a self-identity is, therefore, determined through introspection (by oneself) and observation (by others) of merely the conscious part of the self.

But the unconscious is as much a part of one's self-identity as one's conscious. What if, due to a mishap, the roles were reversed? What if Dan's conscious part were to become his unconscious and his unconscious part - his conscious? What if all his conscious memories, drives, fears, wishes, fantasies, and hopes - were to become unconscious while his repressed memories, drives, etc. - were to become conscious? Would we still say that it is "the same" Dan and that he retains his self-identity? Not very likely. And yet, one's (unremembered) unconscious - for instance, the conflict between id and ego - determines one's personality and self-identity.

The main contribution of psychoanalysis and later psychodynamic schools is the understanding that self-identity is a dynamic, evolving, ever-changing construct - and not a static, inertial, and passive entity. It casts doubt over the meaningfulness of the question with which we ended the exposition: "Who, exactly, then, is Dan?" Dan is different at different stages of his life (Erikson) and he constantly evolves in accordance with his innate nature (Jung), past history (Adler), drives (Freud), cultural milieu (Horney), upbringing (Klein, Winnicott), needs (Murray), or the interplay with his genetic makeup. Dan is not a thing - he is a process. Even Dan's personality traits and cognitive style, which may well be stable, are often influenced by Dan's social setting and by his social interactions.

It would seem that having a memory is a necessary but insufficient condition for possessing a self-identity. One cannot remember one's unconscious states (though one can remember their outcomes). One often forgets events, names, and other information even if it was conscious at a given time in one's past. Yet, one's (unremembered) unconscious is an integral and important part of one's identity and one's self. The remembered as well as the unremembered constitute one's self-identity.

IV. The Memory Link

Hume said that to be considered in possession of a mind, a creature needs to have a few states of consciousness linked by memory in a kind of narrative or personal mythology. Can this conjecture be equally applied to unconscious mental states (e.g. subliminal perceptions, beliefs, drives, emotions, desires, etc.)?

In other words, can we rephrase Hume and say that to be considered in possession of a mind, a creature needs to have a few states of consciousness and a few states of the unconscious - all linked by memory into a personal narrative? Isn't it a contradiction in terms to remember the unconscious?

The unconscious and the subliminal are instance of the general category of mental phenomena which are not states of consciousness (i.e., are not conscious). Sleep and hypnosis are two others. But so are "background mental phenomena" - e.g., one holds onto one's beliefs and knowledge even when one is not aware (conscious) of them at every given moment. We know that an apple will fall towards the earth, we know how to drive a car ("automatically"), and we believe that the sun will rise tomorrow, even though we do not spend every second of our waking life consciously thinking about falling apples, driving cars, or the position of the sun.

Yet, the fact that knowledge and beliefs and other background mental phenomena are not constantly conscious - does not mean that they cannot be remembered. They can be remembered either by an act of will, or in (sometimes an involuntary) response to changes in the environment. The same applies to all other unconscious content. Unconscious content can be recalled. Psychoanalysis, for instance, is about re-introducing repressed unconscious content to the patient's conscious memory and thus making it "remembered".

In fact, one's self-identity may be such a background mental phenomenon (always there, not always conscious, not always remembered). The acts of will which bring it to the surface are what we call "memory" and "introspection".

This would seem to imply that having a self-identity is independent of having a memory (or the ability to introspect). Memory is just the mechanism by which one becomes aware of one's background, "always-on", and omnipresent (all-pervasive) self-identity. Self-identity is the object and predicate of memory and introspection. It is as though self-identity were an emergent extensive parameter of the complex human system - measurable by the dual techniques of memory and introspection.

We, therefore, have to modify our previous conclusions:

Having a memory is not a necessary nor a sufficient condition for possessing a self-identity.

We are back to square one. The poor souls in Oliver Sacks' tome, "The Man Who Mistook his Wife for a Hat" are unable to create and retain memories. They occupy an eternal present, with no past. They are thus unable to access (or invoke) their self-identity by remembering it. Their self-identity is unavailable to them (though it is available to those who observe them over many years) - but it exists for sure. Therapy often succeeds in restoring pre-amnesiac memories and self-identity.

V. The Incorrigible Self

Self-identity is not only always-on and all-pervasive - but also incorrigible. In other words, no one - neither an observer, nor the person himself - can "disprove" the existence of his self-identity. No one can prove that a report about the existence of his (or another's) self-identity is mistaken.

Is it equally safe to say that no one - neither an observer, nor the person himself - can prove (or disprove) the non-existence of his self-identity? Would it be correct to say that no one can prove that a report about the non-existence of his (or another's) self-identity is true or false?

Dan's criminal responsibility crucially depends on the answers to these questions. Dan cannot be held responsible for Jack's murder if he can prove that he is ignorant of the facts of his action (i.e., if he can prove the non-existence of his self-identity). If he has no access to his (former) self-identity - he can hardly be expected to be aware and cognizant of these facts.

What is in question is not Dan's mens rea, nor the application of the McNaghten tests (did Dan know the nature and quality of his act or could he tell right from wrong) to determine whether Dan was insane when he committed the crime. A much broader issue is at stake: is it the same person? Is the murderous Dan the same person as the current Dan? Even though Dan seems to own the same body and brain and is manifestly sane - he patently has no access to his (former) self-identity. He has changed so drastically that it is arguable whether he is still the same person - he has been "replaced".

Finally, we can try to unite all the strands of our discourse into this double definition:

It would seem that we accept that someone has a self-identity if: (a) He has the same hardware as we do (notably, a brain) and, by implication, the same software as we do (an all-pervasive, omnipresent self-identity) and (b) He communicates his humanly recognizable and comprehensible inner world to us and manipulates his environment. We accept that he has a specific (i.e., the same continuous) self-identity if (c) He shows consistent intentional (i.e., willed) patterns ("memory") in doing (b) for a long period of time.

It seems that we accept that we have a specific self-identity (i.e., we are self-conscious of a specific identity) if (a) We discern (usually through memory and introspection) long term consistent intentional (i.e., willed) patterns ("memory") in our manipulation ("relating to") of our environment and (b) Others accept that we have a specific self-identity.

In conclusion: Dan undoubtedly has a self-identity (being human and, thus, endowed with a brain). Equally undoubtedly, this self-identity is not Dan's (but a new, unfamiliar, one).

Such is the stuff of our nightmares - body snatching, demonic possession, waking up in a strange place, not knowing who we are. Without a continuous personal history - we are not. It is what binds our various bodies, states of mind, memories, skills, emotions, and cognitions - into a coherent bundle of identity. Dan speaks, drinks, dances, talks, and makes love - but throughout that time, he is not present because he does not remember Dan and how it is to be Dan. He may have murdered Jake - but, by all philosophical and ethical criteria, it was most definitely not his fault.


More about this topic here:

http://philosophos.tripod.com/

http://samvak.tripod.com/film.html

Movie Reviews here:

http://samvak.tripod.com/film.html

http://www.geocities.com/vaksam/film.html

These six movies deal with moral and ethical dilemmas. "The Matrix" - with the nature of reality, "Titanic" - tackles the moral obligation versus self-preservation and "The Truman Show" - tackles the relationship between God and us, humans. "The Talented Mr. Ripley" deals with the thin line between pathological narcissism and self-promotion, ambition and envy, love and obsession. "Being John Malkovich" deals with this question (among others): are we the owners of our brains? "Shattered" deals with the meaning of self identity and personal responsibility.

Part of the "Philosophical Essays and Musings" Web site here:

http://samvak.tripod.com/culture.html

http://musings.cjb.net