Running Head: Human Brains in Virtual Worlds

©1996 - G. Cartwright

Experienced travelers know that the more foreign the destination, the more adaptation that is required mentally, emotionally, physically, culturally, and socially. But what about visiting a non­existent land in a virtual world? Can our brains adapt? Can we? How will human brains function in worlds for which they were not built? And what will they do when the get there? 

Experienced travelers know that the more foreign the destination, the more adaptation that is required mentally, emotionally, physically, culturally, and socially. Often, the more alien the destination, the more tiring and stressful the experience. The sensual cacophony of sounds, spectacles, textures, tastes, and aromas can be overwhelming and contribute to disorientation, dislocation, and disconnection. The combination can range from mild to deadly: from homesickness at best to psychotic break at worst.
 

A Trip to Nowhere

It follows then, that virtual travel to the same foreign destination, by definition, will yield the same sensations. (If it didn't, it wouldn't be virtual.) But what about travel to a non-existent land in a virtual world? Yesterday's trip to a physical somewhere becomes today's VR trip to unearthly nowhere.

Non-existent lands

Travel in virtual reality (VR) presents many more possibilities than real travel. First, in addition to visiting the same venues as in real travel, one may also call on non-existent, imaginary lands, perhaps cartoon-like expanses like Disney's Aladdin VR ride, or artistic, fluid, experiential shores like none have ever seen before, or territories created uniquely for each visitor which no one else can ever share. Second, one can also jump from place to place, saving travel time and creating instant travel gratification with concomitant sensory discrepancies. Like Alice in Wonderland, the sights in those places may be wondrous to behold. But they may also be awesome, nightmarish, or schizophrenic. Like a hippie on a bad drug trip, the cybernaut risks becoming trapped in a world of someone else's making, at the mercy of its creator, with no place to hide, nowhere to go, no way forward, no way out, and no way back.

Can our brains adapt? Can we?

The human brain evolved and continues to develop in response to its natural environment. Based on the processing of a myriad of sensory inputs, it solves problems, creates unique solutions, and continually adapts to reality. But how might the human brain respond in a land where architecture is liquid, music ephemeral, language incidental, communication altered, maps distorted, geometry transitory, history imaginary, geography variable, and space transmuted? Will the discrepant sensory information used to deceive the brain into thinking it is elsewhere, cause other distress? Will the brain lose total contact with reality and if it does what are the consequences for mental health?

Distinguishing virtual from real

Some early movie goers believed that what was projected on the screen was real (Vorse, 1991), some reacted accordingly (Palmer, 1909). Film libraries recount the tale of the first public motion picture exhibition by the Lumière brothers in which terror ensued as audience members confused a motion picture train for an oncoming real train and fled the theater. This would not happen today with film literate people; human brains apparently learned to distinguish the difference between motion picture representations and reality. But how? Might the same process be necessary in VR to prevent mental and behavioral breakdown? Might there some background brain mechanism operating to keep track of the difference?

If there is, then it must operate from sensory inputs. Could such differences be detected in the brain's electrical activity? Does total immersion in a VR illusion make it harder for the brain to distinguish reality from virtuality?

Mental health risk

The question is important because the immersion of children in an illusory world without knowing how to differentiate it from reality may put their mental health at risk (Cartwright & Silva, 1993; Cartwright, 1994). Since mentally healthy individuals can distinguish between reality and illusion, knowing the difference between reality and virtual reality (VR) may prove to be important in avoiding possible psychotic episodes and other mental disorders.

Just early movie goers learned to distinguish between what was real and what was projected, cybernauts might similarly learn to distinguish reality from VR. If so, and if such a distinction relies on sensory detection, then it might be possible to study such a mechanism by examining sensory input. In this case we have chosen vision, particularly because the perception of movement is a major feature of VR. If the perception of movement involves higher cognitive abilities, might these might function (or be trained to function) to detect the real from the unreal?

To begin to answer this question, our lab began to examine how the human brain perceives movement.

Experiment

The debate over the brain's ability to differentiate between apparent motion (Wertheimer, 1912) and real motion began in the 1960s. Several studies (Freedman & Hardy, 1989; Kolers, 1963, 1964; Peterhans & Hydt, 1991) indicated that the brain processes movement in different ways, and in different cortical areas.

Electroencephalograms (EEGs) were used in various experiments in visual perception (Fabiani, Gratton, Karris & Donchin 1987; Rebert, Low & Larsen, 1984) and can be used to examine the cortical activity which occurs during the perception of real and apparent movement. Both the theta and alpha rhythms have been shown to be related to the visual system and visual perception, and associated with attentional demands and degree of task difficulty (Brazier & Casby, 1952; Barlow, 1993); Gastaut, 1950; Osaka, 1984; Mundy-Castle,. 1951, 1957). The frontal area of the brain seems to be related to visual processing as well as to spatial working memory (Jonides, Smith, Koeppe, Awh, Minoshima & Mintun, 1993; McCarthy, Blamire, Puce, Nobre, Bloch, Hyder, Goldman-Rakic, Schulman, 1994; Owen, Downes, Sahakian, Polkery & Robbins, 1990)
 

Examining the research available on the perception of real and apparent motion and the research on EEG we wanted to know: Is there a significant difference in alpha and/or theta rhythm in the frontal and occipital areas when seeing flashing lights perceived to move and seeing real movement?

To include concepts of virtual reality in our study, we modified our definition of movement to incorporate the following:
 
 

Subjects

There were 7 female and 4 male subjects, aged 19 years, 4 months to 44 years, 11 months, with a mean age 24.5 years. All were volunteers from McGill University undergraduate or graduate programs. All were considered normal and did not report any medical problems. Ten subjects reported no use of medication in the previous 24 hours; one subject reported taking an antihistamine for allergy. All reported they had not taken alcohol or illegal drugs in the previous 24 to 48 hours.

Description of stimuli

Animation Studio software (Walt Disney Company, 1991) was used to develop the stimuli, a white, circular object on a black background known to produce maximum stimulation in the occipital area. The stimuli were presented at three different speeds: 1, 5, and 12 frames per second. The stimuli moved across a 13" computer screen in a straight line from left to right. There were four presentation conditions which varied location, frame rate, and duration.

Procedure

Subjects had 16 electrodes placed according to the international 10-20 system and a QSI 9000 computer was used to record the EEG. A baseline recording with eyes closed was used to compare each subject with normative data to scan for abnormalities. Another baseline, eyes open looking at the computer screen, was used to determine if there were any differences between looking at a static computer screen and later perceiving movement on the screen. With subjects watching moving stimuli on a computer screen, their EEGs were collected and later transformed into colored brain maps.
 

The subjects then viewed the stimuli under the four conditions of real movement, close approximation of movement, flashing light, and eyes open at the computer screen. The flashing lights did not produce the perception of movement. After each 3 minute presentation subjects were asked to report what they saw.

Artifact rejection

Sections of EEG containing muscular movement of the jaw and excessive eye movement were not used; the clearest one minute recording for each condition was kept for analysis.

Description of Electrophysiological measures

The EEG frequency spectrum has 5 rhythms: Delta .4 ­3.6 Hz; Theta 4­7.6 Hz; Alpha 8­12.6 Hz; and Beta 13+ Hz. Only Absolute Power (the average expression of magnitude of the electrical signal in each of the frequency bands) was used in this analysis.

Results

Of the 11 subjects tested, 4 were rejected because of some slight abnormality with their EEG in the eyes-closed at-rest position compared to the normative data base and one other subject was rejected who was unable to report a threshold for the perception of movement. Consequently, only 6 subjects were used in the final analysis. Frontal, occipital, and parietal areas of EEG generation were examined. The results are shown in Table 2.

Conditions Compared	Frontal Alpha Power	Frontal Theta Power	Occipital Alpha Power	Occipital Theta Power	Parietal Alpha Power
Eyes Open to Flashing	50% different 3 increased 0 decreased	50% different 1 increased 2 decreased	no difference	no difference	50 % different 2 increased 1 decreased
Eyes Open to Threshold	50% different 3 increased 0 decreased	67% different 3 increased 1 decreased	17% different 1 increased 0 decreased	no difference	17% different 1 increased
Eyes Open to Real	100% different 6 increased 0 decreased	83% different 3 increased 2 decreased	33% different 2 increased 0 decreased	17% different 1 increased 0 decreased	33% different 2 increased
Flashing to Threshold	67% different 3 increased 1 decreased	67% different 2 increased 2 decreased	no difference	no  difference	17% different 1 decreased
Flashing to Real	66% different 4 increased 0 decreased	66% different 4 increased 0 decreased	17% different 1 increase	17% different 1 increased	33 % different 2 increased
Threshold to Real	100% different 4 increased 2 decreased	83% different 3 increased 2 decreased	no difference	no difference	33% different 2 increased

The results suggested the possible involvement of higher level cognitive processing when visual stimuli were perceived to be moving. This is in line with the findings of Tsude and Ueno, (1993). There were differences in alpha power which are associated with memory, task difficulty, and decreases in attentional demands. Differences in theta power were unclear possibly because midline frontal theta is often associated with personality variables and attentional mechanisms.

Initial results indicated significant differences in the brain's frontal area suggesting that perceived movement requires higher order cognitive processes outside the visual area. What remains unknown is whether the same higher order cognitive processes are used in perceiving VR.

Limitations of the Study

The results presented here must be considered cautiously in view of the small number of subjects and the subjectivity involved in reporting the perception of movement.

Implications for Education

The perception of movement and the associated increases in alpha and theta power have implications for various special populations. For example, dyseidetic dyslexia (children with problems in spelling and reading patterns as measured by the Boder criteria) is known to be associated with relative inefficiencies of the right hemisphere and visual gestalt abilities (Flyn & Deering, 1989). 

Attention Deficit Hyperactivity Disorder (ADHD) children show increased theta predominantly in their frontal areas (Mann, Lular, Zimmerman, Milter, & Muenche, 1991) as well as more delta, faster theta waves, and fewer alpha waves (Matsura, Okubo, Tora, Kayima, Hou, Shen, & Lee, 1993). ADHD children lag behind normal children on the Bender Visual Motor Gestalt Test and many manifest problems with integrating visual perception and fine motor functions. Medicated ADHD children exhibit more problems with visual perceptual integration especially when requiring short-term memory (Risser & Bower, 1993). For all these children, the presentation of moving stimuli may impede rather than augment their learning. Visually presented material and computer generated instruction may further hinder their learning abilities. As more and more learning takes place in virtual reality, such children may be at a particular disadvantage. Comparatively speaking, we speculate that learning disabilities may show a marked increase in cyberspace.

On the other hand if the changes in theta are dependent on individual differences, the perception of movement might decrease theta in certain ADHD children. Perhaps a new method can be devised to increase their learning: might computer screens with flashing rather than static material would help them focus better to optimize their learning?

The collection of additional data using more electrodes together with a more complete statistical analysis is recommended.
 

It was concluded from this experiment that the perception of movement involves higher cognitive processes.

If this conclusion holds, it is likely that similar processes are at work in helping us perceive in cyberspace. Finally, it is possible that two such sets of processes are at work simultaneously, one to anchor us in a rudimentary way to the real world, the other to help us perceive virtual wonders. Such dual systems may be necessary to prevent us from becoming totally lost in cyberspace.

Hypothetical Constructs of the Mind

One set of hypothetical constructs developed during the last century to explain the workings of the mind concerned defense mechanisms.

For example, like the tripping of a circuit breaker, the brain can blank out things it doesn't want to see. Fainting and blackouts are but two examples of defensive actions resulting in detachment from reality.

The great potential of VR, what Hatton called a "celebration of the senses", paradoxically holds the possibility of force-feeding the senses culminating in information overload, cognitive and sensory dissonance, judgmental lapses, and emotional and psychological distress in the form of denial and detachment. Such mechanisms are well known in the literature. Might such forced sensory (VR) input produce the same distress?
 

Our recent work with VR has led us to consider these related questions. How is a person immersed in cyberspace -- perhaps floating in space as we observed in the VR program Osmos -- able to keep feet on the ground and not fall over in the real world? Anecdotal reports suggest that few if any of hundreds of subjects who experienced floating fell over in the real world. Might there be some mechanism in the brain which permits the unconscious detection and separation of real from virtual? Might it allow a feeling of immersion to engulf the subject, yet continue working to look after details such as real world balance? If so, this would explain visitors to cyberspace not falling over in the real world. But is it a naturally occurring (automatic) function or must it be trained? Consideration of these questions has led us to propose the following hypothetical construct.

Gyroscopes of the Mind

We submit that just as the brain has hypothetical defense mechanisms, it may also have a series of dynamic mechanisms to keep the mind "upright" and functioning within normal limits. We envision these as functionally equivalent to physical gyroscopes -- stabilizing devices which act to keep the mind in balance.

A gyroscope is a physical, spinning entity which resists attempts to shift it to another plane. A child's top is a simple example. A spinning top, if disturbed, will right itself. Similarly, a bicycle wobbles less (becomes more stable) the faster it goes.

We argue that it may be useful to imagine a whole series of mind gyroscopes: a reality gyro, an moral/ethical gyro, and a spiritual gyro, to name but three. Perhaps there are more. For example, today one often hears that we have "lost our moral compass". The analogy is striking but not quite complete. A compass may set one's direction but it is a gyro that keeps one on course. If both are needed in the real world, both will certainly be needed in the cyberworld.

The Reality Gyro

We will confine our discussion here to the reality gyro. 

The reality gyro functions by its sensory inputs from the real world. When information from one sense is aberrant, the other senses function to correct the input. Like each of the stabilizing computers on board the Space Shuttle, each sense contributes to correcting the errors of the others to produce a stable environment. The reality gyro -- the sum of all sensory input -- works to keep the person on track. Distorting too many of the sensory inputs in an uncoordinated way can result in severe forms of distress, for example the motion sickness which has been noted on out-of-sync VR amusement park rides.

We speculate that when a person enters cyberspace, a new gyro is established to take over perception of the new reality. However the old reality gyro remains "spinning" in the background (the analogy is a computer background task), anchoring us in the real world, keeping a subconscious watch over our real world presence, maintaining our real world balance etc. We argue that such mechanisms are necessary if we are to be able to travel safely back and forth to and from cyberspace.

Following the argument that the brain develops in response to its environment, this suggests that such gyros can be created as needed. We suspect that as people become more and more familiar with VR applications, they will automatically create new gyros to help them keep track of where they are to facilitate transit between virtual and real worlds. Although this may diminish the overall effect of VR (just as seated movie goers now "know" the oncoming train won't hit them), it will provide a safety valve for some of the dangers associated with travel to cyberspace.

1. They will suffer psychotic breaks, disorientation, and disaffect. The dangers of cyberspace are real and potentially deadly.

2. They will experience cognitive and perceptual lapses and an exacerbation of learning disabilities. New perceptions and experiences will distort the usual cognitive and perceptual processes affecting judgment and action.
 

3. They will experience alienation. Like a foreigner in a strange land, the downside may be a tendency toward paranoia.

4. They will enjoy new sensory experiences. New sensory encounters will transcend the ordinary in intensity, duration, and frequency. Psychedelic experience will be more the rule than the exception.

5. They will free themselves from temporal and spatial constraints. One of the most significant aspects of cybereality will be a return to timelessness and freedom from spatial considerations. Motion will be unfettered and easy, enhancing the illusion of freedom.

6. They will communicate effortlessly. The potential exists for communication to be effortless, integrated, and multidimensional.

7. They will generate new and different bodies. Cybernauts will be able to create and don new bodies to participate in wholly new activities.

8. They will experience increased wonder and awe. The potential exists to create the sublime as a matter of course; to make existence angelic and peaceful.

9. They will heal the body and mind. Cyberspace would be a poor place indeed if its resources could not be used to heal both real body and mind. New insights, transformational fantasy, guided imagery and similar techniques will all take on new meaning and efficacy when executed in cyberspace. Their power will increase further when exported back to the real world.

10. They will transcend traditional consciousness. The ultimate trip will be travel to one's own inner core to create and generate new and multiple forms of distributed consciousness. Such activity will result in a transcendence of conventional consciousness to a more expansive, all inclusive awareness entailing an understanding of cosmic origin, an appreciation of the universe, cognizance of universal human values, and the attainment of true wisdom. These are the potentials and promises: these are goals worth pursuing.

This cyberworld research has implications for real world education. In the real world, it may provide new knowledge and innovative solutions to augment learning and assist children with specific learning disabilities like attention deficit hyperactivity disorder (ADHD). For example, deliberately decreasing frontal theta activity may prove to be beneficial to children with ADHD to help increase their attention span. In the cyberworld, we may learn how to overcome the dangers of educational travel to cyberspace, to understand the psychological limits of our stays there, and to determine how to return safely. Ultimately, it may shed light on how our brains can adapt and function in a wide variety of virtual realities and other worlds.

Barlow, J. S. (1993). The electroencephalogram, its patterns and origins. MIT Press, A Bradford Book.

Brazier, M.A.B. & Casby, J.U. (1952). Cross correlation and autocorrelation studies of electroencephalographic potentials. EEG Clinical Neurophysiology, 4, 201-211.

Cartwright, G. F. (1994). Virtual or real? The mind in cyberspace. The Futurist, 28 (2), 22-26.

Cartwright, G. F. and Carson, D. (1995). Work in cyberspace. Paper presented at the western conference of the American Psychological Association, Kona, Hawaii.

Cartwright, G.F. & Silva, M. (1993). Virtual reality and consciousness. Paper presented at the Seventh General Assembly of the World Future Society. Washington, D.C.

Fabiani, M., Gratton, G., Karis, D.,& Donchin, E.(1987). Definition, identification and reliability of measurement of the P300 component of the event related potential. In Ackles, J. R. Jennings & M. G. H.Coles (Eds.), Advances in psychophysiology, 2, (pp 1 -78). London: JAI Press.

Flynn, J.M., & Deering, W.M. (1989). Subtypes of dyslexia: investigation of Boder's system using quantitative neurophysiology. Development Medical Child Neurology, 31, 215-223.

Fredman, A. & Hardy, C.A. (1989). Seeing Versus imaging movement in depth. Canadian Journal of Psychology, 44 (3), 371-383.

Gastaut, H., Roger, y., Corriol. J., & Naquet, R. (1951). Étude électrographic du cycle d'exicitabilité cortical. EEG Clinical Neurophysiology, 3, 401-428.

Jonides, J., Smith, E.E., Koeppe, R.A., Awh, E., Minoshima, S., & Mintun, M.A. (1993). Spatial working memory in humans as revealed by PET. Nature, 363, 623-625.

Kolers, P.A. (1963). Some differences between real and apparent movement. Vision Research, 3, 191-206.

Kolers, P.A. (1964). Some Illusions of movement. Scientific America 211 (4), 98- 106.

Kolers, P.A. (1972). Aspects of motion perception. Oxford: Pergamon press.

Mann, C.A., Lubar, F.F., Zimmerman, A.W., Miller, C.A., Muenchen, R.A. (1991). Quantitative analysis of EEG in boys with attention-deficit-hyperactivity disorder: controlled study with clinical implications. Pediatric Neurology, 8 (1), 30-36.

Matsuura, M., Okubo, Y., Toru, M., Kojima, T., He, Y., Hou, Y., Shen. Y., Lee, C.K. (1993). A cross-national EEG study of children with emotional and behavioral problems: a WHO collaborative study in the Western Pacific Region. Biological Psychiatry, 34, 59-65.

McCarthy, G., Blamire, A., Puce, A., Nobre, A.C., Bloch, G., Hyder, F., Goldman-Rakic, P.G., & Schulman, R.G. (1994). Functional magnetic resonance imagine of the human prefrontal cortex activation during a spatial working task. Neurobiology, 91, 8690-8694.

Mundy-Castle, A.C. (1951). Theta and Beta rhythm in the electroencephalograms of normal adults. EEG Clinical Neurophysiology, 3, 447-486.

Mundy-Castle, A.C. (1957). The electroencephalogram and mental activity. EEG Clinical Neurophysiology, 9,645-655.

Osaka, M. (1984). Peak alpha frequency of EEG during a mental task: task difficulty and hemispheric differences. Psychophysiology, 21 (1), 101-105.

Owen, A.M., Downes, J.J., Sahakian, B.J., Polkey, C.E., & Robbins, T.W. (1990). Planning and spatial working memory following frontal lobe lesions in man. Neuropsychologia, 28 (10), 1021-1034.

Palmer, L.E. (1909). The world in Motion. Survey 22, 355-365.

Peterhans, E. & Heydt, Von Der, R. (1991). Subjective contours bridging the gap between psychophysics & physiology. Trends in Neuroscience 14 (3), 112-119.

Rebert, C.V., Low, D.W., & Larsen, F. (1984). Differential hemispheric activation during complex visual motor performance: Alpha trends and theta. Biological Psychology 19 (3), 159-168.

Risser, M.G. & Bowers, T.G. (1993). Cognitive and neuropsychological characteristics of attention deficit hyperactivity disorder children receiving stimulant medication. Perceptual and Motor Skills, 77, 1023-1031.

Tsude, R. & Ueno, S. (1993). Spatio-temporal properties of evoked potentials associated with visual apparent movement. Proceedings of the third international congress on BET. Brain Topography, 5 (4), 448.

Vorse, M.H. (1911). Some picture show audience. The Outlook, 98, 441-447.

Walt Disney Company,1991. The Animation Studio. (computer software)