The Tao of Virtual Reality

In Tao Te Ching, perhaps the most widely translated Eastern Philosophy texts of all time, we hear of the philosopher Chuang Tzu. One night, Chuang Tzu dreamed that he was a butterfly. He dreamt that he was flying around from flower to flower and while he was dreaming he felt free, blown about by the breeze hither and thither. He was quite sure that he was a butterfly. But when he awoke he realised that he had just been dreaming, and that he was really Chuang Tzu dreaming he was a butterfly. But then Chuang Tzu asked himself the following question: was I Chuang Tzu dreaming I was a butterfly or am I now really a butterfly dreaming that I am Chuang Tzu?

kindpng 4994330 1024x684 - The Tao of Virtual Reality
Fractal butterfly. Unknown artist.

Eastern Philosophy and VR

How we identify, that is, how we define our selves—asking ourselves who we are—has been a central inquiry in Philosophy. It is usually this question that the world religions claim to answer. We seek an answer that may soothe us, or an answer that at any rate makes sense in relation to our general experience of being-in-the-world. This latter point is crucial: we desire meaning, wholeness.  We want our lives to be purposeful. There is supposed to be something to do here, an end to be achieved, and we feel that it should be of importance.

Self & Identity in Virtual Worlds

Now—who we are can be answered in two ways, and they are both principally the same, as either will have to imply the other. If we are to define who we are as humans, then we must, necessarily, define our selves in relation to the world we inhabit. Similarly, if we choose to define the world, the definition of our world would have to include our role towards it, as worlds can not be accessed except through experience.

Fractal background removed 1024x763 - The Tao of Virtual Reality
Fractal tree. Unknown artist.

Let us imagine a virtual world to exemplify this.  Imagine a virtual world,  covered in a vast forest. In this world, we situate a  subject in the avatar of a strong woodchopper. Now,  the concept of the woodchopper being strong is meaningless in itself, he can only be said to be strong in relation to a particular world where the trees are weaker than him. Similarly, if we create a virtual world with very hard trees, the notion of the trees being hard, is absolutely meaningless unless as understood in relation to a certain woodchopper, for whom the trees are hard. 

Now of course, this is just basic logic, but it is nevertheless very useful in ideation of our future virtual worlds. As we have been discussing throuh various entries, such as From thought to realityThe Existential Problem of VRand The Virtually Extended Mind, with the arrival of ultimate VR we will be able to design our selves; manifest our thoughts and desires; and in so doing, realizing our selves outside of ourselves. In doing so, to guide our perspective, we must know that we can not create worlds independent of our selves; what is objective is in relation to our subjectivity. A certain virtual world will have to imply a certain virtual self—the technological material of virtual reality that is shaped in to a certain world will also shape us in our existential relation to that world.

yinyang - The Tao of Virtual Reality
Yin Yang Mandala.

In philosophy and design research, such a perspective on the mediating qualities of technology—that is, how technology co-constitutes perceived subjectivity and objectivity inour experience—is known as postphenomenology. Just like its phenomenological roots (for instance Heidegger), this Western philosophy is quite related to themes from Eastern philosophy, for instance as illustrated by the Yin/Yang. Just as a certain subjectivity implies an objectivity, and a certain objectivity implies another subjectivity—the traditional Eastern symbol of Yin/Yang similarly shows how Yin implies Yang and Yang implies Ying. But what may such Eastern Philosophy and VR have in common in this regard?

These perspectives can be particularly fruitful for researchers who are investigating the mediating qualities of interactive applications, like VR environments. For instance, if I am in a virtual simulation, controlling a supersonic spaceship, it is not only me—that is, my subjectivity—which is altered. As I become able to navigate, the world—that is, what I adhere to as objectivity—also changes to become more accessible to me. Thus, the technology of the spaceship mediates a certain way of relating to the world, constituting me as a certain subject, and simultaneously, the virtual environment as a certain virtual world.

At this point we have laid the philosophical groundwork making us ready to explore the question of this entry:

Who Are We In Virtual Reality?

Chuang Tzu’s dream and subsequent merry doubt is a simple story that illustrates the flexibility of human identity. Although we usually manage to keep a solid persona and perceive our egos as relatively fixed, it is curious how malleable they are. Consider for instance how, when you are with your parents or relatives, how you are someone completely different from when you’re with your friends—and an even more different character may emerge when alone with your significant other. Beyond simple social situations as mediators of personalities and experiences, phenomena like dreams, or psychedelic drugs, also show how the human mind has the capability of radically altering what it identifies with. And not least, meditation and other forms of contemplation, can radically change our outlook on life in terms of how we identify and approach our existence.

Most exciting of all, however, Virtual Reality can also change our selves.

In our piece on Virtual Embodiment, we explored the ways in which thorough, bodily identification with a virtual avatar, can have profound behavioral effects. It can help against racism, by embodying people in other skin colours; help against violence by placing offenders as victims; it can cause a rise in cognitive performance by being embodied as someone smart, and, last but not least, as we discuss in The Virtual Freud, it can be therapeutic in allowing us a more compassionate outlook on our selves. 

Who Do You Want To Be?

The point which we often come to at Matrise, albeit from various perspectives, is to ask our selves a question. Lying in between Philosophy and VR, the question it poses is: who do we want to be? What virtual worlds will we create in the future, and who will we be, as subjects, within those worlds?

For the interested reader, who wants to further pursue this question from various other perspectives, can go on to explore this through these other entries, many of which touch upon Western- and Eastern Philosophy and VR.

In The Existential Problem of Virtual Reality, we discussed how this question that VR poses us, of who we want to be, can be said to be existential and revelatory as to our core identity.

In A Virtual Masquerade, we discussed the peculiar nature of social VR, and how embodying avatars can provide new benefits and creative expression in communication.

In The Virtually Extended Mind, we discussed how we can extend our mind functions into the virtual, and so entertained the question of how we could design our new minds. 

For more Eastern Philosophy and VR, check out our entry on Inner as Outer: Projecting Mental States as External Virtual Reality.

And finally, in A Psychedelic Virtual Reality, we discussed the possibility of visual languages which defied the subject/object dualism. 

The Virtually Extended Mind

In this entry on the VR mind, we will discuss some ideas of (living!) Philosopher David Chalmers in relation to Immersive Virtual Reality. We will discuss how we may virtually extend our minds, and what kind of VR Minds we would like to create.

Let us start off with a few words from the man himself:

Virtual Reality technology is wonderful for a philosopher because it brings alive some of the greatest philosophical questions — David Chalmers

In the installment from which the above quote was taken, the distinguished philosopher and consciousness researcher goes on to mention Descartes’ suspicion of the external world as an evil illusion—in addition to the more recent theories by Nick Bostrom that discusses the probability that our reality is illusory. Indeed, one of the benefits of the virtual is that it may provoke new ways of viewing the real. What Chalmers mention here is one of the reasons why VR is interesting for philosophers.

Matrise has for the last few years been discussing themes like these, for instance in our entry on The Experience Machine as well as in our three-series entry on Heidegger, both in which we discuss the notion of authenticity in relation to being in virtual reality. Beyond such «principal» themes, however, VR can offer way more for philosophers than just a stir in the age-old questions of philosophy. Today we will review some newer ideas and discuss their relevance for the technology of VR.

David Chalmers - The Virtually Extended Mind
David Chalmers. Photo by Demetrius Freeman for The New York Times—originally from this article. Further edited by Matrise.

Chalmers is very right, however, in that VR acts as a catalysator for thought. Why, it can even be said to pose an existential problem to us. As we recently covered in depth, technologies, in certain ways, are a way of human exteriorisation—a way for humans to realise themselves outside of themselves. As VR, at least in conceptual terms, is a technology capable of realising anything, we can perceive of it as an existential problem as this freedom forces us to reflect on what we value and want, and thus who we are—if we accept the premise that what we externalise can tell us who we are.

In this entry, we will take to imagining such future realities, and bring Chalmers ideas with us on the ride. Not his paralleling VR and Descartes’ evil demon, however, but by returning to some of Chalmers’ own ideas and hypotheses on the extended mind.

The Extended Mind

In 1998, together with Andy Clark, David Chalmers published a paper describing an idea under the name of “The Extended Mind.” The central question which they approach is where our minds stop and where the world begins. Where is the boundary, in other words, between the dualism of self and other, and to which degree is the brain the mind, or the mind the brain? The philosophers deal with this theme in arguing that the tools and technologies we use become part of our minds, and so we extend our selves into the world.

409 bond of union 726 546 80 s orig - The Virtually Extended Mind
Where do we end and the world begin? Artwork by M.C. Escher.

A brilliant introduction to this idea is Chalmers’ Ted Talk in which he presents the central thesis. In the talk, Chalmers parallels this view to the perhaps better known examples of bodily extensions or embodiments; for instance in how blind people’s canes work as an extension of their body. This is exactly the same principle as Merleau-Ponty’s  woman with a feathered hat which he presents in his Phenomenology of Perception; in which her bodily consciousness seem to float even out into the tip of the feather, and so she avoids breaking it.

Chalmers point is that we are outsourcing certain mind functions to machines—such as recalling phone numbers—to our phones. Similarly, spatial navigation and information is offloaded to Google Maps. Now, few would argue against the fact that technologies are performing important functions for us. Chalmers point, however, can not be reduced to understanding these technologies as tools, his argument is that they are literally becoming a part of our minds, although they are not wired directly to our brain.

The Mind / Body Problem

Now, as we said, this is touching upon the mind/brain problem: to which extent can the mind be reduced to, or traced back to, the brain? This question is not as easy as it may first seem. Obviously, if we cap someone on the head, they become unconscious and so, apparently, no mind. It may from this, naïvely, be deduced that thus the mind is the brain. When the mind is operating, however, it is harder to reduce it as it extends and uses what it perceives to operate its functions.

In addressing this problem, we can thus ask: what is so special about the inside of the brain, that only this part should have the special features constituting our mind? If something is going on outside it, as long as it is driving the processes in the brain in the same way, there is no principal difference lying in the skull separating it from the world. If the information structures you are using for your processing is stored in your local hard drive or in the cloud: does it matter? Are they not both a part of your computer, in principal terms?

kolakovic1 large - The Virtually Extended Mind
Artwork depicting a Memory or Mind Palace by Stuart Kolakovic.

The Virtually Extended Mind: A Palace?

Here we reach the point of our entry. If our minds can be extended into our physical world, how may we extend it into the virtual? In other words, a perspective that can be had in imagining the virtual worlds of the future, is to view them as extensions of our minds. What tools, then, should we use, develop and adopt during our thought processes? What would constitute our VR minds?

A good example of this that we have previously discussed at Matrise has been Memory Palacesarranging information visuo-spatially in order to better preserve it. Beyond this concrete utilisation, however, we can also imagine not memory palaces, but mind palaces. These can be personalised, meaningful places, from which we can gain sustenance, peace or productivity. They can be filled with various tools: for emotional healing, meditation, work, relaxation or entertainment. These rooms can be viewed as extensions of our minds and a way for us to immersive our selves and synchronise our selves to the various activities which we want to carry out.

Johnson Memory King - The Virtually Extended Mind
Mind Palace Illustration. Unknown artist.

The way this is related to VR as an existential problem, as we recently discussed, is that the concept of VR is asking our opinion in a different sense than normal technologies. It is not a question of whether we want A or B, but fundamentally, from the start, what would we like to see, or in other words, who do we want to be; or as Bruce Mau’s God-like question is formulated: now that we can do anything, what will we do?

What would you have in your Mind Palace, could you choose? What would constitute your VR mind?

Conclusion

I will conclude this entry with a quote from Hannibal by Thomas Harris:

Like scholars before him, Dr. Lecter stores an enormous amount of information keyed to objects in his thousand rooms, but unlike the ancients, Dr. Lecter has a second purpose for his palace; sometimes he lives there. He has passed years among its exquisite collections, while his body lay bound on a violent ward with screams buzzing the steel bars like hell’s on harp.


The interested reader can go on to these entries which are similar in theme:

In The Virtual Freud, we discussed how VR can induce what we can call out of body experiences; by allowing us a view of ourselves from outside, we may allow us to be more compassionate towards our selves.

In Virtual Embodiment we discussed how powerful illusions can be facilitated so that users can identify completely with virtual avatars and so help us overcome prejudices, reduce racism and violence.

In A Psychedelic Virtual Reality we discussed how VR may take inspiration from psychedelic drugs and facilitate for non-dual states of consciousness through the merging of self and other.

The Existential Problem of VR

Søren Kierkegaard, the father of Existentialism, famously described anxiety, or angst, as the dizziness of freedom. Hardly a cheerful fellow—though his brilliant, satirical wit often forces one to smile—Kierkegaard clearly was no stranger to this “dizziness” he so often spoke of:

I see it all perfectly; there are two possible situations—one can either do this or that. My honest opinion and my friendly advice is this: do it or do not do it. You will regret both.

Screenshot 2020 01 28 at 12.13.14 - The Existential Problem of VR
Søren Kierkegaard by Michael Newton.

Oh dear. Despite the gloom, however, these small drops of the Danish philosopher serve to illustrate his role as a great inspiration for the existential movement within Philosophy. Existentialism is existential in the sense that it is not concerned with the thinking cogito as the starting point of human reasoning. Rather, existential philosophy could be said to be more wholesome in that it is concerned with its taking seriously the human condition of alienation—our tendency towards existential dread. The critique that existentialism holds against traditional philosophy is that it has been too locked up in cognitive schemes, thoughts and abstractions, and in so doing distanced itself from the lived experience of being human, with all that this entails, not adhering to the givens of existence.

Existentialism vs Nihilism & Pessimism

Just to be clear, however, existentialism is not the same thing as advocating a kind of nihilism or a philosophical pessimism, as warned by the existentialist Friedrich Nietzche and held by Arthur Schopenhauer and Peter Wessel Zapffe, the latter holding that to bear children into this world is like bringing wood to a burning fire. No, in addition to the gloomy writings of Kierkegaard expressing the problem to be dealt with, he is also concerned with a solution, however hopeless. Though holding that life does not come with a manual, existentialists are nevertheless concerned with authenticity, or meaning—just understood and found in a different way. Even Albert Camus, whose famous essay on The Myth of Sisyphus, in which Sisyphus is condemned to roll a stone up a mountain only to watch it roll down again ad infinitum holds that, though this is absurd in the literal sense, we must imagine Sisyphus happy.

This always reminds me of a Zen story;

When Mu-chou was asked, “We dress and eat every day, and how do we escape from having to put on clothes and eat food?” Mu-chou answered, “We dress; we eat.”

5cfb1512940709.5626f410e5894 - The Existential Problem of VR
Jean-Paul Sartre, another existential thinker. Painting by Patrick Rork

So. The giveaway point of this brief introduction is that existentialism starts from existence. It it is not concerned with any philosophical beliefs or notions prior to this point, in other words, humans and their essential role are not pre-given. As Jean-Paul Sartre put it, which further has been descriptive of the existential movement as a whole: existence precedes essence; we exist before we define ourselves, and moreover, this decision is one that necessarily must be taken alone, without hiding behind any authority in order to outsource our freedom.

It is exactly this—how we define our selves—which will be the topic of this entry: how technologies, and especially the technology of VR, may play a part in the shaping of our identity. We will discuss the relation between Existentialism and VR in howVR may pose an existential problem to us as individuals and together as a society,

The Existential Problem of Virtual Reality

So what may Existentialism say about VR? The existentialists highlight the freedom of the individual subject in altering, or at any rate defining, his or her interpretation of reality, or even ‘rendering’ or ‘creation’ of it. But how may it be that VR can be connected to this? To make our point, we will look to the phenomenological technology criticism movement that followed Heidegger’s exposition of technology.

In his book series Technics and Time, Bernhard Stiegler argues that technology show us who we are. Stiegler discusses technologies as a way of human externalisation in which we are realising our selves outside of ourselves. Moreover, for Stiegler, this also works the other way around: having externalised our self through technology, we also internalise it yet again, adopting it as parts of our identity. Effectively, therefore, VR can be said to be an expression of our selves in fundamental terms—and this expression we later use as imagery for our selves. 

stighler - The Existential Problem of VR
Bernhard Stiegler.

Now, what does this mean to us? The point which we wish to argue here, is that in being presented with a medium that is, conceptually at any rate, infinite in terms of what it can do, and further who we can become within it, we are faced with an existential problem.

The existential problem that we are faced with, and will be faced with ever more so when the technologies get more sophisticated, can be framed as follows: Now that we can do anything; what will we do? If we consider the point of Stiegler, that technology show us who we are, when we now have the technology to create anything in terms of our experience, VR is essentially asking us who we are, or at any rate, who we want to become. We are being given the question of who we want to become through what we want to do, by the technology allowing this freedom.

IMG 2525 1 1024x1001 - The Existential Problem of VR
Jaron Lanier, depicted in this painting by Darwin Price, has been a giant influence to VR technology. Not just in terms of his contributions to the commercialisation of the technology, but also experimentation and philosophy.

Conclusion: Existentialism and VR

In this piece, we discussed VR and Existentialism, in how VR asks us an existential question. Naturally, to answer this question is as a thought experiment an individual concern. But in terms of our shared reality, it will be a collective future in how it is conducted and externalised. Jaron Lanier in his book The Dawn of the New Everythin,  similarly to Stiegler wrote how VR more than any other technology, will show us who we are. The technology of VR is asking us to choose, not in an either/or situation—not a question of yes and no—but asking us to define everything by will.

The question for us is who we want to be and further what worlds we would like to dwell in; an even more radical freedom than our current situation of free choice given the circumstances. With future technology perhaps, or today through enacting it as a thought experiment, entertaining the notion may be an interesting way to ponder on who we are through what we want to do.


For the interested reader, Matrise has written entries that are similar in theme to this entry, that of VR and Existentialism. For instance:

In The Experience Machine, we discussed how Nozick’s thought experiment of the potential Experience Machine, a thought experiment aimed to test Hedonism, is related to VR.

In The Virtual Freud, we discussed how VR can induce what we can call out of body experiences; by allowing us a view of ourselves from outside, we may allow us to be more compassionate towards our selves. This may be regarded as a partial answer to the thought experiment that this entry has discussed: what would we like to do presented the opportunity?

Similarly, In Virtual Embodiment we discussed how powerful illusions can be facilitated so that users can identify completely with virtual avatars. It can help us overcome prejudices, reduce racism and violence.

In Hinduism and Virtual Reality we paralleled  the broad metaphysics of the East with the potentiality of VR tech. The religious-philosophical system similarly concerns the dilemma of omnipotence: what would you do if you were God?

Hinduism and Virtual Reality

In this piece, we discuss the relationship between Hinduism and Virtual Reality. The cosmology of the religion category “Hinduism”; the broad metaphysic of the East, is a very interesting one. Radically different from that of the West, it is a refreshing albeit heavy shower of new ideas. The best pitch to the worldview at large, I find is best put by “spiritual entertainer” Alan Watts, who put it something like this:

Imagine you are God. Or rather — imagine you could be anything you wanted. Your will is the law of the universe. What would you do if this was the state of affairs? Well, obviously, you would throw a few parties. Really stretch it out, go crazy and mess things up for the laughs of it. The universe is your experience machine; so you do whatever you like.

C164D32B BB6D 4642 9D8B 9EAF260E11DE - Hinduism and Virtual Reality
Digital illustration of Alan Watts by Jean-Francois Painchaud.

So you continue to throw these crazy parties and daring adventures for a couple hundred, or million, of years. Simply testing the limits, doing everything in your mind that can give you pleasure or kick. After a while, however, you find that you have gone out of things to do in «God mode». At least, you want something radically different. A surprise.

So you try and plan to surprise yourself. But as an omnipotent being, this is kind of hard. The curse of being all-knowing and omnipotent is, of course, despite the supreme bliss, that it’s hard to get a true kick out of it anymore. You lack the element of surprise. Surprise, as reaction, needs duality, but you are One. Just as we can’t tickle ourselves, we can’t sneak up on yourself and say «Boo!». There is another option, however; the option of deliberate illusion as to your self. You can create the illusion of splitting — and create a seeming duality within the oneness that constitutes your being.

Through abstraction, you can form the opposite of the distinct quality of your being. From the absolute one, you can conceive of the relative two — by contrast of your endless revealing as God, you conceive of a finite concealment as Man. By hiding your true nature from yourself, its revealing would, in turn, be magnificent; you enter down low to later enjoy your own highness. Though with the potential of the gruesomeness that may result from this fall, you know in the decision, that you will always wake up again to eternal bliss. The ecstasy is inevitable.

EDE053DD 51CA 4A19 953C 51A32F37EC9C - Hinduism and Virtual Reality

Now — at first, you may only dare to go into the depths of time and space for a few hours. The experience is intense: the contrast of transitioning from mortality to godhood was quite ecstatic. Now — your courage grows stronger the more experienced you are, and your adventures go on to wilder and wilder dreams. You go on more and more adventures where you forget who you are, until you find yourself — right here and right now — as a human being reading blogs online.

Thus, according to Hindu cosmology each of us lives in illusion as to what is the core reality of our selves. Life can be seen as a play, and we are still playing — Brahman, the actor that plays all the parts, totally immersed and engaged in them so it forgets its real self, and instead is amusing itself in its ignorance. Reality, then, is a game of hide and seek, where you are both the hider and the seeker, playing for eternity.

The Parallels between Hinduism & VR

So how is Hinduism related to Virtual Reality? The parallels between this ancient creation myth and our dream of ultimate virtual reality may be almost too obvious: it is that of deliberate illusion. Naturally, human beings are not like to gods, but VR as a powerful illusion comes with the power to create and control worlds, to instantiate our thoughts, and actualize our designs. In our recent entry, «From Thought to Reality», we commented and discussed this technological tendency in humans in depth, in how technology in general, but VR in particular, represents «the dream of being able to define reality, to create a representation: the same dream that inspired cave paintings several ten thousands of years ago.» Essentially, VR is a product of the creative element in humans, for good and for worse. The dream of absolute control over matter, but also, the dream of a creative medium without limitations.

3F3EA7E3 D657 4108 A372 A2AB19D0A8A6 - Hinduism and Virtual Reality
The word Avatar, frequently used in VR contexts, has its origin in the Sanskrit word «Avatāra», which means «to descend». It usually refers to when the Hindu gods to take an earthly incarnation.

So, while the Hindu myth had its aim to go from control to chaos and adventure: our dream with VR may be to go from chaos to control. We do not go from One to Dual, and although we may not yet be able to use it to get from Dual to One — it is worthwhile to consider its potential for art and change of our selves.

As Jaron Lanier put it in his book “The Dawn of the New Everything”, VR will, more than any other medium, show us who we are. It will quite simply be interesting to encounter our will and desires as expressed through the worlds we create. We have already begun this investigation and below we mention and interrelate what we have discussed here at Matrise in regards to VR’s potential for art and change of our selves.

Virtual Reality and the Self

The potential of VR for art, expression, and deep impression has been the topic in many of our entries:

In Virtual Embodiment we discussed how powerful illusions can be facilitated so that users can identify completely with virtual avatars. It can help us overcome prejudices, reduce racism and violence.

In The Virtual Freud, we discussed how VR can induce what we can call out of body experiences; by allowing us a view of ourselves from outside, we may allow us to be more compassionate towards our selves.

Last but not least, however, in our last few entries, we have looked to VR as a possible way to escape our enframing, classifying control over nature, as discussed in our three-series entry on Heidegger’s technology critique:

In Existentialist Design, we accommodate Heidegger and Kierkegaard’s concerns and try to imagine, perhaps once again, how we may surprise our selves. The danger of surrounding ourselves in our designs, and classifying the world and its materials as means to our ends, is perhaps that we may not meet anything new — our as put by Heidegger, that it may be denied to Man to enter into a more original revealing.

tumblr p5w6horYa91tqtfrjo1 1280 - Hinduism and Virtual Reality
VR in floatation tanks is one idea meant to exemplify the idea of “Existentialist Design”, designing for a controlled accident in which the outcome is not known. Illustration by Jean-Francois Painchaud.

In the position paper the entry depicts, we imagine the use of VR in sensory deprivation tanks. The design is meant to be a facilitator for a controlled accident in which the medium itself is explored, and one’s self through the medium — where the overarching purpose is the exploration itself. This line of thinking is inspired by existential philosophers such as Kierkegaard and Heidegger, and its aim can be illustrated in how Kierkegaard discusses life not as a ‘problem to be solved, but a reality to be experienced.’ The hope is that the technology may be seen as a lens through which reality can be presented free from an otherwise culturally enframing narrative. The aim is, therefore, to design technology as a catalysator for a more original revealing of truth.

So then, perhaps the Hindu myth isn’t the worst parallel after all. Hinduism and Virtual Reality have striking parallels in conceptual terms, and the Sanskrit word «Maya», may mean illusion, but equally as much Art. And it is Art which may prove the saving power according to Heidegger, as it opens us up to new interpretations — and do not fix us and the world in a prison of our making. Perhaps in the tank, we can once again sneak up on ourselves and say: “Boo”!

Virtual Embodiment

The most praised ability of Virtual Reality is its capability to immerse the user in a Virtual Environment — to the degree that the subject feels present in it. The magic is to be fooled by the system so that one feels present where one actually does not physically reside. This effect can, however, turn even more magical. A deeper step into the effects of technological immersion is found in the concept of Virtual Embodiment. If a subject is embodied virtually,  not only is the virtual environment accepted as such; the subject also identifies with a virtual body or avatar inside the virtual environment. This differs from realizing which character you control in a game — within Virtual Embodiment it is the same processes that make you identify with your real body that makes you identify with a virtual one. This is a key point, as it is why research into virtual embodiment is important.

Screenshot 2018 07 20 at 13.05.27 - Virtual Embodiment
Peeling layers of the onion: VR can be a tool to discover who we are, through investigation of what and how we identify with our bodies. Illustration: “Mask of Day by Day” by Paulo Zerbato.

Hacking and Experimenting with Consciousness

What is fascinating about both of these possibilities of illusion, then — is how, and that, they are possible at all. Knowledge on how to achieve such immersion is obviously relevant for all VR developers, but the knowledge that can be obtained by researching these phenomena goes far beyond knowing how to apply it in VR technology. By creating experiments in VR, we can generate, and investigate, phenomenas of the mind under various experimental conditions. Exploring Virtual Embodiment, for instance, can enable us with a better understanding of our self-consciousness and the relationship between body and mind. Because of this wider span, research on Virtual Embodiment attracts neuroscience researchers, psychologists, information scientists and philosopher’s alike.

The Rubber Hand Illusion

The Rubber Hand Illusion (RHI) is an excellent example of the kind of ‘brain hacks’ that can be achieved by sensory manipulation. The illusion, as illustrated below, is a perfectly simple experiment that does not even require the use of VR technology to perform.  The RHI was introduced by Ehrson, Spence & Passingham (2004) and has been an ingenious way to illustrate how we identify with our bodies. More importantly for this entry, the results of the experiment has inspired further research on Virtual Embodiment.

160330 The Rubber Hand Illusion grande - Virtual Embodiment
Illustration from Thomas Metzinger’s book “The Ego Tunnel: The Science of The Mind and The Myth of the Self”

In the RHI, the hand of the subject is replaced by a rubber hand, while the normal hand is blocked from sight by a separating wall. When the subject is sitting as such, a researcher will stroke each hand, both the rubber and the physical hand, simultaneously. Now, the question is what happens when experiencing the sensory impression of stroking, all the while seeing a corresponding stroke on the rubber hand?

Put very simply, the brain does a ‘reasonable guess’ that this hand is indeed the correct physical hand attached to your body.  You feel that the rubber hand is yours, with nerve-endings and all — and you couple your physical feelings to the vision of the hand. This means that in your subjective experience, the rubber hand is the hand that has the sensation. Ehrson et. al write that their results suggested that “multisensory integration in the premotor cortex provides a mechanism for bodily self-attribution”. When our brains receive sensory information from two differing sensory inputs (sight+feel), these are coupled: the brain is coupling the stroking-sensation with imagery of a nearby-hand being stroked, and this is enough for the brain to attribute its self with the hand, to acknowledge it as its own.

This simple experiment share a lot of principles with the concept of Virtual Embodiment, and has inspired research in the field that we will present in this entry.

img34 1822 - Virtual Embodiment
Some experience out of body experiences (OBEs) on the onset of sleep or waking up. Often they may feel that they are floating over their bodies. VR may help to study such states of consciousness by systematically inducing them.

Virtual Body Illusion

In a later experiment by Lenggenhager et.al (2007), not only the hands of the subjects — but their whole bodies were replaced with virtual representations. Moreover, in the experiment they present, the bodies are seen from behind. In effect, they were simulating out-of-body experiences, with very interesting results.

The experiment was conducted as such: the subjects wore a Head-Mounted Display which projected imagery from a camera located behind the subjects. As such, the subjects could see a representation of their bodies “live”, but from behind. Of course, this is deviating slightly from how we normally experience life. Although the subjects saw their body responding and performing actions in real time as under normal conditions — there is a logical dissonance due to the mismatch between the location of the subjects’ eyes in the virtual environment, and what these eyes see. Effectively, the user is seeing inside a pair of “portal” binoculars (HMD), which display the light from, if not another dimension, then at least a few feet away. And this will be a part of the point.

What is interesting about this experiment is not necessarily simply that the users feel present where they do not reside physically, but how the distance is only a few feet off. The users feel present right outside of their bodies. The situation is similar, the body and the environment is there, but everything is a bit off. What is interesting to investigate then, is how the body adapts to this. Will it accept that it now controls its body from a third person perspective, similarly to how Stratton’s subjects got used to seeing the world upside down?

What they studied was basically whether this change of perspective had an impact on where the users felt embodied. To investigate this, the researchers stroked the subjects as they did in the Rubber Hand Illusion, except at their backs — so that it was perceivable by them. The question is then where this physical feeling will be attributed to — how will the phenomena of the subjective experience present themselves to the subjects?

obe - Virtual Embodiment
Out of Body experiences can be achieved virtually by using sensory impressions from other locations, for instance five meters behind you as in the experiment by Ehrson (2007). You can then effectively look at yourself from the outside.

First of all, to be clear on this — the sensory data of being stroked will initially be provided by the nerves in the physical shoulder of the user. The problem of the brain, however, is that the shoulder is out of sight — blocked by the Head-Mounted Display. There is, however, the visual impression of a shoulder on a person standing in front — being scratched in exactly the same way. Although the nerve-endings definitely feel the stroking, the problem is that where this feeling will be placed in our subjective experience is not the responsibility of the shoulder, but rather the brain. And, as the placement of the physical feeling in the bodily self-consciousness is largely dependent on vision for coordinates, what will happen? How will the brain fix this sensory discord?

In this beautifully written article by The New Yorker, its author Rothman describes one of the co-authors of the research paper, Thomas Metzinger’s, own experience undergoing the experimental conditions:

Metzinger could feel the stroking, but the body to which it was happening seemed to be situated in front of him. He felt a strange sensation, as though he were drifting in space, or being stretched between the two bodies. He wanted to jump entirely into the body before him, but couldn’t. He seemed marooned outside of himself. It wasn’t quite an out-of-body experience, but it was proof that, using computer technology, the self-model could easily be manipulated. A new area of research had been created: virtual embodiment.”

new yorker - Virtual Embodiment
Are We Already Living in Virtual Reality?” — The New Yorker has a brilliant, long, read on Virtual Embodiment that features interviews with VR and Consciousness researchers Prof. Mel Slater and Prof. Thomas Metzinger.

Phantom Pain

Another curious potential effect of Virtual Embodiment, is the possibility of phantom sensory impressions as well. Handling virtual objects while being embodied, for instance, may convince your body to expect pain or touch — and so this is, somehow, actively generated. Because of this, VR may be a way to study how phantom pain is created, and further how it can be alleviated. For instance, several studies show how VR can embody a subject missing a leg in a body with two legs, similarly to traditional mirror therapy treatment, which is effective in reducing phantom pain. Again — what may be most interesting here is the possibility of systematically creating the phenomena and studying it afterwards. For instance, as Metzinger is quoted on in The New Yorker’s article, it may be supposed that phantom pain is created by a body model not corresponding to the physical reality. This will be the case for phantom pain in VR: it is not based on the physical reality, you are only relating to a virtual reality instead. Similarly, those those with real phantom pain may also be relating to a certain kind of “virtual reality”, but rather one in the format of their skewed narratives — maintained by their minds instead of a computer.

That the narrative, worldview and consciousness that our brain’s experience and generate is often not the best match with reality is not something new. As for Matrise, these concepts reminds us of the conclusion from our three-series entry towards a metaphysical standpoint on VR, in which we discussed VR as rather examplifying of our abstracting tendencies of mind. These entries can be read at Matrise, and were called: 1) On Mediums of Abstraction and Transparency, 2) Heidegger’s Virtual Reality, and 3) The Mind as Medium.

Virtual Embodiment for Social Good

Now that we have discussed the concept of Virtual Embodiment, it may be natural to discuss what this knowledge can be used for. As discussed already, generating experiments in VR that hacks our self models, may provide useful knowledge on the structure of our self-consciousness. Apart from this general knowledge, some may also have practical utilisation in applied VR for specific scenarios.

Racial Bias

A very exciting paper that describes work utilizing virtual embodiment, is one by Banakou, Hanumanthu and Slater. In the project, they embodied White people in Black bodies, and found that this significantly reduced their implicit racial bias! The article can be found and read in its entirety here (abstract available for all).

Domestic Violence

Another interesting project by Seinfeld et. al, is one in which male offenders of domestic violence became embodied in the role of a female victim in a virtual scenario. At first in the experiment, the male subject is familiarized with his new, female, virtual body and the new virtual environment. When the body ownership illusion, or virtual embodiment, has been achieved, a virtual male enters the room and becomes verbally abusive. All this time, the subject can see his own female body reflected in a mirror, with all his actions corresponding to his. After a while, the virtual male starts to physically throw around things and start to appear violent. Eventually it escalates and he gets closer into what feels like the subjects personal space, and appear threatening.

They write:

Our results revealed that offenders have a significantly lower ability to recognize fear in female faces compared to controls, with a bias towards classifying fearful faces as happy. After being embodied in a female victim, offenders improved their ability to recognize fearful female faces and reduced their bias towards recognizing fearful faces as happy”

The article can be read in its entirety at ResearchGate.

Staying Updated in the field of Virtual Embodiment

Research on Virtual Embodiment is happening continuously. To stay updated on this area of VR research, I enjoy following Mel Slater, Mavi Sanches-Vives and Thomas Metzinger on Twitter. Last but not least, I would stay updated on Virtual Bodyworks at Twitter, of which both Sanchez-Vives and Slater are co-founders of.


N.B: This entry lies at the centre of Matrise’s interests, and we are planning on writing several entries on this topic further in philosophical directions. Have any ideas or want to contribute? Please contact us.

Literature list

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSaveSaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

The Web of VR Will Change Everything

The World Wide Web (WWW) need no further introduction. The greatest innovation of the Information Age is now essential to the world like no other technology. Before the WWW – computers, programs and information were not linked. The computers were lonely, and users could not browse the millions of interconnected computers the way we do today.

The Web has been changing ever since its dawn in the 90s, and has seen its distinct phases. What we call “Web 1.0”, for instance, was a static web. Websites could be visited and navigated, but they were static in the sense of not affording any user interaction. Web 2.0 opened up for more dynamic web applications that could be altered by user input. These did not just allow download, they could also be uploaded to – a feature that is now an essential underpinning of social media and web-based applications. Companies like Facebook, Snapchat, Instagram, etc., do not provide content to be downloaded, but rather a computer service to be used where the users provide the content.

spider web - The Web of VR Will Change Everything
Illustration by Lancelot Speed from Andrew Lang’s “The Red Fairy Book”.

It should be noted that “Web 1.0” and “Web 2.0” are just terms: there are constantly being added changes to the Web. The terms does, however, signify when these changes are inducers of a paradigm shift in the use of the Web. What is to be classified as Web 3.0 is therefore also discussed. Although not necessarily a feature of the Web, the Internet of Things (IoT) is a candidate for what has become a paradigm shift within web technologies, as more and more devices and artifacts are connected, allowing for ubiquitous computing. Others discuss the personalised Web we see today, as it is influenced by social media, while some are joining the AI hype, and claim that the Web has now become smart. The latter is far from a paradigm shift as of yet.

The Virtual Web
In this entry we will discuss WebVR, or Virtual Reality through, and on, the Web. The title of this entry, which claim that “The Web of VR Will Change Everything”, may indicate a stand towards the debate that I have introduced, of what will be the “Web 3.0” –  but that is not the point of this entry. I do not wish to make a claim of WebVR as a paradigm shift of the way the Web operates, but it most significantly will be a paradigm shift in how we experience the web technology as it is.

The concept and role of the Web, nevertheless, is the same: we have a dynamic web which features download and upload of web documents. In the case of VR the difference is that what we download and upload, are perceived as realities for us: the web is the mediator of realities, and this new way of using the web changes VR more than VR necessarily changes the Web. What characterizes the Web, is its simplicity, its openness and the innate element of surprise. Anything can be found, and the exploration as such is an important part of it. These features are the same that will be valuable in VR as well: to discover open virtual worlds, created by anyone.

Screenshot 2018 06 06 at 13.27.51 - The Web of VR Will Change Everything
Mozilla’s A-Frame is now ready for Link Traversal through hyperportals

Creating VR for the Web
WebVR, a framework for browsers to support Three-Dimensional Stereoscopic Virtual Environments, is already developed and supported by many browsers. As HTML, CSS and JavaScript already have the powers to create and render graphics through frameworks such as WebGL, the web languages have increasing power to support such scenarios. Lately, frameworks combining these different frameworks to make implementation of WebVR even easier. Mozilla’s A-Frame, lets the user set up a Virtual Environment only with less than 20 lines of code (see their Hello WebVR example), using ‘normal’ HTML tags, which they call primitives, to create 3D objects in 3D space. A-Frame utilizes Three.JS to do this, and Three.JS uses the WebGL.

It is now easy to create Virtual Environments on the Web, even arguably easier than creating them through Unity. The great benefit of this is that they can be connected to each other, by a standard hypertext reference, instead of uploading to Steam or Oculus Store, etc. A-Frame introduced hypertext support, which they call “Link Traversal”, in July of 2017, but the browsers are only just catching up. As of now, it is only supported by Firefox and Supermedium on PC, however, as of February 2018 Oculus Browser has supported it on GEAR VR, and most likely also on Oculus Go.

A-Frame’s Diego Marcos called this a great achievement, as A-Frame finally achieved their ‘Web badge’. For this they deserve congratulations, A-Frame has now completed an essential step towards the Web of Realities. In their introductory blogpost, they introduce a “hyperportal” example, which provides you with a preview of the VR world you are about to enter, and which redirects you to the page when you virtually walk through it. This is a piece of very fun code to play around with. A neat feature is that the portal itself is “transparent”, and so provides a preview of the virtual environment to which you are travelling.

The future of WebVR
As with anything within VR, we are still a few years behind its potential. WebVR has had a solid boost the latest few years, but before a Head-Mounted Display is commonplace, we probably wont find a VR search engine or enough websites for exploration to be truly amazing. This is not bad news, however, it means that this is just the right time for creative ideas. We see the inevitable emergence of the VR Web, and can help shape it. For instance, at Matrise, we have previously discussed Virtual Reality Memory Palaces. This would be great to incorporate for sharing on the Web, so each memory palace could be interconnected, creating vast banks of knowledge for memorization.

Do you have any good ideas for any WebVR apps?
Feel free to comment below.

old canal - The Web of VR Will Change Everything
The ancient Greeks created “Memory Palaces” to recall important information.