Sensory Deprivation — Floating in Virtual Reality

If we look to our glossary, we see Presence within Virtual Reality (VR) defined as the degree to which the subject feels present in the virtual world. What is interesting to note, is that this naturally has to be viewed relatively to the degree that the subject feels present in the physical world — as we usually receive information from both our physical and our virtual environments.

There can thus be two separated approaches to designing for presence in virtual reality environments: one is to provide sensory stimulus of the virtual environment, and the other is to block sensory stimulus from the physical environment. Both approaches work towards the same goal of immersion — the encapsulation of the user in the VE. Slater and Wilbur (1997) recognise this in their definition of Immersion, which is closely related to the notion of Presence. They define immersion in terms of four qualities the system can afford, the first one of which is called inclusiveness. Inclusiveness they define as the extent to which physical reality is shut out.

Obviously, the principle of adding and removing sensory experience go hand in hand; by equipping a Head-Mounted Display you are blocking the physical impressions and replacing them with virtual impressions, all the while shielding for incoming light from the surroundings. Blocking light, however, is not the only way to deprive the senses of information from the physical environment. In this entry, we will discuss how we can maximize the inclusiveness of the immersion by achieving sensory deprivation in floatation tanks. Floating in Virtual Reality!

Floatation Chambers

Floatation chambers, or sensory deprivation tanks — are pools of water with copious amounts of epsom salt (≈600kg). The tanks are sealed for any incoming light and sound, and the air- and water temperature is equal to that of your body. When you lie down, you will feel how the salt makes you float even though the pool is very shallow. As you lie there, you notice how the ripples you created when lying down start to slowly subside as you sink down into weightlessness. After a while, because of the air- and water temperatures are the same as that of your body, you can no longer pinpoint where the water ends and the air around you begins. In fact, it gets hard to distinguish anything from anything else, including your body from the air and water. There is really nothing that is easy to grasp as isolated, save perhaps your breath. And as the minutes go, with total physical relaxation and lack of much sensory impression at all, things start to change.

“Alone With Your Thoughts”, Illustration by Cole Ott

The most significant, explicit change one may notice in the tank  is that after a while your bodily self-consciousness is not what it used to be. Your mental model of where your body is in relation to the world around you starts to become blurred. Normally reinforced by tactile stimuli of air and water (of varying temperatures), and visual and auditive stimuli from the environment, your body model is now lacking information on which to create it. Your sense of spaciousness has also changed, that is the feeling of your position as defined relatively to say, the walls, mountains and sky has disappeared. You now really experience nothing around you, but neither any edges to this lack of information in your surroundings. You may get the feeling of floating in empty space — but where are you in all of this? What, in this stream of conscious experience is matter and what is mind?

Inner vs Outer

In our entry — ‘Inner as Outer: Projecting Mental States as External Reality‘ — we discussed the potential of using VR for meditation purposes in experimental ways. In the introduction to the entry, we discussed our feeling of Self as a duality of Inner and Outer, of which our everyday experiences usually comprise. We discussed how technology may have the power to transform our consciousness away from this traditional subject-object hierarchy and into a non-dual one, where the Inner is seen as the Outer, and the Outer as Inner. In this entry we are building further on these ideas. Similarly to visualising inner states in VR through biometrics, using VR in floatation tanks might provide illusory experiences where the conscious experience is significantly altered.

One other entry relevant to our experiments with VR in floatation tanks should be mentioned before we go on: the entry on Virtual Embodiment. In the entry, we discuss the great potential of VR to hack our consciousness; why it is possible, and what it can be used for. The research is highly relevant for floatation in VR, as both floatation tanks and VR alter our self-model, as both alter the sensory impressions necessary to maintain it.

Research on Virtual Embodiment in Floatation Tanks

Matrise partnered up with Bergen Flyt, a local company offering floatation therapy in the heart of Bergen city. We used a Samsung Gear VR with a Samsung S8 phone. We did not use a HTC Vive (Pro) as it would be more risky exposing the cable to water. Also, no room tracking or even much head orientation was needed, and in terms of resolution the HMD is quite high in ppi. We chose to first try out some abstract visualisations through the application “Fractal Lounge”, that shows varying psychedelic visuals and floating through space.

My Experience

“After I had showered, I put on the GEAR VR headset, started the application, and slowly entered the floatation pool. I held my hands towards the wall, as I did not see anything else than the visuals in the headset. When I was inside, I closed the glass door, and slowly lowered myself into the water — back first. It took a few seconds before I dared to lower my head all the way down, but very soon I was totally relaxed. As expected, the electronics in the display was kept well above water, due to the intense amount of salt in the water …”

The kind of visualisations provided from Fractal Lounge, the application that was tried in the floatation tank.

“The visualisation pulsated, floated, drifted along — and often totally changed in colours and shapes. It took probably about ten minutes before my feeling of body totally vanished, to the degree that it was a larger gap between wanting to move the body and actually being able to move it than usual. I felt like perceiving a great drama and scene, and I got engaged in the forms and ways of the visualisations, sometimes quite invested in it, as it felt close and reality-defining for me. After about twenty minutes in, I felt as if I was drifting along in space at high speed, because of the steady movement of stars away from me. At the same time, there was no sound, which made the quick travel feel peaceful and smooth. As with normal floating, about every ten minutes there is a sort of reality-check moment where you remember you are in the tank and contemplate how weird it is. This also happened in VR, and was … equally as weird”

Reflections and Future Work

My first experiment with floatation in VR lasted for about 45 minutes. Sometimes, unfortunately, the VR headset glided slightly off my face, and I had to reposition it with my wet, salty fingers. After this happened about three times, I had to leave the tank in order to save the equipment.

Thank God that we have floatation pools instead of this creepy stuff.

My first experience of floating in virtual reality was very promising. The largest surprise was the feeling of movement through space at high speed. The largest frustration was the lack of any sort of interaction with ones surroundings at all, except the possibility to open and close one’s eyes. A great experiment would be to use eye tracking technology as a way of navigation in the vast, abstract psychedelic spaces. If one travelled towards where one saw, one could even be interactive while lying still in the floatation tank. This could also possibly have curious effects on which parts (perhaps the eyes), we identify with our selves. Perhaps the placement of our self could be altered by changing the agency for transportation.

Matrise will continue the cooperation with Bergen Flyt, and both try and develop different applications. Our plan is to measure the feeling of presence and self-identification and consciousness while in the tank.

 

Literature list

Virtual Embodiment

The most praised ability of Virtual Reality is its capability to immerse the user in a Virtual Environment — to the degree that the subject feels present in it. The magic is to be fooled by the system so that one feels present where one actually does not physically reside. This effect can, however, turn even more magical. A deeper step into the effects of technological immersion is found in the concept of Virtual Embodiment. If a subject is embodied virtually,  not only is the virtual environment accepted as such; the subject also identifies with a virtual body or avatar inside the virtual environment. This differs from realizing which character you control in a game — within Virtual Embodiment it is the same processes that make you identify with your real body that makes you identify with a virtual one. This is a key point, as it is why research into virtual embodiment is important.

Peeling layers of the onion: VR can be a tool to discover who we are, through investigation of what and how we identify with our bodies. Illustration: “Mask of Day by Day” by Paulo Zerbato.

Hacking and Experimenting with Consciousness

What is fascinating about both of these possibilities of illusion, then — is how, and that, they are possible at all. Knowledge on how to achieve such immersion is obviously relevant for all VR developers, but the knowledge that can be obtained by researching these phenomena goes far beyond knowing how to apply it in VR technology. By creating experiments in VR, we can generate, and investigate, phenomenas of the mind under various experimental conditions. Exploring Virtual Embodiment, for instance, can enable us with a better understanding of our self-consciousness and the relationship between body and mind. Because of this wider span, research on Virtual Embodiment attracts neuroscience researchers, psychologists, information scientists and philosopher’s alike.

The Rubber Hand Illusion

The Rubber Hand Illusion (RHI) is an excellent example of the kind of ‘brain hacks’ that can be achieved by sensory manipulation. The illusion, as illustrated below, is a perfectly simple experiment that does not even require the use of VR technology to perform.  The RHI was introduced by Ehrson, Spence & Passingham (2004) and has been an ingenious way to illustrate how we identify with our bodies. More importantly for this entry, the results of the experiment has inspired further research on Virtual Embodiment.

Illustration from Thomas Metzinger’s book “The Ego Tunnel: The Science of The Mind and The Myth of the Self”

In the RHI, the hand of the subject is replaced by a rubber hand, while the normal hand is blocked from sight by a separating wall. When the subject is sitting as such, a researcher will stroke each hand, both the rubber and the physical hand, simultaneously. Now, the question is what happens when experiencing the sensory impression of stroking, all the while seeing a corresponding stroke on the rubber hand?

Put very simply, the brain does a ‘reasonable guess’ that this hand is indeed the correct physical hand attached to your body.  You feel that the rubber hand is yours, with nerve-endings and all — and you couple your physical feelings to the vision of the hand. This means that in your subjective experience, the rubber hand is the hand that has the sensation. Ehrson et. al write that their results suggested that “multisensory integration in the premotor cortex provides a mechanism for bodily self-attribution”. When our brains receive sensory information from two differing sensory inputs (sight+feel), these are coupled: the brain is coupling the stroking-sensation with imagery of a nearby-hand being stroked, and this is enough for the brain to attribute its self with the hand, to acknowledge it as its own.

This simple experiment share a lot of principles with the concept of Virtual Embodiment, and has inspired research in the field that we will present in this entry.

Some experience out of body experiences (OBEs) on the onset of sleep or waking up. Often they may feel that they are floating over their bodies. VR may help to study such states of consciousness by systematically inducing them.

Virtual Body Illusion

In a later experiment by Lenggenhager et.al (2007), not only the hands of the subjects — but their whole bodies were replaced with virtual representations. Moreover, in the experiment they present, the bodies are seen from behind. In effect, they were simulating out-of-body experiences, with very interesting results.

The experiment was conducted as such: the subjects wore a Head-Mounted Display which projected imagery from a camera located behind the subjects. As such, the subjects could see a representation of their bodies “live”, but from behind. Of course, this is deviating slightly from how we normally experience life. Although the subjects saw their body responding and performing actions in real time as under normal conditions — there is a logical dissonance due to the mismatch between the location of the subjects’ eyes in the virtual environment, and what these eyes see. Effectively, the user is seeing inside a pair of “portal” binoculars (HMD), which display the light from, if not another dimension, then at least a few feet away. And this will be a part of the point.

What is interesting about this experiment is not necessarily simply that the users feel present where they do not reside physically, but how the distance is only a few feet off. The users feel present right outside of their bodies. The situation is similar, the body and the environment is there, but everything is a bit off. What is interesting to investigate then, is how the body adapts to this. Will it accept that it now controls its body from a third person perspective, similarly to how Stratton’s subjects got used to seeing the world upside down?

What they studied was basically whether this change of perspective had an impact on where the users felt embodied. To investigate this, the researchers stroked the subjects as they did in the Rubber Hand Illusion, except at their backs — so that it was perceivable by them. The question is then where this physical feeling will be attributed to — how will the phenomena of the subjective experience present themselves to the subjects?

Out of Body experiences can be achieved virtually by using sensory impressions from other locations, for instance five meters behind you as in the experiment by Ehrson (2007). You can then effectively look at yourself from the outside.

First of all, to be clear on this — the sensory data of being stroked will initially be provided by the nerves in the physical shoulder of the user. The problem of the brain, however, is that the shoulder is out of sight — blocked by the Head-Mounted Display. There is, however, the visual impression of a shoulder on a person standing in front — being scratched in exactly the same way. Although the nerve-endings definitely feel the stroking, the problem is that where this feeling will be placed in our subjective experience is not the responsibility of the shoulder, but rather the brain. And, as the placement of the physical feeling in the bodily self-consciousness is largely dependent on vision for coordinates, what will happen? How will the brain fix this sensory discord?

In this beautifully written article by The New Yorker, its author Rothman describes one of the co-authors of the research paper, Thomas Metzinger’s, own experience undergoing the experimental conditions:

Metzinger could feel the stroking, but the body to which it was happening seemed to be situated in front of him. He felt a strange sensation, as though he were drifting in space, or being stretched between the two bodies. He wanted to jump entirely into the body before him, but couldn’t. He seemed marooned outside of himself. It wasn’t quite an out-of-body experience, but it was proof that, using computer technology, the self-model could easily be manipulated. A new area of research had been created: virtual embodiment.”

Are We Already Living in Virtual Reality?” — The New Yorker has a brilliant, long, read on Virtual Embodiment that features interviews with VR and Consciousness researchers Prof. Mel Slater and Prof. Thomas Metzinger.

Phantom Pain

Another curious potential effect of Virtual Embodiment, is the possibility of phantom sensory impressions as well. Handling virtual objects while being embodied, for instance, may convince your body to expect pain or touch — and so this is, somehow, actively generated. Because of this, VR may be a way to study how phantom pain is created, and further how it can be alleviated. For instance, several studies show how VR can embody a subject missing a leg in a body with two legs, similarly to traditional mirror therapy treatment, which is effective in reducing phantom pain. Again — what may be most interesting here is the possibility of systematically creating the phenomena and studying it afterwards. For instance, as Metzinger is quoted on in The New Yorker’s article, it may be supposed that phantom pain is created by a body model not corresponding to the physical reality. This will be the case for phantom pain in VR: it is not based on the physical reality, you are only relating to a virtual reality instead. Similarly, those those with real phantom pain may also be relating to a certain kind of “virtual reality”, but rather one in the format of their skewed narratives — maintained by their minds instead of a computer.

That the narrative, worldview and consciousness that our brain’s experience and generate is often not the best match with reality is not something new. As for Matrise, these concepts reminds us of the conclusion from our three-series entry towards a metaphysical standpoint on VR, in which we discussed VR as rather examplifying of our abstracting tendencies of mind. These entries can be read at Matrise, and were called: 1) On Mediums of Abstraction and Transparency, 2) Heidegger’s Virtual Reality, and 3) The Mind as Medium.

Virtual Embodiment for Social Good

Now that we have discussed the concept of Virtual Embodiment, it may be natural to discuss what this knowledge can be used for. As discussed already, generating experiments in VR that hacks our self models, may provide useful knowledge on the structure of our self-consciousness. Apart from this general knowledge, some may also have practical utilisation in applied VR for specific scenarios.

Racial Bias

A very exciting paper that describes work utilizing virtual embodiment, is one by Banakou, Hanumanthu and Slater. In the project, they embodied White people in Black bodies, and found that this significantly reduced their implicit racial bias! The article can be found and read in its entirety here (abstract available for all).

Domestic Violence

Another interesting project by Seinfeld et. al, is one in which male offenders of domestic violence became embodied in the role of a female victim in a virtual scenario. At first in the experiment, the male subject is familiarized with his new, female, virtual body and the new virtual environment. When the body ownership illusion, or virtual embodiment, has been achieved, a virtual male enters the room and becomes verbally abusive. All this time, the subject can see his own female body reflected in a mirror, with all his actions corresponding to his. After a while, the virtual male starts to physically throw around things and start to appear violent. Eventually it escalates and he gets closer into what feels like the subjects personal space, and appear threatening.

They write:

Our results revealed that offenders have a significantly lower ability to recognize fear in female faces compared to controls, with a bias towards classifying fearful faces as happy. After being embodied in a female victim, offenders improved their ability to recognize fearful female faces and reduced their bias towards recognizing fearful faces as happy”

The article can be read in its entirety at ResearchGate.

Staying Updated in the field of Virtual Embodiment

Research on Virtual Embodiment is happening continuously. To stay updated on this area of VR research, I enjoy following Mel Slater, Mavi Sanches-Vives and Thomas Metzinger on Twitter. Last but not least, I would stay updated on Virtual Bodyworks at Twitter, of which both Sanchez-Vives and Slater are co-founders of.


N.B: This entry lies at the centre of Matrise’s interests, and we are planning on writing several entries on this topic further in philosophical directions. Have any ideas or want to contribute? Please contact us.

Literature list

 

 

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSaveSaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

Inner as Outer: Projecting Mental States as External Reality.

Introduction to Mysticism

Within Mysticism, the merging of Self and World — Inner and Outer — is seen as the utmost aim. Mysticism can be found within most of the world religions, such as Buddhism, Christianity, Hinduism and Islam — and its aim is often formulated as union with God. Depending on the religion, however, the degree to which Mysticism is the common way of practicing the religion varies. Although many religions have such contemplative practices, they are not always adopted by the religion’s followers at large.

When discussing «Union with God», it should be noted that the term «God» varies in its meaning between these religions. The contemplative practices often have significantly varying metaphysics, for instance Monotheistic (Christianity), Polytheistic (Hinduism), and relatively Atheistic or Agnostic (Buddhism). Be this as it may, their descriptions of the experience of this merging of Self and God is often strikingly similar. These states of enlightenment are often described as ecstatic, in which the conscious experience can not be placed within our normal frames of language or understanding.

What also unites the different traditions, is that such states of consciousness is usually  worked towards through contemplative practice such as yoga, meditation or other disciplines of focus or conscious attention. Other techniques for achieving these ecstasies have have been ascetic ones, such as fasting, waking, isolation — or other ways of stirring the Self to war.

The experience of seeing the Inner as Outer, and the Outer as Inner, is often described as the feeling that living itself is an experience of seeing and perceiving Oneself and/or God. Within this worldview, there is no Self relating to anything external.

Non-duality: synchronization of Inner & Outer

The concept of merging Inner and Outer, or Self and God, can each be viewed either in very material or spiritual terms. Although materiality and spirituality do not have to differ metaphysically, separating these gives us some communicative benefits — and Mysticism may be explained and spoken of from both these perspectives. Discussing the Inner as Outer purely «scientifically», if you will, makes sense in that all our perceptions of the Outer world is indeed created Inner, and as such — Reality will always be a synergy of Inner and Outer. We know that we do not see, or have ever seen, anything which we ourselves do not actively generate. As neuroscientist and consciousness researcher Anil Seth put it, “our brains are actively hallunicating our conscious reality”.

States where a subject experiences the Inner and Outer as ‘one’, is often referred to as «non-dual».  Often while speaking of Inner and Outer, we tend to implicitly reinforce the Self and the World as a duality (when pitching a solution we often have to pitch the problem first). By using the word «non-dual» instead of ‘one’, we may pinpoint the nuance that it is not a duality in separation, but neither completely “same-same”. Although it is non-dual, neither is it all same or flat — least of all static!

Although we classify and divide our reality, fundamentally what we perceive is a stream of experience, which in every sense is simply “reality” before divided, and, again, actively created by us. This is not to say that there are no external reality or world — but it definitely is to say that all which is external is perceived first and foremost, solely, internally. Experientially — externality has never been perceived, except as a subcomponent of internality.

A vase, or two faces? Each defines the other, and neither exist without the other.

Experiencing and Sensing the Non-Dual

This causal explanation, however, leaves out the experiential aspects of the non-duality. Although it may make sense on paper, it matters little to us as we absolutely perceive the world as dual — as subjects relating to a World. Within Philosophy, this traditional way of adhering to and speaking of the world,  is referred to as the subject-object dichotomy. Although, between different cultures and continents, the degree to which we adhere to this way of thinking vary in its intensity, it is nevertheless definitely an essential part of the human experience which we share.

How the material explanation can be said to be different from the spiritual in this sense, is that the spiritual concern is to experience the Inner as Outer, not to understand it cognitively. As such, and towards that, meditation practices such as Mindfulness and Yoga have existed, to increase wellbeing by increasing the degree to which one feels in union with God, or for those who do not fancy the term; to the degree which one has peace with oneself and the world.

Contemplative practices such as yoga and meditation, has the last fifty years become more popular in western societies. Although they have been subject to a certain degree of metaphysical raffination the last years, these methods are nevertheless largely old and traditional. The most common of these contemplative practices we see today is adopted from the Vippassana practice, commonly known as Mindfulness. These methods are now commonly used in psychological treatment of anxiety and depression, and research has the latest years started to uncover the benefits of learning to be able to sit quietly with your mind and, well, deal with shit, or seeing it for what it is.

In the next section, we will discuss an approach utilizing Virtual Reality to aid in Mindfulness meditation — which can help to perceive the Inner as Outer.

A common belief is that the aim of contemplative practices is to empty the mind. In a sense, it can be said to be correct, in that meditation practices often seek to eradicate, dilute or cancel the self-referential narratives.

The effects of Mindfulness meditation

The essence of Mindfulness or similar contemplative practices, lie in their manipulation of identity. We stated “the problem” of Mysticism as the gap between Self and Other — and for this separation to be there, we must necessarily have a relatively thoroughly defined sense of self. For most of us, this tend to be limited to the cognitive processes that constitute our mental narrative (the personalized voice in our heads, our formulated will, and how it appears to direct our actions and plans our lives). It is actually to a far lesser extent our bodies, although this also attributes to our self-consciousness.

Mindfulness is about being present attentively in each moment to one’s state of mind. When doing such focus excercises directed at the mind, and observing these mental processes closely, the idea or view of them as solid things starts to unravel. When rather seeing them as thoughts from a distance, they appear untangled to us, and we perceive our own existence as distinct from those thoughts.

Virtual Reality Biofeedback as Meditation aid

One of the great benefits of VR is its ability to project and represent data in the format of the reality encompassing us. Within the context of this entry, we could say therefore that VR can simulate what we perceive as the Outer. The question may then be asked: how can we project our Inner in to this medium of Outer?

Although I believe we will see more work on VR biofeedback within this domain in the future, in this entry we will focus only on one research paper in particular to examplify our case. At last years CHI conference, the world leading conference on Human Computer Interaction, Joan Sol Roo and his colleagues presented their work on Inner Garden: a mixed reality sandbox for mindfulness. The artifact is a physical sandbox, which the user can shape to a given terrain. The sandbox is given generally visually augmented by a projector with colors and shapes — and physical changes to the sandbox will also alter the output of the projector, which deliver terrain information such as sea levels and green growth.

The sandbox is just not physical, however; by placing a physical avatar in the physical sandbox, you can enter into the land you created in Immersive VR. A 3D-model of the land you created physically can be seen virtually, from the viewpoint of your placed avatar.

The Sandbox, which heights of the sand have been turned into an island by the projector.

Attached, to measure your inner states, is both breathing- and heart rate sensors — which are coupled to provide visual and auditive feedback. In this way, you can synchronize your breath to control the environment and rythm and breaking of the waves. The Inner Garden represents your inner state, and. by practicising breathing techniques, the flora of your world will get greener and more animals will appear.

In this way, Inner Garden works as a great example of representing Inner phenomena as External Reality. Very conceptually interesting, and hopefully one day we will also see empirical studies on similar artifacts.

You can read more about Inner Garden, which received an honorable mention at CHI’17, here.


What do you think? Do you have any ideas for VR applications using biofeedback?  Please comment below.

Literature list

 

 

 

 

 

SaveSave

SaveSave

 

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave