Posted on

The History of Virtual Reality

In recent years, Virtual Reality (VR) technology has finally reached the masses. 2016 was called “The Year of VR” as several actors released their Head-Mounted Displays (HMDs) on the consumer market. While HTC, Oculus and Playstation delivered high quality HMDs that require external computers to run, the year also opened up for high quality mobile VR. Both Google with their Daydream View and Samsung/Oculus with their GEAR VR have provided an easier step for consumers to enter the world of VR. These mobile VR solutions  offer better internal measurement units than the simpler Cardboard devices, and also feature simple controllers for interaction. We now see the market spreading out both  in quality and accessibility: in 2018 we have both seen the coming of the HTC Vive Pro, a more expensive high-end HMD with increased resolution, and the Oculus Go, which is a reasonably-priced ($200) stand-alone 3DOF (3 Degrees of Freedom) HMD for the starters.

It is natural to wonder how all of this started. Why did we for instance not see much VR before 2016? When it now seems to be relatively easy for commercial actors to push out HMDs down to $200, why did it not happen sooner? Of course, we have had Oculus’ development kits since 2013 — but even this is very recent.  When Google released their Cardboard (a simple HMD made out of cardboard  and some lenses), it seemed incredilous that VR could be attainable for the smartphone for only 50 cents. This, however, only points us toward how fascinatingly simple the underlying pinciples of VR technology actually are.

In this entry, we will trace the VR tech we see today back to its roots. We will go back about two hundred years, and work ourselves jumpingly forward to the very recent innovative technologies.

Stereoscope

A drawing of the Lothian Stereoscope, released in 1895; one of many different models.

In 1838, Sir Charles Wheatstone developed what would be the first Stereoscope. Even before the camera was invented, people were seeing (drawn) images with 3D effect through stereoscopes.  Stereoscopy, that is, perceptory illusion of depth, is achieved by displaying a slightly different segment of an image to each eye. Wheatstone achieved this by separating the two images by a piece of wood, and providing a lens directing the light, between each eye and the corresponding image. While looking through the stereoscope, our brains perceive the two images as one image, with the added 3D effect due to the varying segments of the images. This effect is simply caused by an utilization of how our eyes and brain work, by combining the sensory data from each eye into one. We may, for instance, most likely be able to recall  sometimes «seeing double», when our brains have yet not our varying visual impressions.

Since Wheatstone, different stereoscopes have been produced all the way up to the Google Cardboard or other HMDs; which instead of drawn images, or later photographs,  utilizes a screen to deliver the imagery to the eyes. Actually, in the early 1900s, Stereoscopes functioned as home entertainment devices, and «stereo cards» such as the image seen below could be purchased from photography companies.

Stereo card of St. Peters Church in the Vatican. Such cards, picturing tourist attractions all over the world, could be purchased and viewed at home in a Stereoscope.

Stereoscopes and modern day Virtual Reality HMDs share the essential feature of stereoscopic depth illusion (3D). Apart from that, however,  a lot has obviously happened since 1838, which we now regard as essential for the feeling of presence and realism, and which makes the technology capable of simulating realities. The most important of these have been moving images, 3D environments, interaction, and 360 degrees of orientation. With the stereoscope, images very static in every sense.

Sensorama

In the mid 1950s, however, some people saw the opportunity to spice up their stereoscopes a bit. A bold attempt at enrichening this, was the Sensorama. In addition to providing a stereoscope with motion pictures in 3D and color,  all quite revelutionary, the device had fans for simulating wind, odor transmitters for smell of the environment, stereo sound, and even a moving chair!

The Sensorama, or «Experience Theather». Illustration from Morton Heilig’s 1962 US Patent.

Pygmalion’s Spectacles

The idea of the Sensorama, or VR in general, can as many other innovative future-defining ideas, be found in science fiction literature. Before its conception, in the 1930s, the science fiction writer Stanley G. Weinbaum introduced the idea of «Pygmalion’s Spectacles». By wearing these, the user could experience a fictional, or virtual world, with holographs, smell, taste and touch, and make the virtual come alive. Pygmalion, which «Pygmalions Spectacles» were named after, were a Greek sculptor who fell in love with his sculpture, and so begged Venus that it would come alive. The Myth sheds an interesting light on VR as an ultimate dream of humanity, to create realities for ourselves to inhibit, or to create images in the format of reality.

Pygmalion, which «Pygmalions Spectacles» were named after, were a Greek sculptor who fell in love with his sculpture. He begged Venus that it would come alive. Painting by Jean-Baptiste Regnault.

Information Technology

To take a leap towards another paradigm shift in VR tech, we must enter the land of 1s and 0s.  The Stereoscope slowly moved from drawn images, to photographs, and further to moving images with the Sensorama. None of these, however, supported spherical environments that could be perceived in all their 360˚. To achieve this,  certain sensors and further computation based on their sensory input has been necessary.  The most important and interesting of these sensors, has been the Gyroscope.

The Focault Gyroscope, created by physicist Jean Bernard León Focault.

The Gyroscope was given its name by Phycisist Jean Bernard León Focault in 1852 who used the device as a means to prove the rotation of the Earth.  The gyroscope is a device consisting of a spinning top with a pair of gimbals. Its origin can not be traced to a single invention or inventor, as tops have originated in many ancient civilizations — however, unlike the «complete» Gyroscope, these were not necessarily used as instruments.  Although Focault’s gyro were not the first that were used as a measuring instrument, its affordances work well to examplify the usefulness of gyroscopes in VR HMDs; the important feature it affords is the measure of rotation, which key lies in the Gyroscope’s tops’ possibility for free rotation.

Gyroscopes are fun artifacts to play with as they seem to defy gravity. While spinning, they can remain stable in most positions. If placed on a platform, that unlike the gyro remain stable, the position in terms of rotation can be measured relatively to the platform, and as such we can also measure the rotation of a HMD. It should be noted, however, that the gyroscopes of today are not pretty mechanical objects of brass anymore, which, although they do no longer satisfy our aesthetic appetite, at least have the benefit of fitting into our smartphones and HMDs. Today, gyroscopes have heights, widths and lengths of only millimeters, which opens the possibility for placing them inside smartphones and HMDs.

The Sword of Damocles

The Sword of Damocles, an old greek cultural symbol of Mortality — ever close to those in power. We see the sword hanging from a single horse hair over the head of Damocles.

Fifty years ago, in 1968, Ivan Sutherland and his student Bob Sproull created the first computer-driven stereoscopic (3D) Head-Mounted Graphical Display with 360˚ head-tracking. The HMD was not exactly lightweight, and was named after the «Sword of Damocles» because of the heavy stand hovering over its users head. As can be seen in the illustration below, the head-tracking was mechanical, and did not in fact use a Gyroscope. Later, however, this became a more fruitful approach, so as to avoid the massive device rotating over the users head.

The field of view and graphical fidelity of the Sword of Damocles were obviously quite low, yet the Sword of Damocles is the first widely known HMD, and has since its dawn inspired and launched further decades of VR research.

The first Virtual Reality Head-Mounted Display, named after the Sword of Damocles, because of its great weight hanging over the user’s head.

Towards the modern HMD

Since the invention by Sutherland and Sproull,  creation and use of HMDs was seen more and more within research. As computational power became faster and cheaper, the HMDs decreased in size, and increased in field of view, graphical fidelity and refresh rates. Yet — even back in the 1990s for instance, the technology was still expensive, and poor in terms of graphical realism. It often caused cybersickness due to low refresh rates, and high motion to photon latency. Of this reason, as with any really powerful computer from that time, VR was reserved for research universities that could invest into the technologies, or businesses with resources to experiment with the technology. There were some attempts at commercializing VR for gaming purposes, such as the SEGA Genesis and Nintendo Virtual Boy — however, both of these remained largely as prototypes and were later discontinued. To this day, none of these companies has since experimented with the technology, although Nintendo in 2010  released the Nintendo 3DS which utilizes a stereoscopic display that does not require any glasses.

 

Image of a 3D model of the HTC Vive Pro.

Conclusion

Since the Sword of Damocles, VR technology has undergone small incremental changes leading to where we are today, mainly as a result of general computer and graphics research, and the natural progression of Moore’s Law; today our processors are smaller and more powerful, and our screens of higher resolution.

In addition to this, however, there are certain very recent technologies that have impacted the VR as we know it today as well. In Matrise’s glossary, we briefly present and define some of these technologies. Some that can be read about is Foveated Rendering and  Low Pixel Persistence Modes.

Did we miss anything? Any thoughts are welcome in the comments section.

SaveSave

SaveSave

Posted on

Virtual Reality and Exposure Therapy

The most essential feature of VR is its ability to simulate what is not real. This is its core concept, and what causes its radical exclusivity and novelty. The benefits of ‘avoiding reality’ in this sense, is most often that virtuality is more cost-effective than reality. For instance, corporations worldwide train their employees in VR as it saves money to avoid renting a physical location and hiring physical trainers. ‘Money’ in this case, is of course just a measure of effectivity: it takes less resources to achieve certain objectives virtually than physically. The cost is not the only benefit, however; the virtual may also be safer. We see this especially within surgery, where a failed operation on a virtual patient is much preferred than on a real one.

“Scream”, by the expressionist painter Edvard Munch. Want to experience Munch in VR?  Read our entry on Art in Virtual Reality.

Exposure Therapy
Another scenario where virtuality may be preffered  is psychological treatment of anxiety disorders. Anxiety is a terrible disorder in the way it is eating away the lives of the sufferers, and is hard to treat to by non-addictive pharmaceutical medicine. Psychological treatment, however, is in general very successful towards certain anxiety disorders.  Agoraphobia, arachnophobia, glossophobia, etc., can be treated by what we call “exposure therapy”.

Under exposure therapy, the patient usually get together with a psychologist, and is asked to express their fears of the situation of exposure. Here they answer what they think will happen, and how they think they will react. Their fears are pinpointed, and their catastrophic thinking is outlined. In these cases, it is not uncommon that patients believe they will literally stop breathing, or die, etc.: the narrative which operates is something they buy heavily in to, and the key of exposure therapy is to challenge their acceptance of this narrative. To a certain extent, this is a central problem of anxiety disorders: patients very seldom challenge these fears, of obvious reasons, and so their map of how the world works is not challenged and updated by reality. This is, through exposure therapy, systematized.

When the patient has been exposed to their fear scenario — the psychologist confront the patient with their initial fears that were written down prior to the exposure. The patient is then encouraged to reflect on the gap between their fears and what actually happened, something which we refer to as inhibitory learning. This kind of treatment falls under what is depicted as Cognitive Behavioral Therapy (CBT); by actively challenging the patients’ mental model of the world by reflection on facts.

Long story short — exposure therapy works. The largest problem with exposure therapy is, as usual, the cost. Having highly educated psychologists dedicated to the task is expensive enough in itself — but arranging the exposure to a fear scenario is an often greater challenge, practically and economically. It is not really convenient to summon spiders into the psychologist’s office, for instance. Arranging complicated fear scenarios and executing them is not convenient, and at high cost, which is a hinder for an otherwise effective treatment.

Virtual Reality Exposure Therapy can be used to treat, for instance, arachnophobia (fear of spiders). If you are not afraid of spiders, you might need to increase their size. I can recommend trying Farpoint for PSVR, which features giant space spiders similar to those in this illustration by Alphonse de Neuville.

Virtual Reality Exposure Therapy
This is where the concept of VR enters our story, as we start talking of Virtual Reality Exposure Therapy (VREP). By using virtual environments instead of actual physical locations, effective exposure therapy can be offered to more people at lower cost. At the University of Bergen, through the research project INTROMAT, we develop and do research on VR Exposure Therapy for adolescents with fear of public speaking. The INTROMAT project aims to introduce personalized treatment of mental health problems using adaptive technology.

The question that often raises itself when we discuss the concept of VRET, is whether we can fear what we know is not real. Although we know what it is like to be nervous before talks, it is perhaps hard to imagine being afraid of speaking in front of virtual subjects in which ‘nobody’s home’. On this point, however, the research is very clear. As Lindner et. al (2017) writes, “decades of research and more than 20 randomized controlled trials show that [VRET] is effective in reducing fear and anxiety”. The reason why VRET is interesting now, today, is then not necessarily because VR is finally good enough to deliver realistic virtual scenarios. VRET has been shown to be effective with VR technologies far inferior to those setups we have commercially available today.  The reason for its relevance as a research subject now, is because the technologies are finally cheap enough to successfully be used in large scale treatment. It is therefore time to revisit the previous research, and look at how this treatment can be improved further.

Participatory Design of VR Scenarios for Exposure Therapy
We are presenting our paper on VR Exposure Therapy at the ACM CHI Conference on Human Factors in Computing Systems — the premier international conference of Human-Computer Interaction — in May 2019 (Flobak et. al.) The paper is available at ResearchGate, and has in my unbiased opinion two very interesting contributions: 1) it documents prototyping of VR with 360 cameras, which enables less technically skilled persons to produce VR (for exposure therapy, and for other areas), and 2) through the fact that it enables a larger group to produce VR, it simultaneously enables adolescents to communicate their social scenarios through the medium of VR. As we have previously discussed in The Capture of Reality, it easier to avoid the uncanny valley effect and simultaneously maintain high standards of realism when using image capture equipment, than it is to create it with 3D animation. Thus, this approach to creating VR Exposure Therapy has both the benefit of being extremely cost-effective, whilst still being developed based on good sources for the social scenario it should depict.

You can read the paper at ResearchGate. Below is the conclusion of the paper:

Conclusion
“This paper has presented a participatory approach to prototyping virtual reality scenarios for exposure therapy for fear of public speaking. In the study, we demonstrate how adolescents can be involved in the design of VR scenarios enabled by 360° videos. We also show how the participants draw on their lived experiences when creating the scenarios. The paper also illustrates how 360° video is a viable tool for making the design of immersive experiences more accessible, as the method involves far less technically demanding skills than you need for constructing CGI-based environments, and is less time- consuming. Further, the expert evaluation highlighted the authenticity and realism of the scenarios, and the scenarios were seen as having a potential for use in a therapeutic context. In addition, we have discussed how this approach can be used to make tailored VR experiences for exposure therapy”

Literature list

SaveSave

Posted on

The Mind as Medium

N.B: This post is the third and final post in a series that comments upon a metaphysical stand towards VR technology. The entries are based upon an essay that was written for a doctoral course on the philosophy and ethics of the social sciences. The two previous post that precedes this one, are linked here:  1) “On Mediums of Abstraction and Transparency“, and  2) “Heidegger’s Virtual Reality“

3 / 3

Now that we have arrived at the final post of the series, it is time to revisit our initial problem of Virtual Reality and Authenticity. Initially, we introduced our problem as the abstracting tendencies of Information Technology, and the unique position of VR technology in this case, as it abstracts all the while displaying a high degree of transparency and coherency with the real world – while simultaneously hiding the real world as much as possible. We wanted to ask whether this was indeed a real problem, and if the technology could in effect distance ourselves from the reality of the world, and thus distance ourself from truth, or an authentic living.

An old illustration of a Stereoscope, the illusive technology responsible for 3D effects in our modern VR Head-Mounted Displays. The first stereoscope was created in 1838 by Sir Charles Wheatstone.

Discussion

After reviewing Heidegger’s essay “The Question Concerning Technology”, we noticed several questions that we could use in our existential approach to our technology critique. Following the lines of Heidegger, we could say that we could have a free relationship towards VR when we know what it is. We can for instance ask whether we can see any “Enframing” tendencies of VR technologies.

In Being and Time, Heidegger’s concern for authenticity is a concern for individuality: a concern for Dasein’s possible impossibility of leading its own life: the death of authenticity. In the they-self, no individual is thoroughly relating to its Being, and so the truths and the “goings-on” that is defined “culturally” in the they-self, are in a sense accepted blindly and left unexplored: they are abstractions as they are not defined relatively to each Dasein; they are not experiential, not resolutely made up. Effectively, we are not in control when we have given ourselves up to let the they-self decide the possibilities of what we can project upon.

“The Fairies Flew Away”, by Charles Henry Bennett.

Similarly, In Heidegger’s Questioning Concerning Technology, we introduced the term of Enframing. Enframing is dangerous because we create things that enframe us, we instantiate our enframing orientation in technology. If we do not relate to technology as exactly this which it is in its essence, Enframing, we do not relate to it as it is, and so it may hinder us to perceive the world as it is. Both by relating to the agenda set by the they-self, or the framework set by ourselves indirectly through our enframing technology, we do not relate to the world and our active projection upon it: we do not resolutely enact our nature of actualizing the possibilities that can lead us to our authentic self.

So how is this relevant for the technology of VR? Should we interpret it as that we are not in control over our possibilities, if they are presented to us through VR, rather than in real life? Can , in this respect, VR be seen as an instantiation of the they-self, as it similarly provides abstractions, terms and conventions? If we remember to follow Heidegger deeper than the image of the problem, we may see that it is not VR that we should be afraid of. VR, like other modern technology, carries the mark of its author: and similarly we can see the creation of VR as the ultimate dream of Man. We spoke earlier of the characters of challenging-forth as inspired by the view of modern physics as an exact science: we wanted to view the world as chopped up in parts and materials that we could understand, enframe, and use as means to ends. If the dream accompanying this Newtonian metaphysic ever was lost with the rise of quantum physics, VR can certainly become the free space where man’s illusive control over the world could be rekindled: finally we have a world, not of atoms, but of bits, that we can know totally through an actual access to its source code.

We can have a free relationship towards VR, when we understand what the essence of VR is. To what is the essence of VR, we will not answer “enframing”, but rather abstraction, and more specifically, abstraction in the mode of transparency. The tendencies of abstraction, was, as with Heidegger’s technology, perhaps not inherently something technological, but something human: only technology made it obvious and explicit enough for us to see clearly. VR may perhaps be an interesting way for us to look at the real problem figuratively; the technology stands in between (medium) you and your senses, in the same way that our mental terms and classifications obscure the otherwise non-reducable reality.

To create VR is a human activity, and in VR we find much of ourselves: similarly to technology in general, we see that VR is a means to an end, and that it is an instantiation of this purpose. Sometimes, this purpose is “re-presenting” an abstraction of reality, and so it is not the genuine authentic reality itself that we see. We can therefore say that it is a human activity to create representations as means to an end, and even further, we can say that it is human to abstract, it is human to deal in images, it is human to connote terms and concepts, similarly to “putting things to boxes” in the enframing attitude of mind. With VR, these boxes are presented to us as reality, or at least in the format of reality, and the result is a realm of abstraction, a realm of representation, that blocks naturally-occurring presentation. Similarly to how Heidegger’s technology illustrated our enframing tendencies, VR may show us our desire to create our own bubbles of reality to inhibit, our own terming and associations and concepts of the self. In this respect, the essence of VR may not even be anything new. In this sense, we have created “mental virtual realities” for a long time, and the technological, the “material” expression of this does not provide anything new.

Conclusion

If we can understand the essence of VR as abstraction in the mode of transparency, and, similarly to Heidegger’s technological essence, believe that this essence is inherited from our own tendencies of mind, we will view it as such that it is human to create transparent abstractions. Through our terms, conventions, and definitions, we abstract, and through relating to these abstractions, we perceive them as real. Our thoughts and defined concepts, and the conventions we adhere to, work as our transparent user interface’s which we use to navigate the world. Our initial fear was that IT, examplified in its extreme case of VR, would act as a wall between us and the world; inhibiting a true, authentic relationship towards it. This is, however, not fundamentally something that we find in especially in a certain technology – instead, we find that when we look at this technology we are instead looking at ourselves. The mediums and interfaces that classifies and simplifies is inspired by our minds that classify and simplify. Technology is not the separation between us and the world, at least not any less than the enframing and abstracting orientation of our minds to the world are: the one mirror the other. We have thus reached a new question to replace the first, one step further on the hermeunetical spiral, and that is whether our own abstracting tendencies of mind keep us from authenticity.

Literature list

Posted on

Heidegger’s Virtual Reality

N.B: This post is the second out of three, in a series that comments upon a metaphysical stand towards VR technology. The entries are based upon an essay that was written for a doctoral course on the philosophy and ethics of the social sciences. The first post,  preceding this one, is “On Mediums of Abstraction and Transparency“, while the third and subsequent to this post, is “The Mind as Medium“.

2 / 3


In the previous post, we argued why existentialism was fit to explore our problem of VR authenticity. Although the post quoted several existential philosophers, we will further only go in depth into the work by Martin Heidegger (1889 – 1976). Heidegger was deeply concerned with authenticity, and also discussed the role of technology in our relation to the world. Although he never lived to see the true emergence of IT, Heidegger’s 1954 essay, “The Question Concerning Technology”, discusses the essence of technology in general, and our relationship towards it.

Martin Heidegger, Portrait, Oil on Canvas.

Questioning concerning Technology

In his essay, Heidegger does not just analyse technology itself – as part of this, he writes about how the technology implicitly alters the metaphysics of our world; how technology may change ourselves and our relation to the world. In the essay, which title is “The Question Concerning Technology”, Heidegger discuss the essence of technology as it appears in our relationship towards it. The reason for his questioning concerning technology, is that he believes we can have a free relationship towards technology when we know what it is, that is, its essence. Throughout his essay, Heidegger draws a distinction between traditional technology, which is ‘bringing-forth’, and modern technology on the other hand, in which the technology is ‘challenging-forth’ in a more brutal and hacky way. It is this latter technology that Heidegger is critical towards and wishes to discuss, more “traditional” technology, such as for instance a bridge, is not an example of a technology that is challenging-forth. Heidegger for instance writes of windmills, that, though they draw energy from the wind, do not extract the energy for storage as we do with coal: the wind can still “do it’s thing”, being as-it-is, unlike coal, which for us no longer exist as-it-is; its only given meaning is being a means to our end. Through rigorous analysis, that is where Heidegger ends up: through such challenging-forth technology, our relationship towards nature is changed, as nature is interpreted only as a means towards our end, the world is no longer interpreted as it is, as it manifests.

To fully understand how Heidegger comes to this point, however, we must back up a bit. We will start “in media res” with a quote by Heidegger that is relatively concluding as to the main point of the essay, and then further navigate backwards to introduce the terms necessary to understand what he means.

Heidegger writes:

The rule of enframing threatens man with the possibility that it could be denied to him to enter into a more original revealing and hence to experience the call of a more primal truth.


We see then, that Heidegger’s main concern is that man may be separated from truth, a more “primal truth”, or a more “original revealing” of such. He does not say bluntly that technology threatens man with this possibility, but rather speaks of “the rule of enframing”, which will be an important term for us to understand Heidegger’s proper meaning. Throughout his essay, Heidegger builds his philosophical argument with his self-composed terms from the ground up before getting to the point, and so to understand “the rule of enframing”, we must therefore go back to thoroughly define Heidegger’s concepts of bringing-forth and challenging-forth. Both bringing-forth and challenging-forth are modes of revealing, although they differ in how and what they reveal. Heidegger’s “bringing-forth”comes from the Greek “poiesis”, which means to bring something out from concealment to unconcealment. Heidegger speaks of this revealing rather poetically, and asks the readers to imagine what is being brought forth as already “being on its way” to existence, from unconcealment to concealment; and so we can say that poeisis brings that which was not present into presence. This “revealing” or “unveiling” that happens with poeisis, Heidegger describes by the Greek word “aletheia”, literally meaning “revealing”, but which also is the Greek word for “truth”. In this way, according to Heidegger, technology can reveal truth through poeisis, although, as Heidegger points out, this is exactly what modern technology fails to do in its challenging-forth.

Rotary Boilers, by Broux, P, from Les merveilles de l’industrie, vol. 2.

Bringing-forth vs. Challenging-forth

According to Heidegger, we find the difference between bringing-forth and challenging-forth in that challenging-forth treats “modern physics as an exact science”. This can be interpreted as such: modern science was on its arrival a dream come true for the human impulses to classify and understand the world: now we could understand the workings of the world according to our own scheme, and therefore also “own” them in the sense that they were perceived to be within our control. On the discovery of the atom, for instance, we finally had a graven image of the source code of reality, and we were closing in on a framework of the whole world (this was, of course, before quantum physics shattered our image of reality as something that could easily be pinned down). It is this “attitude”, according to Heidegger, which separates bringing-forth from challenging-forth: the implicit metaphysic we accepted along with the principles of Newtonian physics, that the world could be viewed as a clockwork of cause and effect, where the world is made up of manipulable materials that could serve as means to our ends.

Heidegger notes, however, that challenging-forth also reveals, but that it does not reveal truth in the same way that bringing-forth does. The point is to be found in that “the essence of [modern] technology, is not something technological”. By this, Heidegger means that technology bear a resemblance to its Maker; there is something in the essence of technology that it has inherited by us. Early in the essay, Heidegger notes that technology can be thought of both as a “means to an end”, and a “human activity”. We can therefore also say that it is a human activity to think in the context of means to ends, and it is this which technology represents to us, as in a mirror: our attitude towards the world, where everything is viewed and classified in terms of potential means to an end. Through the vision based on modern physics as an exact science, human beings view the natural world as materials for their endavours. It is a human tendency to look at the world and ask: “what is it good for?”; in the words of Albert Camus, “the world evades us when it becomes itself again”; we do not really relate to the world, but to our interpretation of it, which is an interpretation of utility or means to ends. Technology is in this respect mirroring ourselves as an expression and example of how we interpret things in light of our narrative, where things in the world fit in the degree to which they can be means to our ends. This is what Heidegger considers when he writes that the essence of technology is not anything technological, rather, its essence lies in how we approach and orient ourselves towards technology, and through technology, to the world.

With this, we are approaching the point of Heidegger’s criticism of technology: if we only interpret the world as potential means to an end of ours, we don’t really see the world as it is, or as it reveals itself. Heidegger uses the example of technology to pinpoint that it is the human aspect of technology that is dangerous about it: because we as humans are used to interpreting the world to concepts, terms and classifications, these now stand in the way as a medium or interface between us and the world. This is Heidegger’s definition of the essence of modern technology, and it is this that we shall mean by the term Enframing that we set out to define. Enframing comes from the German “Gestell”, and has associations to that of order, system or framework. It should be noted, however, that Heidegger uses it as an active verb, and so instead we talk of “ordering”, or “gathering together”, or an “enframing” of the world. Enframing is in this case a mode of human existence, it is how we navigate and present the world to ourselves, relative to ourselves.

Illustrator: León Benett. From Jules Verne’s “Les cinq cents millions de La Bégum”

We set out to answer what the challenging-forth reveals, as both bringing-forth and challenging-forth are modes of revealing. It is by its enframing that challenging-forth reveals, and the enframing categorizes human and machine alike, and what it reveals, it reveals as standing-reserve. The concept of standing-reserve is the utmost point of criticism for Heidegger, and is what will be discussed in the next section.

 

The Danger


Heidegger’s critique then, is that the attitude of enframing is dangerous. Technology as examined here, have worked to reveal this dangerous tendency in humans. The humans themselves are not safe from their orientation of enframing: the enframing of the world also include the enframing of human beings, and humans are also in this system reduced to “human resources”; the degree to which they are beneficiary resources in this system of their own creation. Enframing is the consequence of what happens when this attitude of ours is enforced and instantiated through technology; “standing-reserve” is a role that the world and its inhabitants has been given in its enframing.

This category or classification of “standing reserve” is illustratively speaking exactly of this: technology, with its essence, does not, and can not, view anything in the world “as it is”, it is only judging in terms of utility and means to ends; it is seen not as good, but good for. This is being in the standing-reserve: an airplane has no meaning or value out of itself – it is purpose incarnate, and its value is therefore only in relation to human beings and a certain activity, and so it’s role is otherwise as standing reserve. For Heidegger, however, it does not end there, as humanity is also caught up in this system of technology: the system of the world is increasingly becoming a system in which technology is heavily incorporated. Within sociology, this kind of society with non-human agents as equal affectors, has seen new theories reflecting these relationships, for instance in Bruno Latour’s Actor-Network Theory. The theory describes one system, where technology and humans are both treated as actors in the society. Similarly, according to Heidegger, humans are equally likely to be put into standing reserve to fit this system as any other: he describes the forester as put in standing- reserve by the paper industry: his role and worth is defined there only in so much as he produces for them; when he does not, his being is one of standing-reserve, not as-he-is.

The Solution

At this part of the point, we should revisit Heidegger’s initial point of inquiry: that we inquire into technology in order to establish a free relationship to it. Heidegger is not just concerned with the existence of the technology, he is also definitely concerned with our orientation towards it, our standpoint towards it, and so our relationship with it. We initially set out to establish a free relationship towards technology, and now we see that it in fact is our relationship with the world, and the technology’s relationship to the world as well as to humans and other agents that is the issue at hand. This is a critical point, because it also means that this is not a problem that can be solved even in the extreme case choosing to discontinue technology as a whole; the essence we want to avoid comes from us. On this, Heidegger writes:


We shall never experience our relationship to the essence of technology so long as we merely conceive and push forward the technological, put up with it, or evade it. Everywhere we remain unfree and chained to technology, whether we passionately affirm or deny it.


We are limited then, in that we can not change technology in its essence. Heidegger does not want to create “better” technology: he wants to see technology for what it is, to prepare a relationship to it – and it is here that we find Heidegger’s curious path of “technological determinism”; a view that can be defined as the lack of control by humans over technology, that the development of technology is somewhat deterministic, instead of perhaps more commonly viewed as an expression of our needs through tools that we create (i.e. social constructivism). Therefore Heidegger does not want to change technology, but change our relationship towards it. The approach is almost stoic in its basic philosophy: it is only concerned with one’s own relationship to the situation, the situation itself is at any rate given. And again, the orientation which we normally have to both normal technology and to the natural world, is one of enframing. It is not just in technology, it is a human mode of being and connecting to the natural world in itself: enframing is the human tendency of categorizing the world, into objects, events, or other terms or conventions. We “frame” things in the sense that we “box” them, and this Heidegger considers to be a human activity in general, not a technological one in particular.

Heidegger does not offer a plan for humanity to solve this, but rather encourages us to stray from the enframing attitude of mind, to the more poetic: in poeisis, we can see the world as it reveals itself, not just as raw materials to be used by us. By adopting the vision of the artist, we can take part in the world instead of analysing and categorising it from the outside, indirectly alienating us from the world. Although not very specific in the approach, Heidegger is still clear on the role of action towards the problem, as he writes that “humanity is needed and used for the safekeeping of the essence of truth”.

Literature list

Want to read more? This essay is continued and finalized in “The Mind as Medium“. In the final entry we revisit VR and Heidegger’s relevance for VR technologies. The entry that you just read was a continuation of the entry On Mediums of Abstraction and Transparency.

SaveSave

Posted on

On Mediums of Abstraction and Transparency

N.B: This blog post is the first of three posts in a series that comments upon a metaphysical stand towards VR technology. The entries are based upon an essay that was written for a doctoral course on the philosophy and ethics of the social sciences, and was presented in a panel at Global Fusion 2018 at the University of Virginia. The posts following these as the second and third, are “Heidegger’s Virtual Reality” and “The Mind as Medium“.

1 / 3

Technology, in all its facets, has increasingly changed our way of life. It changes how we approach life and tasks in that we distribute agency and commands to artifacts and computers. Coffee is made with a button, transport is performed by sitting. Our toil with the world is outsourced to technology, and so we are indebted to our dishwashers and robo-vacuums. What this essay will discuss, however, is whether it matters that technology in many ways is the mediator of our reality. The question is, broadly, if we lose something when we ourselves avoid getting our hands dirty by touching the unpractical reality ourselves. Can it be that technology is responsible for distancing ourselves from a more authentic way of living? Or more concretely – would that matter, and if so, why?

Technology (the Brunton Boring Machine)

We now communicate with each other, and the world, through interfaces. For technology to be relevant for us, it naturally has to embrace and cooperate with the world. Therefore, when using technology, we are not separated from the content of the world: the technology is simply representing the world to us through different mediums. To “present” originally comes from the latin word “praesent”, meaning “being at hand”, and to re-present, then, is to present something as “being-at-hand”, where it perhaps did not naturally originate. To represent something, we need a medium; we need matter by which to re-create an ideal form or pattern, and so the data which express this pattern is presented through a medium of either e.g. text or images. We here see the concrete meaning of “medium”, which literally means “in the middle”; an appropriate term for the re-presenting technology, which is standing between ourselves and the world.

Technology is within the world, and as such a part of it, and so it can not represent the whole world without compressing it and leaving out details. Therefore, when representing the world, with each medium, there follows an abstraction. In the way the information is simplified and represented through abstractions, different mediums vary in their degree of transparency, or in other words, their conformity with the real world. Throughout this essay, we will keep this separation of abstraction and transparency, and we will define transparency within this essay as the degree to which the abstraction corresponds with reality; the more it corresponds, the more transparent the abstraction is. A photograph is far more transparent than a drawing, for instance, but both are abstract representations of the reality they represent. Examples of abstractions can be, as mentioned, when the complex coffee brewing process is reduced to, and represented by, a button with an icon; a minimalistic silhouette of a steaming cup of Joe. Similarly, in our communications, our emotions are also reductionally abstracted to emoticons, and we see our personal identity and self as abstracted and reduced to social media profiles, for easy export over the Web.


Physical vs. Virtual

Hiroshi Ishii criticizes the abstracting tendencies within Information Technology (IT) in his paper “Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms” from 1997. Critically discussing the abstraction of information in to the standard Graphical User Interfaces, Ishii proposes a new vision for the research field of Human Computer Interaction. By introducing the concept of “Tangible Interaction”, Ishii were to take back the physical, and in that, acknowledging our human history of some million years dealing of dealing with physical objects and artifacts. This, then, would provide us with a more “authentic” or “traditional” way of interacting with IT. According to his vision, bits and atoms would then live side by side in a glorious harmony. In this, we also see that Ishii’s motivation is touching on the question which our essay will discuss: there is a sort of gap between space and cyberspace. Where Ishii postulates his problem as that there is a gap between the virtual and the real, we shall rather investigate whether it can be that cyberspace is the gap between ourselves and the actual.

We will use the example by Ishii to illustrate our point a bit further. Ishii wanted to bridge the virtual and the physical, so that we could interact with computers in the same way that we interact with physical artifacts by employing tangible interaction. Rather ironically, however, the most realistic way of realizing Ishii’s vision of coupling bits with atoms, may be through the medium of VR. When immersed in a virtual environment through a head-mounted display, the whole world is made of bits: they work as the atoms comprising the world. In VR, our reality is mediated by a medium that can theoretically contain an abstract representation of the world in digital form, all the while the abstraction is presented to us in the same way that traditional reality is, avoiding the immediate feeling of the medium as separating. In this case, what we traditionally view as physical, then, is represented by bits, and the atoms are brought into the virtual, instead of the virtual being brought into the physical.

Ishii’s vision confronts the unnatural abstraction of information in that we view IT through various interfaces. VR escapes this degree of abstraction, and what separates VR as a mediator of our world from other digital representations such as social media, is exactly in its own relation between abstraction and transparency; it is at its point on the abstraction-transparency continuum that VR is unique. Placed on the utter end of the abstraction-transparency continuum, VR has the capability to be extremely transparent in its abstract simulation of the world. Often we turn to and equip the technology of VR because of its transparency: we do not feel that we interact with an abstraction of the world, but with a genuine representation of it. This does not mean that we are out of the abstraction altogether, however; a virtual world most certainly is a reduction in its representation – the interesting thing about VR thus is that it is an abstraction that presents itself as transparent, and at the same time is the first truly immersive, encompassing technology which hides you from the real world. We leave the world in absence, and are in return given an illusion of presence in an abstraction that claims its rightful transparency. To use the word “illusion” in this case, is not an overstatement: by facilitating for stereoscopic (three-dimensional) vision into the virtual reality, the brain is tricked to perceive the environment as an environment of depth, which combined with a 360° degree presentation of the environment, often leads to a feeling of presence to it. It’s illusory tendencies are caused by the fact that we see the virtual as we normally see the physical.

Problem

We have now articulated the problem issued by the unique technological medium of Virtual Reality. Abstracting in disguise, the medium is unique in its the relation of its approach to abstraction and transparency. In and out of itself, however, this does not mean anything. It is not as if we loose our access to oxygen in VR: the detachment is not physical, it is only our minds that are immersed and feel present. The question this essay will discuss is not whether it is the case that technology distances us from the world, as this technology per definition does; but whether this matters. Does it matter if sexual urges are satisfied through “VR Porn” in a 360° stereoscopic (3D) video, filmed from a first-person view? Does it matter that elderly with dementia are given VR experiences, because this is cheaper than arranging a visit outdoors? Put short: does it matter that our worldly needs are simulated rather than actualized? Does the concrete reality have a value in itself, or is its mirroring mediums capable of transferring the essentials? Does authenticity; whether what we adhere to is true, real, or genuine, matter?

To tackle this question, this essay will not feature empirical studies on the wellbeing of humans with virtual sexpartners, nor will we perform studies on the use of VR in nursing homes. We will discuss the concept of authenticity philosophically, and implicitly close in upon commenting on a metaphysic of technology in general, and VR in particular. To this purpose, we will discuss VR in terms of the philosophical tradition of inquiry called “Existentialism”, a field of philosophy that is especially concerned with the notion of authenticity or authentic living. In the coming section, we will define existentialism and present how it can help us to shed light on the human concern of authenticity in relation to technology. When we have defined the term, we will move on to present the two works of philosophy that we will use to comment on our problem, namely Heidegger’s essay: “The Question Concerning Technology”, and his book “Being and Time”.

Existentialism

In Existentialism, the human is viewed as the starting point of philosophical reasoning in general. Therefore, it also takes the human experience and concern for authentic living seriously. For instance, in discussing the concept of Being, Heidegger is quick to identify the process of inquiry into Being, as first and foremost a human activity, as humans are the only beings for whom Being is an issue. Of this reason, the human problem of suffering also comes quickly as an object of inspection, including the receding (or preceding) sense of absurdity. We are now approaching the concept of existential angst, which is terribly related to the sense of responsibility over ones own life. In respect to this, Søren Kierkegaard, often called the Father of Existentialism, spoke of anxiety (Angst), as “the dizziness of freedom”: and the concern for authenticity is also the concern over choices. The concern for authenticity, and in some cases as individuality, naturally only matter in the degree to which humans are free to affect their position towards it. It is in this case that Jean-Paul Sartre speak of “radical freedom”, like Kierkegaard he also emphasizes the responsibility of each individual for the choices that he or she makes. Existentialism may be useful to examine our relationship to technology, because of its genuine and felt concern for authenticity. In the sense that the field of inquiry is humanly oriented, existential philosophy at times comes close to the practical, as what it discusses at its core, is each person’s concern at its core. In this sense, to be an existentialist can necessarily not be just an outward doing: the problem it discusses has to be understood personally and experientally, because it is only where it exists. Because of existentialism’s genuine concern for authenticity, it may be able to shed light on how we should relate to technologies such as VR, and how this may impact our relationship between self and world, and this potential relationship’s corresponding state of authenticity or inauthenticity.

Want to read more? This essay is continued in “Heidegger’s Virtual Reality“.

Literature list

SaveSave

SaveSave

Posted on

The Web of VR Will Change Everything

The World Wide Web (WWW) need no further introduction. The greatest innovation of the Information Age is now essential to the world like no other technology. Before the WWW – computers, programs and information were not linked. The computers were lonely, and users could not browse the millions of interconnected computers the way we do today.

The Web has been changing ever since its dawn in the 90s, and has seen its distinct phases. What we call “Web 1.0”, for instance, was a static web. Websites could be visited and navigated, but they were static in the sense of not affording any user interaction. Web 2.0 opened up for more dynamic web applications that could be altered by user input. These did not just allow download, they could also be uploaded to – a feature that is now an essential underpinning of social media and web-based applications. Companies like Facebook, Snapchat, Instagram, etc., do not provide content to be downloaded, but rather a computer service to be used where the users provide the content.

Illustration by Lancelot Speed from Andrew Lang’s “The Red Fairy Book”.

It should be noted that “Web 1.0” and “Web 2.0” are just terms: there are constantly being added changes to the Web. The terms does, however, signify when these changes are inducers of a paradigm shift in the use of the Web. What is to be classified as Web 3.0 is therefore also discussed. Although not necessarily a feature of the Web, the Internet of Things (IoT) is a candidate for what has become a paradigm shift within web technologies, as more and more devices and artifacts are connected, allowing for ubiquitous computing. Others discuss the personalised Web we see today, as it is influenced by social media, while some are joining the AI hype, and claim that the Web has now become smart. The latter is far from a paradigm shift as of yet.

The Virtual Web
In this entry we will discuss WebVR, or Virtual Reality through, and on, the Web. The title of this entry, which claim that “The Web of VR Will Change Everything”, may indicate a stand towards the debate that I have introduced, of what will be the “Web 3.0” –  but that is not the point of this entry. I do not wish to make a claim of WebVR as a paradigm shift of the way the Web operates, but it most significantly will be a paradigm shift in how we experience the web technology as it is.

The concept and role of the Web, nevertheless, is the same: we have a dynamic web which features download and upload of web documents. In the case of VR the difference is that what we download and upload, are perceived as realities for us: the web is the mediator of realities, and this new way of using the web changes VR more than VR necessarily changes the Web. What characterizes the Web, is its simplicity, its openness and the innate element of surprise. Anything can be found, and the exploration as such is an important part of it. These features are the same that will be valuable in VR as well: to discover open virtual worlds, created by anyone.

Mozilla’s A-Frame is now ready for Link Traversal through hyperportals

Creating VR for the Web
WebVR, a framework for browsers to support Three-Dimensional Stereoscopic Virtual Environments, is already developed and supported by many browsers. As HTML, CSS and JavaScript already have the powers to create and render graphics through frameworks such as WebGL, the web languages have increasing power to support such scenarios. Lately, frameworks combining these different frameworks to make implementation of WebVR even easier. Mozilla’s A-Frame, lets the user set up a Virtual Environment only with less than 20 lines of code (see their Hello WebVR example), using ‘normal’ HTML tags, which they call primitives, to create 3D objects in 3D space. A-Frame utilizes Three.JS to do this, and Three.JS uses the WebGL.

It is now easy to create Virtual Environments on the Web, even arguably easier than creating them through Unity. The great benefit of this is that they can be connected to each other, by a standard hypertext reference, instead of uploading to Steam or Oculus Store, etc. A-Frame introduced hypertext support, which they call “Link Traversal”, in July of 2017, but the browsers are only just catching up. As of now, it is only supported by Firefox and Supermedium on PC, however, as of February 2018 Oculus Browser has supported it on GEAR VR, and most likely also on Oculus Go.

A-Frame’s Diego Marcos called this a great achievement, as A-Frame finally achieved their ‘Web badge’. For this they deserve congratulations, A-Frame has now completed an essential step towards the Web of Realities. In their introductory blogpost, they introduce a “hyperportal” example, which provides you with a preview of the VR world you are about to enter, and which redirects you to the page when you virtually walk through it. This is a piece of very fun code to play around with. A neat feature is that the portal itself is “transparent”, and so provides a preview of the virtual environment to which you are travelling.

The future of WebVR
As with anything within VR, we are still a few years behind its potential. WebVR has had a solid boost the latest few years, but before a Head-Mounted Display is commonplace, we probably wont find a VR search engine or enough websites for exploration to be truly amazing. This is not bad news, however, it means that this is just the right time for creative ideas. We see the inevitable emergence of the VR Web, and can help shape it. For instance, at Matrise, we have previously discussed Virtual Reality Memory Palaces. This would be great to incorporate for sharing on the Web, so each memory palace could be interconnected, creating vast banks of knowledge for memorization.

Do you have any good ideas for any WebVR apps?
Feel free to comment below.

The ancient Greeks created “Memory Palaces” to recall important information.
Posted on

The Experience Machine

In  a previous entry, we discussed the potential of using VR technologies for journalistic purposes. In this post, we will both examplify and abstract the underlying principles of this theme. We will start with a philosophical introduction to the concept of “The Experience Machine”, before presenting and discussing four new VR Journalism prototypes developed during a course at the University of Bergen, Norway.

The Evil Deceiver
Since Rene Descartes published his Meditations in 1647, the West has been interested in what is real or authentic, and whether any knowledge of this is at all possible. In his text, Descartes outlines the idea of a potential “Evil Deceiver”; a demon that could alter his impressions of the external world. He writes that he can not know whether what he sees and perceives is real, or if it is a demon that is fooling him.  Although Descartes later in his Meditations provides arguments for why this can not be the case, based on his provided “evidence of God”, this argument of his does not bring solace to most, and as such the skepticism he postulated have outlived him, and is just as relevant a theme today.

Descartes’ Illustration of Stereoscopic Vision

When this is discussed philosophically today, the imagery of an Evil Deceiver comes a bit short. Rather, it is discussed in more technological or “physical” terms – instead the demon, we think of the potential scenario of our brains encapsulated in a vat. We know (although it always feels unnatural to use the word “know” when discussing the grounds of epistemic reasoning),  that our perception of reality is sensory impressions interpreted by the brain. We can then imagine the “Brain-In-A-Vat” as the philosopher Gilbert Harman describes it; a brain disconnected from its body, yet being fed electrical pulses that simulate sensory impressions. This has been the theme of several popular films, such as The Matrix series, in which AI has imprisoned humanity in a Virtual Reality. The first film brilliantly examplifies our philosophical points, as it depicts both an Evil Deceiver in the form of the AI, and the imprisonment of human brains in a vat. Conceptually, Harman’s reformulation does not bring anything new to this thought experiment, except a sort of updated imagery as a vessel on which to explain it. Of that reason, however, it is more relevant to our associations, as the “Brain-In-A-Vat” idea reminds us of the Immersive Virtual Reality technology that we see emerge today.

Clockwork Eyes by Michael Ryan

 

 

 

 

 

 

 

 

And this is the direction that we are heading in this entry – towards VR rather than epistemology.  Instead of discussing whether knowledge about the external world can be had, we will investigate this theme further through a philosopher that take a different approach to the concept of artificial stimuli: he is not speaking of the impending doom of AI demonic domination, but rather uses these concepts in thought experiments, as a means to reason on what matters to us. What matters to us is deeply related to what we do, and what we do is deeply related to who we are. As such, these questions can be illuminating in exploring who we are, or  want to be, as human beings.

The Experience Machine, Unknown Illustrator.

The Experience Machine
In 1974, Robert Nozick introduced the term “Experience Machine”, or “Pleasure Machine”, as a thought experiment. The experiment is a brilliant way to put the idea of Hedonism in individuals on trial. Now, hedonism is the view that happiness or pleasure is, the only thing that is in its essence good. Other things can be “instrumentally good”, as they may lead to happiness or pleasure, but they are only given their value then, by being the means to the end. The thought experiment is this: if you had a machine that could give you any experience that you ever wanted, including pleasure for the rest of your life, would you use it? Would you plug in?

We are here awfully close to the idea of the Brain-In-A-Vat, but yet the perspective is shifting: in this case we would not be trapped in a false reality against our will, but according to it. The value of the thought experiment is clear – in answering to it, we answer to whether we value truth as an “attribute” of sensory impressions in and out of itself; if authenticity and reality or whatever we want to call it has a value in itself, and is to be preferred despite its potential, say, gruesomeness. If one accepts Hedonism, the whole world is enframed as being a means to the end of pleasure, and as such, if such a Pleasure Machine presented itself, we could abandon the means in favor of the ends, going straight for the juice. Nozick’s idea is that if one does not want to plug into the machine, one effectively ‘proves’ that pleasure is not all Man wants or needs, and Hedonism can as such be refuted.

Some, however, feel that this formulation of the thought experiment is not sharp enough in its ability to try out the idea of Hedonism. The critique is that concern for authenticity or “realness” is not the only possible motive one may have to decline: many would prefer to not plug into the machine, not necessarily because they really care for truth, but because they are creatures of habit, and simply prefers the world they have accustomed themselves to. There is in other words other motivations for negating the question than the preference for truth. A better formulation that avoids this pitfall, is this: if you were told that your life up to now had been such an illusion, a pleasure machine, would you then like to wake up? 

Hinduism
These thoughts can be traced back to long before the rise of the particular Western philosophy as followed Descartes. According to Hindu cosmology, each of us lives in maya (illusion) as to what is the core reality of Self, or Atman. Life can be seen as a play, and Brahman is the actor that plays all the parts, totally immersed and engaged in them so it forgets its real self, and instead is amusing itself in its ignorance (for instance by passing time reading blogs online). Reality then, is a game of hide and seek, where you are both the hider and the seeker, playing for eternity. Similarly to the Experience Machine, the subject is ignorant to the true nature of reality, by choice.

The Five Aspects of Shiva

Why, then, would God hide from himself? The idea is simple: being God gets boring after a while. Imagine that you are God, and could have any experience you ever wanted. You would perhaps start by throwing some crazy parties, creating some planets, etc., for a few million years. Eventually, you have kind of “been there, done that”. The curse of being all-knowing and omnipotent is, of course, despite the supreme bliss, that it’s hard to get a true kick out of it anymore. You lack the element of surprise. Surprise, as reaction, needs duality, but you are One. The idea strikes you: what if we were not One? You decide to go for it: split into multiples, and so each can surprise the other. Atman, becomes Brahman, the player of all the parts in the play of life. At times it gets terrifyingly real, after all you have to die to wake up.  But this is what you came for, because no surprise is greater than to wake up, and to realize that you were Atman, the Permanent, all along.

Modern Day Experience Machines
In 1974, for Nozick, the experience machine was a thought experiment far away from reality. Today, it is perhaps far too easy to see the potential of its realization. When VR applications are released today, they are often presented as “experiences”; not always games, not necessarily videos, but experiences. We have in a previous entry discussed Art in Virtual Reality, where the focus on “experience” is particularly present, and shows the potential of the medium for artistic expression. The same is the case with Immersive Journalism, where the first person perspective is especially exploited as a means to provide a user with a different experience than what otherwise is had in everyday life. Within VR Journalism, the discussion of experience machines has almost been had, as Milk in his 2016 Ted Talk described VR as the ultimate empathy machine. In Immersive Journalism, the 360° (sometimes 3D) camera work as the eyes of the observer, and lets one virtually step into someone elses shoes, or at least to put on their eyes.

In a course that we have ran at the University of Bergen this spring, we have taught Immersive Journalism within an Innovation Pedagogy regime. As the rules and practices within the new concept is not very well established, we do not teach the students exactly to solve their tasks, but rather how to experiment with the novelties of the medium and try to innovate and create new genres. The end result have been four prototypes, that was presented yesterday, at the Norwegian Centre of Excellence (NCE) Media’s media lab in Media City Bergen. All of these productions are well fit for the topic of “modern day experience machines”, and as such they will be presented here in this entry.

Drug addict
The first of the VR experiences is called “Narkomani” which from Norwegian can roughly be translated to “Drug addict”. The aim of the production is to see the world from the point of view of a drug addict, perhaps living on the streets in Bergen. How is it to be frowned upon, walking around the streets, uneasy to get the next shot of dope? As my colleague Nyre stated in the introduction the projects, this VR project features “not a first person shooter from Los Angeles, but first person social realism from Bergen”.

Schizophrenia
The second of the VR experiences, attempts to create understanding on how it is to live with schizophrenia. In the experience, the user perceives visual hallucinations, and audio of up to five different personalities. The concept is brilliantly illustrated by the poster, and the experience tries to portrait a subjective reality falling apart.

Plastic
The third VR experience has children as its target group, and aims to introduce the world problem of plastic to them. In this case, the VR medium is being exploited for its capabilities for visualization. For children, numbers may be too abstract, and visual images may be more appealing to get the message across.  The experience features combinations of 360° video footage with 3D objects.

Cryonics
The final VR experience has the topic of cryonics, i.e., freezing oneself down immediately upon death. The production aims to educate on the topic with interviews from different philosophers, religious leaders, etc., which you hear and see while chilling in your freeze tank. When the production starts, you find yourself lying down on an operation table before insterted into the capsule and frozen.

Images and more information about the event, can be seen at the websites of research project ViSmedia.

Conclusion
The relation between Self and Other, the Internal and the External, Inner and Outer, has always fascinated us. This dualism comes with problems, and we are skeptic to whether the seemingly similar values on both sides of the Self – Other equation, can be cancelled out and forgotten. These problems enter a new fascinating light when we encounter new technology that has the capability of actualizing them. All sorts of experience machines are being created today, although we still are quite far from escapism into VR as a societal issue.

The topic of authenticity in relation to VR, will be further discussed at Matrise in a three-entry series that steers towards a metaphysic of VR. The entries will discuss the topic in light of, particularly, Heidegger’s existentialism. The entries will be published next week.

Literature list

SaveSave

SaveSave

SaveSave

SaveSaveSaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

Posted on

Art in Virtual Reality

A couple of weeks ago, I watched the fantastic “Loving Vincent” at the cinema. Unique in its cinematography and visual effects, it depicts the life and death of Vincent van Gogh. There is, however, not much depiction of the great painter himself, as the film starts with the news about his death. Despite of this, the Van Gogh-hungry are not left unsatisfied — the whole film is oil-painted in his style, and so he is present in every stroke.

Robert Gulaczyk as Vincent Van Gogh, in Loving Vincent

Before this turns into a movie review, I should pinpoint that this entry is in fact about Virtual Reality (VR). Loving Vincent instantly reminded me of one of my favorite VR experiences. Depicted in the film itself, “The Night Café”, originally a painting by Van Gogh, has been turned into a VR experience by Borrowed Light Studios.

A Gif from the virtual experience of “The Night Café”, from Borrowed Light Studios.

The Night Café
On enteringThe Night Café in Immersive VR, one starts to hear piano music. Further, the scene fades from black, and one finds oneself present in a somewhat psychedelic bar depicted in “Van Gogh style”: the lights flare and pulse, and the animated characters definitely come straight from the “Uncanny Valley“. The fascinating aspect of the experience is to feel  present in a live painting: while moving around one definitely feels apart of the virtual environment. Because of the rather simple style, the resolution and refresh rates can be maximised even on mobile VR goggles, which turns it into a very graphically rich VR experience in the context of mobile VR. The experience of “The Night Café” definitely immerse you in an ecstatic, dynamic Van Gogh painting, which Van Gogh-fans and layman alike should check out. I would not be surprised if the whole  film was inspired by this VR experience.

The original painting of “The Night Café” by Van Gogh.

The Virtual Van Gogh
Something similar to “The Night Café” was created by The Amsterdam Van Gogh Museum in 2015, where they invited their visitors to enter the bedrooms of Vincent Van Gogh and Edvard Munch in VR. Although the event has now passed, the application “Virtual Bedrooms” is available for download for Google Cardboard-devices, at App Store and at Google Play. Unlike The Night Café, however, none of these offer any navigation within the virtual environment.

The Amsterdam Van Gogh Museum’s Illustration of their Virtual Van Gogh.

The expression of Van Gogh is enrichened, heightened, augmented and arguably also changed through the medium of VR.  VR is a medium with a lot of expression potential, and this is the immediate relevance between art and virtual reality. To broadcast their message, artists utilize every new medium in their power. In this context, it is very useful to pinpoint that the medium of VR is an especially powerful one: by enabling users to adopt different identities within any conceivable programmed world, the potential for expression is endless. Another artist we will use to examplify VR’s potential within art, is Pushwagner.

Pushwagner Self-Portrait. which has seen a re-creation also in VR.

Pushwagner
The Norwegian contemporary artist “Pushwagner”, who unfortunately died last month, is another artist whose art is seen reimagined in the medium of VR. As part of a master’s project at the Western University of Applied Sciences, Runar Tistel has recreated the art of Pushwagner in a 360° 3D Virtual Environment. Pushwagner’s works often feature repetitive patterns and an abundance of geometrical shapes, which made Tistel approach the generating of these virtual environments mathematically. By adopting this mathematic-reductionist approach to the art, the re-creation of Pushwagner’s art to VR, is performed by a script that procedurally generates Pushwagner’s work. Essentially, the art is rendered on site, from source code text to imagery. I have myself had the joy of stepping into a recreation of Pushwagner’s self portrait in VR and found it to be a very interesting experience.

Bergen International Festival
As a resident of Bergen, I am anxiously awaiting Festspillene (Bergen International Festival); an annual event in Bergen, Norway, which is the greatest music- and theater festival in the North. The aim of the festival is to present art in every genre: music, theater, dance, opera, and the visual arts. For the first time in 2018, the Festival is introducing a “VR Lounge”, a free exhibit of VR experiences in Grieghallen. Here you can come to enjoy first class VR experiences for free: the Festival will set up several HTC Vive’s in the lounge.

Of the works being presented are The Virtual Orchestra by London Philharmonic Orchestra, which takes you to Southbank Centre, London, while they play the last part of Sibelius’ 5th symphony. In addition to this, you can see the theater show “My Name is Peter Stillmann”, inspired by Paul Austers’ City of Glass. Similarly to Loving Vincent and The Night Café, it features handdrawn animation. The experience is delivered by 59 productions, who arranged the video- and design production during the Olympic opening in London in 2012, so this should be really exciting.

BIFF
“Festspillene” is not the first event of its kind in Bergen to embrace the visual expression power of VR technologies. Residents of Bergen were equally lucky during Bergen International Film Festival (BIFF) at the exhibit “BIFF Expanded”, where they previewed various VR and AR experiences. The scope of the exhibit was to display the cross section between film, visual arts and new technology, where VR was well represented. Amongst others, the guys and gals at Crossover Labs displayed their VR experiences there, in addition to holding a workshop on Unity in VR for us at the local library. Lucky us.

Art in Virtual Reality
To summarize this entry, we can definitively say that the medium of VR proves a great venue for artists, designers, etc., to present art in 3D and 360 degrees. Not only for presenting older art, obviously, as this post presented, but to display new art that plays with established principles of visual presentation. In addition to this, VR can aid users in the creation of new art as well (check out Google Tiltbrush, or Kodon by the Bergen-based TenkLabs, which lets you draw in 360 3D).  It will be exciting to see what creative projects that emerge during the next years. For now, we have the Bergen International Festival to look forward too. If you want to join me as a VR assistant volunteer at the Festival, you can sign up here.

Oilpaint style in Unity
If you enjoy the oil-painted style in Virtual Environments, I would strongly encourage to play with the NPR Oil Paint Effect for Unity 3D, available in Unity Asset Store. The effect can turn any game or VR experience into oil-painted style. A youtube video illustrating the concept, can be seen here.

To make it a hat-trick of Van Gogh-screenshots, here is The Night Café as depicted in Loving Vincent as well. Go buy the film if you want to see it in motion — it’s worth it.

 

SaveSave

SaveSave

SaveSave

SaveSave

SaveSaveSaveSaveSaveSave

SaveSave

SaveSave

SaveSave

SaveSaveSaveSaveSaveSaveSaveSave

Posted on

Virtual Reality Memory Palaces

The ancient mnemonic “method of Loci” has been in use for thousands of years. From memory competitors to students, the method helps to memorise items by combining visual and spatial cues. Many have perhaps heard of the technique through popular TV shows such as Sherlock or Hannibal, where their “Mind Palaces” or “Memory Palaces” are portrayed as a genius trait.

Illustration based on “Mind Attic” by wonderful artist Tisserande. Original version without Head-Mounted Display can be found here.

The real method, however, is perfectly simple. You start by finding a location to act as your Memory Palace. It should be a place you know well, for instance your apartment or your university. You further isolate several rooms, and sub-parts of those rooms, that amount to the number of things you want to recall. Next, you may want to have a certain route which you mentally walk through your Memory Palace. This is where the method comes in: you visualise what you want to recall at the given places in your Mind Palace. Need to go shopping? Imagine coffee beans poured out on the floor, orange juice cartons in the sofa, and bananas hanging on the TV. If you need to recall something more abstract, you can get creative and use the visualisation as an association to what you want to recall instead.

This method is believed to be powerful because of the combination of information with visual cues (the visualisation of the memory items), and with spatial cues (their given place in the environment). Curiously, however, this exploitation of the visual and spatial cues in the brain, does not involve our vision, or true perception of spatial three-dimensional spaces at all — only through our “inner eye” and our already-established memory of spatial environments.

“Get out — I need to go to my Mind Palace”. Benedict Cumberbatch in BBC’s Sherlock  — Quite busy in his Memory Palace. Illustration of by edthatch.

But what if we could perform this method with raw, fresh vision that gives us both the visual and spatial cues? Such a task is perfect for the medium of Immersive Virtual Reality. In VR, we are provided with highly optimisible a visuo-spatial environment made of software, which gives us the opportunity  to tailor the Memory Palaces to our own needs. By exploiting the visuo-spatial elements of the method of Loci in a much more explicit sense,  the method may perhaps work even better, or be more fun and easy to use.

Addressing this, I created an a VR memory palace app as part of my master’s thesis in Information Science. The application is called “Mnemosyne”,  after the Greek goddess of memory, of which country the Method of Loci originated. Developed with A-Frame, the application presents you with a text field where you can enter the things you want to recall. You then put on your VR headset, and are presented with an apartment with five rooms where you can place these memory items you entered. The algorithm returns images based on your text preferences from a search engine, and places these in the virtual environment. When this is done, you just walk around and watch to memorise the items.

Mnemosyne is still just a prototype, and we can imagine it will have room for many improvements in the future – after more research has been done. Currently, we at the Centre for the Science of Learning & Technology (SLATE) have performed a pilot study with 18 participants. The short paper discussing the findings was published in June 2018 in the Springer’s series «Lecture Notes on Computer Science», in the «Augmented Reality, Virtual Reality and Computer Graphics» edition. The conference paper was presented in the beautiful village of Otranto at the Salento AVR conference in July.

Our first paper on VR Memory Palaces was posted in the LNCS on Augmented Reality, Virtual Reality and Computer Graphics.

Our paper, ‘Mnemosyne: Adapting the Method of Loci to Immersive Virtual Reality’, can be accessed at Springer. Stay put at Matrise for further updates on our work regarding Virtual Reality Memory Palaces.

Literature list

Posted on

VR Journalism at Watching in the Media 2018

During Joakim’s talk on “Virtual Reality Journalism” at the Watching in the Media conference, ViSmedia handed out close to 200 Cardboard VR goggles. In this post, you can get some tips on how to use them to experience Immersive Journalism. Don’t have a cheap pair of Cardboard yourself? You could always get a pair for less than 1 USD at eBay.

If you need an introduction to the theme, or want to read more on the subject, you can read our previous article on VR Journalism here. In the entry we briefly discuss ethical and conceptual issues with VR Journalism, for those interested.

To see a few 360 images from the conference, you can go to this link.
Remember to tilt your phone sideways and click the VR Goggles icon before you put the phone into the cardboard viewer. Below is a small list of recommended content for you to explore.

Watch Joakim’s talk here:

1. View 360 images from the ViSmedia conference.
Students at the University of Bergen took a few 360 images during the Watching in the Media conference. These images are unfortunately no longer available.

2. New York Times’ VR app.
The NYT VR app can be found at the app store or at Google Play
We can recommend the stories Fight for Fallujah, Donald Trump Rally and The Displaced.

3. The Guardian VR.
The Guardian also has their own app for iPhone and Android.
Their most popular VR story is 6×9: a VR experience of solitary confinement, but their app also offer a whole lot more.

4. YouTube.
YouTube offers 360 videos in VR mode. Download the app, and browse with keywords such as “360” and “VR”. In our experience, especially in the Playstation VR Head-Mounted Display, these do not deliver very high resolution as with most 360 streaming.

5. Other apps
Your VR goggles can be used for other things than just journalistic viewing. The app store and play store is filled with VR apps for your goggles. Just search for keywords such as “360” and “VR”, and download what you like.

6. The VR Web
Although still in its infancy, with the rise of WebVR support in more and more browsers, we can browse VR applications on the web. Matrise has experience developing WebVR apps with A-Frame (read our entry on Virtual Reality Memory Palaces here), and would recommend going to A-Frame’s website for examples on what the framework is capable of.

Happy exploring.