The Web of VR Will Change Everything

The World Wide Web (WWW) need no further introduction. The greatest innovation of the Information Age is now essential to the world like no other technology. Before the WWW – computers, programs and information were not linked. The computers were lonely, and users could not browse the millions of interconnected computers the way we do today.

The Web has been changing ever since its dawn in the 90s, and has seen its distinct phases. What we call “Web 1.0”, for instance, was a static web. Websites could be visited and navigated, but they were static in the sense of not affording any user interaction. Web 2.0 opened up for more dynamic web applications that could be altered by user input. These did not just allow download, they could also be uploaded to – a feature that is now an essential underpinning of social media and web-based applications. Companies like Facebook, Snapchat, Instagram, etc., do not provide content to be downloaded, but rather a computer service to be used where the users provide the content.

spider web - The Web of VR Will Change Everything
Illustration by Lancelot Speed from Andrew Lang’s “The Red Fairy Book”.

It should be noted that “Web 1.0” and “Web 2.0” are just terms: there are constantly being added changes to the Web. The terms does, however, signify when these changes are inducers of a paradigm shift in the use of the Web. What is to be classified as Web 3.0 is therefore also discussed. Although not necessarily a feature of the Web, the Internet of Things (IoT) is a candidate for what has become a paradigm shift within web technologies, as more and more devices and artifacts are connected, allowing for ubiquitous computing. Others discuss the personalised Web we see today, as it is influenced by social media, while some are joining the AI hype, and claim that the Web has now become smart. The latter is far from a paradigm shift as of yet.

The Virtual Web
In this entry we will discuss WebVR, or Virtual Reality through, and on, the Web. The title of this entry, which claim that “The Web of VR Will Change Everything”, may indicate a stand towards the debate that I have introduced, of what will be the “Web 3.0” –  but that is not the point of this entry. I do not wish to make a claim of WebVR as a paradigm shift of the way the Web operates, but it most significantly will be a paradigm shift in how we experience the web technology as it is.

The concept and role of the Web, nevertheless, is the same: we have a dynamic web which features download and upload of web documents. In the case of VR the difference is that what we download and upload, are perceived as realities for us: the web is the mediator of realities, and this new way of using the web changes VR more than VR necessarily changes the Web. What characterizes the Web, is its simplicity, its openness and the innate element of surprise. Anything can be found, and the exploration as such is an important part of it. These features are the same that will be valuable in VR as well: to discover open virtual worlds, created by anyone.

Screenshot 2018 06 06 at 13.27.51 - The Web of VR Will Change Everything
Mozilla’s A-Frame is now ready for Link Traversal through hyperportals

Creating VR for the Web
WebVR, a framework for browsers to support Three-Dimensional Stereoscopic Virtual Environments, is already developed and supported by many browsers. As HTML, CSS and JavaScript already have the powers to create and render graphics through frameworks such as WebGL, the web languages have increasing power to support such scenarios. Lately, frameworks combining these different frameworks to make implementation of WebVR even easier. Mozilla’s A-Frame, lets the user set up a Virtual Environment only with less than 20 lines of code (see their Hello WebVR example), using ‘normal’ HTML tags, which they call primitives, to create 3D objects in 3D space. A-Frame utilizes Three.JS to do this, and Three.JS uses the WebGL.

It is now easy to create Virtual Environments on the Web, even arguably easier than creating them through Unity. The great benefit of this is that they can be connected to each other, by a standard hypertext reference, instead of uploading to Steam or Oculus Store, etc. A-Frame introduced hypertext support, which they call “Link Traversal”, in July of 2017, but the browsers are only just catching up. As of now, it is only supported by Firefox and Supermedium on PC, however, as of February 2018 Oculus Browser has supported it on GEAR VR, and most likely also on Oculus Go.

A-Frame’s Diego Marcos called this a great achievement, as A-Frame finally achieved their ‘Web badge’. For this they deserve congratulations, A-Frame has now completed an essential step towards the Web of Realities. In their introductory blogpost, they introduce a “hyperportal” example, which provides you with a preview of the VR world you are about to enter, and which redirects you to the page when you virtually walk through it. This is a piece of very fun code to play around with. A neat feature is that the portal itself is “transparent”, and so provides a preview of the virtual environment to which you are travelling.

The future of WebVR
As with anything within VR, we are still a few years behind its potential. WebVR has had a solid boost the latest few years, but before a Head-Mounted Display is commonplace, we probably wont find a VR search engine or enough websites for exploration to be truly amazing. This is not bad news, however, it means that this is just the right time for creative ideas. We see the inevitable emergence of the VR Web, and can help shape it. For instance, at Matrise, we have previously discussed Virtual Reality Memory Palaces. This would be great to incorporate for sharing on the Web, so each memory palace could be interconnected, creating vast banks of knowledge for memorization.

Do you have any good ideas for any WebVR apps?
Feel free to comment below.

old canal - The Web of VR Will Change Everything
The ancient Greeks created “Memory Palaces” to recall important information.

The Experience Machine

In  a previous entry, we discussed the potential of using VR technologies for journalistic purposes. In this post, we will both examplify and abstract the underlying principles of this theme. We will start with a philosophical introduction to the concept of “The Experience Machine”, before we relate this to Virtual Reality.

The Evil Deceiver
Since Rene Descartes published his Meditations in 1647, the West has been interested in what is real or authentic, and whether any knowledge of this is at all possible. In his text, Descartes outlines the idea of a potential “Evil Deceiver”; a demon that could alter his impressions of the external world. He writes that he can not know whether what he sees and perceives is real, or if it is a demon that is fooling him.  Although Descartes later in his Meditations provides arguments for why this can not be the case, based on his provided “evidence of God”, this argument of his does not bring solace to most, and as such the skepticism he postulated have outlived him, and is just as relevant a theme today.

353d8b20e154fd856de668440bda49b9 - The Experience Machine
Descartes’ Illustration of Stereoscopic Vision

When this is discussed philosophically today, the imagery of an Evil Deceiver comes a bit short. Rather, it is discussed in more technological or “physical” terms – instead the demon, we think of the potential scenario of our brains encapsulated in a vat. We know (although it always feels unnatural to use the word “know” when discussing the grounds of epistemic reasoning),  that our perception of reality is sensory impressions interpreted by the brain. We can then imagine the “Brain-In-A-Vat” as the philosopher Gilbert Harman describes it; a brain disconnected from its body, yet being fed electrical pulses that simulate sensory impressions. This has been the theme of several popular films, such as The Matrix series, in which AI has imprisoned humanity in a Virtual Reality. The first film brilliantly examplifies our philosophical points, as it depicts both an Evil Deceiver in the form of the AI, and the imprisonment of human brains in a vat. Conceptually, Harman’s reformulation does not bring anything new to this thought experiment, except a sort of updated imagery as a vessel on which to explain it. Of that reason, however, it is more relevant to our associations, as the “Brain-In-A-Vat” idea reminds us of the Immersive Virtual Reality technology that we see emerge today.

abf94fae0ecdde8b7d6753e315c3bd47 - The Experience Machine
Clockwork Eyes by Michael Ryan

And this is the direction that we are heading in this entry – towards VR rather than epistemology.  Instead of discussing whether knowledge about the external world can be had, we will investigate this theme further through a philosopher that take a different approach to the concept of artificial stimuli: he is not speaking of the impending doom of AI demonic domination, but rather uses these concepts in thought experiments, as a means to reason on what matters to us. What matters to us is deeply related to what we do, and what we do is deeply related to who we are. As such, these questions can be illuminating in exploring who we are, or  want to be, as human beings.

635187350773856993 the happiness machine by tris31 d37ajkp - The Experience Machine
The Experience Machine, Unknown Illustrator.

The Experience Machine
In 1974, Robert Nozick introduced the term “Experience Machine”, or “Pleasure Machine”, as a thought experiment. The experiment is a brilliant way to put the idea of Hedonism in individuals on trial. Now, hedonism is the view that happiness or pleasure is, the only thing that is in its essence good. Other things can be “instrumentally good”, as they may lead to happiness or pleasure, but they are only given their value then, by being the means to the end. The thought experiment is this: if you had a machine that could give you any experience that you ever wanted, including pleasure for the rest of your life, would you use it? Would you plug in?

We are here awfully close to the idea of the Brain-In-A-Vat, but yet the perspective is shifting: in this case we would not be trapped in a false reality against our will, but according to it. The value of the thought experiment is clear – in answering to it, we answer to whether we value truth as an “attribute” of sensory impressions in and out of itself; if authenticity and reality or whatever we want to call it has a value in itself, and is to be preferred despite its potential, say, gruesomeness. If one accepts Hedonism, the whole world is enframed as being a means to the end of pleasure, and as such, if such a Pleasure Machine presented itself, we could abandon the means in favor of the ends, going straight for the juice. Nozick’s idea is that if one does not want to plug into the machine, one effectively ‘proves’ that pleasure is not all Man wants or needs, and Hedonism can as such be refuted.

Some, however, feel that this formulation of the thought experiment is not sharp enough in its ability to try out the idea of Hedonism. The critique is that concern for authenticity or “realness” is not the only possible motive one may have to decline: many would prefer to not plug into the machine, not necessarily because they really care for truth, but because they are creatures of habit, and simply prefers the world they have accustomed themselves to. There is in other words other motivations for negating the question than the preference for truth. A better formulation that avoids this pitfall, is this: if you were told that your life up to now had been such an illusion, a pleasure machine, would you then like to wake up? 

Hinduism
These thoughts can be traced back to long before the rise of the particular Western philosophy as followed Descartes. According to Hindu cosmology, each of us lives in maya (illusion) as to what is the core reality of Self, or Atman. Life can be seen as a play, and Brahman is the actor that plays all the parts, totally immersed and engaged in them so it forgets its real self, and instead is amusing itself in its ignorance (for instance by passing time reading blogs online). Reality then, is a game of hide and seek, where you are both the hider and the seeker, playing for eternity. Similarly to the Experience Machine, the subject is ignorant to the true nature of reality, by choice.

deity 1299533  340 1 - The Experience Machine
The Five Aspects of Shiva

Why, then, would God hide from himself? The idea is simple: being God gets boring after a while. Imagine that you are God, and could have any experience you ever wanted. You would perhaps start by throwing some crazy parties, creating some planets, etc., for a few million years. Eventually, you have kind of “been there, done that”. The curse of being all-knowing and omnipotent is, of course, despite the supreme bliss, that it’s hard to get a true kick out of it anymore. You lack the element of surprise. Surprise, as reaction, needs duality, but you are One. The idea strikes you: what if we were not One? You decide to go for it: split into multiples, and so each can surprise the other. Atman, becomes Brahman, the player of all the parts in the play of life. At times it gets terrifyingly real, after all you have to die to wake up.  But this is what you came for, because no surprise is greater than to wake up, and to realize that you were Atman, the Permanent, all along.

Modern Day Experience Machines
In 1974, for Nozick, the experience machine was a thought experiment far away from reality. Today, it is perhaps far too easy to see the potential of its realization. When VR applications are released today, they are often presented as “experiences”; not always games, not necessarily videos, but experiences. We have in a previous entry discussed Art in Virtual Reality, where the focus on “experience” is particularly present, and shows the potential of the medium for artistic expression. The same is the case with Immersive Journalism, where the first person perspective is especially exploited as a means to provide a user with a different experience than what otherwise is had in everyday life. Within VR Journalism, the discussion of experience machines has almost been had, as Milk in his 2016 Ted Talk described VR as the ultimate empathy machine. In Immersive Journalism, the 360° (sometimes 3D) camera work as the eyes of the observer, and lets one virtually step into someone elses shoes, or at least to put on their eyes.

In a course that we have ran at the University of Bergen this spring, we have taught Immersive Journalism within an Innovation Pedagogy regime. As the rules and practices within the new concept is not very well established, we do not teach the students exactly to solve their tasks, but rather how to experiment with the novelties of the medium and try to innovate and create new genres. The end result have been four prototypes, that was presented yesterday, at the Norwegian Centre of Excellence (NCE) Media’s media lab in Media City Bergen. All of these productions are well fit for the topic of “modern day experience machines”, and as such they will be presented here in this entry.

Drug addict
The first of the VR experiences is called “Narkomani” which from Norwegian can roughly be translated to “Drug addict”. The aim of the production is to see the world from the point of view of a drug addict, perhaps living on the streets in Bergen. How is it to be frowned upon, walking around the streets, uneasy to get the next shot of dope? As my colleague Nyre stated in the introduction the projects, this VR project features “not a first person shooter from Los Angeles, but first person social realism from Bergen”.

Schizophrenia
The second of the VR experiences, attempts to create understanding on how it is to live with schizophrenia. In the experience, the user perceives visual hallucinations, and audio of up to five different personalities. The concept is brilliantly illustrated by the poster, and the experience tries to portrait a subjective reality falling apart.

Plastic
The third VR experience has children as its target group, and aims to introduce the world problem of plastic to them. In this case, the VR medium is being exploited for its capabilities for visualization. For children, numbers may be too abstract, and visual images may be more appealing to get the message across.  The experience features combinations of 360° video footage with 3D objects.

Cryonics
The final VR experience has the topic of cryonics, i.e., freezing oneself down immediately upon death. The production aims to educate on the topic with interviews from different philosophers, religious leaders, etc., which you hear and see while chilling in your freeze tank. When the production starts, you find yourself lying down on an operation table before insterted into the capsule and frozen.

Images and more information about the event, can be seen at the websites of research project ViSmedia.

Conclusion
The relation between Self and Other, the Internal and the External, Inner and Outer, has always fascinated us. This dualism comes with problems, and we are skeptic to whether the seemingly similar values on both sides of the Self – Other equation, can be cancelled out and forgotten. These problems enter a new fascinating light when we encounter new technology that has the capability of actualizing them. All sorts of experience machines are being created today, although we still are quite far from escapism into VR as a societal issue.

The topic of authenticity in relation to VR, will be further discussed at Matrise in a three-entry series that steers towards a metaphysic of VR. The entries will discuss the topic in light of, particularly, Heidegger’s existentialism. The entries will be published next week.

Literature list

SaveSave

SaveSave

SaveSave

SaveSaveSaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

Art and Virtual Reality

A couple of weeks ago, I watched the fantastic “Loving Vincent” at the cinema. Unique in its cinematography and visual effects, it depicts the life and death of Vincent van Gogh. There is, however, not much depiction of the great painter himself, as the film starts with the news about his death. Despite of this, the Van Gogh-hungry are not left unsatisfied — the whole film is oil-painted in his style, and so he is present in every stroke.

Loving Vincent 1024x683 - Art and Virtual Reality
Robert Gulaczyk as Vincent Van Gogh, in Loving Vincent

Before this turns into a movie review, I should pinpoint that this entry is in fact about Virtual Reality (VR). Loving Vincent instantly reminded me of one of my favorite VR experiences. Depicted in the film itself, “The Night Café”, originally a painting by Van Gogh, has been turned into a VR experience by Borrowed Light Studios.

f8155ce7780b92f0ea56c52e1e36fc84 - Art and Virtual Reality
A Gif from the virtual experience of “The Night Café”, from Borrowed Light Studios.

The Night Café
On enteringThe Night Café in Immersive VR, one starts to hear piano music. Further, the scene fades from black, and one finds oneself present in a somewhat psychedelic bar depicted in “Van Gogh style”: the lights flare and pulse, and the animated characters definitely come straight from the “Uncanny Valley“. The fascinating aspect of the experience is to feel  present in a live painting: while moving around one definitely feels apart of the virtual environment. Because of the rather simple style, the resolution and refresh rates can be maximised even on mobile VR goggles, which turns it into a very graphically rich VR experience in the context of mobile VR. The experience of “The Night Café” definitely immerse you in an ecstatic, dynamic Van Gogh painting, which Van Gogh-fans and layman alike should check out. I would not be surprised if the whole  film was inspired by this VR experience.

thenightcafe 1 - Art and Virtual Reality
The original painting of “The Night Café” by Van Gogh.

The Virtual Van Gogh
Something similar to “The Night Café” was created by The Amsterdam Van Gogh Museum in 2015, where they invited their visitors to enter the bedrooms of Vincent Van Gogh and Edvard Munch in VR. Although the event has now passed, the application “Virtual Bedrooms” is available for download for Google Cardboard-devices, at App Store and at Google Play. Unlike The Night Café, however, none of these offer any navigation within the virtual environment.

maxresdefault 1024x683 - Art and Virtual Reality
The Amsterdam Van Gogh Museum’s Illustration of their Virtual Van Gogh.

The expression of Van Gogh is enrichened, heightened, augmented and arguably also changed through the medium of VR.  VR is a medium with a lot of expression potential, and this is the immediate relevance between art and virtual reality. To broadcast their message, artists utilize every new medium in their power. In this context, it is very useful to pinpoint that the medium of VR is an especially powerful one: by enabling users to adopt different identities within any conceivable programmed world, the potential for expression is endless. Another artist we will use to examplify VR’s potential within art, is Pushwagner.

OCA Pushwagner self portrait 1024x1024 - Art and Virtual Reality
Pushwagner Self-Portrait. which has seen a re-creation also in VR.

Pushwagner
The Norwegian contemporary artist “Pushwagner”, who unfortunately died last month, is another artist whose art is seen reimagined in the medium of VR. As part of a master’s project at the Western University of Applied Sciences, Runar Tistel has recreated the art of Pushwagner in a 360° 3D Virtual Environment. Pushwagner’s works often feature repetitive patterns and an abundance of geometrical shapes, which made Tistel approach the generating of these virtual environments mathematically. By adopting this mathematic-reductionist approach to the art, the re-creation of Pushwagner’s art to VR, is performed by a script that procedurally generates Pushwagner’s work. Essentially, the art is rendered on site, from source code text to imagery. I have myself had the joy of stepping into a recreation of Pushwagner’s self portrait in VR and found it to be a very interesting experience.

Bergen International Festival
As a resident of Bergen, I am anxiously awaiting Festspillene (Bergen International Festival); an annual event in Bergen, Norway, which is the greatest music- and theater festival in the North. The aim of the festival is to present art in every genre: music, theater, dance, opera, and the visual arts. For the first time in 2018, the Festival is introducing a “VR Lounge”, a free exhibit of VR experiences in Grieghallen. Here you can come to enjoy first class VR experiences for free: the Festival will set up several HTC Vive’s in the lounge.

Of the works being presented are The Virtual Orchestra by London Philharmonic Orchestra, which takes you to Southbank Centre, London, while they play the last part of Sibelius’ 5th symphony. In addition to this, you can see the theater show “My Name is Peter Stillmann”, inspired by Paul Austers’ City of Glass. Similarly to Loving Vincent and The Night Café, it features handdrawn animation. The experience is delivered by 59 productions, who arranged the video- and design production during the Olympic opening in London in 2012, so this should be really exciting.

BIFF
“Festspillene” is not the first event of its kind in Bergen to embrace the visual expression power of VR technologies. Residents of Bergen were equally lucky during Bergen International Film Festival (BIFF) at the exhibit “BIFF Expanded”, where they previewed various VR and AR experiences. The scope of the exhibit was to display the cross section between film, visual arts and new technology, where VR was well represented. Amongst others, the guys and gals at Crossover Labs displayed their VR experiences there, in addition to holding a workshop on Unity in VR for us at the local library. Lucky us.

Art in Virtual Reality
To summarize this entry, we can definitively say that the medium of VR proves a great venue for artists, designers, etc., to present art in 3D and 360 degrees. Not only for presenting older art, obviously, as this post presented, but to display new art that plays with established principles of visual presentation. In addition to this, VR can aid users in the creation of new art as well (check out Google Tiltbrush, or Kodon by the Bergen-based TenkLabs, which lets you draw in 360 3D).  It will be exciting to see what creative projects that emerge during the next years. For now, we have the Bergen International Festival to look forward too. If you want to join me as a VR assistant volunteer at the Festival, you can sign up here.

Oilpaint style in Unity
If you enjoy the oil-painted style in Virtual Environments, I would strongly encourage to play with the NPR Oil Paint Effect for Unity 3D, available in Unity Asset Store. The effect can turn any game or VR experience into oil-painted style. A youtube video illustrating the concept, can be seen here.

To make it a hat-trick of Van Gogh-screenshots, here is The Night Café as depicted in Loving Vincent as well. Go buy the film if you want to see it in motion  it’s worth it.

Do you know of any great art in Virtual Reality? Please comment below, and check out our videos on YouTube featuring VR artist Kevin Mack:

the night cafe - Art and Virtual Reality
Art in Virtual Reality. Illustration from Loving Vincent.

SaveSave

SaveSave

SaveSave

SaveSave

SaveSaveSaveSaveSaveSave

SaveSave

SaveSave

SaveSave

SaveSaveSaveSaveSaveSaveSaveSave

Virtual Reality Memory Palaces

The ancient mnemonic “method of Loci” has been in use for thousands of years. From memory competitors to students, the method helps to memorise items by combining visual and spatial cues. Many have perhaps heard of the technique through popular TV shows such as Sherlock or Hannibal, where their “Mind Palaces” or “Memory Palaces” are portrayed as a genius trait.

newname 1 - Virtual Reality Memory Palaces
Illustration based on “Mind Attic” by wonderful artist Tisserande. Original version without Head-Mounted Display can be found here.

The real method, however, is perfectly simple. You start by finding a location to act as your Memory Palace. It should be a place you know well, for instance your apartment or your university. You further isolate several rooms, and sub-parts of those rooms, that amount to the number of things you want to recall. Next, you may want to have a certain route which you mentally walk through your Memory Palace. This is where the method comes in: you visualise what you want to recall at the given places in your Mind Palace. Need to go shopping? Imagine coffee beans poured out on the floor, orange juice cartons in the sofa, and bananas hanging on the TV. If you need to recall something more abstract, you can get creative and use the visualisation as an association to what you want to recall instead.

two hannibals skull - Virtual Reality Memory Palaces
Hannibal illustration. Virtual Reality Memory Palace.

This method is believed to be powerful because of the combination of information with visual cues (the visualisation of the memory items), and with spatial cues (their given place in the environment). Curiously, however, this exploitation of the visual and spatial cues in the brain, does not involve our vision, or true perception of spatial three-dimensional spaces at all — only through our “inner eye” and our already-established memory of spatial environments.

faktiskstorsherlock - Virtual Reality Memory Palaces
“Get out — I need to go to my Mind Palace”. Benedict Cumberbatch in BBC’s Sherlock  — Quite busy in his Memory Palace. Illustration of by edthatch.

But what if we could perform this method with raw, fresh vision that gives us both the visual and spatial cues? Such a task is perfect for the medium of Immersive Virtual Reality. In VR, we are provided with highly optimisible a visuo-spatial environment made of software, which gives us the opportunity  to tailor the Memory Palaces to our own needs. By exploiting the visuo-spatial elements of the method of Loci in a much more explicit sense,  the method may perhaps work even better, or be more fun and easy to use.

Addressing this, I created an a VR memory palace app as part of my master’s thesis in Information Science. The application is called “Mnemosyne”,  after the Greek goddess of memory, of which country the Method of Loci originated. Developed with A-Frame, the application presents you with a text field where you can enter the things you want to recall. You then put on your VR headset, and are presented with an apartment with five rooms where you can place these memory items you entered. The algorithm returns images based on your text preferences from a search engine, and places these in the virtual environment. When this is done, you just walk around and watch to memorise the items.

Mnemosyne is still just a prototype, and we can imagine it will have room for many improvements in the future – after more research has been done. Currently, we at the Centre for the Science of Learning & Technology (SLATE) have performed a pilot study with 18 participants. The short paper discussing the findings was published in June 2018 in the Springer’s series «Lecture Notes on Computer Science», in the «Augmented Reality, Virtual Reality and Computer Graphics» edition. The conference paper was presented in the beautiful village of Otranto at the Salento AVR conference in July.

978 3 319 95270 3 - Virtual Reality Memory Palaces
Our first paper on VR Memory Palaces was posted in the LNCS on Augmented Reality, Virtual Reality and Computer Graphics.

Our paper, ‘Mnemosyne: Adapting the Method of Loci to Immersive Virtual Reality’, can be accessed at Springer.

If you would like a more practical, visual approach, check out our video on YouTube showing how you can create your own Virtual Reality Memory Palace on almost any VR headset here:

Literature list

Virtual Reality Journalism

Journalism is largely defined by which medium it uses to convey its message. The last hundred years, it has moved from medium to medium: from text to radio, and further from photographs to video. With each new medium the fidelity of the message it is providing is steadily increasing. This is perhaps especially clear now with the use of Virtual Reality (VR) technologies for journalistic purposes. By using 360° stereoscopic (3D) cameras, we are getting very close to capturing subjective realities at given points in spacetime.

The Ultimate Empathy Machine

In his Ted Talk, Chris Milk describes the potential of VR for creating “the ultimate empathy machine”, which later has been the subject for extensive debate. The message is that we may be more empathetic towards others if we can “literally”, or at least virtually, view the situation through their eyes. Many journalistic projects have focused on refugees or war zones, such as the stories “Fight for Fallujah” and “The Displaced“. In these stories, the camera works as the eyes of the observer, as the screen in the VR goggles is mounted directly on to the eyes. The user is presented to the story through a first person perspective, and may feel present in the story as if he or she is actually being transferred to the environment.  This presence to the stories, and the following perceived realism, is what is believed to be able to increase empathy in the viewers. Of this reason also, Google News Lab’s Ethnographic Study on Immersive Journalism  describes VR news as more fit for the term “storyliving” rather than “storytelling”, indicating that the user feels a part of the story that is being conveyed.

poster thedisplaced 774x1024 - Virtual Reality Journalism
‘See for yourself’: Is the NYT VR pitching their VR production by indicating that you can see the situation yourself, instead of adhering to the mediated version by a journalist who’ve been there?

Critics of VR as the “ultimate empathy machine”, or as capable of delivering “storyliving”, say that you can not possibly know how it is to be in a refugee camp while lying on the couch a friday evening with your VR goggles and a glass of wine. And, of course, you can not. It may, however, prove to be a more empathetic instrument than a regular video as it may seem more real and thus affect us differently.  It is no wonder that it may seem more ‘real’, when it is presented in the ‘format of reality’. Some do agree, however, and believe that it may have the power to make viewers more empathetic – nevertheless, they think that it may be unethical to use it as such. Should journalists have this power, to distribute realities to news consumers, and by new technologies we are not used to, affect people this strongly? These critics may be afraid of the “brainwashing” potential of the technology. It is easier to distance one self from a 21 inch screen than it is to distance yourself from an encompassing, immersive virtual reality you inhibit. Perhaps the most crucial question is whether it is brainwashing if what you are showed in fact is real?

Fake News

This brings us to another point of VR Journalism: how hard it is to manipulate the content. The journalist can not hide behind the camera. There is no artificial lighting or too narrow segment of the shot. The camera shoots in 360° horizontally and vertically — it is a totally observant witness at that given space and time.  In a sense, news in 360° is far less directed than in traditional flat videos: the viewer chooses which part of the video he or she wants to see. Naturally, the journalist is still an active role which chooses where to shoot, and does the final edit, however, it is a great shift from traditonal video footage. As we all know, during the last few years the concept of ‘fake news’ and mistrust in the media has arisen. The transparency that shooting with 360° 3D offers may help combat this. Perhaps also Journalism has to adapt to these changes, and rather deliver (immersive) content on which its readers themselves can decide and conclude upon. Euronews, one of the largest European news agencies, argues that this is why they have produced several 360° videos each week for now two years. In the VR session of the Digital World Conference in 2016, Editor-in-Chief of Digital Platforms, Duncan Hooper, stated they “want[ed] to let [the users] make their own decisions, not tell them what they should be watching, not to tell them what they should be thinking”.

It is, however, rather naïve to believe that immersive content alone can deliver objective truths — no matter how close the images correspond to reality. When the videos themselves lack in clear message or narrative, it is natural to imagine how they may rather be used as building bricks for constituting a narrative elsewhere. Besides, imagine the concrete example of the news coverage of the Israel and Palestine conflict. In this case, we may ask whether the journalist will choose to show immersive footage of a knife attack in Jerusalem, or deadly shots by the border patrol in Gaza? Both would be correct to show, but by this example, we see that to a certain extent, in the problem of news objectivity and fake news, it is not a problem of facts vs. non-facts; but which facts are focussed on. Immersive Journalism is no silver bullet in this regard, however, that is not to say that it may not find a natural place in news coverage. 

VR Journalism at the University of Bergen

During the spring of 2018, I taught 20 undergraduate students in VR Programming, 360° video shooting- and editing and photogrammetry. The aim was that the students should be able to create their own prototype delivering Immersive Journalism. As the rules and practices within the new concept is not very well established, we did not teach the students exactly to solve their tasks, but rather how to experiment with the novelties of the medium and try to innovate and create new genres. This is often called ‘Innovation Pedagogy’. The end result have been four brilliant prototypes, that was presented at the Norwegian Centre of Excellence (NCE) Media’s media lab in Media City Bergen. We discuss two of these here. Interested in the other two? These are mentioned in an entry where we go more in depth, philosophically, on the concept of Experience Machines.

Drug addict
The first of the VR experiences is called “Narkomani” which from Norwegian can roughly be translated to “Drug addict”. The aim of the production is to see the world from the point of view of a drug addict, perhaps living on the streets in Bergen. How is it to be frowned upon, walking around the streets, uneasy to get the next shot of dope? As my colleague Nyre stated in the introduction the projects, this VR project features “not a first person shooter from Los Angeles, but first person social realism from Bergen”.

Schizophrenia
The second of the VR experiences, attempts to create understanding on how it is to live with schizophrenia. In the experience, the user perceives visual hallucinations, and audio of up to five different personalities. The concept is brilliantly illustrated by the poster, and the experience tries to portrait a subjective reality falling apart.

Conclusion

Journalism through the medium/technology of VR has great potential. Immersive Journalism is still in its infancy, but the projects done so far shows promising. Much will depend on VR goggles entering into people’s homes, as with any other technology. For insights into where we are in the terrain of VR technology in 2018 — take a look at our entry discussing the History of Virtual Reality.

Literature list


Are you interested in more reading on Virtual Reality Journalism?

This executive summary on “Virtual Reality Journalism” by Owen & Pitt at the Tow Center for Digital Journalism is one of the first reports on VR Journalism.

Further, the report by Doyle, Gelman & Gill at the Knight Foundation is a good background read.

Finally, the Reuters report by Zillah Watson is more recent and sheds light on the more current situation of the medium for journalistic purposes. This report  illuminates a change we have seen recently, with the use of consumer/”prosumer” cameras for easier production by newsroom. This will definitely turn out, as with traditional cameras, to be a prerequisite for the adoption of this medium across Journalism as whole. When it is easier to produce content in 360° video, more newsrooms will do it.

As the reports by The Knight Foundation, The Tow Center, and further Sirkunnen et. al indicate, Immersive Journalism has not been so prevalent in less-affluent media houses. We may know of VR stories such as 6×9 by The Guardian, but have not necessarily heard of any from our local newspaper. This may change in the near future due to better consumer products.