‘Headspace VR’ Study Examines Audience and Empathy

By Erin Reilly and Amara Aguilar

Ajene in awe with his 1st time experiencing VR.

As 20-year old Ajene Harris slipped on a Samsung Gear VR headset for the first time at the University of Southern California Innovation Lab, his mouth dropped.

The USC communications major began to be immersed in the New York Times’ “Fight for Fallujah,” a virtual reality story that takes users to a war-torn area filled with conflict.

He watched the piece, along with seven of his classmates in USC Annenberg’s Advanced Journalism for Mobile and Emerging Platforms class.

The experimental journalism class focuses on 360 storytelling, artificial intelligence, mixed reality, augmented reality, chat bots, voice command devices and storytelling with other emerging technologies. Research is also an integral part of the class.

During the spring 2017 semester, the students in the class participated in a preliminary study that looked at how six virtual reality stories with the theme of war and conflict affect users’ emotions, specifically: Apathy, Sympathy and Empathy.

Most of the students had only experienced 360 stories on their mobile devices, and not through a headset, which triggered mouths to drop and “wows” as they realized immersing oneself through a headset was a completely different experience.  

“Because all of the stories focused on war and conflict — a theme that can feel quite far away and impossible to grasp for a person who has not personally experienced the conflicts of war — it seems that generating true empathy in a viewer is rooted in the development of some shred of a personal connection — a common ground — between the viewer and the characters in each respective story," USC student Emily Czachor said in a focus group.

The study, entitled “Headspace VR,” aimed to connect the heart to the head as students stepped inside virtual reality. It combined qualitative and quantitative data—including surveys, interviews, and data collection using neuro-technology, biosensors that tracked heart rate, and skin conductivity.

“Overall, from viewing all of these VR pieces, I learned that a VR experience through a headset makes a world’s difference than just playing that same VR piece online in a web browser,” USC student Moera Ainai wrote in a Medium post. “It is almost like an out-of-body experience in some cases. No matter how much I liked or disliked the storytelling techniques in each piece, every time I took off the gear, I would have to adjust back to the classroom setting I was actually sitting in the entire time. I felt real emotion, whether it was for Sidra or someone in solitary confinement, and that was what turned what was just a few minutes in a headset into a meaningful experience.”

Students critically reflecting on the
narrative style of the VR stories.

The results show that virtual reality breathes with story and character.  For many of the participants, it was more important that the journalistic stories were narrated in the third person point of view than from a live action main character in order to foster empathy.  Participants would rather virtually walk beside another person, than walk directly in that person’s shoes. Human characters were the lifeblood of the stories, according to preliminary research results.  

Post survey results showed “Clouds over Sidra” had the strongest storytelling quality and biosensor data affirmed participants had a significant higher emotional intensity with it than with the other narratives.

Many shared walking alongside Sidra, the main character, and hearing her story gave a sense of inhabiting the space and better identifying and sharing in the complex emotional feelings characters portrayed.  This, in return, stirred affective empathy in each participant.

“A personal narrative makes people care about the story and, thus, feel immersed and involved in the outcome of their lives. I could relate to the VR experiences and empathize with the characters’ struggle or living situation when I had a subject to latch onto emotionally,” Gabriela Vidal said.

However, the environment is an emerging leading character in a VR experience, according to the study results.  It is the heartbeat of a 360 story.  For example, though “Project Syria” didn’t resonate across all participants as immersive, it did have the highest emotional activation with the three narratives used to collect data with the neuromarketing biosensors.

Emotional Activation is events (something that happens) that instantaneously captured the attention. Project Syria correlation of all data shows that one moment (when a bomb goes off) had the highest emotional activation.  

In addition, though many participants realized they were in a classroom sitting in a chair viewing the story through a VR headset, participants shared that of all the varied media creating the VR experience, the audio took them to another place and made them feel lost in the moment. The more powerful VR stories had spatial audio, a technique that captures sound in all directions of the environment helping to direct the viewer’s attention in the environment.  From a medium perspective, audio is the key to ensuring a sense of “being there” and encourages the most emotional impact.  

“As far as empathy is concerned, I really liked when the narratives were genuine and used real characters or even audio to move the narrative,” USC student Tiffani DuPree wrote in a Medium post. “For example, I was able to empathize with the characters while watching Project Syria before I even knew the audio had been taken from real events. Somehow, just listening to the audio and watching the computer-generated characters in the story made me understand the story even more.”

So if the environment is the heartbeat of a VR storyworld and spatial audio helps to build and support emotional impact, then the old adage of KISS it (keep it simple stupid) may apply.  Creators have an entire toolset to pull from in shaping a story.  Yet, layering additional text, animations or additional video (ie: picture within picture) should be considered as unnecessary complexities to avoid. Being more simplistic with a focus on higher production value and quality continued to appear as important to participants. Students did not relate to animated characters as much as they did to cinematic ones, where they could see human faces in a real world environment.

”I felt as though reliance on graphics and animation to tell the story actually diminished its immersive quality, and served as a constant reminder to me — the viewer — that the world I was experiencing was, in fact, merely a simulation, rather than a replicated world of which I was a part,” Czachor said in a focus group interview.

Putting audience first

The study was done to better inform the news industry about audience engagement, best practices for journalistic virtual reality storytelling and to look at what motivates users to become more engaged citizens.

Many news organizations have recognized the importance of focusing on audience engagement. With this virtual reality study, we applied the Leveraging Engagement framework to examine in what ways audiences would take action after viewing VR stories.  What motivated participants behavior the most was an interest in watching (entertainment), immersing themselves in the environment for a moment (immersion), and at best, creating a personal connection to who they are as a person (identification).  

However, the experiences were limiting in providing enough time to really understand the topic (mastery) or material to get behind it and take action (advocacy).  With VR being an emergent technology, one would think that sharing it socially with others would be a strong motivator (social connection).  However, with the few participants that shared with friends, it was always in person rather than what we might assume, online via our social networks.  This means two things.  The sharing functionality within the VR environment needs to be further designed and developed.  And, the sharing in person with friends meant an importance of watching reactions over viewing it individually in a VR headset.

After watching “Clouds Over Sidra,” Anai wrote in a Medium post that her view on the subject matter was “heightened immensely” and it made her “want to look more into the crisis as well as share the stories and spread just how important this matter is.” She added, “It kills me that [I] alone can’t make a difference.”

It’s unclear that VR is enough of an experience to stand alone for viewers to take immediate action on the cause or subject shared.  Each time, the participants watched a VR story, we offered a printed Take Action worksheet with reminders and follow up links for four weeks after viewing.  This gave participants additional opportunities to further engage in the subject matter from watch it again, share it with a friend, read more, join, donate… all the way to connecting with the creators and distributors with write a note to download their app and watch more.

Erin discusses research with YBVR.

How we did it

The research project was led by ReillyWorks director Erin Reilly and professor Amara Aguilar in collaboration with YBVR implemented at USC’s Annenberg Innovation Lab.

Our methods of collecting data included surveys, video diaries, focus groups and interviews, combined with head-tracking heat maps.  The study examined six pieces. Below are the six narratives with descriptions pulled from each website.

• PROJECT SYRIA, United Nations, USC & Nonny de la Pena

"Nearly one half of Syria’s 23 million people have been displaced in its civil war and no group has been as severely affected as children. Children make up more than half of the three million refugees living in camps or makeshift housing and some news reports indicate that children are actually being specifically targeted in the violence."

• CLOUDS OVER SIDRA, United Nations, WithIn

"Meet Sidra. This charming 12-year-old girl will guide you through her temporary home: The Zaatari Refugee Camp in Jordan. Zaatari is home to 130,000 Syrians fleeing violence and war, and children make up half the camp's population. In this lyrical VR film, Sidra leads you through her daily life: Eating, sleeping, learning and playing in the vast desert city of tents."

• THE FIGHT FOR FALLUJA, NY Times

"Embed with Iraqi forces as they retake a city from ISIS – and experience the battle's aftermath."

• ZAMBIA, GIFT OF MOBILITY, StoryUP / The Washington Post

"Shot from the perspective of people who live on the ground in Zambia, this 360-degree video provides a unique immersion into what it’s like to live there without adequate mobility, and the obstacles endured trying to navigate the country’s rugged terrain. But it also highlights one solution that is, quite literally, lifting people such as Emmanuel Chilufya off the ground."

• 6 X 9: A VIRTUAL EXPERIENCE OF SOLITARY CONFINEMENT, The Guardian (Abbreviated Version)

"What’s it like to spend 23 hours a day in a cell measuring 6x9 feet for days, weeks, months or even years? 6x9 is the Guardian's first virtual reality experience, which places you inside a US solitary confinement prison cell and tells the story of the psychological damage that can ensue from isolation."

• PEARL HARBOR ATTACK RE-ENACTMENT, VRtually There, USA Today Network

"Get an inside look at an infamous Japanese midget sub and experience a stirring Pearl Harbor reenactment."

By far, participants favored “Clouds Over Sidra” as a virtual reality story and experience. Participants felt they could relate to the main character and almost feel as though they were right next to her experiencing her daily challenges and triumphs as a refugee. The story was told using only the main character’s voice, with nearly zero text overlays or graphics. Participants did not react as favorably to the Pearl Harbor piece because many said it was a piece that could have been done as a traditional documentary, instead of a virtual reality experience. Participants also said the Zambia story was too much like a broadcast piece and “Project Syria” was too much like a video game.

“Two of the narratives, however, were not very engaging,” Dupree wrote in a Medium post. “I was happy when they were over. Zambia: Gift of Mobility and Pearl Harbor were not narratives that did well with ongoing engagement. Pearl Harbor felt like a museum piece and the storytelling, transitions, and commercials were not appealing to me. Zambia: Gift of Mobility felt a lot like a traditional broadcast piece. The 360 aspects looked like something the producers did as an afterthought.”

Coming full circle (360 degrees)

This small pilot study was to establish the methodology and test out technology with a small set of participants, so there’s more research to be done, and students want more of this type of applied learning. However, we came away with some tips to improve VR stories which correlate with core attributes to being mindful and fostering empathy.

• STOP - This is not traditional filmmaking so stop trying to recreate what you can do in other mediums. Be open to what VR has the potential to be. You have this moment in time for the viewer to lose oneself and shift focus from real life -- take advantage of it by exploring all senses and not just what the eyes can see.

• LOOK - Allow things to be what they are and let the sense of space speak for itself.  Don’t try to overproduce with different techniques or bells and whistles.  For the pilot participants, less is more but focusing in on making the details of your chosen media work best is important.

• ACCEPT - Accept this reality and be in the moment to grab the viewer’s attention and turn this moment of empathy into compassion that’s long-lasting.  Without turning the experience into an advertisement with lots of pop ups, consider subtle triggers at the close during credits to extend viewer’s attention directly after the experience.

Tiffani learned many ways VR can be effective and will use
this knowledge in her future VR pieces she creates.

This pilot study gave students in the Advanced Journalism for Mobile and Emerging Platforms class the opportunity to look at virtual reality critically from a user experience point of view. They also were trained in 360 storytelling and were able to apply what they learned as audience members to their storytelling in the field. We hope this preliminary study will help others interested in VR storytelling to do the same.

“From viewing all these VR pieces, I have learned the many ways 360 and VR can be effective in telling a story and portraying aspects of a story that viewers were not able to experience before. I learned that if done correctly, a VR piece can really tell a powerful story or leave the viewer thinking about what they had just experienced,” Dupree wrote in a Medium post.

-----------------------

Read the full “Headspace VR” report here.

For more Medium posts from the course, visit: https://medium.com/journalism-and-emerging-digital-innovation

Photo credits: Amara Aguilar

What is Distributed Cognition?

I posted this originally on New Media Literacies blog, and thought I'd re-post it here for you. This gives you an idea of what I'm currently working on at work.

This post is pulled from NML's 6th webinar from our monthly series on the new media literacies. What is Distributed Cognition? was a big success! I'd like to share with you the presentation we gave for those of you who couldn't attend. This presentation was created and made possible through the collaboration of Henry Jenkins, Katie Clinton, Vanessa Vartabedian and myself. Over the past few weeks, we came together (via Skype and email conversations) to reflect on what was written in the white paper and to further explore what distributed cognition means and how to foster this new media literacy with educators and students.

-------------------------------------------------------------------------------

We define distributed cognition as the ability to interact meaningfully with tools that expand mental capacities. One of our past webinars focused on the new media literacy, collective intelligence -- the ability to pool knowledge and compare notes with others toward a common goal. Collective intelligence focuses on the ability of humans working together and is a complementary skill to the new media literacy, distributed cognition which can push our notion of pooling knowledge and expanding our capacity to include not just humans but the tools we use in sharing and expanding our knowledge.

In having us talk more deeply about distributed cognition, I want to share that I think this literacy is different than the others in NML's list. For one, we saw that it wasn't a skill educators and students gravitated towards as an entry point in beginning to understand the new media literacies. Perhaps, its because distributed cognition is more of a philosophy of mind, meaning its ever present in something we practice, an unconscious practice that we're hoping more people become aware of. It's different than our other nmls, for example transmedia navigation that is more tangible and applicable. Including distributed cognition, as one of the new media literacies is our tipping of our hat to the education research that we think the new media literacies aligns with and a chance for you to better understand what cognition is in the 21st century.

To better understand distributed cognition, the first thing to grapple with is --What is a Tool? In the 21st century, our minds might immediately go to the digital technology that has become an extension of ourselves and provides us with the ability to sample music, capture video, and edit media to socially construct meaning of the world. It is these tools that are talked about and are becoming the tools that we are comfortable with in shaping our idea of the world.

However, if you look back in history, you can see that the tools of today were not available back then, and so in thinking of the definition of distributed cognition, we need to broadly define the word tool as a device used to communicate, perform, make or facilitate. These devices work in conjunction with our mental capacities, a combination of "hybrid systems" interacting with one another. These tools can take many forms of externalized memory. For example, a database holds a lot of information in one place and alleviates humans from having to remember or store all of it in their own brain. We all can't be Rain Man but we can work with databases to remember large quantities of information, and free our minds to be used for other things - such as asking the right questions when we are analyzing that data.

Or we can use the tool to do work with us in gathering new information - like Facebook or Wikipedia, or the periodic table. We use these tools to expand the pool of knowledge we access. The ability to use these tools becomes increasingly important as the amount of information available to tap into becomes bigger and bigger! An example of this is the spell checker. We work with the spell checker to check our spelling. If we were to take everything the tool said at face value, than we wouldn't be using it at its capacity and our spelling wouldn't be right all the time, especially if you take into account the different ways to spell words like their or two. The spell checker shouldn't be seen as just a crutch; it can support our learning, especially if it's used within writing that the child is engaged in. It offers an intrinsic goal of aligning learning how to spell with something the child is interested in.

Henry Jenkins admits that he is a terrible speller and has learned how to spell words after the spell checker has caught his mistakes on many occasions. It's the reminder of being shared this new information by the spell checker that has him fix his spelling errors going forward and he's learning the words in conjunction with the subjects he is passionate about.

Language is a central tool in intellectual activity. We can think of language as a tool. Take for example a book you read. I'm currently reading Harry Potter to my son who is decoding and making sense of the story through performing as Harry Potter in the backyard as he makes up his own wand tricks or draws pictures of quidditch matches. This sense of play helps him to better understand the stories we read together. Or you can look at the millions of young fans who've joined communities, like FictionAlley to chat in detail about every character, and who have written fan fiction to extend the stories of many scenarios in Harry Potter. All of this doesn't happen in isolation. It is a cultural and social practice that uses the delivery technologies available today to be understood and remixed by others.

Lisa Gitelman offers a model of media that works on 2 levels. 1) The Tools that enable communication, like a television and 2) the Protocols, which are the social and cultural practices, that grows up around it. These protocols include the shifting content and the changing audience.

Do you own one of these tools?
* Record player
* Radio
* Still Analog Camera - like a Polaroid
* VHS

If so, how are you using these tools?

Are you using them with their original intent (such as a record player is a place to listen to music) or have you looked at today's social and cultural practices and shifted the content with the changing audience. DJs sampling music with the record player is an example of the tool taking on new protocols.

I don't think that when the Internet began, they foresaw how people use it today. It's those that used distributed cognition and asked the right questions to encourage new protocols to emerge, like social networking, and new audiences, like Cosplayers, to find their niche in this delivery mechanism. So as teachers, a key attribute to foster is to look at the tools that you have available and ask new questions on how they can be used to meet your learning objectives. But this doesn't have to reside solely on the teachers, What do you think would happen if you offered your students a tool and ask them what they would do with it without a pre-determined learning outcome?

As new tools develop, society shifts to include these new tools to tell new stories; and yet the old tools remain and take on new purposes than what they originally were for. The medium that tells the story may shift and the audiences that participates in that medium also shifts... so it is in our classrooms similar with new tools and how we engage our audience, the students, in these new practices.

But distributed cognition doesn't just use language as a tool. Using tools allows us to look at bigger problems and expand our knowledge because we now have new capacities to do things that were not available to us before.

Are you wearing a watch?

You might not realize this because it is second nature to us now, but telling time is a form of distributed cognition. How many of you are wearing a digital watch versus an analog watch where you have to know how to read the clock hands?

Before we had clocks, people still knew the time of day. Perhaps it wasn't exact as we have it now, down to the millisecond but through reading the sun and having sundials, they knew if it was mid-day or evening. The watch, whether it's the clock on our computer or the watch on our wrist or because our favorite television program is on tells us the time and we believe it. We take it at face value and we don't consider the cognition that is happening in that tool to keep track of that process for us.

Do any of you know Ken Robinson, he often speaks at Ted and he tells this great story about his 20-year old daughter who doesn't understand why anyone would wear a watch, it only does one thing - tells time. Think about the young people in your life ...do they wear a watch? More importantly, when they ask what time it is in class, do they ever look up at the clock on the wall? There are so many ways now to find out the time of day ...it's often embedded into many of the new tools that we have at our fingertips. This understanding of what time it is becomes second nature because there are so many means of gaining this knowledge.

As I was thinking through this presentation, I decided to ask my son, what time it was? On his own, he knew to just read the digital time on my computer, but then I asked him to read the watch on my wrist and he didn't understand why I was showing him that way when he could gain the information another way. So much for that watch I bought him for Christmas, I don't think he'll ever want to wear it. But should I be upset that he might not learn how to read an analog clock? Here is a new example of a shift in our culture. New tools have given way to letting go of old tools - and that thinking is still being done for us by a tool.

There's a healthy, on-going debate that relates to distributed cognition. People who study distributed cognition are not technological determinists nor social determinists. They're interested in what cognition is. In all versions of the theory it is about how people and tools act together to accomplish thinking and doing. The debate for them is in whether the "essence" of cognition is solely in people's heads or whether it is actually *in* the integrated brain-body-world systems of extended processing, including combinations of thinking with our brain, physical gestures, our relationship with others and the objects we use.

When I joined NML, I came because in my role as a practitioner, I had gotten to a point where when I created new ways to engage children in their learning, it worked and they loved it. I knew using social media as a tool for learning was powerful but I wanted to better understand why it was working. In taking up the new media literacies, distributed cognition was not one that I actually understood at first. It was one that I had to grapple with over time. But having done that, I think that this new media literacy is part of answering the why in my initial questions that put me on this track in my career.

This is a list of guiding ideas for how NML uses the term, distributed cognition:

* Technologies don't change how we think; it's how we use technologies that changes how we think.

McLuhan notes, technologies are often put out before they are thought out and thus they can have unintended consequences. These consequences can also be good, especially when people see beyond the original intent of a tool and use it for new benefits to humanity.

* Even when technologies can do many things, it's humans who are at the helm and who need to take responsibility for their actions.

At NML, we worry about the tendency to make causal claims about how technologies change how we think. It bothers us when we read articles that say technology makes us dumber, makes us more violent, and so forth -- including Obama's recent mistake in his address to students graduating. I think right now there is a far greater tendency to exaggerate the impact of technology than to under-estimate it. Humans for a range of reasons want to deny their own agency and accountability for their actions where technologies are concerned.

* Human nature is dynamic; as we act within new sorts of extended networks of people and tools, human nature, itself changes.

Looking at the history of storytelling showed that what it is to be human can be qualitatively different in different eras and tools play a key role in enabling change (always keeping in mind that it is how humans use the tool that matters).

* We need to resist a humanizing of technologies; machines have capacities, not minds.

Technologies do have agency though they have capacity only in regards to what a human does with it. This idea is the one we most debate internally at NML.

* Think holistically and be aware of the entire system.

Work to identify and understand everything that is contributing to the process of thinking.

Participants in the webinar added to the discussion and I encourage you to listen to the archived recording in Elluminate to hear the whole discussion that happened. Also, feel free to pass this link along to others: https://sas.elluminate.com/p.jnlp?psid=2010-07-15.1632.M.ABCA45F9F0C0248640F8705DA5109D.vcr&sid=voffice.

But most importantly, from reading this -- can you identify distributed cognition in your life? If so, share -- I love to have more examples on this topic.