Mixed Reality Escape Room — Yovanna Cifuentes-Goodbody

Overview

This mixed-reality (a blend of virtual reality and the physical world) interactive game immerses Spanish learners in the play “The House of Bernarda Alba” by Federico García Lorca. The game is an escape room simulation where, as a team, players have to uncover clues to break free from the house described in the play. Clues are hidden in both the virtual and physical worlds. To find all clues, one player wears a virtual reality (VR) headset and describes everything in the experience to their partner, while the partner finds matching clues in the physical world.

Problem and Solution

The SMEs, a team of Spanish literature professors at Occidental College, reached out to me because they wanted to incorporate 360° images of the house of Federico García Lorca in Granada, Spain, into their course on the author’s work. Specifically, they wanted an interactive way for learners to experience Lorca’s house after having read and analyzed his play in class. Given that this was a language class, they also wanted to make sure the activity required spoken interaction between learners.

The SMEs opted for an immersive experience (as opposed to seeing the 360° images on a 2D screen), but that meant learners would be wearing a headset, making it an individual task with no human interaction. My solution was to gamify the experience by turning it into an escape room, where learners wearing the headset had to work with a partner in the physical world in order to successfully escape. This not only solved the SMEs’ problem; it also fit perfectly with the narrative of the book, where characters are trying to escape the metaphorical house in which they’re trapped.

Tools used

Pano2VR
VRTourviewer
Padlet

Process

I set up weekly meetings with the SMEs for content development and check-ins. First, we identified the topics in the play that would be included in the game, and then matched them with spaces in Lorca’s house. This way, we could create clues that were tied specifically to the virtual home built from the 360° image. Given that the clues followed a particular order, I built constraints into the design that limited players access to only two virtual rooms at a time. This would keep players focused on the task rather than touring the whole house.

Storyboard

Next, I created a storyboard to help the SMEs visualize the order in which the game had to be played, the type of activities that would be integrated, and the interactions that would occur between learners wearing the headset and their teammates outside. This also helped me keep track of all the elements and props that had to be created for both the VR experience and its analog counterpart.

Development

Based on the storyboard, we started writing clues and selecting objects (both in VR and in the physical world) that would be part of the experience. I began by loading the 360° images into an app called Pano2VR. This software allows you to add interactivity to 360° photos, and I chose it because it allowed me to add important affordances like walking from room to room, spatial audio, and seamless integration with a VR viewer. To avoid breaking presence in the VR immersion, I edited the images to remove objects like fire extinguishers and “Exit” signs — anything that might make the player realize the images were from the 2020s and not the 1930s, when Lorca’s play takes place.

Prototype

I also added auditory and visual clues to the experience such as the sound of a church bell in the distance and textual clues above objects in the rooms. There were two main affordances I wanted the player to do: move from room to room and discover the written clues I had added. I used a visual cue for teleporting between rooms because I wanted players to have a sense of agency as to which room they would inspect. I made clues discoverable only when the player’s gaze hovered over an object because I wanted them to look around without giving away the answers. The first scene, that is, the first 2 rooms, served as a prototype to show the SMEs how the other scenes would work.

Prototype of the first 2 rooms played using VRTourviewer.

Testing and Production

Finally, we recruited volunteers from advanced Spanish courses to test the game. After incorporating their feedback, the project was deployed in two intermediate Spanish courses. As players completed the experience, we asked them to complete a survey. 75% of respondents rated the experience as excellent, and agreed that the experience had significantly helped them understand the play. 100% found the experience enjoyable, educational, and would recommend it to someone else.

Follow-up

Since I was present during all of the game sessions, I was able to take notes about the time it took teams to complete the game, the words they had trouble understanding or recalling, and the scenes where they needed help from their professor. Using this data, I suggested shortening clues. I also compiled a list of words that needed to be reinforced in class, and I rearranged the order in which clues appeared in some scenes.