It’s been almost six years since Niantic released the augmented reality game Pokémon Go in the summer of 2016, but if you talk to people who played the game that summer, they’ll often speak of that time with a great deal of wonder. A student in the digital storytelling class I’m teaching this semester called it “magical,” and my own memories of that summer are glazed and golden. I remember the excitement of finding my first Pikachu, and the way that normally empty local parks were suddenly bustling. I remember organizing raid battles with strangers on social media, and then driving across town, visiting gym after gym with people I’d never met before, but whom I’d probably passed in the grocery store for years. Mostly, I remember walking and walking, sometimes alone while listening to music, sometimes with my dog or my partner, sometimes even making dates with friends to traipse across parts of the city, sharing our stories and squealing with glee at each new discovery. I moved differently in the world that summer, and it was all because of a game.
(The 2015 Pokémon Go Trailer)
For those unfamiliar with augmented reality, Pokémon Go is a useful entry point. Extended reality (XR) is a term used to describe environments that combine the real and the virtual, whether that be virtual reality (VR), or augmented reality (AR). In VR, digital elements mimic the experience of engaging physically with a world, usually through a headset, like the Oculus Quest. When players put on the headset, the real world is replaced with a fully virtual world, which players can interact with through physical motion, whether that be slashing at colored blocks with light sabers in Beat Saber, or simply mimicking the experience of sitting under a star-filled sky. In contrast, in AR, digital elements
are overlayed on or integrated into reality, with the goal being to merge the digital and the physical elements as seamlessly and realistically as possible. Companies (like Apple) are currently developing wearable technology, but for now, most people engage with augmented reality experiences through their phones or tablets. When I play Pokémon Go, I use my phone as I wander through my neighborhood to discover digital Dragonites and Magicarp, but many AR experiences are rooted in practical applications. For instance, furniture companies often have AR features that allow you to place a digital chair in your own living room to see how you like it, and apps, like Papago, allow you to translate foreign languages in real time by pointing your phone camera at menus and signs. AR experiences often also incorporate sound and interactivity and can respond to elements of the user’s environment. In Pokémon Go for instance, the app uses geolocation to track how far you’ve walked, and things like weather and time of day influence the kinds of Pokémon that appear in your location. At its best, AR blurs the boundaries between the digital and the real, influencing the ways we view the world and engage with it, or even (as with face filters on Snapchat and Instagram) shaping how we see ourselves.
(Heather Dunaway Smith “Adobe Aero Earth Day AR Artwork”)
Artistically, AR has a lot of potential to create unique and memorable digital storytelling experiences. In Heather Dunaway Smith’s Earth Day AR Art Experience, users place a digital image of a face in their own surroundings and then watch as the waters rise around them, an experience intended to create a fear response. Then, viewers can use their devices to interact with digital objects such as soda cans and plastic bags, triggering visual representations of the time it will take for each object to decompose. Dunaway Smith writes, “One of [the] superpowers [of mixed reality] is its ability to help us wrap our heads around large numbers…One of my goals was to allow people (myself included) to really SEE the lifespan of ordinary products. We are surrounded by (almost) immortal objects.” AR enables Dunaway Smith to draw viewers into the experience, triggering physical and empathetic responses, while also maintaining a close connection to the actual world. This narrative experience takes place in a setting that is familiar to viewers, thereby encouraging us to consider the objects around us and our own role in sustainable consumption. Nancy Baker Cahill uses a similar approach in her artistic installation “Mushroom Cloud,” an AR experience intended emphasize community connections in the face of climate change. In the Minneapolis-based AR installation “Dakota Spirit Walk,” Marlena Myles uses augmented reality to connect viewers to local Dakota culture, heritage, and history. As they walk along the trails of the Bruce Vento Nature Sanctuary, visitors interact with a series of AR art installations, which explore the role of nature in Dakota cultural practices and encourage visitors to consider indigenous perspectives of the land around them. As a narrative approach, AR storytelling has the potential to create a variety of experiences that are new, immersive, and difficult to replicate in any other medium, but one of the most profound is the potential for AR to alter the ways that participants interact and experience the space around them.
(Marlena Myles “Dakota Spirit Walk Promo Video”)
As a storyteller who is also a gamer, I spend a lot of time thinking about the ways that digital experiences can do the things that great stories can do, things like build empathy, broaden understanding, or help us to arrive at more complex ideas of what it means to be human. It seems to me, though, that AR narratives offer unprecedented possibilities for using digital elements to shape the ways that participants think of themselves and the world. There are a lot of frightful narratives
surrounding technology in contemporary society—concerns that XR experiences might distract us from reality, that the virtual world might be so compelling that we ignore the real one, that digital worlds might make us more isolated or less empathetic, or that companies might use virtual spaces to further corporate interests that are at odds with common values. AR interests me precisely because of the ways it both shapes and is shaped by real world elements. Playing a game like Pokémon Go encourages a person to experience the world differently, often in ways that have lasting implications. It encourages players to visit new parts of town, sometimes at unusual times of day, and often communally. It’s made me more aware of the statues and monuments and churches in my town, and it’s pushed me to make connections with people I would never have met otherwise. Further, it encourages conversations about the places we inhabit every day, raising questions about appropriate behaviors—whether it is okay to play a game like Pokémon Go in a cemetery, for instance—and pushing us to examine the degree to which we feel comfortable in certain spaces.
In one Pokémon Go group I participated in on Facebook, a woman raised concerns about playing downtown after her work shift ended, noting how she sometimes felt uncomfortable wandering the streets alone late at night. Her concerns prompted another player, a black man, to share how he too often played late at night, and how he was frequently approached by law enforcement, who wondered why he was loitering and were concerned he might be engaged in a crime. Conversations like these happened all over the internet, as people discussed the ways that privilege and fear influenced the different ways that players could engage with the game. The fact that these discussions happened on social media, about local areas that many of us felt we knew but hadn’t taken the time to consider how our experiences might be different, seems important to me. We weren’t considering gender and race and public places in the abstract; we were thinking about a specific park, and a particular street, about two people we knew, and a police force we supported with our taxes. These were our stories, and when these two players made a plan to team up and play together, it felt like we’d helped accomplish something, that small, personal actions could stand in the face of an overwhelming and impossible system.
Similarly, I remember an early evening raid caravan I participated in in Cincinnati. We were far out in the suburbs, in a place I’d never been before, and it can be difficult to tell in Pokémon Go exactly where a gym is located. This was how it happened that sixty players, almost all of us white, converged all at once on the Guru Nanak Society, a resource center for local Sikhs. We leapt out of our cars, intent on the raid battle, surprised to see two men hurry out of the building, fear bright on their faces. For a moment we all stood there, stunned. We’d come to battle Articuno, but suddenly we were confronted with something else—the danger we represented as a horde of white people, the privilege that allowed us to arrive on the grounds of a place dedicated to a culture that was not our own, the fear that we’d inspired, and the larger systemic hostilities that it represented. No one spoke, but the moment was thunderous and shocking. It was not lost on any of us. Eventually the tension dissipated. The men seemed pleased to learn that the society had been chosen as a Pokémon Go landmark, and they invited us to the festival they were hosting that weekend. We battled Articuno, and some of us managed to capture him. Then we moved on to a pizza place so we could fight Lugia. On the surface, we were playing a game, but the tension between the digital world and the real one prompted striking reflections about race and culture and our larger communities. It made us think carefully about where we feel safe and why others might feel differently. It prompted larger conversations about complex topics, while also rooting these discussions in specific local communities. Further, this experience happened among strangers, people whose main unifying interest was a digital game. It might be tempting to dismiss this encounter as another instance of digital technology prompting antisocial behavior, but doing so, to my mind, ignores a larger, more interesting conclusion, which is that AR experiences have the unprecedented potential to change the ways that participants think about and interact with the physical world, and that these interactions can have a lasting and profound social influence.
Over the past year, funded as part of the UWGB Strategic Initiatives, Dr. Chris Williams and I have been working on the Phoenix AR project, an effort to use AR to tell stories of underrepresented voices on our campus. Our ultimate vision for the project includes multiple sites spread across all four UWGB campuses and surrounding communities, but in this first iteration, our focus is somewhat smaller. We’ve been collaborating with Dr. Lois Stevens in First Nations Studies and with Foresight Studios, a Milwaukee-based, community-focused XR developer, to create a series of AR sites on the main campus that highlight First Nations culture and experience. These sites will be location-based, meaning that participants will need to travel to various sites on campus, possibly even places that are new to them. Visiting the Phoenix Statue, for instance, will allow participants to learn about the Menominee village that once existed on campus land, while visiting the foyer outside Common Grounds Coffeehouse will allow viewers to engage with a digital version of the land acknowledgment and to learn about Wisconsin tribes. The Phoenix AR installations are not a game like Pokémon Go. There won’t be bosses to fight or points to earn. Instead, we hope that the experience will highlight the potential for AR to tell stories that transform physical spaces, particularly in ways that amplify underrepresented voices and encourage participants to think more carefully about the ways that they participate in and connect with physical spaces. This year, we’ve spoken with many professionals working in AR, and all of them have remarked on how new this field is, how uncertain, how poised it is to transform the ways we experience stories and the world. It’s difficult to predict where we are headed, but perhaps we are only headed toward familiar places, to places we know well and feel comfortable in. Perhaps this emerging technology will help us to see these places and ourselves differently, will complicate the things we thought we knew, and we will never be the same again.
By Dr. Julialicia Case
Julialicia Case is an Assistant Professor of the Humanities and English at the University of Wisconsin – Green Bay, where she teaches courses in fiction, game writing, and digital media, and is a co-director of the UWGB Center for Games and Interactive Media. You can learn more about her writing and scholarship at www.julialiciacase.com.