A defining feature of augmented reality is how much the technology’s promise has outpaced its real-world use. Despite years of glossy demos, AR’s predominant consumer experiences are still Pokémon Go and an iOS app that measures things. Advancing the medium will take not only impactful hardware—like Apple’s U1 chip and loudly rumored glasses—but innovative games and experiences. The kind that Punchdrunk, the theater company behind immersive shows like Sleep No More, knows better than anyone.
Now Punchdrunk and Niantic, the developer of Pokémon Go and Harry Potter: Wizards Unite, are announcing a collaboration that will span multiple projects. Details such as what the debut effort will look like, when it will launch, and what it might cost remain a closely guarded secret, but as far as crossover episodes go, the cast of characters looks encouraging.
Punchdrunk is no stranger to immersion. Sleep No More, the group’s signature piece, invites audience members to walk through a series of rooms at their own pace, in whatever order they like. The actors perform hour-long loops; as you wander, you cobble together their story fragments into a whole.
The trick of Sleep No More is to give the viewer a sense of free will while maintaining an environment that does not respond to their choices; immersive here does not equal interactive. The performance is fixed, regardless of what the audience does.
“Although it feels spontaneous, everything is meticulously choreographed,” says Punchdrunk founder and artistic director Felix Barrett (no relation). “Our process even at the most transitional beats is completely set. With that sort of precision, and the ability to set and rehearse different scenarios … It’s those sorts of learnings, almost like binary coding in a way, that enable us to think about how one could use that information.”
It’s no stretch to see how that philosophy might extend to AR. Punchdrunk has already dabbled in virtual reality, with a project for the Samsung Gear VR called #believeyoureyes, as well as mobile gaming with Silverpoint, an app that directed players around the streets of London.
“Quite early on I became fascinated in the potential for game mechanics,” Barrett says. “We’ve scratched the surface with a few projects, but I always knew that for an audience to have real agency over the landscapes they traverse, if they could actually grant them that, they could become the hero of their own living game, their own movie. It felt like that was the next hurdle to cross.”
Working with Niantic gives Punchdrunk a leg up. The company has spent years mapping a digital layer on top of most parts of the world, ever since the early days of Ingress, its first game. This rich AR infrastructure is the engine driving its subsequent hits, Pokémon Go and Wizards Unite, and it’s what Punchdrunk will tap into for their collaboration, whatever form that takes.
“There’s so many beautiful and interesting and inspiring things out in the world, and thrilling and mysterious things, that we’re programmed to ignore,” says Niantic’s CEO John Hanke. “The goal of a project like this, in my opinion, is to add those points of interaction and that layer of story and gameplay that really causes you to pay attention all the time. It’s not the very particular interactions that this thing is going to deliver, it’s that state of having your eyes open.”
The math here does not quite seem to add up: Can you really make people more in tune with their surroundings by funneling their experiences through a phone? Again, hard to say, given how few details are currently available about the collaboration. But Hanke describes an experience that does not depend on keeping your eyes locked on a screen.