The best augmented reality I saw this year was live theater funded by a liquor company

Commentary: Whiskey-sponsored immersive theater. Snickers escape rooms. Immersive branding is just getting started.

Scott Stein Editor at Large
I started with CNET reviewing laptops in 2009. Now I explore wearable tech, VR/AR, tablets, gaming and future/emerging trends in our changing world. Other obsessions include magic, immersive theater, puzzles, board games, cooking, improv and the New York Jets. My background includes an MFA in theater which I apply to thinking about immersive experiences of the future.
Expertise VR and AR | Gaming | Metaverse technologies | Wearable tech | Tablets Credentials
  • Nearly 20 years writing about tech, and over a decade reviewing wearable tech, VR, and AR products and apps
Scott Stein
8 min read
Scott Stein/CNET

It began in a bar.

The experience -- a project called Behind the City -- asks for two people, by name, no more or less per appointment. My friend and I meet down in Manhattan's Tribeca neighborhood, and walk to our designated address, a bar. With a check-in desk. The Macallan logo — the Scotch company that was sponsoring the entire affair — is projected on a wall. Dim lighting pools around quiet tables. A woman types poetry on demand in a glass-walled vestibule. We order cocktails -- provided by The Macallan, of course -- and wait.

We're eventually called up, led downstairs, and enter a basement room past a kitchen where we wait longer, with another pair of people. Photos and unknown objects are everywhere. We're served another drink.

Then, we are led to a massive refrigerator. It's not on. An office is inside. A woman in pristine color-coded clothing that matches the desk is poring through papers. We are given things to hold in a bag. Told a story. I am separated from my friend, and led down a hallway... and here, things get fuzzy. Time gets fuzzy. Now I'm with a stranger, one of the two in the room. We're led up an elevator into the sunlight of the city at dusk, and someone meets us. We're taken for the beginning of a long tour.

Scott Stein/CNET

World bleed

Welcome to the world of immersive theater. There is no HoloLens or Magic Leap-style digital headset technology here -- this version of "augmented reality" is much more literal: You enter into a voluntary illusion, an experience where you're both a viewer and a participant. 

It's a world I've been to before, but not everyone has. Unlike escape rooms, self-contained multiplayer puzzle boxes that have spread to thousands of local spaces, immersive theater's best performances tend to happen in larger cities and over short spans of time. In this case, my experience lasts about as long as a movie -- two hours or so -- and extended over several blocks of downtown Manhattan.

The most fascinating shows are easy to miss. VR companies like Oculus are already trying to take immersive theater to living rooms, but the best experiences still remain low in tech and very site-specific, staged in real-world places, and kept mostly phone-free (I turn off my phone during shows).

Created by Third Rail Projects, a New York-based theater company that made the acclaimed and still-running intimate immersive theater piece Then She Fell, Behind The City was an open, relentlessly coordinated work of impossible time bends. I entered rooms in random buildings. Had snacks from strangers. Listened to odd messages. Wore headphones that directed me to places. Sat in lobbies. Stared up from the street into windows where performers acted, and listened on my headphones from below. Plugged wires into strange circuit boards with a group of operatives that claimed to send messages through time.

I went through parts of the real city, waited on street corners and in hotel lobbies for events to happen, had conversations with a stranger, a fellow audience member with whom I walked for blocks and blocks, and who I realized halfway through lived in the same town as me. Moments happened in ways that felt like reality was now in flux. Was that movie theater in the hotel always there? Who is that person across the street? Is my companion a performer, or another "audience" member?

Third Rail Theater Projects

In a year where I've seen a lot of attempts at making augmented reality become a magical thing, this theater piece was probably the best augmented-reality experience of all.

The aforementioned Then She Fell, produced by Third Rail Projects and still running in Brooklyn, was -- in many ways -- the best VR I experienced in 2016. In similar ways, Behind The City stands as an amazing and improbable demonstration of the possibilities of realities blending in AR.

"Then She Fell is to virtual reality as Behind the City is to augmented reality," agrees Zach Morris, co-artistic director of Third Rail. The way the performance layered live theater on top of the living streets of New York City, it caused a blend of realities. It bent time.

"One of my co-artistic directors has a great term she calls 'world bleed.' Most of the time you're trying to avoid world bleed, you're trying to build a 360-degree terrarium, and you don't want the outside world getting in. It's a bummer if someone pulls out an iPhone, or a fire truck goes by. We realized early on we wanted to embrace that."


Maps projected on maps: part of Behind The City's experience, in a strange room up a long flight of stairs.

Rebekah Morin, Ethan Covey Photography

Funding improbable art

The show's gone now. It ran for just a few weeks, with a limited run of tickets given away for free. It was also a sponsored work, a piece of funded art completely paid for by Macallan.

It was the most incredible, unusual theater piece I've seen this year, and also, maybe, a harbinger of where immersive branding might be heading next -- for pop-up, unexpected, interactive experiments that walk a fine line between entertainment and advertising. The show stood as an impressive experience on its own. Does the fact that it was sponsored infiltrate my memory, in some way? Does my sharing of the experience act as a form of advertising, too?

We're entering a weird world where immersive experiences are being sponsored by brands. It's not even the first immersive theater experience created by an alcohol brand: Bacardi had an immersive theater performance about the history of its rum earlier this year in New York, hiring a Cuban-American playwright to tell the story. And way back in 2011, Stella Artois commissioned Punchdrunk, the UK-based creators of the long-running immersive show Sleep No More, for a free immersive project called The Black Diamond.

Third Rail Theater Projects' Morris explains how Behind The City came about: "The idea was hatched sometime early last year when I was thinking about what it would be like to create an experience with just two people that you could gift to someone," he says. "I loved the idea of what it might mean to create something for just two people."

Behind The City was a performance spanning blocks of New York City, taking place in rooms in separate buildings, hotel lobbies, and performers coordinated perfectly. It was even more amazing because it was intimate, made for miniature audiences of two at a time, like a bespoke performance.

"At the end of the day, we realized that this project was beautifully improbable. Were we to sell tickets for it, they would be thousands of dollars," Morris says of the show.

Third Rail Theater Projects doesn't have current plans to restage the experience, mostly because its scale is so improbable and expensive. But Behind The City might indicate where things could be headed for future performances. "We're interested in exploring different modes in creating work: what are the alternate ways that work might be supported in this cultural moment, especially in the wake of so many of our arts funding being systematically dismantled," says Morris. "We settled on The Macallan underwriting this really improbable project and offering tickets free to the public, and also making The Macallan speakeasy, a gathering place in its own right."

The brand behind the curtain

Getting a brand and an artistic company to coexist isn't easy. Morris said the goal was to "be very transparent about the relationship: they're sponsoring, and the scotch is over here, and the art is over here."

The Macallan's Brand Director, Samantha Leotta, explained it in terms of an arts commission as well: "In that spirit, we have sought out ways to celebrate artists who are making work as remarkable as The Macallan itself."

But it also won't be the last immersive experience to come from The Macallan. The brand's exploring its own immersive advertising: a pop-up immersive space at New York's Grand Central station, coming later this month, looks to be a headset-free holodeck-type way to experience the company's distilleries, a possible model for what might get installed at retail stores.

The beautiful part of Behind The City was its success as a pure, artistic theater experience. And, yet, even that experience had its hidden branding moments.

At the end of the performance, when I found myself in the dim lighting and holding a beverage at the speakeasy bar, drinking a scotch cocktail, my friend and I were approached and invited to try an additional experience. Would we be up for joining her? I followed the woman who invited us, and we found ourselves in a small room with its own tiny bar. My friend and I sat on stools as the woman, a host at the experience, started to tell a story of how The Macallan whiskey was made. We saw tiny dioramas of forests, and held pieces of wood. We were invited to sip a bit of a new variety, Rare Cask. We held our glasses, indulged in a small micro-tasting, a piece apart from the show.

This part had a separate name -- Behind The Secret Door -- and was researched and performed apart from the rest of the theater performance.

I think it was my first time in an immersive advertisement.

Scott Stein/CNET

Year of the Snickers Escape Room

Another experience, just a few days later, reminded me that these immersive experiences are only increasing.

Snickers set up an out-of-the-way, one-day-only escape room just a day after my immersive theater adventure: The Hunger Bunker. I went, because I like escape rooms. The pop-up was a clear, obvious advertisement for new Snickers flavors. The escape room was a 10-minute challenge, set up in an enclosed area designed to look like a vault in the warehouse-size space that was laid out in industrial decoration and lots of bowls of mini Snickers.

I solved the escape room challenges with two "influencers" and few employees of Mars. I watched a video on an old TV screen in front of a sofa, that explained our mission. We put together pieces of tubing on a puzzle-table. We answered trivia questions. We put our hands in mysterious holes to activate buttons. We eventually beat the clock and won the challenge.

It wasn't a great escape room, but it was free. Josh Olken, brand director for Snickers, said that while this was a one-off experience, more things like it could come. If a thing like this ended up in a mall or a park or an airport -- well, I'd probably enjoy playing.

All the world a Westworld?

Brands as artistic sponsors, to help create new strange and wonderful immersive art pieces? I like that idea, for art's sake. Or, are we headed towards pop-up branded events, half-entertainment, half-ad? We'll probably end up seeing a mix of both, in VR, AR, and in real spaces. We're already seeing it: VR promotional apps, AR advertising, immersive pop-up venues and "Instagram Traps" (Casper's pop-up nap store, most recently).

Behind The City was a great example of the best we can hope for. But I'm also reminded that a lot of Snickers-like escape rooms will probably be the result, too.

The best VR I experienced all year had no tech at all: How an immersive theater experience blew away any headset to date.

See Super Mario through a HoloLens AR headset: What it's like playing a mixed-reality version of the classic Nintendo game.