First Drive: BMW M Mixed Reality M2 Is a Real-World Racing Sim

From motion-controlled VR rigs to Volvo's 3 million euro R&D chassis simulator, I've tested a wide range of racing simulator setups. Yet as close as the best get to replicating the feel of driving, all fall just short of the real thing. M racing and performance division's latest project comes at this problem from the opposite direction, bringing a real accelerating, cornering, power-sliding 2023 M2 sport coupe into the simulation using mixed reality technology.

I was able to test BMW's M Mixed Reality M2 prototype on a gray afternoon in late November at the M Driving Academy at Fürstenfeldbruck Air Base, just a half-hour's ride from the automaker's Munich headquarters. For the most part, this is just a normal 2023 M2 with a dark techno-livery. Opening the door, I spy the large stereoscopic camera pod occupying most of the dashboard and the mixed reality headset waiting for me in the driver's seat.

The headset is the Varjo XR-3, a pro-level mixed reality unit that features lidar-based 3D spatial positioning, passthrough cameras and hand and eye tracking. It's the latest version of the XR-1 rig that Volvo used a few years ago to test driver safety, and it's one of the most advanced VR/MR headsets that I've ever strapped to my melon. The XR-3 has been modified slightly with six small positioning spheres that further boost head tracking accuracy.

As I settle into the racing bucket seats and secure the four-point harness, BMW's engineer explains to me that by combining  virtual reality with actual driving, the M team is taking an inside-out approach to creating the most realistic simulator. There's also less motion sickness than with a fully virtual experience because your body (specifically, your inner ear) will be experiencing real-world physics. He also explains that he's got a remote brake pedal on his side of the car — like a driver's ed car — but promised only to use it in an emergency. 

Through the XR-3's displays, my vision is filled with live video of the M2's actual dashboard in front of me, complete with my real hands on the very real wheel. It's as if someone replaced my eyes with the MR headset's cameras. It's a little weird, but there's no perceptible latency. I'm instructed to shift into drive and move to the starting line.

Suddenly, the view outside the car changes. The gray sky and empty airstrip I'd lined up on disappear and they're replaced by a futuristic racetrack with a cyberpunk nighttime cityscape in the distance. As I move my head, the virtual world moves perfectly in sync; I can even lean forward or tilt my head to peer around the roof up at the night sky — at least, as far as the harness will let me. My real hands and the. M2's dashboard are still there — that' s the mixed part of this reality — but beyond them stretches a neon-lit course, marked by tall, glowing barriers. The large camera pod, I notice, has also seemingly disappeared. 

My co-driver explains that the elongated figure-eight circuit fits nicely within the paved runway — which they surveyed and GPS-marked with a drone at the start of the event much like how you'd mark the safe boundaries of room-scale VR at home. The system uses a combination of camera and laser head tracking, precision accelerometer data and GPS, along with data pulled from the M2's onboard sensor network, to accurately line up the vehicle's position and my augmented field of view.

Then it's "ready, set, GO!" as I'm instructed to take a slow lap to get familiar with the system. The moment I get rolling, all of the mixed reality weirdness disappears and I'm immersed in the sim. Muscle memory takes over as I pilot the M2 smoothly through the video game landscape. BMW added a bit of Mario Kart gamification to the course with BMW roundel tokens to collect to improve my score and large red digital barriers to avoid. It's fun. 

Of course, there are limitations to the tech. It can't create camber or elevation changes where there are none — a big parking lot is never going to feel like the Nürburgring's banked carousels or Laguna Seca's corkscrew. And so the two times this tech has been demonstrated to journalists have been flat simulations on smooth runways. The opposite is also true: If there's a bump or pothole on the real driving surface, you'll feel it — and if a puddle or oil slick develops or a deer runs into the driving area, you won't see it coming in the simulation, which is why the human co-driver and their emergency brake pedal are there.

Later, BMW's engineers excitedly explain their vision for using technology developed in the M Mixed Reality prototype to teach novice drivers performance techniques or to allow casual enthusiasts to test the real-world limits of a car like the M2 without the fear of putting it into a wall or wrapping it around a pole.

Expert drivers can also benefit from mixed reality driving. Say there's a tricky segment on the Nürburgring you want to practice. Load it up on a skid pad and hit it over and over again without the need to complete the entire Nordschleife between attempts. You could spend the day autocrossing with a different course, each run without having to lay out or pick up dozens of cones.

On my second lap, I push the speed a little further and the illusion remains unbroken. The car moves around the simulated course precisely, naturally following my inputs and matching perfectly with the seat-of-the-pants feel. Only the weight of the headset reminds me that I haven't been magically transported inside an arcade racer. 

The BMW M Mixed Reality prototype is a ton of fun, but for now it's just a development platform with no plans to produce or offer the tech to M car owners. At the end of my session, all that I really want is one more lap.