Since the launch of Pokémon Go in 2016, and Apple and Google’s rollout of their respective augmented reality SDKs in 2017, conventional wisdom has noted that most consumers will experience AR first through the screen of their smartphone. Lenovo turned that expectation on its head with the recent launch of its Jedi Challenges product, which includes a Mirage headset into which you insert your phone. I’ve been playing with the hardware, fresh off a viewing of The Last Jedi, and while it’s a little rough around the edges and the content is very limited, there’s no denying that the experience offers a little taste of our AR-enabled future.
Lenovo, along with partner Disney, wisely chose Star Wars as the launch vehicle for this experience full well knowing that many of us would jump through quite a few hoops for the opportunity to wield a lightsaber. The $200 package includes the headset, a lightsaber controller, and a tracking beacon. The setup requires numerous steps, starting with the download of the smartphone app (I used an iPhone 8 Plus for my testing). The app walks you through each step, which includes calibrating the lightsaber and room tracking beacon, and placing the phone inside of a tray, sliding it into the headset, and connecting it via a cable. Setup was largely painless until the final steps, where I kept running into issues that required me to access the phone. The problem is, at this point in the process you’ve locked the phone inside the headset, which necessitates taking the whole thing apart again. Pro tip: Make sure your sound, whether through the phone’s speakers or Bluetooth, is set at the right level before you entomb the phone.
Once I finally got through the setup, however, things picked up. The headset is relatively comfortable, and the field of view is limited but sufficient. The opening animations looked good, and the presentation is polished. Finally, you are instructed to turn on your lightsaber. When that blade appears, you’d have to be a scruffy nerf herder not to feel a genuine pang of excitement.
I walked through the first set of training scenarios, battled a series of increasingly bold battle droids, and then faced my first “living opponent” in Darth Maul. Game mechanics are rudimentary but well-conceived, the game does a good job of keeping the motion in a well-defined space, the haptic feedback is excellent, and sounds seemed immersive and directionally accurate using my AirPods.
While playing Jedi Challenges is a hoot, from a technical standpoint the limitations of using a smartphone, a single controller, and a tracking beacon become apparent quickly to anyone who’s had the opportunity to use one of today’s high-dollar commercial AR headsets. Most notable, for me, was the fact that the lightsaber blade was often just a hair behind the movement of the hilt. Equally notable is the fact that nothing in this world is particularly tethered to the floor upon which you stand nor bounded by the physical walls of your room.
For gameplay experiences such as Jedi Challenges, these aren’t deal breakers. But for real-world AR devices must be able to detect surfaces if they’re going to inject digital objects into our physical world. It’s one of the things that Apple seems to have gotten remarkably right with the first iteration of ARKit, which manages to do the job using just one camera on the iPhone.
After about 20 mins of gameplay, I shut down the system and took the phone out of the headset and was astounded at just how warm it was running. While the Mirage headset only uses a portion of the smartphone screen, there is obviously a fair amount of graphics processing and battery usage occurring. I’ve experienced similar issues with smartphone-based virtual reality using Samsung’s Gear VR and Google’s Daydream VR, but the advantage of Mirage is that the phone isn’t radiating that heat directly onto your face. I also found the Mirage experience superior in that I could still see what was going on in the room around me, versus the total immersion required of VR, which dramatically decreases the likelihood of a lightsaber going through the family flatscreen during battle.
A View of the Future
It’s important to point out the AR issues and limitations with this product, but it’s also important not to get too hung up on them. A year ago, I expected my only day-to-day AR interactions to happen by holding my screen out in front of me. But now I can have a very basic head-mounted experience at home without having to spend thousands of dollars for a cutting-edge headset. Today the platform is quite limited, but I look forward to seeing what else Lenovo and Disney might show us through the Mirage in 2018. And I also think the product will also help drive developers’ imaginations around what’s possible with consumer AR.
In the near-term, commercial AR will continue to drive much of the headset development in the world as companies are willing to pay the high prices necessary to acquire the hardware and software needed because they see a clear path to return on investment. But with Lenovo’s Mirage and the likely stream of copycat products it will engender, it seems increasingly clear that consumer AR headsets will find their way to some mainstream users in the not-to-distant future.