Promise of Magic Leap AR now powered by NVIDIA

After four long, long years of development in which many in the outside world (myself included) doubted that the product would ever see the light of day, Magic Leap held an even this week to give some final details on its development kit. First, the Magic Leap One Creator Edition will be shipping “this summer” though nothing more specific was given. Pricing is still unknown, though hints at it being in the realm of a “high end smartphone” point to this as a ~$1,500 item.

For the uninitiated, Magic Leap is the company behind countless videos that appeared to show the most amazing augmented reality demonstrations you can imagine. The demos, some of which claimed to be captured rather the created, were so mind blowing that it was easy to dismiss them as fantasy. Through this series of live streams the Magic Leap team is attempting to demonstrate the capability of the hardware, leaving behind the negative attention.

Magic Leap showcased a demo called Dodge, in which the wearer combines the use of their hand as a pointing and action device with the real world. By looking around the room, a grid was projected on the floor, table, and couch, indicating it recognized the surfaces thanks to the depth camera integration. Using an ungloved hand to point and pinch (replicating a click action), the user is setting locations for a rock monster to that emerges from the surface in a fun animation. It then tosses stones your way, which you can block with your hand and push away, or move to the side, watching it floats harmlessly past – one time even hitting a wall behind you and breaking up.

The demo is a bit jittery and far from perfect, but it proves that the technology is real. And the magic of watching a stone thrown past your head and virtually breaking on a real, physical surface is…awesome.

The other new information released included the hardware powering the One. For the first time outside of the automotive industry, the Tegra X2 from NVIDIA makes its way into a device. The Magic Leap One requires a substantial amount of computing horsepower to both track the world around the user as well as generate realistic enough looking imagery to immerse them. The previous generation Tegra X1 is what powers the Nintendo SHIELD, and the X2 can offer as much as 50% better performance than that.

Inside the TX2 is an SoC with four Arm Cortex-A57 CPU cores and two more powerful NVIDIA-designed Denver2 ARMv8 cores. A Pascal-based GPU complex is included as well with 256 CUDA cores, a small step below a budget discrete graphics card for PCs like the GeForce GT 1030. This is an impressive amount of performance for a device meant to be worn, and with the belt-mounted design that Magic Leap has integrated, we can avoid the discomfort of the heat and battery on our foreheads.

The division of processing is interesting as well. Magic Leap has dedicated half of the CPU cores for developer access (2x Arm A57 and 1x NVIDIA Denver2) while the other half are utilized for system functionality. This helps handle the overhead of monitoring the world-facing sensors and feeding the graphics cards with the data it needs to crunch to generate the AR imagery. No mention of dividing the resources of the 256 Pascal cores was mentioned but there is a lot to go around. It’s a good idea on Magic Leap’s part to ensure that developers are forced to leave enough hardware headroom for system functionality, drastically reducing the chances of frame drops, stutter, etc.

The chip selection for Magic Leap is equally surprising, and not surprising. NVIDIA’s Tegra X2 is easily the most powerful mobile graphics system currently available, though I question the power consumption of the SoC and how that might affect battery life for a mobile device like this. Many had expected a Qualcomm Snapdragon part to be at the heart of the One, both because of the San Diego company’s emphasis on VR/AR and mobile compute, but also because Qualcomm had invested in the small tech firm. At least for now, the performance that NVIDIA can provide overrides all other advantages competing chips might have, and the green-team can chalk up yet another win for its AI/graphics/compute story line.

There is still much to learn about the Magic Leap One, including where and how it will be sold to consumers. This first hardware is targeting developers just as the first waves of VR headsets did from Oculus and HTC; a necessary move to have any hope of creating a software library worthy of the expensive purchase. AT&T announced that it would the “exclusive wireless distributor” for Magic Leap One devices in the US but that is a specific niche that won’t hit much of the total AR user base. As with other VR technologies, this is something that will likely need to be demoed to be believed, so stations at Best Buy and other brick and mortar stores are going to be required.

For now, consider me moved from the “it’s never going to happen” camp to the “I’ll believe it when I try it” one instead. That’s not a HUGE upgrade for Magic Leap, but perhaps the fears of vaporware can finally be abated.