How Apple might Deliver AR on the iPhone

Tim Bajarin / March 1st, 2017

If you follow the world of tech, you know that two of its new big things are AR and VR. VR got a major push with the introduction of Oculus Rift and made Oculus a household name once Facebook bought the company. Since then, HTC’s Vive and Sony’s Playstation VR have delivered VR headsets and Microsoft is working hard to deliver a mixed reality solution around HoloLens. Samsung and Google are both trying to deliver a VR experience through smartphones aided by low-cost VR headsets. At the moment, VR is mainly aimed at gaming but it has had some buy-in with vertical markets such as travel, entertainment, sports, and even advertising. However, VR is in its very early stages and requires a head mounted display and will take many years to get into the broader consumer market.

Last year, a large consumer audience was introduced to AR via Pokemon Go, which allows for characters to be superimposed on real life settings as part of the game. This game gave consumers a small taste of what AR is about and has left them wanting more of this technology on their smartphones.

Google has realized the smartphone is an important vehicle for delivering AR and has created the Tango AR platform that is currently deployed in Lenovo’s Phab 2 Tango phone and will soon be in other smartphones as well. In this case, the AR experience is just delivered on the smartphone and the Tango platform is designed to help develop AR apps for use on Tango-supported phones. The Tango AR platform is still in its early stages and few Tango apps are even available to take advantage of this platform. But it is an important AR program for the Android crowd and needs to be watched closely to see how Google and their partners use this platform to bring AR to more Android phones in the future.

There is a school of thought that says the best way to deliver AR is through some type of glasses or goggles. In the end, a mixed reality set of eyewear will be the best way to make VR and AR deliver on the promise of bringing this technology to the masses. The problem is this eyewear is expensive now and, especially in VR cases, they have to be powered by a PC with a graphics card to get the full effect.

Tim Cook has repeatedly stated Apple sees AR as the more interesting product at the moment and, while not discounting VR, he seems to suggest that, if Apple does get into this new area of VR and AR, AR will be the technology they will drive first to their platforms.

There have been rumors Apple is working on a set of glasses that could be part of their AR solution but, even if they have this in the works, I just don’t see that coming this year, or even next year, given the costs and lack of AR-based apps to support them.

If Apple decides to bring AR to iPhones soon, I believe Apple’s initial move into AR will be at the platform level and delivered on some next generation iPhones. This is just speculation on my part but it is highly plausible Apple tackles the AR opportunity by creating a special AR SDK kit for iOS that takes full advantage of the two cameras in the iPhone 7 Plus and, most likely, will be in some new iPhone models they release in the fall. There are additional rumors Apple has a special 3D camera coming in some high-end models. If that is true, this camera may also play a key role for user-created AR content on this special AR platform.

By creating an iPhone that supports a special AR SDK, Apple could be well positioned to expand the idea of AR-based apps and features to millions of users almost overnight. Like other SDKs of the past, first generation AR apps could be pretty straightforward and, like Pokemon Go, allow a person to just place virtual objects or specialized information on top of a live image. Imagine going into a museum and pointing the iPhone at a woolly mammoth and seeing information about this animal on your screen. Or, if you are in NYC and have the Empire State Building in your view, you point the camera at it and see data about its dimensions or info on its history.

It could utilize the cameras in innovative ways for anyone to create specialized AR content of their own. Over time, and with a powerful AR SDK kit to work with, developers could innovate on this special platform and create AR content we can’t even imagine at the moment.

Although I have no clue if Apple will actually do an AR SDK optimized for new iPhones, Tim Cook’s fascination with AR at least suggests AR is very much in their crosshairs. If they do, I expect it to follow Apple’s proven playbook in which they develop innovative new hardware, tie it to an enhanced OS, then create a special SDK for developers to allow them to create innovative AR apps and make the iPhone a window to the world of AR-based functions and applications.

Side note: If Apple does deliver an iPhone optimized for AR, this could start a new super cycle for iPhone replacement and drive huge numbers of iPhone sales for another three years. Many financial analysts believe Apple kickstarted this super cycle of replacement growth with the iPhone 7. But I suspect an AR-based iPhone would pretty much kick this super cycle into high gear and last well into 2020.

Tim Bajarin

Tim Bajarin is the President of Creative Strategies, Inc. He is recognized as one of the leading industry consultants, analysts and futurists covering the field of personal computers and consumer technology. Mr. Bajarin has been with Creative Strategies since 1981 and has served as a consultant to most of the leading hardware and software vendors in the industry including IBM, Apple, Xerox, Compaq, Dell, AT&T, Microsoft, Polaroid, Lotus, Epson, Toshiba and numerous others.
  • Last year, a large consumer audience was introduced to AR via Pokemon Go, which allows for characters to be superimposed on real life settings as part of the game. This game gave consumers a small taste of what AR is about and has left them wanting more of this technology on their smartphones.

    You wrote that paragraph above as thought it was a statement of fact but provided no factual basis for it. I have observed some consumers — namely myself, and my kids — disabling the AR feature in Pokemon Go because it reduced the accuracy of tossing Pokeballs.

    • jfutral

      “I don’t see this happening in museums”

      Depends on the museum and also the particular exhibit at the museum. None allow flash. Many do allow photos. The Met, MoMA, and National Gallery all allow pictures. There are some exhibits they will exclude, but not often. The Van Gogh allowed photos, but the Ufizi did not, or at least not of Botticelli’s Birth of Venus. The sculptures and paintings in the hallways were not a problem.

      Last time I was at the Met 3 years ago, a lot of kids were walking around with their iPads taking pictures and looking up info on paintings. It did give me a false positive on the potential of the iPad (because they were everywhere), but it was pretty cool to see.

      Galleries, on the other hand tend to be a bit more picky, though, even then not always.

      Joe

      • isitjustme

        It will be interesting to see information display on a phone when I point it at a picture or a thing instead of asking Siri or make a web search for it.
        I believe it will and should be coming but then who am I to say that.

  • Vadim Dumin

    Since Apple is very much into music I firmly believe that the next step for them (VR) would be creating a brain wave controlled headset for the purposes including Apple music voting and sound editing. There is some anecdotal evidence that the hardware for this is already proven to exist.

    https://www.ted.com/talks/tan_le_a_headset_that_reads_your_brainwaves

Protected by Gerben Law