iPhone X Study Follow Up

I’m sure by now most have you have seen the iPhone X survey I published went viral. Many thanks to John Gruber who linked to the article and made it the second most read article in Tech.pinions history. As of now, over 75,000 people have read that article, and at this rate, it will be 100,000 by the end of the day. On the back of that article, I intended to add some follow up commentary given there is plenty of data from that study I have not shared yet. But, the public reaction to my article brought up some new thoughts as well worth sharing.

Siri’s Low Score Was No Surprise
I want to start by sharing some of the pubic’s reaction to the data. In particular, the chart with a customer satisfaction breakdown by feature. Here is the chart again for reference.

I received dozens of emails and many dozens of Twitter mentions and direct messages over this chart. The fascinating consensus was the low score for Siri was no surprise. In fact, the general sentiment was total agreement the iPhone X is great overall and “yep” Siri sucks. I can’t say this part surprised me, but I did expect some Siri defenders to come at me. There were zero people defending Siri. This strikes me as a critical observation.

Now, I’d like to remind everyone that these were mostly tech-savvy people taking our survey and the ones who took the time to email or send me a message on Twitter. But, I maintain, this audiences approval of a product or service carries a great deal of gravitas with the mainstream consumer market. Something I will share more data on shortly. When we did our Voice Assistant 2.0 study last year (3.0 will be done in May), Siri did not rank quite as low in customer satisfaction. Customer satisfaction of Siri by a mainstream representative consumer sample was 70%. Quite it higher than the 10% rank early adopters gave it in our iPhone X study. That being said, only 20% of mainstream consumers in our sample said they were very satisfied with Siri. If you recall, I said the number of very satisfied answers is the key stat to look at in customer satisfaction. Most, and I mean the vast majority, of Apple products (mostly hardware) have a customer satisfaction of very satisfied consumers north of 80%.

I consider Siri a product, not a feature, and I’m sure people may want to debate that, but this is how we need to look at and evaluate Siri. In my opinion, Siri as a software product is as important as any hardware product Apple makes. Let’s hope Apple’s recent hires and continued investment in Siri pay off and can convince their customers it is an invaluable part of the Apple experience.

The Secret about Net Promoter Score
If you are not familiar with Net Promoter Score, it is a measurement technique to quantify if people are happy enough with your product or service to recommend it to someone else. There is a lot of controversy with this measurement, mostly because people don’t do it well or don’t understand how best to get actionable results. I have been involved in many net promoter score studies in the past, and generally, when you look at the early adopter cohort, you never see a high net promoter score. This is particularly true when a product is very new, or unproven. Case in point, Apple Watch.

When we did the first Apple Watch study with Wristly, we all watched in amazement as the customer satisfaction results trickled in. We knew we had an early adopter cohort for that first study and were surprised they were so satisfied with a brand new product in a brand new category. Then we watched in bewilderment as we saw the net promoter results trickle in at a staggeringly low number of low 30s. If consumers taking our study were highly satisfied with the product, it would stand to reason they would be willing to recommend it to other. The vast majority of these highly satisfied Apple Watch owners were not going to recommend it to friends or family? Why did we wonder?

Then through follow up conversations we uncovered a critical insight. This cohort knew they were early adopters and were self-aware of the fact that they often like to get technology early to try it and are tolerant if it all the kinks aren’t worked out. In the case of Apple Watch, they felt Apple nailed a 1.0 experience but also knew it still had kinks, it wasn’t perfect, and therefore weren’t comfortable recommending it to friends and family yet. This was fascinating but also incredibly insightful and important.

Now fast forward to two new products from Apple, and the early adopter cohort net promoter score (NPS). The first is Apple Airpods. In our study, Airpods had a 98% customer satisfaction. That paired with a surprising net promoter score of 75. From what we learned with Apple Watch, when the early adopter cohort is highly likely to recommend a product to friends and family then you know it’s ready for the mainstream market. Apple accomplished this with Airpods on their first try.

Now, looking at iPhone X. Early adopters in our panel gave the iPhone X a net promoter score of 69. Just for reference here is how that ranks against the industry benchmark from SurveyMonkey out of an average NPS score of over 200,000 organizations who tested their products or services net promoter score.

Anything above 50 is a pretty solid NPS number. Apple’s products typically are in the 80s and are a part of a rare few consumer products with that high of NPS. For iPhone X, a radical redesign with some significant new features and software design paradigms to score a high NPS from early adopters is really remarkable.

Assuming Apple plans to build upon the iPhone X and trickle many of its features down the lown, even to lower-priced products, I think we can safely assume that even with mainstream non-techies get their hands on this technology there won’t be any frustrations or resistance to the user interface changes. This is a good sign and impressive hardware and software work from all at Apple who worked on it.

Published by

Ben Bajarin

Ben Bajarin is a Principal Analyst and the head of primary research at Creative Strategies, Inc - An industry analysis, market intelligence and research firm located in Silicon Valley. His primary focus is consumer technology and market trend research and he is responsible for studying over 30 countries. Full Bio

Leave a Reply

Your email address will not be published. Required fields are marked *