J.D. Power Explains Its Inscrutable Tablet Rankings

on November 1, 2013

jdpower-tabletJ.D. Power caused a bit of a stir today when it put out a new survey in which Samsung had displaced Apple as the leader in U.S. tablet satisfaction. While the headline of the McGraw-Hill Financial unit’s press release declared “Samsung Ranks Highest in Overall Tablet Customer  Satisfaction with Tablet Devices,” the featured table (above) seemed to tell a different story: Apple outscored Samsung in every category except cost.

It gets even stranger when you drill down a bit. J.D. Power weights each of the categories for importance: Performance 0.26, ease of use 0.22, physical design 0.19, features 0.17, and cost 0.16. The unweighted average of Apple’s scores was 4.4 to Samsung’s 3.6. Weighted, Apple outscored Samsung 4.52. to 3.52.

So how did Samsung come out on top? According to a J.D. Power spokesperson, the flashy “Power Circles” chart doesn’t really have much to do with the overall satisfaction. Rather than try to explain this, here’s the relevant section of the email I received in response to my inquiry:

Thank you for your interest in the  2013 U.S. Tablet Satisfaction Study–Volume 2. It’s important to note that award is given to the brand that has the highest overall index score, not the company with the most Power Circles.  In this study, the index score is comprised of customer’s ratings of five key dimensions or factors.

The Power Circles Rankings are something we provide to consumers to understand the relative rank of brands within each of these five dimensions. The Power Circle Rankings  denote the brand that has the highest score within each factor regardless of how much higher their score is.

In the case of Apple in the tablet study, although it did score higher on four out of five factors measured, its score was only marginally better than Samsung’s.  At the same time, however, Apple’s score on cost was significantly lower than that of all other brands.  In comparison Apple’s ratings on cost was more than 100 points lower than Samsung’s.  As such, even though its ratings on other factor was slightly higher than Samsung’s,  Apple’s performance on cost resulted in an overall lower score than Samsung.

In this cost-conscious environment, cost is a key factor in many products purchase and services they use.  Tablets are no exception, where cost is a key driver of the overall customer experience with their device. Although “cost” has the lowest weight among the five factors that drive satisfaction, the notable difference between Samsung’s and Apple’s score in the cost factor was enough for Samsung to rank highest in the study.

Mathematically, this does’t make much sense. Apple outscored Samsung by two “power circles” in each of two categories with a combined weight of 0.48 and by one circle in each of two categories with a combined weight of  0.36. Samsung outscored Apple in one category with a weight of 0.16. Unless the score difference between two circles and four is much, much bigger than the difference between three and five–in which case the whole circles chart is grievously misleading–there is no way Apple’s loss on cost could outweigh it winning in the four other, more heavily weighted, categories.

J.D. Power does not reveal the raw scores by category, but gives Samsung’s overall satisfaction score as 835 (out of 1000) and Apple’s as 833. Curiously, Amazon and Asus, which received only three circles for overall satisfaction, had scores of 826 and 821–only a 1.8% spread between first and fourth place. Asus–two circles–was a fairly distant fifth at 781.

I don’t put much stock in J.D. Power ratings, but they do get used heavily in advertising by whatever company comes out on top. If people pay attention to this sort of “data,” they should be getting more transparent methodology.