Confessions of a Reviewer: How the iPhone Maps Mess Was Overlooked

by Steve Wildstrom   |   September 28th, 2012

Sample Apple mapHow did they miss it?

If you go back and read the reviews of the iPhone 5  published on the first day that Apple allowed recipients of early samples to write, you would be stunned by the furor the soon broke out over the quality of Apple’s Maps app. Of the major early reviewers, only The Wall Street Journal‘s Walt Mossberg  noticed major flaws in Maps, noting its “big minuses” in his second paragraph. Others either failed to mention Maps at all, or praised the app. The New York Times‘s David Pogue may have set some sort of record by writing an initial column in which he called the Maps app one of the iPhone’s “chief attractions” and coming back a week later with a column calling that same app “an appalling first release.”

I spent 15 years reviewing tech products for BusinessWeek and the depressing truth is that oversights and errors like these are painfully easy to make. I wrote reviews of major products that had been in my hands for as little as 36 hours, which barely gave me enough time to figure out what the major features were, let alone test them thoroughly. (My best guess is that iPhone reviewers had their test units for five or six days before they had to write.)  I relied too heavily on the review materials and demos provided by the manufacturers, and these tend to steer reviewers to the features that work best and away from the ones that are dicey.

It’s hard to say whether the extreme time pressure facing reviewers is the result of a deliberate strategy by companies or a necessity of the product cycle; it probably varies from case to case. The companies are often working under very tight schedules themselves and the products, particularly their software, is often not final until just before release. I was always grateful to companies that provided products extra-early, even on the understanding that the hardware was a pre-production sample the software wasn’t quite finished. I could log the defects and see if they were fixed in the final product. (My favorite was very early in  my tech career, when I spent more than a year watching Windows 95 evolve through its testing.)

The Maps problems were particularly easy to miss. iPhone reviewers were focused on the new hardware and Maps is actually part of the general update of iOS software for iPhones and iPads. Testing mapping software is time consuming and the problems of the apple app are much more serious in some locations than others, so it depended on where you looked. I was lucky. I installed iOS on my iPad, opened up Maps, and discovered a glaring flaw right in my neighborhood: the main campus of the National Institutes of Health was missing. That got me looking and I quickly discovered lots of other problems.

I think the bottom line here is pretty simple: The pressures reviewers are under all but guarantee that mistakes will be made, though I can’t remember one quite as glaring or near-universal is this. Don’t put too much faith in the fevered first batch of reviews, especially of products that inspire a reviewer feeding frenzy. The problems will be discovered quickly enough.

 

Steve Wildstrom

Steve Wildstrom is veteran technology reporter, writer, and analyst based in the Washington, D.C. area. He created and wrote BusinessWeek’s Technology & You column for 15 years. Since leaving BusinessWeek in the fall of 2009, he has written his own blog, Wildstrom on Tech and has contributed to corporate blogs, including those of Cisco and AMD and also consults for major technology companies.
  • http://twitter.com/Mangy_Dog Vicious Cur

    Frankly, I haven’t seen any problems with Apple’s maps app in my neck of the woods. I can see how reviewers would not necessarily see issues. The vast majority of locations are correct, so unless you’re looking for mistakes you might not notice them unless you’re unfortunate enough to be sent to an incorrect location.

    • Grwisher

      +1

      I, like you, have had a good a good experience using iOS6 maps. This situation, like others have pointed out, will probably be like the Flash situation. Predictors of the “End of the World” will grab the headlines for a while and then, over time, fade into the background.

  • Grwisher

    Great points. And this is not a new situation where software reviews fail.

    I have been a programmer for over 40 years and have converted some pretty big data bases along the way. In my experience, you can test and review all that you want. But it is only when the users get their hands on your product, with the real data, that you will know the true extent of success or failure. Reviewing often fails to show the true picture for the reasons that you pointed out. Good article.

  • Graham J

    Ya I read that TechCrunch article, too.

  • http://www.facebook.com/THExREALxTACO Jeremy Taco Patterson

    Great points. I am still amazed just how BIG a deal was made about it. I understand inaccurate maps are a big deal, but I knew (or assumed) that this being Apple’s first foray into the Mapping world, there would be some growing pains.

    I also know that on the rare occasion Apple does drop the ball, they tend to correct any issues wholly and in a timely manner.