How did they miss it?
If you go back and read the reviews of the iPhone 5 published on the first day that Apple allowed recipients of early samples to write, you would be stunned by the furor the soon broke out over the quality of Apple’s Maps app. Of the major early reviewers, only The Wall Street Journal‘s Walt Mossberg noticed major flaws in Maps, noting its “big minuses” in his second paragraph. Others either failed to mention Maps at all, or praised the app. The New York Times‘s David Pogue may have set some sort of record by writing an initial column in which he called the Maps app one of the iPhone’s “chief attractions” and coming back a week later with a column calling that same app “an appalling first release.”
I spent 15 years reviewing tech products for BusinessWeek and the depressing truth is that oversights and errors like these are painfully easy to make. I wrote reviews of major products that had been in my hands for as little as 36 hours, which barely gave me enough time to figure out what the major features were, let alone test them thoroughly. (My best guess is that iPhone reviewers had their test units for five or six days before they had to write.) I relied too heavily on the review materials and demos provided by the manufacturers, and these tend to steer reviewers to the features that work best and away from the ones that are dicey.
It’s hard to say whether the extreme time pressure facing reviewers is the result of a deliberate strategy by companies or a necessity of the product cycle; it probably varies from case to case. The companies are often working under very tight schedules themselves and the products, particularly their software, is often not final until just before release. I was always grateful to companies that provided products extra-early, even on the understanding that the hardware was a pre-production sample the software wasn’t quite finished. I could log the defects and see if they were fixed in the final product. (My favorite was very early in my tech career, when I spent more than a year watching Windows 95 evolve through its testing.)
The Maps problems were particularly easy to miss. iPhone reviewers were focused on the new hardware and Maps is actually part of the general update of iOS software for iPhones and iPads. Testing mapping software is time consuming and the problems of the apple app are much more serious in some locations than others, so it depended on where you looked. I was lucky. I installed iOS on my iPad, opened up Maps, and discovered a glaring flaw right in my neighborhood: the main campus of the National Institutes of Health was missing. That got me looking and I quickly discovered lots of other problems.
I think the bottom line here is pretty simple: The pressures reviewers are under all but guarantee that mistakes will be made, though I can’t remember one quite as glaring or near-universal is this. Don’t put too much faith in the fevered first batch of reviews, especially of products that inspire a reviewer feeding frenzy. The problems will be discovered quickly enough.