I believe most analysts, including those that monitor Apple’s every move, are seriously underestimating the ramifications of Apple baking Shazam’s music identification service into iOS 8. This is not merely about increasing song downloads. Rather, this move marks Apple’s determined leap to re-position the iPhone in our lives. The digital hub metaphor is now much too limiting. As the physical and digital worlds mix, merge and mash together to create entirely new forms of interaction and new modes of awareness, the iPhone will become our nerve center. It will guide us, direct us, watch, listen and even feel on our behalf.
A bold statement, I know, especially given the prosaic nature of the rumor. Let’s start then with the original Bloomberg report:
(Apple) is planning to unveil a song discovery feature in an update of its iOS mobile software that will let users identify a song and its artist using an iPhone or iPad.
Apple is working with Shazam Entertainment Ltd., whose technology can quickly spot what’s playing by collecting sound from a phone’s microphone and matching it against a song database.
Song discovery? Ho hum. Only, look beyond the immediate and there’s potential for so much more. That late last year, Shazam updated its iPhone app to support an always-on, always-listening ‘Auto Shazam’ feature is no coincidence. Our phones are becoming increasingly aware of their surroundings. I expect Apple to leverage this technological confluence for our mutual benefit.
Today, Song Discovery.
Apple’s move no doubt satisfies a near term need. While Shazam has been around since 2008, and the company claims 90 million monthly users across all platforms, having their service baked into the iPhone will almost certainly spur increased sales. Song downloads have slowed — not just with iTunes, the world’s largest seller of music — but across the industry.
Instead of having to download the Shazam app, iPhone users will now simply point their device near a sound source and summon Siri: “what song is playing?” So notified, they can then buy it instantly from iTunes.
Little surprise music industry site MusicWeek was generally positive about the news. Little surprise, also, the tech industry could not muster much excitement. Thus…the Verge essentially summarized Bloomberg’s report.
Daring Fireball’s John Gruber offered little more than “sounds like a great feature.”
Windows Phone Central readers offered only gentle mocking, reminding all who would listen this feature is already embedded in Windows Phone.
That’s about it. Scarcely even a mention Shazam has a similar, if less developed TV show identification feature which could also prove a boon for iTunes video sales.
Place me at the other end of the spectrum. I think the rumored Shazam integration is a big deal and not because I care about the vagaries of the music business. This is not about yet another mental task the iPhone makes easier. Rather, this move reveals Apple’s intent to enable our iPhones to sense — to hear, see and inform, even as our eyes, ears and awareness are overwhelmed or focused elsewhere.
Tomorrow, Super Awareness.
Our smartphones are always on, always connected to the web, always connected to a specific location (via GPS) and, with minimal hardware tweaks, can always be listening, via the mic, and even always be watching, via the cameras.
What sights, sounds, people, toxins, movements, advertisements, songs, strange or helpful faces, and countless other opportunities and interactions, some heretofore impossible to assess or even act upon, are we exposed to every moment of every day? We cannot possibly know this, but our smartphones can, or soon will. I believe this Shazam integration points the way.
It’s not just about hearing a song and wanting to know the artist. It’s about picking up every sound, including those beyond human earshot, and informing us if any of them matter. Now apply this same principle to every image and face we see though do not consciously process.
Our smartphone’s mic, cameras, GPS and various sensors can record the near-infinite amount of real and virtual data we receive every moment of every day. Next, couple that with the fact our smartphone’s ‘desktop-class’ processing will be able to toss out the overwhelming amounts of cruft we are exposed to, determine what’s actually important, and notify us in real-time of that which should demand our attention. That is huge.
Going forward, the iPhone becomes not simply more important than our PC, for example, but vital for the successful optimization of our daily life. This is not evolution, but revolution.
The Age Of iPhone Awareness
Yes, it’s fun to have Siri magically tell us the name of a song. Only, this singular action portends so much more. At the risk of annoying Android and Windows Phone users, Apple’s move sanctions and accelerates the birth of an entirely new class of services and applications which I call ambient apps.
Ambient apps hear, see and record all the ‘noise’ surrounding us, instantly combine this with our location, time, history, preferences — then run this data against global data stores — to inform us of what is relevant. What is that bird flying overhead? Where is that bus headed? What is making that noise? Who is the person approaching me from behind? Is there anything here I might like?
Your smartphone’s mic, GPS, camera, sensors and connectivity to the web need never sleep. Set them to pick up, record, analyze, isolate and act upon every sound you hear, every sight you see.
This has long been the dream of some, though till now was impossible due to limited battery life, limited connectivity, meager on-board processing and data access. No longer.
Let’s start with a simple example.
Why ask Siri “what song is this”? Why not simply say, for example, “Siri, listen for every song I hear (whether at the grocery store, in the car, at Starbucks, etc.). At the end of the day, provide an iTunes link to every song. I’ll decide which ones I want to purchase. Thank you, Siri.”
Utterly doable right now. Except, why limit this service to music?
For example, perhaps our smartphone can detect and take action based upon the fact that, unbeknownst to you, the sound of steps behind you are getting closer. It can sense, record and act upon the fact you walk faster each time you hear this particular song. Or you slowed down when passing a particular restaurant. What do you want it to do based upon its “awareness” of your own actions — actions which you were not consciously aware of?
Our smartphone can hear and see. It is always with us. It makes sense then to allow it to optimize and prioritize our responses to the real and virtual people and things we interact with every day, even those outside our conscious involvement.
Ambient Apps Are The New Magic
The utility of our smartphone’s responses will only get better. Smartphones sense by having ears (mic), eyes (cameras), by knowing our exact location (GPS) and by being connected to the internet. These continue to improve. It is smartphone sensors, however, that parallel our many nerve endings, feeling and collecting all manner of data and notifying us when an appropriate action should be taken.
Though still a relatively young technology, smartphones have added a wealth of new sensors with each iteration. The inclusion of these sensors should radically supplement the recording, tracking and ambient ‘awareness’ of our smartphones, and thus further optimize our interactions, both online and offline.
Jan Dawson posted this Qualcomm chart which illustrates the amazing breadth of sensors added to the Samsung Galaxy line over just the past five years. What becomes standard five years from now?
Hear, see, sense. The smartphone’s combination of hardware, sensors, cloud connectivity, location awareness and Shazam-like algorithms will increasingly be used to uncover the most meaningful bits of our lives then help us act upon them, as needed. This is not serendipity, this is design. I think Apple is pointing the way.
I really enjoyed your article. It got me thinking in many different directions, from the more mundane “digital politics” to the more important everyday impact sensor integration brings.
-It’s not new, not even close. This is what sensors have been doing for decades. The cloud integration, location awareness, and other stuff are being done by many, especially Google. You did note this.
-It better have an off switch. The privacy issues are HUGE. How would you like it if your webcam came on when, IT (not you) decided to turn on?
Google (or anyone else) may be good, or Google may be evil, but they give you no cost services in exchange for your information. When you buy a phone (or other computer), you paid money and you own it. Which means, you better control how it works, and what it’s doing. (For the “design submissives” out there-yes, within the capabilities and functional expectations of the device).
Remember the inadvertent cell tower tracking controversy that was stored in the iPhone? That was the very most molecular tip of the iceberg. What I’m trying to say is, sensors have the capability to make things even more powerful. This means more complex, with much more user responsibility to have it work to the owners favor. Sure, we can put a pretty face on it, make it easy (or even without user intervention), but it does not abdicate the user from responsibility. OEM’s, of course, will try to conceal that. They want to own your data.
Great comment. Thanks. I must say, I still trust Apple with my data (including my theoretical data) than nearly any other tech company. My article is speculative so I did not touch on this but I think Apple is best positioned to *empower* us with all the data we are generating, rather than simply use it to sell us product.
Spot on Brian. Apple is perhaps the only company that can deliver a trustworthy experience, because the ecosystem is curated and controlled, and because Apple has a profit margin. Apple does not need to sell my information to make money. They have a strong financial incentive to protect my information.
I trusted them for a period of time (2008-2010), never again. I don’t and shouldn’t trust anyone selling me anything really. I reserve trust for a more substantive relationship.
Definitely good to be cautious. The point the others were making was that trustworthiness is relative in this context; and, in the absence of something more substantive, selling for a healthy margin makes a company “more trustworthy” than the company giving you a service for free — the incentives to protect your data are different. Apple has an incentive to protect your data, Google has an incentive to sell your data or misuse it. That’s the simple fact.
That’s part of what caused the Google/Apple split over maps, IIRC. Apple wanted turn by turn. In exchange, Google wanted the user information. Apple wouldn’t budge. That’s not all of it, but a major point of it, AFAIK.
Joe
Google badly overplayed their hand in that one.
Exactly. You can’t really trust any company all the time (Apple included), but you can count on profit motive. Apple will do dumb things from time to time, and they’ll piss me off from time to time, but on the whole their strong financial incentive to serve my needs keeps me very satisfied. Is there any other company in tech with that kind of direct customer relationship and financial incentive? Certainly not Google. And not really Microsoft either, they aren’t fully focused the way Apple is. Amazon maybe (but the profit motive is weak)? Not Samsung either, the customer relationship is naturally fractured.
I don’t necessarily agree that you shouldn’t trust, but certainly mis-placed trust is dangerous. Otherwise, each transaction is an irrational leap of faith. I like the motto “Trust, but verify”. Unfortunately, I more often than I would like to admit drop the ball on that second half, and I always regret it when I do. Without fail.
Joe
“This means more complex, with much more user responsibility to have it work to the owners favor. Sure, we can put a pretty face on it, make it easy (or even without user intervention), but it does not abdicate the user from responsibility”
Fortunately or unfortunately, this would pretty much mean the death of what Brian suggests. Complexity is not the forward trend, either from tech companies or from consumer expectations, from what I can tell.
Or from a user unfriendly perspective, complexity is Facebook’s shield, for instance. The more complex they can make the privacy settings the less inclined people are to use them.
Otherwise, I keep trying to figure out how to undermine the data-mining of personal information. Maybe start a website where we can publish/offer our information for free to Google’s or Facebook’s real customers. Otherwise, I’m with Brian with Apple being the more trustworthy of the current crop.
Joe
“Fortunately or unfortunately, this would pretty much mean the death of what Brian suggests. Complexity is not the forward trend, either from tech companies or from consumer expectations, from what I can tell.”
True. Not everyone is qualified to fly the airplane. Those that are, get maximum advantage, and responsibility for it.
I, for one, welcome our Skynet overlord.
as you should!
what I am envisioning for the iPhone is absolutely empowering. Of course, there will be many who will want to profit from all this personal data. Sadly, some scary neighborhoods in our new world.
Nah, I’m more of a John Connor proponent… 🙂
Forget those who seek to profit. The NSA, et al. will love this always connected GPS, camera, and mic too.
Brian, this is your coinage? Ambient apps? Nicely played.
“A class of apps, christened “ambient apps” by the blogosphere, are endowing mild-mannered smartphone users with what might have passed for superpowers a few years ago. They work by keeping your smartphone’s mic, GPS and even its camera listening, watching and seeking out every signal coming from your surroundings.” Search The amazing, inspiring and scary world of ambient apps
Cf. Ambient music is said to evoke an “atmospheric”, “visual”[2] or “unobtrusive” quality.[3] To quote one pioneer, Brian Eno, “Ambient Music must be able to accommodate many levels of listening attention without enforcing one in particular; it must be as ignorable as it is interesting.”
I actually have those original Brian Eno *albums* so +1 for me 😉
(Ambient 1: Music for Airports is still worth a listen.)
M7 and Primesense…. Apple is building a low-powered, always on sensory system for the phone.
I didn’t think about Primesense when writing this, though my limited understanding of what it can do absolutely fits into this notion, and your labeling this a “sensory system” is a great phrase.
I did cut out some bits, including thoughts on M7 as I think that currently biases people to think of fitness band tracking. But, I do think it belongs in this overall discussion of how iPhone is becoming our personal nerve center.
The biggest questions are how low powered and how trustworthy can it be built? Essentially, it seems, we are talking of phones that are constantly aware of their surroundings, we will want them to be, but we won’t want to have that affect the battery power for when we want to use them. I believe that Apple is ahead in the hardware integration area, but Google has shown us more on the software side with Google Now. Will Apple show us an updated Siri at WWDC that delves into more contextual awareness?
The other big limiting factor on these is the use of the radio. That’s a big drain on battery. Apple has already been showing off the technology that coalesces radio use to improve battery performance. I think so much of the technology is there now, it just needs to come together.
So true. All these grand ideas and the battery remains the barrier.
It will remain that way for the next about 5 years. The research being done in terms of heat generated electricity will start to bear fruition in a few years. I would love to imagine that electronics are so efficient that a kinetic drive system (Seiko Kinetic) would be sufficient to power a computer wrist watch. It really feels like we are on the cusp of the next shift in computing.
Whatever happened to the methanol fuel cell? Theoretically, it could power a laptop for a week. I imagine a mobile device for a month or more.
I used to closely track innovations in batteries, including methanol fuel cell and hydrogen fuel cell, but there’s just so much problems with these devices (safety, reliability,. access). For the foreseeable future, seems like everything else inside a smartphone must get both better *and* smaller, to allow for ever larger batteries (that are not advancing as rapidly).
On the trust is issue, there was a tiny outrage about privacy on the Google Play Store, but it ended up being standard procedure at the time.
Please correct me if I’m wrong.
The issue was that when a user buys an app (maybe even free ones?) on the Google Play Store, Google handed over the email address and billing zip code for the user, to the developer of the app.
The Apple App Store keeps that user info private and doesn’t share that info at all.
Of course both platforms have apps that ask for access to users’ address books / contacts, that is a separate issue.
That happened, yes. I do not know what the current situation is with it though.
I’ve long thought Apple is working towards its own network of things, including expanded sensor capability within the iPhone as well as accessory sensors outside of the iPhone. This natural extension of jobs-to-be-done requires a high level of privacy and data protection, as well as vertical integration and curation.
Agreed.
I am genuinely thankful to the owner of this web site who has shared this wonderful piece of writing atat this place.
you are in reality a just right webmaster. The web site loading pace is incredible.It sort of feels that you are doing any distinctivetrick. In addition, The contents are masterpiece.you’ve performed a wonderful task on this topic!
What does honey do for your skin? My website: kerassentials
Error 212 origin is unreachable
Some really excellent info, I look forward to the continuation.