I did a quick post on my personal blog earlier this week called “Ten quick thoughts on WWDC” and threatened to expand on some of those thoughts in more detail in the coming days. Here, then, is the first expansion on one of those thoughts, and it’s the last one on my list: Apple is retaking control of content.
Apple adrift
In my piece earlier this week, I wrote:
Apple has been big in content for twelve years, since the launch of the iTunes Store in 2013, and continuing with major launches like TV shows and movies in iTunes, iBooks, Newsstand, and so on. However, for the last several years Apple has seemed adrift in content, a victim rather than a driver of trends, and has seen its content revenues stagnate and fall even as third party apps explode (along with the associated revenue stream for Apple).
There was a time when Apple itself was the conduit for a lot of the content we consumed, whether buying music, renting movies, or watching TV shows. When the download model dominated online content consumption, Apple dominated in the major categories. However, what we’ve seen over the last several years is the download model has been replaced with subscription and free, ad-supported streaming for content. Spotify, Deezer, Rdio and others have led the charge in music; Netflix, Hulu, and Amazon have done the same in video; and subscription and streaming models have popped up for everything from books to games. For the first several years of this trend, Apple did very little to respond and it felt as if Apple was adrift. The acquisition of Beats was the first big sign Apple intended to change this but it wasn’t until this week we really got clear signals of how it would respond.
Apps, Music, News… and TV?
The first signs of better curation actually came in the App Store over the last few weeks. Apple essentially overhauled the way games in the App Store are presented in the US the week before WWDC and the major change was the role of human curation rather than automatically generated lists. This trend of Apple re-engaging in the layer between users and their content was evident in other areas at WWDC itself, notably with the new News app and with Apple Music. With News, Apple is replacing the relatively hands-off approach it took with Newsstand (which really added no value above and beyond the underlying apps) and moving into a more direct role with its own app, dictating design and feature functionality to partners and doing the curation on behalf of users. With Music, Apple is not only providing standard subscription service features, but adding its own radio station and its own custom content destination in the form of Connect. Again, Apple is taking a more direct role in not only content curation but ownership. What we didn’t see at WWDC, of course, was Apple announce its upcoming TV service, but that will likely fit this now familiar pattern.
“For You” a significant theme
One of the other common themes in all of this is not just generic curation but personalized recommendations. Both the News app and the Music app have sections titled “For You” which contain personalized recommendations based on your explicitly stated and implied interests. One important difference between the way this works in Music and in News is, in Music the emphasis is on human curation while in News, it appears to be largely algorithmic. Based on Jimmy Iovine’s remarks on stage at WWDC, it appears Apple believes music has a strong emotional component which machines alone can’t understand – presumably the same can’t be said for news. But it almost certainly can be said for video content, which is why I suspect we’ll be seeing a similar “For You” section in Apple’s TV service when that lands. Apple has, of course, done recommendations for a long time, mostly under the Genius brand in iTunes, but it hasn’t always been effective. Whether it’s any better now will be a major deciding factor in whether Apple Music, News app, and ultimately its TV service will be successful.
Exclusive content and formats
The other big trend in Apple’s re-energized content play is exclusive content and formats. Music has Beats 1 and the Connect platform for artists and fans and News has custom formats for articles. Apple has never really been a content provider before – it’s always simply acted as a channel for third party content – so this is a significant shift. I’m particularly curious to see how this approach gets applied to its TV service, because we’ve already seen significant investments by Netflix, Amazon, Hulu, and HBO in original content over the last few years. I still can’t see Apple investing to that extent in exclusive content, but I can see Apple extending the Connect concept to TV shows and movies and allowing creatives to create and share exclusive content with viewers.
There’s still an app for that
Despite all this, Apple isn’t going to be locking out or shutting down any third party apps that compete with these services. Spotify, Pandora, and so on will continue to exist in the App Store and may even make their way to the Apple TV if Apple does indeed open up the SDK at some point later this year. Apple is more open than ever to competing apps on its platforms and this raises the bar for Apple’s own content services. While Apple has an inherent advantage with these services by virtue of being able to pre-install them on every iPhone, iPad, and Mac, that’s no guarantee everyone will suddenly dump competing apps and come running back. Apple’s content play can’t rely on the home field advantage alone but has to actually differentiate on quality. That’s the big question I have coming out of WWDC. We saw so little of how well these new apps and services will work on stage and there are so many questions remaining. We really will have to wait and see how these things – and especially the curation and recommendation side – work in practice. For now, all we know is Apple is finally ready to try to recapture our content attention, but we still don’t know if it will work.
One thing that I would like to know regarding the human curation versus algorithmic recommendation argument is the human labour associated with Google maps. That is, I think human curation is much more important than many people consider it to be.
Maps is an area where Google, the forerunner of artificial intelligence, has invested many years in improving. However, if my understanding is correct, they have a huge army of humans manually checking and correcting errors that occur when the machines are left to themselves. If this is true, and if this level of curation is truly needed also for high-quality results in areas other than maps, then it suggests that to provide high-quality content, human curation is essential, even when you have Google’s AI capabilities.
My hypothesis and experience is that algorithms are great when you are allowed to make mistakes. For example, a Google search is considered a success if you get the right one from the 10 candidates listed. A batting average of say 0.300 is acceptable. For Ads, the batting average can be even lower to be considered a success because we all know that Ads have a very low batting average. This was not the case with maps. Even when Apple Maps was OK for 95% of the time, the remaining 5% was ridiculed and maybe contributed to the ousting of Scott Forestall. A vastly higher batting average was required for maps. The same with music. A single out of place tune will destroy the mood, even if the five tunes preceding it were perfect.
The problem with AI is not that it doesn’t understand emotions. The problem is that the error rate is not low enough. Knowing how Google maps operates will help me understand this better.
In the field of genetics and genomics research, human curation is again very, very important despite the wealth of statistical tools. Coming from this field, I have observed how important human curation is to create a body of knowledge accurate enough to serve as a foundation for future research. I imagine the same is true still in many areas to which AI is being applied to, and the maps example would be an ideal case to study.
That’s a very interesting insight Naofumi. AI doesn’t fully replace human intelligence and the curating value it brings to bear. We know this but can get carried away pushing the limits. Just like all technology and software — it is just a more advanced version of what has come before. It doesn’t replace human decision making, just shifts the inflection point.
Technology just creates more valuable points of leverage. It does the lower level heavy lifting, freeing people to do the more specialized work that really does require human decision making or at least is more likely to produce consistent results. In the past, we could only solve this by hring armies of people to do this sort of job. They would deliver information to the decision makers, reducing the things they needed to decide and shortening the time to do so.
The promise moving forward is that with ongoing iteration of machine learning and AI, we can continue to refine that point of inflection, continuing to free us to do only the most value added of things. Fascinating stuff. Great time to be alive.
Thanks, that is exactly my point. “Shift the Inflection point” is the word that I was looking for!
I think one of the basic mistakes of automatic curation is that they’re only looking at your history, correlating it with others’, and filling gaps from that. I’ve noticed it from Amazon for books, and from all music services. They’re completing overlooking timing, mood (they probable can’t help that), day of week, weather… I might be wrong, but I get the feeling the recos are totally endogenous, with 0 input besides listening/buying lists. They’re proposing steack fries and wine for breakfast on hot day because last winter I had a lot of that for dinner.
Statistically, they’re not wrong, but practically, they’re utterly wrong. I’m curious as to what parameters algorithms do take into account.
Yup, it’s one big grand reversion to the mean. Algorithmic curation focuses on the commonalities across the sample points when in reality, all the fun and excitement is in the differences.
Automatic curation. Exactly the problem. Your comments make me realize that companies (Google and especially Apple) could greatly improve their algorithms for each of us individually if they allowed us to opt in to a quiz type input which they would tie to us. For example, an algorithmic music quiz would ask about types, artists, and favorite songs then play short pieces of music for us to rate and perhaps also to describe it according to mood, etc. The moods tags — happy, sad, pensive, exercise, energetic, romantic, etc., — could be used by us for telling the app suggest music for me in xxx mood. Instead of a programmed algorithm for everyone, they would have a personalized algorithm for anyone who wanted it. Such a modification of an automatic algorithm to make it a personal algorithm should not be hard to program. This personalization could really lock-in a user to whomever decides to provide it.
Yes, automatic curation is a problem. With my Apple Radio station the matches are pretty good, but sometimes I get a song that does not match the theme in my opinion. If an automatic curation worked well, the dating sites would match couples effortlessly based on questionnaires, but as they admit themselves their matching algorithms do not work.
In human matching there are both rational and emotional components. Unless the automatic curation finds a way to algorithmize the emotional component it will not be sufficient in my opinion.
You mention that your problem is that you sometimes get a song that does not match. My question is, is the mismatch so subtle that you need an emotional component to weed it out? Or is the mismatch rather glaring and obvious? If there were ten ordinary people (not DJs), how many do you think would agree that it was a mismatch?
If it is indeed a subtle mismatch, then maybe you do need emotions. However, I expect that more often than not, the algorithms fail to exclude a tune that rather obviously does not fit in. This would be a case where AI failed at common sense.
I would like to know whether emotion is the key or not. This is because my hypothesis is that human curation is still needed even in areas where emotion is less of an issue. News articles for example (and Apple is also hiring curators for News). I do not expect AI to be good enough for high quality curation, even in non-emotional content. Your experience would help me understand.
I would say the mismatch was more subtle. To answer your question I would need to rationalize my emotions, which is hard to do. Most of the times it was related to my mood, but then I just switched to another station. I remember that one time the lyrics was too harsh in the song and I was riding in a car with a lady friend, so I felt slightly uncomfortable. Though most of the times I am pretty happy with song selection.
Thank you. That’s interesting. Being located in Japan where neither Pandora, Spotify, nor iTunes Radio are available, I unfortunately can’t experience algorithm-selected playlists directly. I’m hoping Apple Music will be available though.
To be honest I am surprised the least about song matching to be more or less accurate. Emotional component in a voice communication can be extracted from a tone alone. There is a study done by Stanford that found a direct correlation between a surgeon’s tone of the voice and the malpractice claims history for example. And the content that triggers word recognition can be removed from a song by removing high frequencies. To do the emotion extraction from a textual content would be much more difficult I assume.
I agree in that understanding the emotions expressed in a particular song might be actually quite easy to analyse. Which brings me back to the point that if a human curated playlist is significantly better than current algorithms (which, not having compared the two, I have no idea if this is true or not), it will probably not be because the former was able to understand the general emotional tone.
I am looking forward to when we can compare the two and make informed decisions about which is better (humans or algorithms) and why.
The problem with AI is that in the end, whatever body of ‘knowledge’ it assembles about people’s preferences, it will be an aggregation of the data on preferences as accumulated and analyzed by the algorithm. Unfortunately, the mythical or archetypal consumer whose preference profile the machine has constructed does not exist. So for most people, the recommendations cranked out by the machine would be glaringly off.
[This is the same thing that happens when you base your child-rearing practices on strict adherence to what is prescribed by parenting books. A lot of people who read such books come away thinking they’re doing it all wrong or there’s something wrong with their child because they can’t make the books’ recommendations work. They don’t realize that the recommendations on those books are based on the average (or amalgamated?) child or infant, and that perfect dear does not exist. Kids are all over the map and what works for one could be completely different from what works for another.]
What a human curator can do that AI is hard-pressed to duplicate is to detect how a particular person’s preferences deviate from the archetypal profile that the algorithm constructed, and why. Once you figure out the why, then the rest comes easy.
Yes, algorithms can generate complicated conditionals on the AI-generated preference map: If listener A likes X, he is likely to like Y. But a machine can never know why. All the recommendations it makes are based on correlations, perhaps thousands or millions of different dumb correlations mined from data with little guidance as to what correlation is relevant to which real human customer.
From the dark side…
There’s Turkish that says “Tell a lie forty times, and it becomes the truth”. We humans are infinitely more adaptable than any machine. We get acclimated, and eventually start expecting things to be “a certain way”. This applies to algorithmic versus human curation equally well. Point is we can be curated and manipulated to the point of liking crap.
Look at the state of music today. Yes, there are some glimmers of excellence, but IMO it’s pretty lousy. I know I must therefore be getting older, but “the day the music died”, or at least got really sick, was sometime in the early 90s. We’re being told what to like.
I constantly resist the urge to denigrate my kids’ musical choices because I know that what I feel now is what my parents felt about my own choices years ago. It helps that one of my kids is a music major who patiently explains to me which contemporary stuff is musically very sophisticated and why as well as which ones are rather boring and rudimentary. In the end, beauty is in the ear of the listener.
It has also been documented in careful research that most people stop listening to new music around their mid-twenties. Though I don’t know how prevalent my case is where my musical tastes have actually gone backward in time as I got older. My top listening choices now are the standards by Sinatra, Bennett et. al., singers that I certainly couldn’t bear to listen to when I was younger.
A final observation. We think now of classical music and the musicians of old as boring, staid, extremely formal, and lacking spontaneity. I just read a biography of Beethoven and found out that during his era, pianists regularly engaged in joint performances where they would take turns improvising on a theme that the other performer throws out as a challenge. Classical jamming. I don’t know why contemporary classical musicians don’t do that. They would certainly attract more young listeners if they did.
Naomi I, you have an intriguing way of “opertionalizing” qualitative features through introducing the parallel notion of not making mistakes
Apple Watch, Apple Pay, Apple Music, iAd…even the best CEOs may eventually succumb to excessive ambition. If that happens we may lose some of the Apple that used to be, and that would not make me happy.
More than any other company, Apple seems to be constantly in peril of greatly displeasing its customers if it strays out of a box that said customers have decided is where they want Apple to be. It’s a curious phenomenon.
To some of us techies, that day came sooner.
Great article. For those who live outside US like me, you can access Netflix, Hulu and similar media stations on your Apple TV by using UnoTelly or similar tools.