Author: Anthony Simon

Dr. Anthony Simon is VP in charge of DTV Ecosystems Strategy & Partnerships efforts for CSR. CSR, headquartered in the UK, is a leader in semiconductor and software platform solutions for the Automotive, Consumer Electronics and Locations based markets. Anthony develops third party partnerships with interactive new media companies related to the digital TV market space. Anthony comes to CSR as part of CSR’s acquisition of Zoran Corporation where he led the pay operator set top box business. Prior to CSR Anthony has held management roles in both marketing and sales at technology companies such as Pixelworks and Conexant focusing on the consumer TV, operator set top box and new media markets. Over his career, Anthony has served multiple expatriate positions in Europe and Asia and speaks Japanese, French and Hungarian. Anthony is based in Sunnyvale, CA.

Interactive TV Trends – How the TV Experience is Changing, Part III

This is the third article in a three part series discussing key trends in TV. The first article looked at how new interface technologies are enabling new ways to control our TVs. The second article focused on the multi-screen TV experience. This article focuses on how interactive TV trends are driving the need for improvements in TV image quality.

Full HD is not enough for Future TV
Some might believe our latest flat panel televisions represent the zenith of picture quality. This is not surprising given we often hear that 1080P resolution or “Full HD” are “future proof” technologies. The oft-cited reasoning is that for a given screen size, viewed from a normal watching distance, the acuity of the eye cannot discern resolutions beyond Full HD. Another reason why Full HD is considered future proof is because actually a very small percentage of video content is even broadcast at this resolution. Most digital pay TV broadcasting systems transmit in lower resolution formats – the industry is still catching up.

Certainly, for those looking to buy their next TV set – no one should be concerned that 1080P is not good enough. Considering the horizon of time people buy and keep a TV set which is about 8 years– a consumer cannot go wrong with “Full HD”. But for people interested in where the industry is going in the long term –looking out over the next ten years, our image quality is going to see massive improvements making today’s TV technology look primitive.

Part of the reason why we can expect big improvements in TV video quality has to do with our superior eyesight. Our capacity to see is many multiple orders of magnitude greater than what our TVs can display. For example, a Full HD TV displays about 2 million pixels of video information. In real life, one of our eyes processes about 250 million pixels – but since we have two eyes channeling vision to our brains – our effective vision makes use of greater than 500 million pixels of video information. And while it is true that we can only discern a limited resolution from a given distance – our eyesight is also sensitive to contrast, color saturation, color accuracy and especially movement. All these areas are where TV systems can improve.

Detractors may argue TVs do not have to be perfect – just a reasonable representation. Others may argue that consumers only care about TV size and price that TV quality is not a selling point. But I argue TV image quality does matter – quality has always had to keep pace with the growing size of TV screens. TVs will continue to get larger – requiring improvements in resolution as our room sizes will start to limit viewing distance. Also, the nature of interactive TV and future 3D systems will make us want to sit closer to the TV set – again mandating video quality improvements.

Interactive TV’s Make You Sit Closer
Interactive TVs will bring games, virtual worlds and new video applications drawing us physically closer to the TV screen. Gaming is a huge industry- with almost $50B spent on gaming consoles, software and accessories. Virtual world games are increasingly popular. “World of Warcraft” is a massively multiplayer online role-playing game with over 10 million subscribers. All kinds of social virtual worlds such as the Sims, Second Life, IMVU and Club Penguin are attracting millions of players. IMVU, with over 50 million registered users, is a social game where people can develop personal avatars and spend time in virtual worlds chatting and interacting. While many of these games are still played on PCs – migration to the living room TV is inevitable. Console games have already shown the way – the size and immersive nature of the large screen TV will draw others into the living room as well.

3D display will also drive the need for improving display resolution and image quality. Sure, everyone hates 3D glasses – but technology will continue to evolve and glassless 3D displays will continue to improve and come down in price. There will be applications that consumers will demand in 3D such as sports – people will see the advantages of watching close up sports games on the large screen display in vivid, artifact free video.

OEMs and broadcast equipment companies are investing heavily in supplying the infrastructure to make this happen. 3D advertising will take on more importance – imagine having the option to tour a car or a house in extremely vivid 3D. On the entertainment side, movie and video directors will become much better at using 3D perspectives in such a way to take advantage of image quality improvement. Today 3D effects are more like a gimmick – watch the arrow fly into the room for example. But going forward directors will make more subtle use of 3D adeptly drawing viewers into to the film or the show. On a beautiful large screen display with ultra high resolution and image quality, viewers will practically feel like they are part of a movie or scene.

3D also opens up a world that we could only dream about when matched with the power of the internet. For example, the evolutions of virtual worlds and their capabilities becomes much more compelling with large screen displays. A simple example is virtual tourism and world exploration. Just as Google has taken a picture of all the street views of the world, there is no reason we cannot build a 3D model of the whole terrestrial experience on earth in a few years. Imagine then the capability to walk around the world as a virtual tourist and view the world from the comfort of your 3D television.

As virtual worlds improve and evolve, new immersive ways to interact with large screen TVs will continue to evolve. Many social activities come to mind as well as the concept of participating or viewing in e-sports. E-sports are virtual sports games that can also be viewed by others. The prospects for e-sports are boundless and limited only by imagination. Virtual bullfights, gladiator battles, racing events will be watched on-line the same way we watch football games today.

The display-use model will also change over time. Today our concept of a display is a TV set that sits in the living room – a piece of functional furniture. With the advent of new display materials like OLED, display will transform from furniture to architectural material. In fact there is no reason why the wall in your den cannot become a display. In fact, why stop with the wall? Imagine the immersive feeling of the ceiling, floor, and walls all around built of display – it’s the video equivalent of surround sound. In fact, the architectural use of display could add interesting use cases beyond entertainment.

For example, inlaid architectural materials can appear in almost in any room around the house. Touch screen uses in the kitchen, can provide not only control but also interactive recipe applications and videos on cooking instructions. Bathroom walls can provide wallpaper backgrounds or any kind of networked information that we already see on our PCs. Inlaid display technologies will appear on appliances as well as anywhere people need information or help with controls. The point of all this is that again there will be many reasons in the future of us needed to get close to the screen – and all this near proximity will demand increases in display quality.

TV Development Underway
Already major TV OEMs are working on the next step up in resolution over Full HD. There are multiple propositions in development for higher order resolution TV systems. TV OEMS are already demonstrating “4KX2K” systems that provide 4096 X 2160 pixel arrays. Even beyond “4KX2K” is Ultra High Definition (UHD) which provides 7,680X4320 pixels resolution which equals 33 million pixels or about 16 times the number of pixels used by Full HD systems. UHD was first introduced by Japan’s national TV broadcaster NHK in 2003. NHK, marketing the resolution as “Super Hi-Vision” had to build the cameras and display technology from scratch to be able to create a UHD demonstration system. Since then NHK has displayed the system at numerous broadcasting shows. Toshiba, LG and Panasonic showed UHD systems at CES 2011 – likely more UHD sets will be shown in 2012. UK’s BBC also is interested in this format. The BBC announced plans to provide UHD coverage of the 2012 London games.

In addition to higher resolution, OEMs continue to invest in superior display technologies like organic light emitting diode (OLED) displays. OLEDs have several advantages over LCD and plasma display technologies. For example, OLED do not make use of a backlight and emit light directly. Direct emission results in a much more vivid display of color, contrast and viewing angle over LCDs. Since there is no backplane in OLED TVs, OLEDS are a much more power efficient and lower in weight. OLED displays are also flexible – opening up new opportunities to use displays in various new applications in architectural display and even clothing.

OLEDs also have a very high response time over LCD. In fact, the relative low response time of LCD, required the industry to introduce all kinds of approaches to compensate by introducing frame rate conversion techniques. OLEDs response time increases response time by a factor of 1000 over LCD allowing for a much better display motion performance.

Improvements will also need to continue on the broadcast side. Higher resolution TVs consume bits at an alarming rate. For example, uncompressed ultra high HD would demand 24Gbps a major jump over ~1.5Gbps required for Full HD. Any increases in resolution will demand major improvements in data compression as well as networking, storage and broadcasting capacity.

But the march of improvements will continue. As TV screens get larger and the way we use these screens draw us in closer – the need for improved image quality will also continue to improve.

Our TV experience will change dramatically over the next ten years. As these series of articles have discussed the whole TV experience will continue to morph the way we spend our time watching large screen displays. 2012 will bring some interesting signs about how all this will play out. 2012 we will see OEMs developing much better ways to interact with TVs – our ability to control the TV through new remote technologies and improvements in finding and sharing content will make major advances. We can expect more use of our hand held tablets and smart phone devices joining us in front of our TV sets. Interactive TV will bring, not only more sources of content, but also new tools to help recommend as well as share content and media that we really want to see. Finally, the way we use TV will be much more immersive demanding major improvements in the video quality in TVs over what we have today.

Interactive TV Trends – How the TV Experience is Changing – Part II

This is the second article in a three-part series discussing key trends in TV. The first article looked at how new interface technologies are enabling new ways to control our TVs. This article focuses on how the TV experience is changing as we begin to use multiple screens of our PC, phone and tablets together with our TV sets. The third and last article will discuss new trends in image processing and why major improvements in picture quality are still necessary.

Entertainment Multitasking – How our TV experience is changing
For years now we have had multiple screens in the home – at least two anyway, the PC and TV – though they never had much to do with each other. This is now changing. TVs are starting to connect – not just to a PC but more importantly to your smart phone and tablet. In fact, our hand-held systems used in conjunction with the interactive TV represents a major change in how we will digest entertainment going into the future.

Earlier this year Nielsen made a study of the use of tablets, smart phones and e-readers in the home. Nielsen’s survey found that the tablet and smart phones are more likely to be used while watching TV. (E-readers on the other hand were more likely to be used in bed –no surprise there.) In fact, 70 percent of tablet owners and 68 percent of smart phone owners said that they use their devices while watching TV. Moreover, using the tablet while watching TV constitutes the largest portion of time spent on the device – representing about 30% out of the total time spent. After TV, tablet owners spend about 21 percent of their time using the tablet in bed. The Nielsen report also surveyed what people are actually doing on their tablets and smart phones while watching TV. Most popular activities are checking email and searching for either related or unrelated content. Again not surprisingly, women are more likely to be connecting to social applications while men are more likely to be looking up sports scores.

The Nielsen report by itself shows clearly that we like to multi-task our tablet and smart phone use while watching TV. But is there something more compelling about the multi-screen phenomenon? In fact is it really a phenomenon at all? Whether or not there is truly a greater trend at work – TV networks, advertisers and technology companies are trying hard to put the multi-screen to better use.

Advertisers in particular are interested in managing ad campaigns that coordinate across the multiple screens in the home. ComScore recently published a report that measured advertizing effectiveness of ads that are coupled with synergistic multiple platform campaigns or “touch points”. The research shows that using synergistic touch points can actually reduce advertising cost in reaching TV audiences. A TV ad campaign that harnesses digital touch points can increase effective reach by 16% at the same overall budget. This is a compelling number – especially since we are still at the very early stages of this type of multi-front digital advertising activity.

Networks are increasingly looking for ways to increase engagement with their programs. And it is clear that the multi-screen environment accommodates social TV activities if for no other reason that it is easier to communicate over a smart phone or tablet rather than a TV remote. Additionally, who wants to overlay a bunch of Twitter chatter onto the beautiful HD display? Major networks are already making use of Twitter and Facebook to raise the level of dialog, recommendation and engagement of their programs. In fact, most networks these days publish Twitter hashtags associated with their programming – some shows even display hashtags on-screen during a program broadcast. This is especially true for reality entertainment and talent shows like Xfactor where audience participation voting for favorite performers is part of the program. All of this social activity works well when TVs are used in conjunction with multi-screen handhelds.

The trend is not lost on the TV OEMS. One practical use of a smart phone or a tablet is that they make for great TV remotes. Companies developing remote control software for tablets and smart phones enable users to customize the user interface and functionality. For example, your child’s remote UI can be designed to only have selections for children’s programming with animated design of the buttons. Adults in the family can design more complex remotes that provide access to a wide range of programming and applications.

TV companies in China have already started shipping tablets with their TV sets. Both Haier and Hisense have started shipping Android based tablets together with their higher end digital TV sets sold. In the USA Samsung together with Best Buy gave customers an Android based Galaxy Tab 10.1 for every 46 inch 3D Samsung HDTV they bought for a week back in August. Also, Sony offers a bundling discount for their Sony 16GB Tab for every Smart TV sold. It will be interesting to see how tablets with TVs will be promoted by the large OEMs at CES 2012.

Multiple technology start-ups are also proposing new ways to make use of a tablet or smart phone in conjunction with the TV. A good example of a multi-screen use case is the application called Into_Now – a company now owned by Yahoo. In the spirit of Shazam, a much-loved audio application that could record any song and identify the song title and artist, Into_Now can record a TV program or movie from your tablet or smart phone and in addition to identifying a show (down to the specific season and episode) display relevant information and metadata associated with the TV program. Into_Now also includes a social tagging and chatting capability which allows you to discuss shows that your are “Into….Now” with your friends. In fact, Into_Now’s preference engine algorithms use tag information that you and your friends can use to develop recommendations on other shows you may like.

Into_Now was also engaged in some interesting advertising campaigns. Before Yahoo purchased Into_Now (Twelve weeks after the company was started), Into_Now partnered with Pepsi Max where Into_Now users were rewarded with free Pepsi in exchange for tagging the “Clubhouse in the Corn” Pepsi Max commercial. An interesting combination of TV, hand-held device, social networking resulting in strengthening advertisement engagement.

Samsung also recently started promoting their new Galaxy tablet 7 plus, which comes with an application called Peel. Peel turns your smart phone or tablet into a preference based TV remote. The Peel application recommends programming options that may be interesting to you. The application also allows you to share over Facebook and Twitter. Samsung announced that their partnership with Peel is an example of their strategy to create enhanced user experiences.

Peel was started by some folks who worked at Apple and were involved in developing i-Tunes. They have implemented a compelling system to combine the power of a preference- based recommendation engine application for your smart phone or tablet that works together with a device called the Peel “Fruit” a device that sits near your TV and set-top boxes and works in conjunction with your smart phone or tablet to control all the input devices in your home. The Peel application together with the Fruit becomes a recommendation engine that is a universal remote tied to your social sphere – with a very compelling user interface to boot.

The smart phone or tablet nearby will accelerate the social interactivity associated with TV. And the level of social chatter about shows is being watched much more closely. Taking advantage of this trend, a technology company in the UK, called TV Genius, set up a website called Social TV Statistics. This site is updated daily and provides a list of the20 most tweeted TV shows aired in the UK. The statistics include the maximum daily tweets in a week as well as the maximum tweets in an hour. Recently the UK version of the XFactor was the most tweeted show with about 80,000 max daily tweets and 20,000 max hourly tweets. This is followed by “The Only Way is Essex” which has registered about 1400 max daily and 1200 max hourly tweets. (The large discrepancy most likely is also partly due to the nature of the Xfactor show which makes use of multiple engagement strategies such as viewer participatory voting as well as aggressive Twitter engagement through hash tag promotion etc.)

Social chatter is golden information to networks and advertisers and shows a level of engagement in a show or movie– which is more valuable than viewership statistics of old. Also, the general chatter in the tweets can also be aggregated to develop some meaningful insights about how people feel about the show as well as potentially how they feel about advertising associated with the show. TV networks themselves can also use tweet levels to engender further engagement in their programming by publishing tweet levels or other such information.

The use of a handheld device in conjunction with the TV also introduces interesting new use case possibilities. As we discussed in the first article, TVs are going to be able to recognize users and be able to recommend content tailored to viewer preferences. The handheld communication device is a great way for a TV to recognize a user. The hand-held device can also download information to the TV to further aid in preference generation. For example, say you were watching a movie on your tablet on an airplane. Half way through the movie, you need to shut down as the airplane starts to land. After you get home, the tablet can update the TV to let it know that you did not finish your movie. The TV can then provide you an option to see the rest of the movie on the TV set.
In fact, the portable nature of smart phones and tablets could give rise to improved applications that further bind tablets and smart phones to your TV watching. In fact, if OEMs could find a simple way to download your pictures and video from your tablets and smart phones it would make a big impact. Imagine that you come home from a day at the beach where you took amazing videos of your family on your smart phone. Wouldn’t it be nice if as you come into the house your phone asks if you would like to synch your new content to you home network – and it does it automatically. A few short seconds later you can call up the movies and pictures on your TV.
The knitting of the multi-screen enables media to move around the house. Today we already have the base protocols and standards in place to make this happen. Many TV companies have been trying to improve this dynamic. Apple is perhaps the most advanced. Apple’s Airplay is a system that already sets up the use of a tablet or smart phone to mirror what we see on a TV. In addition, it allows us to easily take the content off the PC and display it on our TV. This is a huge advantage. 95% of us are not making use of the HD content we produce with camera and camcorders- how many of us resort to looking at our photographs on a 9 inch computer display when in fact we could be using the high-definition screen in the living room.

Tablets and smart phones will also help us navigate growing cloud- based applications for TV. In fact, the use of the cloud will have huge benefits especially for the future of interactive TVs. By moving content and interactivity to the cloud, content can be viewed on any device. This allows one to watch TV anywhere and on any device.

The cloud is also important because it extends considerably the capabilities and future functionality of your TV. TV is not a platform that lends itself well to constantly updating software and increased CPU requirements. After all, it is easy to replace a PC, a Phone, or a tablet – but we hate to throw away the beautiful 60” OLED if it is already hanging on the wall and looks great.
The cloud in effect serves to future-proof your TV. Now applications can evolve, user interfaces can improve, and applications of all kinds can multiply. The heavy lifting will be taken care of in the cloud while our TV screen will do what it does best – provide a great picture.

With the cloud we will see expanded use of the TV for online sales of movies and TV shows – not to mention other retail sales. To date, other than I-Tunes there has not been very successful systems that allow digital sales of video to flourish. But that may be changing. Recently a large studio consortium announced a digital right locker system called Ultraviolet. The system establishes a streamlined way to buy digital video programming and store it on the cloud. Searching, purchasing and navigation will be much more efficient through the combination of the tablet or smart phone. The hand-held can be used to enter or swipe credit card data while you and your family review movie trailers or whatever it is that you are considering buying.
Again CES 2012 will be a key show to see how TV industry stakeholders will expand the use of the multi-screen multi-tasking. The multi-screen experience is becoming the new TV experience. Looking into 2012 we are sure to see many new applications by networks, advertisers and technology companies to take advantage of this new dynamic.

Note on Part III: Part III will look more closely at TV image quality and expected improvements we will see in TV display technology in the coming years and why the connected TV is driving new requirements in image processing.

Interactive TV Trends – How the TV Experience is Changing

The article below is the first in a three part series describing key interactive TV trends. This first article looks at new technologies to control the TV – and how the TVs future ability to recognize users will allow it to tailor content choices and preferences. The second article in the series will examine how multiple screens of the PC, tablet, smart phone and TV will alter the TV experience. Finally the third article discusses new trends in image processing and why major improvements in picture quality are still necessary

Credit: Soft Kinetic
Part 1: Where’s the Remote? Controlling the TV with your Gestures and Voice
The convergence of the internet and broadcast TV is changing the way we will interact with the TV set. Convergence is enabling the use of the TV for gaming, social interaction and new ways to watch content. As the internet and broadcast TV continue to intertwine – the way we interact to the TV will continue to evolve. This evolution will focus on finding new ways to fuse interactive functions to work well in the “lean back” TV experience. Nobody wants to “lean in” on the TV in the living room; this is why early attempts to simply graft a PC to the back of a TV were never going to create a useable interactive TV. The industry is finding new ways to bridge the internet more naturally into the TV experience. This is the first in a series of three articles explaining key trends in interactive TV and the technologies that are being developed to support them.

Improvements in the way we interact with the TV start with how we control the TV. Gesture recognition technologies are a very promising development – especially command gestures that do not require a remote. The Xbox Kinect is probably the most compelling example of the importance of this trend. The Kinect works by combining the use of a camera, and light emitter and receiver as well as voice control. The combination of these capabilities enables the Kinect to recognize you, watch and understand your physical movements and gestures, as well as understand voice commands. This results in an interactive experience that enables remote free gesture control. You can control the TV and games by using hand gesture. For games, this is great as it allows a more immersive experience.

For example, by detecting your body’s movements and articulation – your movements are translated to your avatar representation on the screen. For action games, you simply mimic the movements as though you were skiing, dancing or playing tennis. This technology can also be used to control the viewing experience on TV. A typical example is viewing menu pages or video thumbnails – you can move options or pages around by a wave of your hand. Future advances could allow for more intuitive controls as well as systems that integrate coordinated gesture controls from your tablet to your PC.

Apple’s new Siri improved voice control is also a promising technology that could have a place in the interactive TV world. Siri enables people to speak to machines in a more natural way. The Siri technology includes a semantic sensitivity – that can find meaning in your statement to help it understand you better. This has huge implications in the interactive TV world where we need this type of personalized control especially when we convey our intentions, preferences and feedback to search and discovery.

There are consistent rumors that Apple is working on an Apple branded TV or at least an improved version of their Apple TV media box. They could easily apply the Siri voice control capability to the control of the TV. (As we will discuss in the next series, Apple’s combination of the multi-screens in the home and their elegant interface to the cloud creates a TV ecosystem that could pose a threat to existing TV OEMs.)

TV OEMs like Samsung have been experimenting with remote free gesture control for a while. Samsung, Toshiba and others have shown these technologies at CES over the years. But there is no large scale market availability. That said, in China, mega TV maker Hisense announced that it will be shipping a remote-free gesture control TV starting this month. On the voice side, besides Xbox, there are several electronics companies that have been working on standalone voice activation TV remotes. Voice activation on a remote or a tablet may have a lot of advantages. For one, it is easier for the TV to isolate who it should listen to when there are several people in the room. It will be interesting to see what will come out at CES 2012 on these technologies.

Remote free gesture and voice control are excellent solutions for overcoming the lean back environment of the living room TV. And these capabilities will only get better as the underlying software, user interface, electronic program guide and menu systems improve. The methods of controlling the TV will also become more efficient as TVs take on their ability to personalize their menus for either an individual or a group in the household. In short, TVs will have the capability to recognize us and present a tailored list of menus and services when we come into their vicinity.

Personalization in general is a key trend on the internet. We see that many interactive programs attempt to improve their services by personalizing their user experiences. Examples include the voting function “thumbs up” or “thumbs down” on Pandora allowing it to tailor an individual’s song selection. The interactive TV will also take on capabilities to personalize menu options as well as content and service preferences. This represents a new level of convenience in terms of controlling the TV, as the TV will only be presenting options that you really care about.

To enable this, smart phones or tablets interacting with the TV through WiFi, Bluetooth or other interactive technologies can also identify users to the TV. Alternatively or in addition, TVs could use camera technology just like cameras are used in our hand held devices and laptop computers. Cameras together with facial recognition algorithms can do a good job to see who walks into the room. Imagine entering your living room or den and the TV automatically brings up options dialed into your specific preferences and interests. The TV can also set up all kinds of services and capabilities that are tied to your needs. The system can stand ready to serve up your favorite TV shows, download music for your run or commute, enable or disable your home security system, regulate the sprinkler system if it is raining, and update you on the whereabouts of family and friends.

Preferences do not have to be limited to individuals. TVs will also be able to recognize groups of people such as your whole family sitting together, the kids only or even the family dog and will serve tailored content and service options appropriate to each group.

The TV can be programmed to personalize its menus when the entire family or various subsets of the family is sitting in the front of the TV. The TV greets the family and immediately serves up some appropriate video, audio or service options. The father can ask the TV through a voice command to display the photographs from the recent family trip to Hawaii and provide some Hawaiian background music. During the slide show, the family can also ask the TV to dial in a distant grandparent to join the review of slides. If a child asks a question such as when Hawaii became a state – the TV can search video, webpage or blog content on Hawaii’s history. Likewise, if the family suddenly has an interest in buying a surfboard, the TV can put together a list of interactive ads from local surfboard shops.

Of course, it can be unnerving for some to contemplate this type of interaction with a TV or any machine for that matter. Thoughts of Space Odyssey 2001 may come to mind. There is no doubt that the preferences and choices made through an interactive TV represent valuable information to advertisers and retailers. The technology should also provide consumers strong privacy controls. But the advantages of personalization will outweigh the concerns of letting “HAL” loose in the home. In terms of control – it is much easier to control what you want if the TV is familiar with your preferences.

The technology driving gesture control, voice commands and cameras with facial recognition are available today. We are likely to see incorporation of these types of concepts in TVs next year. As the internet makes further inroads into our living room TV – we can expect to see the use of these tools to improve our ability to interact and maintain our feet-up laid-back position on the couch.