Interactive TV Trends – How the TV Experience is Changing, Part III

This is the third article in a three part series discussing key trends in TV. The first article looked at how new interface technologies are enabling new ways to control our TVs. The second article focused on the multi-screen TV experience. This article focuses on how interactive TV trends are driving the need for improvements in TV image quality.

Full HD is not enough for Future TV
Some might believe our latest flat panel televisions represent the zenith of picture quality. This is not surprising given we often hear that 1080P resolution or “Full HD” are “future proof” technologies. The oft-cited reasoning is that for a given screen size, viewed from a normal watching distance, the acuity of the eye cannot discern resolutions beyond Full HD. Another reason why Full HD is considered future proof is because actually a very small percentage of video content is even broadcast at this resolution. Most digital pay TV broadcasting systems transmit in lower resolution formats – the industry is still catching up.

Certainly, for those looking to buy their next TV set – no one should be concerned that 1080P is not good enough. Considering the horizon of time people buy and keep a TV set which is about 8 years– a consumer cannot go wrong with “Full HD”. But for people interested in where the industry is going in the long term –looking out over the next ten years, our image quality is going to see massive improvements making today’s TV technology look primitive.

Part of the reason why we can expect big improvements in TV video quality has to do with our superior eyesight. Our capacity to see is many multiple orders of magnitude greater than what our TVs can display. For example, a Full HD TV displays about 2 million pixels of video information. In real life, one of our eyes processes about 250 million pixels – but since we have two eyes channeling vision to our brains – our effective vision makes use of greater than 500 million pixels of video information. And while it is true that we can only discern a limited resolution from a given distance – our eyesight is also sensitive to contrast, color saturation, color accuracy and especially movement. All these areas are where TV systems can improve.

Detractors may argue TVs do not have to be perfect – just a reasonable representation. Others may argue that consumers only care about TV size and price that TV quality is not a selling point. But I argue TV image quality does matter – quality has always had to keep pace with the growing size of TV screens. TVs will continue to get larger – requiring improvements in resolution as our room sizes will start to limit viewing distance. Also, the nature of interactive TV and future 3D systems will make us want to sit closer to the TV set – again mandating video quality improvements.

Interactive TV’s Make You Sit Closer
Interactive TVs will bring games, virtual worlds and new video applications drawing us physically closer to the TV screen. Gaming is a huge industry- with almost $50B spent on gaming consoles, software and accessories. Virtual world games are increasingly popular. “World of Warcraft” is a massively multiplayer online role-playing game with over 10 million subscribers. All kinds of social virtual worlds such as the Sims, Second Life, IMVU and Club Penguin are attracting millions of players. IMVU, with over 50 million registered users, is a social game where people can develop personal avatars and spend time in virtual worlds chatting and interacting. While many of these games are still played on PCs – migration to the living room TV is inevitable. Console games have already shown the way – the size and immersive nature of the large screen TV will draw others into the living room as well.

3D display will also drive the need for improving display resolution and image quality. Sure, everyone hates 3D glasses – but technology will continue to evolve and glassless 3D displays will continue to improve and come down in price. There will be applications that consumers will demand in 3D such as sports – people will see the advantages of watching close up sports games on the large screen display in vivid, artifact free video.

OEMs and broadcast equipment companies are investing heavily in supplying the infrastructure to make this happen. 3D advertising will take on more importance – imagine having the option to tour a car or a house in extremely vivid 3D. On the entertainment side, movie and video directors will become much better at using 3D perspectives in such a way to take advantage of image quality improvement. Today 3D effects are more like a gimmick – watch the arrow fly into the room for example. But going forward directors will make more subtle use of 3D adeptly drawing viewers into to the film or the show. On a beautiful large screen display with ultra high resolution and image quality, viewers will practically feel like they are part of a movie or scene.

3D also opens up a world that we could only dream about when matched with the power of the internet. For example, the evolutions of virtual worlds and their capabilities becomes much more compelling with large screen displays. A simple example is virtual tourism and world exploration. Just as Google has taken a picture of all the street views of the world, there is no reason we cannot build a 3D model of the whole terrestrial experience on earth in a few years. Imagine then the capability to walk around the world as a virtual tourist and view the world from the comfort of your 3D television.

As virtual worlds improve and evolve, new immersive ways to interact with large screen TVs will continue to evolve. Many social activities come to mind as well as the concept of participating or viewing in e-sports. E-sports are virtual sports games that can also be viewed by others. The prospects for e-sports are boundless and limited only by imagination. Virtual bullfights, gladiator battles, racing events will be watched on-line the same way we watch football games today.

The display-use model will also change over time. Today our concept of a display is a TV set that sits in the living room – a piece of functional furniture. With the advent of new display materials like OLED, display will transform from furniture to architectural material. In fact there is no reason why the wall in your den cannot become a display. In fact, why stop with the wall? Imagine the immersive feeling of the ceiling, floor, and walls all around built of display – it’s the video equivalent of surround sound. In fact, the architectural use of display could add interesting use cases beyond entertainment.

For example, inlaid architectural materials can appear in almost in any room around the house. Touch screen uses in the kitchen, can provide not only control but also interactive recipe applications and videos on cooking instructions. Bathroom walls can provide wallpaper backgrounds or any kind of networked information that we already see on our PCs. Inlaid display technologies will appear on appliances as well as anywhere people need information or help with controls. The point of all this is that again there will be many reasons in the future of us needed to get close to the screen – and all this near proximity will demand increases in display quality.

TV Development Underway
Already major TV OEMs are working on the next step up in resolution over Full HD. There are multiple propositions in development for higher order resolution TV systems. TV OEMS are already demonstrating “4KX2K” systems that provide 4096 X 2160 pixel arrays. Even beyond “4KX2K” is Ultra High Definition (UHD) which provides 7,680X4320 pixels resolution which equals 33 million pixels or about 16 times the number of pixels used by Full HD systems. UHD was first introduced by Japan’s national TV broadcaster NHK in 2003. NHK, marketing the resolution as “Super Hi-Vision” had to build the cameras and display technology from scratch to be able to create a UHD demonstration system. Since then NHK has displayed the system at numerous broadcasting shows. Toshiba, LG and Panasonic showed UHD systems at CES 2011 – likely more UHD sets will be shown in 2012. UK’s BBC also is interested in this format. The BBC announced plans to provide UHD coverage of the 2012 London games.

In addition to higher resolution, OEMs continue to invest in superior display technologies like organic light emitting diode (OLED) displays. OLEDs have several advantages over LCD and plasma display technologies. For example, OLED do not make use of a backlight and emit light directly. Direct emission results in a much more vivid display of color, contrast and viewing angle over LCDs. Since there is no backplane in OLED TVs, OLEDS are a much more power efficient and lower in weight. OLED displays are also flexible – opening up new opportunities to use displays in various new applications in architectural display and even clothing.

OLEDs also have a very high response time over LCD. In fact, the relative low response time of LCD, required the industry to introduce all kinds of approaches to compensate by introducing frame rate conversion techniques. OLEDs response time increases response time by a factor of 1000 over LCD allowing for a much better display motion performance.

Improvements will also need to continue on the broadcast side. Higher resolution TVs consume bits at an alarming rate. For example, uncompressed ultra high HD would demand 24Gbps a major jump over ~1.5Gbps required for Full HD. Any increases in resolution will demand major improvements in data compression as well as networking, storage and broadcasting capacity.

But the march of improvements will continue. As TV screens get larger and the way we use these screens draw us in closer – the need for improved image quality will also continue to improve.

Our TV experience will change dramatically over the next ten years. As these series of articles have discussed the whole TV experience will continue to morph the way we spend our time watching large screen displays. 2012 will bring some interesting signs about how all this will play out. 2012 we will see OEMs developing much better ways to interact with TVs – our ability to control the TV through new remote technologies and improvements in finding and sharing content will make major advances. We can expect more use of our hand held tablets and smart phone devices joining us in front of our TV sets. Interactive TV will bring, not only more sources of content, but also new tools to help recommend as well as share content and media that we really want to see. Finally, the way we use TV will be much more immersive demanding major improvements in the video quality in TVs over what we have today.

Interactive TV Trends – How the TV Experience is Changing

The article below is the first in a three part series describing key interactive TV trends. This first article looks at new technologies to control the TV – and how the TVs future ability to recognize users will allow it to tailor content choices and preferences. The second article in the series will examine how multiple screens of the PC, tablet, smart phone and TV will alter the TV experience. Finally the third article discusses new trends in image processing and why major improvements in picture quality are still necessary

Credit: Soft Kinetic
Part 1: Where’s the Remote? Controlling the TV with your Gestures and Voice
The convergence of the internet and broadcast TV is changing the way we will interact with the TV set. Convergence is enabling the use of the TV for gaming, social interaction and new ways to watch content. As the internet and broadcast TV continue to intertwine – the way we interact to the TV will continue to evolve. This evolution will focus on finding new ways to fuse interactive functions to work well in the “lean back” TV experience. Nobody wants to “lean in” on the TV in the living room; this is why early attempts to simply graft a PC to the back of a TV were never going to create a useable interactive TV. The industry is finding new ways to bridge the internet more naturally into the TV experience. This is the first in a series of three articles explaining key trends in interactive TV and the technologies that are being developed to support them.

Improvements in the way we interact with the TV start with how we control the TV. Gesture recognition technologies are a very promising development – especially command gestures that do not require a remote. The Xbox Kinect is probably the most compelling example of the importance of this trend. The Kinect works by combining the use of a camera, and light emitter and receiver as well as voice control. The combination of these capabilities enables the Kinect to recognize you, watch and understand your physical movements and gestures, as well as understand voice commands. This results in an interactive experience that enables remote free gesture control. You can control the TV and games by using hand gesture. For games, this is great as it allows a more immersive experience.

For example, by detecting your body’s movements and articulation – your movements are translated to your avatar representation on the screen. For action games, you simply mimic the movements as though you were skiing, dancing or playing tennis. This technology can also be used to control the viewing experience on TV. A typical example is viewing menu pages or video thumbnails – you can move options or pages around by a wave of your hand. Future advances could allow for more intuitive controls as well as systems that integrate coordinated gesture controls from your tablet to your PC.

Apple’s new Siri improved voice control is also a promising technology that could have a place in the interactive TV world. Siri enables people to speak to machines in a more natural way. The Siri technology includes a semantic sensitivity – that can find meaning in your statement to help it understand you better. This has huge implications in the interactive TV world where we need this type of personalized control especially when we convey our intentions, preferences and feedback to search and discovery.

There are consistent rumors that Apple is working on an Apple branded TV or at least an improved version of their Apple TV media box. They could easily apply the Siri voice control capability to the control of the TV. (As we will discuss in the next series, Apple’s combination of the multi-screens in the home and their elegant interface to the cloud creates a TV ecosystem that could pose a threat to existing TV OEMs.)

TV OEMs like Samsung have been experimenting with remote free gesture control for a while. Samsung, Toshiba and others have shown these technologies at CES over the years. But there is no large scale market availability. That said, in China, mega TV maker Hisense announced that it will be shipping a remote-free gesture control TV starting this month. On the voice side, besides Xbox, there are several electronics companies that have been working on standalone voice activation TV remotes. Voice activation on a remote or a tablet may have a lot of advantages. For one, it is easier for the TV to isolate who it should listen to when there are several people in the room. It will be interesting to see what will come out at CES 2012 on these technologies.

Remote free gesture and voice control are excellent solutions for overcoming the lean back environment of the living room TV. And these capabilities will only get better as the underlying software, user interface, electronic program guide and menu systems improve. The methods of controlling the TV will also become more efficient as TVs take on their ability to personalize their menus for either an individual or a group in the household. In short, TVs will have the capability to recognize us and present a tailored list of menus and services when we come into their vicinity.

Personalization in general is a key trend on the internet. We see that many interactive programs attempt to improve their services by personalizing their user experiences. Examples include the voting function “thumbs up” or “thumbs down” on Pandora allowing it to tailor an individual’s song selection. The interactive TV will also take on capabilities to personalize menu options as well as content and service preferences. This represents a new level of convenience in terms of controlling the TV, as the TV will only be presenting options that you really care about.

To enable this, smart phones or tablets interacting with the TV through WiFi, Bluetooth or other interactive technologies can also identify users to the TV. Alternatively or in addition, TVs could use camera technology just like cameras are used in our hand held devices and laptop computers. Cameras together with facial recognition algorithms can do a good job to see who walks into the room. Imagine entering your living room or den and the TV automatically brings up options dialed into your specific preferences and interests. The TV can also set up all kinds of services and capabilities that are tied to your needs. The system can stand ready to serve up your favorite TV shows, download music for your run or commute, enable or disable your home security system, regulate the sprinkler system if it is raining, and update you on the whereabouts of family and friends.

Preferences do not have to be limited to individuals. TVs will also be able to recognize groups of people such as your whole family sitting together, the kids only or even the family dog and will serve tailored content and service options appropriate to each group.

The TV can be programmed to personalize its menus when the entire family or various subsets of the family is sitting in the front of the TV. The TV greets the family and immediately serves up some appropriate video, audio or service options. The father can ask the TV through a voice command to display the photographs from the recent family trip to Hawaii and provide some Hawaiian background music. During the slide show, the family can also ask the TV to dial in a distant grandparent to join the review of slides. If a child asks a question such as when Hawaii became a state – the TV can search video, webpage or blog content on Hawaii’s history. Likewise, if the family suddenly has an interest in buying a surfboard, the TV can put together a list of interactive ads from local surfboard shops.

Of course, it can be unnerving for some to contemplate this type of interaction with a TV or any machine for that matter. Thoughts of Space Odyssey 2001 may come to mind. There is no doubt that the preferences and choices made through an interactive TV represent valuable information to advertisers and retailers. The technology should also provide consumers strong privacy controls. But the advantages of personalization will outweigh the concerns of letting “HAL” loose in the home. In terms of control – it is much easier to control what you want if the TV is familiar with your preferences.

The technology driving gesture control, voice commands and cameras with facial recognition are available today. We are likely to see incorporation of these types of concepts in TVs next year. As the internet makes further inroads into our living room TV – we can expect to see the use of these tools to improve our ability to interact and maintain our feet-up laid-back position on the couch.