The fast developing potential for truly useful applications that can convert our personal companions, our smartphones, from basic input/output devices into truly thinking machines is dazzling, science fiction-like, and almost staggering in their scope and range of capability. With well over a billion devices deployed, containing a bewildering array of sensors and processing power to design for, thousands of new, useful, and inexpensive applications are being offered daily to exploit these marvelous devices. But most, unfortunately, are still living in the age of Pong — applications designed to mindlessly entertain a user while they are actively interacting with their phone.
A simple litany of these applications would quickly become tedious and boring; and no matter how long the list, it would not reveal the growing influence and power that comes with cognitiveness and connectedness. We’re living in an era of cognitive computing that has the potential to deliver an entirely new type of user experience, one that you don’t need to engage in actively. Technology works best when it’s invisible. For instance, we no longer think about the transmission in our cars, its gears smoothly and unobtrusively give us the right amount of power and, at the most economical level, it just works, we take it for granted, and it makes our lives better.
It’s a few years from now and I have an appointment. My personal companion is talking to my car before I even get into it. My companion really cares about me and wants to make sure the car knows where I’m going and that I am alert while I drive. My car is not totally autonomous yet and it’s best if I’m awake and alert when driving. My personal companion will inform the car I went to bed at midnight, have been up since 5:00am, have only had one cup of coffee, spent two hours checking email, and I’m now slumping in my seat with a glazed look on my face. The worst thing in the world for me at this point is to let the interior temperature get too comfortable or to play soothing music. I’ll be sound asleep in 20 minutes and I have an hour and a half drive ahead of me in mixed to heavy traffic, with a strong possibility of a light tapping of raindrops to lull me to sleep.
My personal companion senses this and goes on high alert. Because my device is cognitively proactive, it is already instructing the car’s driver monitoring cameras and steering wheel vibrators to be alert, while at the same time listening to the radio and changing stations at the end of a news story or song. I could have gone a step further and authorized the phone to prevent me from turning off the radio which would instruct the car to limit my speed to the speed limit—that would really annoy me, so I didn’t opt for that cute little feature.
And then there was the little talk we had on the way to the car. My personal companion, having all the data on my morning activities so far, knowing where I’m going, and having to get back from, said, “Jon, do you really have to make this trip? Can you reschedule it? You have an opening tomorrow and Friday.”
As I get out of the car, the glasses (which are paired to my personal companion) have visited the social media sites and shown me the bio and a photo of the person I’m going to meet. It found a lot about her company and her current projects, and the last three she did. I blink and the string of emails that got me to make this trip show up, thankful I never had to take my hands out of my pockets this chilly, rainy day.
Entering the lobby I take off the glasses and approach the counter, while bringing out my phone to show the attendant who it is I am here to see. At the same time, it sends my information to his terminal. He hands me my badge and my smart companion quickly informs me the badge doesn’t allow me to go beyond the lobby – this is going to be a very short meeting. I quickly review the email thread again, and my companion highlights the unresolved issues. It knew not to do it while driving here because it would have taken my attention away. I suppose that’s another reason to buy one of those autonomous cars that will be available next year.
It’s been a long day, so I go to the local pub to meet a friend for a snack and a few drinks. My watch is informing my personal companion my heart rate is a little elevated from the drive home in rush hour traffic and maybe that extra cup of coffee with the client. “Thanks for the update,” I say snidely. My phone’s not judgmental about my tone but I know at times it can and should be. I like that.
Dinner and conversation at the pub is pleasant and the hours slip by when, suddenly, my phone starts saying “Jon” progressively louder. Pretty soon everyone in the pub can hear my little loud-mouthed companion. I try ignoring it but it won’t stop so I look at it and it says, “Please take a breathalyzer test now if you would like to drive your car home today.” It seems my tattletale smart watch has been keeping track of exactly how many times I’ve lifted something to my face. Even more intelligently, based on my heart rate and respiration (again detected by the tattletale smart watch), my personal companion suggests I start drinking water.
Looking around the room, I notice a young woman discretely dip an iPod sized device in her drink and then a few minutes later excuse herself to go to the toilet. She’s sampled her drink and is going to analyze it. What does that say about what she thinks of her date? Why doesn’t she just leave? Then I remember, people can also check for sugar content with those samplers and she might be diabetic.
I hesitate. I don’t want to fail my own test, so maybe a little water is a good idea. I’d order coffee but even the meditation app in my phone can’t overcome late night caffeine ingestion. My personal companion is on the job already and suggests a little walk would be a good idea. I say good night to my friend and march outside to pace the parking lot and think about the day. I start wondering if I could probably get rid of my doctor, my lawyer, and my mechanic, and have my phone do everything. Maybe I have had too much to drink.
With the new low power consumption processors, my companion is always listening and monitoring. It knows if I’m sitting, walking, running, sleeping, in the office, my car, or a restaurant. Knowing that, it makes recommendations and is able to take independent actions.
That’s all done in the phone along with the security ID stuff that checks my eyes, face, and heartbeat via my watch and, if necessary, my companion will ask to check my fingerprint. I haven’t had to have my fingerprint checked for some time due to the behavior sensing my companion does. Talk about being paired, this thing really knows me.
As I approach home, the lights in the bedroom and hallway come on and the garage door opens as I enter the driveway — no need to push buttons like in the old days. In bed, reading a book on my phone, the lights in the room start to dim. It was 11:00pm and my companion lets me know it’s time to try to get some sleep. I put the phone down and let its induction charger work while I snooze. It will be a comforting sleep, as relaxation sounds play. I don’t have to worry about the alarm, based on my calendar; my personal companion will wake me at the appropriate time.
The lights come on the next morning and I can smell the coffee. My companion has turned on the machine and signaled my PC to boot up and collect email. These were not timer settings; these were intelligent functions based on what I had to do today including what time I got up. I slept in this morning till 6:15am.
The day went pretty fast, no major problems. The call with the client as suggested by my personal companion really helped. She had come up with a way to get around the major obstacle and, as luck would have it, could use my help with the new approach.
By 6pm, I was ready to knock off and take a little diversion. I’d been working on an idea to catch rainwater, taking the little bit we get from the roof and route it to a little pond and then draw on that water to feed the plants. The grass was long gone, replaced with succulents we maintained to support the birds, bees, and tiny critters that clean up the place at night. I had the idea but it was hard getting my mental visualization across to the roofing people and I couldn’t express it properly on paper. I took the phone out and pointed at the sloping, flat-topped roof and walked the perimeter of the house. My personal companion faithfully captured every detail with its built in 3D camera, measuring every inch and all the obstacles like doorways, trees, vertical support, nearby fences, building an accurate 3D model as we went.
Back in my office, I tossed the phone on the desk and it charges itself. As I opened up an app on my computer, the phone automatically connected itself and I was pulling the 3D model out of it in minutes. Now, with a clever app I was able to add my design. The app was smart enough that I was cautioned not to make the cantilevered gutter too wide. After about two hours of trial and error, all done with a mouse and some verbal commands, it was finished and perfectly accurate. I hit the print button and a PDF file was created, complete with dimensions and images of the house. It was good enough for framing, or so I thought. I emailed it to the roofer and went on to dinner.
Later I looked at my work of art on my phone. The six inch 4K screen was fantastic and the eye tracking cameras adjusted the image’s perspective as I looked at it, checking different angular views. It was like 3D but without the glasses. Putting the phone in its docking station, it connects wirelessly and automatically to my 4K TV, which I also use for work.
I wasn’t tired and didn’t want to watch TV, so I decided to finish my review of a new AR e-book that had been sent to me. I used to think these were kid’s books but this one was about WWII and tapping on a paragraph would give me a choice of a 3D map or Google street views. With the eye tracker I could look around in the scene. It really made a difference. The book was well written in addition to having a rich database of images behind it.
The night lights came on and my personal companion started playing soft music, a cue to wind things down. I had a big day tomorrow, a new game to test that was more likely to test me.
By the time I got to the office, the mail (yes we still got regular post office mail) was on my desk. I still looked forward to the mail. It was always a pleasant surprise to look at postage stamps and open an envelope. One of the envelopes was in Chinese, addressed to me in English. Opening it revealed an official looking letter, but in Chinese except for the date and the line with a dollar sign and a nice number in it. I had bid on a project with a major Chinese computer company and I was hoping this was the approval. I picked up my phone and scanned the document. It was immediately translated, storing the results as I read so I could print it out later. This was a happy day. I did get the contact, now I had to do the job. I looked over the document to see if they had accepted my delivery date. They hadn’t and moved it up two weeks. That was annoying and, as I was working up my anger, my phone grabbed by attention. The screen had turned green and a message in a light yellow italic font floated across the screen: “Relax, you don’t have to accept it, make a counter offer. Take a breath. Stand up. Walk around; your heart rate is elevated.” “Yes dear,” I say to my personal companion, and take the advice.
Later in the afternoon, after I had thought about it, I came up with what I felt was a reasonable compromise. I started dictating to my phone, having told it I wanted the file to be in English and Chinese. After a few minor corrections, I said, “sign it, send it, and file it.” Done, done, and done.
It was the end of the day and time for a little relaxation. I slid my phone into a housing and then put on the HMD. I was going in, into a 3D world and kick some dragon butt. The HMD used all, and I mean ALL, the sensors in the phone, to locate me, track my eyes, track my head, and even watch where my hands were. The wide screen phone gave me pretty good peripheral vision and its super high res 4k screen with 120 Hz refresh rate and full UHD color gamut made the world look as realistic as possible. Certainly good enough to get me, Mr. Pixel, to suspend disbelief. OK, here. dragon dragon, come on out, dragon.