Next Major Step in AI: On-Device Google Assistant

The ability to have a smartphone respond to things you say has captivated people since the first demos of Siri on an iPhone over 7 years ago. Even the thought of an intelligent response to a spoken request was so science fiction-like that people were willing to forgive some pretty high levels of inaccuracy—at least for a little while.

Thankfully, things progressed on the voice-based computing and personal assistant front with the successful launch of Amazon’s Alexa-powered Echo smart speakers, and the Google Assistant found on Android devices, as well as Google (now Nest) Home smart speakers. All of a sudden, devices were accurately responding to our simple commands and providing us with an entirely new way of interacting with both our devices and the vast troves of information available on the web.

The accuracy of those improved digital assistants came with a hidden cost, however, as the recent revelations of recordings made by Amazon Alexa-based devices has laid bare. Our personal information, or even complete conversations from within the privacy of our homes, were being uploaded to the cloud for other systems, or even people, to analyze, interpret, and respond to. Essentially, the computing power and AI intelligence necessary to respond to our requests or properly interpret what we meant required the enormous computing resources of cloud-based data centers, full of powerful servers, running large, complicated neural network models.

Different companies used different resources for different reasons, but regardless, in order to get access to the power of voice-based digital assistants, you had to be willing to give up some degree of privacy, no matter which one you used. It was a classic trade-off of convenience versus confidentiality. Until now.

As Google demonstrated at their recent I/O developer conference, they now have the ability to run the Google Assistant almost entirely on the smartphone itself. The implications of this are enormous, not just from a privacy perspective (although that’s certainly huge), but from a performance and responsiveness angle as well. While connections to LTE networks and the cloud are certainly fast, they can’t compete with local computing resources. As a result, Google reported up to a 10x gain in responsiveness to spoken commands.

In the real-world that not only translates to faster answers, but a significantly more intuitive means of interacting with the assistant that more closely mimics what its like to speak with another human being. Plus, the ability to run natural language recognition models locally on the smartphones opens up the possibility for longer multi-part conversations. Instead of consisting of awkward silences and stilted responses, as they typically do now, these multi-turn conversations can now take on a more natural, real-time flow. While this may sound subtle, the difference in real-world experience literally shifts from something you have to endure to something you enjoy doing, and that can translate to significant increases in usage and improvements in overall engagement.

In addition, as hinted at earlier, the impact on privacy can be profound. Instead of having to upload your verbal input to the cloud, it can be analyzed, interpreted, and reacted to on the device, keeping your personal data private, as it should be. As Google pointed out, they are using a technique called federated learning that takes some of your data and sends it to the cloud in an anonymized form in order to be combined with others’ data and improve the accuracy of its models. Once those models are improved, they can then be sent back down to the local device, so that the overall accuracy and effectiveness of the on-device AI will improve over time.

Given what a huge improvement this is to cloud-based assistants, it’s more than reasonable to wonder why it didn’t get done before. The main reason is that the algorithms and datasets necessary to run this work used to be enormous and could only run with the large amounts of computing infrastructure available in the cloud. In addition, in order to create its models in the first place, Google needed a large body of data to build models that can accurately respond to people’s requests. Recently, however, Google has been able to shrink its models down to a size that can run comfortably even on lower-end Android devices with relatively limited storage.

On the smartphone hardware side, not only have we seen the continued Moore’s law-driven increases in computing power that we’ve enjoyed on computing devices for nearly 50 years, but companies like Qualcomm have brought AI-specific accelerator hardware into a larger body of mainstream smartphones. Inside most of the company’s Snapdragon series of chips is the little-known Hexagon DSP (digital signal processor), a component that is ideally suited to run the kinds of AI-based models necessary to enable on-device voice assistants (as well as computational photography and other cool computer vision-based applications). Qualcomm has worked alongside Google to develop a number of software hooks to neural networks they call the AndroidNN API that allows these to run faster and with more power efficiency on devices that include the necessary hardware. (To be clear, AI algorithms can and do run on other hardware inside smartphones—including both the CPU and GPU—but they can run more efficiently on devices that have the extra hardware capabilities.)

The net-net of all these developments is a decidedly large step forward in consumer-facing AI. In fact, Google is calling this Assistant 2.0 and is expected to make it available this fall with the release of the upcoming Q version of Android. It will incorporate not just the voice-based enhancements, but computer vision AI applications, via the smartphone camera and Google Lens, that can be done on device as well.

Even with these important advances, many people may view this next generation assistant as a more subtle improvement than the technologies might suggest. The reality is that many of the steps necessary to take us from the frustrating, early days of voice-based digital assistants, to the truly useful, contextually intelligent helpers that we’re all still hoping for are going to be difficult to notice on their own, or even in simple combinations. Achieving the science fiction-style interactions that the first voice-based command tools seemed to imply is going to be a long, difficult path. As with the growth of children, the day-to-day changes are easy to miss, but after a few years, the advancements are, and will be, unmistakable.

Published by

Bob O'Donnell

Bob O’Donnell is the president and chief analyst of TECHnalysis Research, LLC a technology consulting and market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on Twitter @bobodtech.

4 thoughts on “Next Major Step in AI: On-Device Google Assistant”

  1. Very often, ladies really feel pressured to stay in unhealthy marriages, and men benefit from it, usually abusing and causing suffering to those women as a result of they know their new brides cannot just depart them and discover someone else. Anyway, Kiev has the very best feminine choice in Ukraine – you simply have to go away the principle space and go straight to the middle and give it some thought a little bit bit, you can get the highest quality, identical to anyplace on this planet. It is simple, free paid possibility as properly and is straight forward. Being an inexperienced driver and with no public transport working, she had no possibility however to drive her family to Lviv, a metropolis in western Ukraine, for her father-in-law’s mind tumour operation. Following the 2014 Russian aggression in opposition to Ukraine, the league misplaced its Donetsk team which was disbanded due to the Russian occupation of eastern Ukraine in addition to Crimea.

    Up till the 2000s, ukraine as a former soviet nation had a very completely different cultural setting. The Ukrainian Women’s Union (Ukrainian: Союз українок) (1920-1938) was probably the most influential women’s group operating amongst Ukrainian women outside of Soviet Ukraine. Between 1921 and 1930, their objective was to ascertain financial and cultural buildings which may very well be used to improve the standard of life of all Ukrainians and produce their nationalist cause to the eye of worldwide organizations and by 1930 the union had between 50,000 and 100,000 members. Some 80 regional branches ukrainian women representing round 1,a hundred native organizations were formed. Later the staff at first current as a division of the principle Metalist membership, in 2006 was taken over by a neighborhood construction company. Notice that Zhilstroi-1 is a Soviet name for building firm out of Kharkiv which in its title actually carries a number. The brand new Ukrainian league included a number of teams that participated in the former Soviet championship reminiscent of Lehenda Chernihiv, Area Kyiv, Olimp Kyiv (former Soviet champion Nyva Baryshivka), Dynamo Kyiv, ZHU Zaporizhia, Luhanochka Luhansk, Dnipro Dnipropetrovsk, and lots of others.

    Some six groups that beforehand competed in Soviet competitions (in 1990 and 1991) chose to not enter the brand new Ukrainian league in 1992. Extra teams withdrew from competitions after 1993 inflicting the second tier (First League) to be disbanded for the subsequent 20 years. They are a lot more romantic than you believe. It goes with out saying that numerous girls try for monetary stability of their lives, and girls from Ukraine are not any exception. Vol. 41, no. 607. Lviv, Ukraine. Origins girls’s football within the territory Ukraine. Channel of Daria Rakitska (About girls football and sport). Also Russian language continues to be dominant in Ukrainian sport and significantly in women football where a lot of teams named in Russian method i.e. Donchanka, Zhilstroi, Voskhod, Iuzhanka and others. The first Ukrainian championship introduced a brand new Donetsk group (as Tekstylnyk Donetsk) that started from the primary League and ultimately would become one of the vital successful clubs in the league. The first Ukrainian championship consisted of 18 teams that have been cut up into two divisions, the upper League (10 groups) and the first League (8 teams). Ukrainian Championship. High League. Ukrainian ladies are perfect for marriage, as they’re nice cooks, loyal partners, caring mothers, and lovely ladies.

    In case you have time and finance, romance tours might be ideally suited for you, and you’ll meet your future wife in particular person. There is no time for any person to provide you with training and classes. At the identical time the Bubka college of Olympic reserve which provided athletes for the workforce was evacuated to Bakhmut. Whereas the main Donetsk team declined, Chernihiv footballers acquired a notable competitiveness boost from Kharkiv, Prykarpattia and Azov regions. Whereas most ladies choose males who provoke the connection, Ukrainian women are generally not prone to do. For certain, you do not must be a “wifebooty” to draw a Ukrainian belle, but it’s best to never be arrogant to her or treat her as if she is inferior to you. And people who’ve married Ukrainian women are by no means disenchanted with their wives. Because they represented Ukrainian nationals living in different sovereign states, the organization solidified their battle for gender recognition with one for nationalist goals.

Leave a Reply

Your email address will not be published. Required fields are marked *