Arm Doubles Down on AI for Mobile Devices

While many people still aren’t that familiar with semiconductor IP stalwart Arm, most everyone knows their key customers—Qualcomm, Apple, Samsung, MediaTek, and HiSilicon (a division of Huawei) to name just a few in the mobile market. Arm provides chip designs that these companies and others use to power virtually every single smartphone in existence around the world.

As a result, if you care the least bit about where the mobile device market is headed, it’s important to keep track of the new advancements that Arm introduces. While you won’t experience them immediately, if you purchase a new smartphones 12-18 months from now, it will likely be powered by a chip (or several) that incorporates these new enhancements. In particular, expect to see a big boost in AI performance, across a range of different chips.

Those who are familiar with Arm know that, like clockwork every year, the company announces new capabilities for its Cortex CPUs, Mali GPUs and, most recently Ethos NPUs (neural processing units). As you’d expect, most of these include refinements to the chip designs and resulting increases in performance. This year, however, Arm has thrown in a few additional twists that serve as an excellent roadmap for where the smartphone market is headed at several different levels.

But first, let’s get the basics. The latest top-end 64-bit Cortex CPU design is the Cortex-A78 (up from last year’s A77), a further refinement of the company’s ARMv8.2 core. The A78 features 20% sustained performance improvements versus last year’s design, thanks to several advanced architectural refinements. The biggest focus this year is on power efficiency, letting the new design achieve that 20% improvement at the same power draw, or allowing it to achieve the same performance as the A77 with just 50% of the power, thereby saving battery life. These benefits result in better performance per watt, making the design of the A78 well suited for both power- and performance-hungry 5G phones, as well as foldable and other devices featuring larger displays.

In addition to the A78, Arm debuted a whole new branch of CPUs with the Cortex-X1, a larger, but more powerful design. Recognizing the growing interest in gaming-focused smartphones and other applications that demand even more performance, Arm decided to provide an even more performant version of their CPU core with the X1 (it features a 30% performance boost over the A77).

Even more interesting is the fact that the X1 doubles the performance for machine learning and AI models. Despite the appearance of dedicated AI accelerators (like the company’s Ethos NPUs) as well as the extensive focus on GPUs for AI, the truth is that most neural network and other AI models designed for mobile devices run on the CPU, so it’s critical to enhance performance there.

While the X1 isn’t intended for mainstream usage and won’t represent a particularly large segment of the market (particularly because of its larger and more power-hungry design), its appearance reflects the increasing diversity and segmentation of the smartphone market. In addition, the Cortex-X looks like it would be a good candidate for future versions of Arm CPUs for PCs and other larger devices.

On the GPU side, the company made two different introductions: one at the top end of the performance chain and the other emphasizing the rapidly growing opportunity for moderately priced smartphones. The top-of-the-line Mali-G78 features a 25% increase in standard graphics performance over its Mali-G77 predecessor, as well as a 15% boost in machine learning application performance. Given the interest in achieving PC and console gaming-like quality on smartphones, the G78 adds support for up to 24 shader cores, but leverages a clever asynchronous power design that allows it to create high-level graphics without drawing too much power.

The other new design is the Mali-G68, which Arm classifies as being targeted to a “sub-premium” tier of phones. Leveraging essentially the same design as the G78, but limited to a maximum of 6 shader cores, the G68 allows its chip customers and then smartphone makers in turn to create products with premium-like features but at lower price points. Given the price compression that many expect to see in smartphones over the next several years, this seems like an important step.

The final new design from Arm was their Ethos-N78, just the second generation of their dedicated line of AI co-processors for mobile devices. Featuring more than 2x the peak performance of the N77, as well as a greater than 25% improvement in performance efficiency, the N78 also offers more flexibility in configuring its core elements, letting companies more easily use it across a wide range of different mobile devices.

Even more important than raw performance in the AI/ML world is software. Not surprisingly then, the company also announced new enhancements to their Arm Development Studio and other tools that make it easier to optimize AI applications not only for the N78, but for its full line of Cortex CPUs and Mali GPUs as well. In fact, Arm is offering a unified software stack that essentially allows developers to create AI/ML models that can run transparently across any combination of Arm CPUs, GPUs or NPUs. Conceptually, it’s very similar to Intel’s One API idea, which is intended to provide the same level of flexibility across a range of different Intel silicon designs. Real-world performance for all of these “write once, run anywhere” heterogenous computing models remains to be seen—and the challenges for all of them seem quite high—but it’s easy to see why they could be very popular with developers.

As expected, Arm brought a range of new mobile-focused chip designs to the table once again this year, but thanks to the debut of the Cortex-X1, the sub-premium Mali-G68, and the overall emphasis on AI and machine learning, they still managed to shape things up a bit. Clearly, the company sees a growing demand for all these market sub-segments and, because of the pivotal role they play, their efforts will go a long way toward making them real.

The ultimate decisions on how all these new capabilities get deployed and the features they enable get implemented is up to the company’s more famous customers and, in some cases, their customers’ customers, of course. More “intelligent” devices, more immersive augmented reality (AR) and virtual reality (VR) enhancements, and significantly improved graphics performance all seem like straightforward outcomes they could enable. Nevertheless, the groundwork has now been laid for future mobile devices and it’s up to other vendors in the mobile industry to see exactly where it will take us.