One of the most interesting, and perhaps significant in its impact to the semiconductor ecosystem, was Amazon’s surprise announcement of an Amazon-built custom Arm based server option for AWS customers. There is a lot to unpack about this announcement, and while we are hoping for more information about the details of the processor, Amazon’s strategy with it is the fascinating part.
Amazon’s public posture with the Arm server chip is their desire to offer a competitive set of AWS instances from a price standpoint which shines a light on how the pricing structure from Intel, Nvidia, and to a degree AMD, continues to lead to relatively high server costs. A brief bit of perspective here on the problem may be helpful.
Inevitable Cloud Platform Verticalization
With the growing trend of machine learning, which turns into AI in some capacity, it is a well-known fact that training machines are still very costly. Because of this, many companies buy a rack of Nvidia or AMD graphics cards and work to train their networks locally then upload that training to the cloud. That works fine and well in experimental modes, and some early stage product modes, but that doesn’t work as cleanly as companies grow and start to need more data trained in real-time. Real-time training is where the world is headed. It is true it will take a few more years to get there, but in that world, the role of the silicon in cloud servers will become even more important than they are today.
For this reason, Google is working on some custom silicon around their Tensor processing unit (which will only get beefier and more powerful). In this light, I see the Amazon Graviton. It is starting with humble beginnings, offering a more competitive price option of AWS instances, but I fully expect this chip to get more powerful over time and in the case of both Google and Amazon, I am convinced they will pose a continued threat to Intel and Nvidia. Intel more than Nvidia, but both will be impacted.
The future of cloud computing is looking like it will be run on Amazon, Microsoft, and Google. As these companies start to create exclusive hardware (with silicon) to compete, differentiate, and offer a variety of customer options, they lessen their dependence on companies like AMD, Intel, and Nvidia. This is not to say those companies won’t have a role, only that their share of the market may decrease.
In many ways, it is inevitable that the dominant cloud platform go more vertical in both proprietary software and hardware by way of custom silicon. Apple has made it clear that any significant platform player must own some software and some silicon that is customized to their overall strategy.
Many are familiar with Alan Kay’s famous quote, which Steve Jobs seemed to take seriously, which was: “People who are serious about software should make their hardware.” What is clear about that quote, in hindsight, is how vital custom silicon is that reality. It is clear now that the hardware part of his quote extends into silicon (which is also a lot of software). From a competitive advantage part, custom software tied to custom hardware is essential to compete, and it is something every platform company will inevitably be forced to do which is where the question about how Intel, Nvidia, and AMD evolve in an era where the dominant platforms start to tread on their turf.
A Boost for Arm Servers
The other significant part of Amazon’s announcement is the boost it will be for the Arm server narrative. Arm’s server initiatives to date have been so poor it has been hard to take them seriously. Many companies were launched trying to develop Arm-based servers only to close their doors because they couldn’t find any customers. Even Qualcomm launched a huge initiative around their Arm server platform only to hit a similar wall in demand and scale down the group and their overall investment in the solution.
Amazon may very well be the boost for Arm servers the technology needs. If Amazon can do it, compete, and offer competitive solutions on Arm it is likely it could lead to a reinvigorated movement for Arm-based servers. At the very least, I’d expect both Microsoft and Google to start to go deeper down the custom silicon rabbit hole and their belief they can do so with the Arm platform may be helped by Amazon’s move.
This is early days, but I’m more optimistic now on Arm’s potential in cloud/server than I was last year. Which also will fuel the bearish view on Intel as well since Arm is, by nature of its openness and modularity, the single biggest disruptive threat to Intel and several others in the semiconductor industry.