Key points
- Maia 300 uses four chiplets on a two-nanometer node with 288GB of HBM4 memory.
- The chip delivers 40% more tokens per megawatt compared to leading GPU-based systems.
- Microsoft plans to deploy the accelerators in liquid-cooled racks containing 96 units.
Microsoft has officially unveiled Azure Maia 300, a custom artificial intelligence accelerator designed for internal and partner workloads at multi-gigawatt scale, according to Neowin. The company states this silicon will become its primary source of inference capacity over time. This move targets high-efficiency processing for large language models and other AI tasks that require significant computational power.
In plain English
The Maia 300 chip is optimized specifically for inference, which is the process of using trained AI models to generate answers or content. Microsoft estimates it can process 40% more tokens per megawatt than leading GPU-based systems, likely referencing Nvidia Corp. products, according to SiliconANGLE. This efficiency gain is crucial for organizations managing the rising energy costs associated with running large-scale AI services.
The background
Maia 300 features four computing chiplets made using Taiwan Semiconductor Manufacturing Co.’s latest two-nanometer node. These chiplets are paired with eight HBM4 modules providing 288 gigabytes of combined capacity, matching Nvidia’s flagship Rubin accelerator. However, Maia 300 has about 20% less memory bandwidth than Rubin. Memory bandwidth measures how much fast data moves to and from RAM, a key factor in processing speed.
What changes now
To minimize data movement slowdowns, Maia 300 uses a dataflow architecture that keeps information in an internal SRAM cache when possible. When accessing HBM is unavoidable, data moves to nearby memory stacks rather than distant ones. The chip also compresses data into lightweight formats like MXFP4. Microsoft plans to deploy these accelerators in trays of four, housed in liquid-cooled racks with 96 accelerators total.
What to do and how to stay safe: Microsoft
- Review your organization’s AI infrastructure energy costs to assess if custom silicon offers better efficiency than current GPU setups.
- Monitor vendor announcements regarding support for new data compression formats like MXFP4 in your existing software stacks.
- Evaluate whether liquid-cooled rack deployments are feasible for your data center’s current cooling and power infrastructure.
- Track the timeline for Maia 300 availability in Azure regions to plan workload migrations accordingly.
Step-by-step guide: Smartphone Security Updates for Small Business Operations
General security guidance from the Patch Review newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
How does Maia 300 compare to Nvidia’s Rubin accelerator?
Maia 300 has the same 288GB memory capacity as Rubin but offers 40% more tokens per megawatt, though it has 20% less memory bandwidth.
What is the physical layout of the Maia 300 chip?
It features four computing chiplets on a two-nanometer node, flanked by eight HBM4 modules and supported by control and auxiliary chiplets.
How does Microsoft plan to deploy these chips?
Microsoft will deploy Maia 300 in trays of four accelerators, which are then placed in liquid-cooled racks containing a total of 96 accelerators.
