Microsoft has unveiled Maia 300, its next-generation artificial intelligence accelerator, as it prepares to expand its in-house computing infrastructure to support increasingly demanding AI workloads. The chip is designed to improve AI inference efficiency, potentially making services such as Microsoft 365 Copilot, Microsoft Teams, and other AI-powered applications faster and more capable.
The company expects to begin deploying Maia 300 in 2027. According to Microsoft, the accelerator will deliver 40% more tokens per megawatt than other leading GPU-based systems, with a particular focus on long-context reasoning, responsive conversational AI and multi-step tasks handled by AI agents.
Microsoft also claims a rack with 96 Maia 300 accelerators could achieve up to seven times the token throughput of a Maia 200 rack. The improvements are intended to help the company serve more AI requests while using electricity, memory, and computing resources more efficiently in its data centres.
Microsoft targets more efficient AI inference
Microsoft developed Maia 300 primarily for AI inference, the process by which trained AI models generate responses, analyse information, and complete tasks. As AI features become more widely integrated into business software and consumer services, inference efficiency has become an important consideration for companies operating large-scale AI infrastructure.
Microsoft expects the new accelerator to support a broad range of applications across its ecosystem. These include AI-assisted coding through GitHub, model deployment through Microsoft Foundry and improvements to products such as Copilot and Teams. The company also anticipates benefits for its internal Microsoft AI team, particularly when generating data and running demanding models.
The chip uses what Microsoft describes as an “inference-first architecture”. Rather than concentrating solely on raw computing power, this approach aims to reduce unnecessary data movement and improve coordination between processing units, memory and networking components. Keeping data closer to the resources that need it can reduce delays and help improve performance during complex AI operations.
Microsoft has combined a software-defined dataflow architecture with AI-assisted kernel optimisation to support this approach. These technologies aim to use available hardware resources better by directing data efficiently through the system. Such improvements could prove particularly useful for AI agents that must process lengthy inputs, maintain context and perform several connected operations before producing a result.
Memory capacity and bandwidth also play a central role in the design. Maia 300 will use eight stacks of high-bandwidth memory based on the HBM4 standard, with a claimed peak bandwidth of 18 terabytes per second. The accelerator has a thermal design power of 950 watts, reflecting the substantial energy requirements associated with high-performance AI computing.
The chip will also use on-chip static random-access memory as a temporary workspace. Microsoft says specialised software can use this memory to reduce pressure on HBM during demanding workloads. By managing data storage and access more efficiently, the company aims to improve throughput while supporting larger context windows, which let AI models work with more information in a single task.
New chiplet design aims to improve performance at scale
Maia 300 introduces a multi-chiplet architecture that separates computing, memory-controller and input/output functions. This arrangement lets different parts of the accelerator be designed around their specific responsibilities, helping Microsoft balance processing performance, memory access, and communication between components.
Microsoft will manufacture the compute chiplets using Taiwan Semiconductor Manufacturing Company’s N2P process. The technology is part of Microsoft’s effort to develop specialised hardware that meets the performance and efficiency requirements of large AI deployments.
The accelerator will also support low-precision numerical formats, including an enhanced version of MXFP4. These formats let AI calculations use fewer bits to represent certain values, potentially reducing memory use and the amount of data that must move through the system. This can help accommodate larger workloads and improve processing throughput without requiring a proportional increase in power consumption.
The overall architecture also incorporates networking. Maia 300 will use high-bandwidth connections between accelerators to support tensor parallelism, a technique that distributes parts of an AI model’s calculations across multiple processors. Microsoft is also introducing a customised AI transport system built on Ethernet infrastructure.
By integrating networking into its dataflow design, Microsoft aims to scale Maia 300 from individual racks to larger data-centre deployments. Efficient communication between accelerators is essential when AI models require multiple processors to work together, particularly when serving numerous users or processing complex requests simultaneously.
A Maia 300 rack will contain 96 accelerators within a total power envelope of 167 kilowatts. Microsoft lists the configuration as providing 27.6 terabytes of HBM capacity and 1.7 petabytes per second of aggregate HBM bandwidth. This concentration of computing and memory resources is intended to let each rack handle more simultaneous AI interactions and larger workloads.
The figures highlight Microsoft’s focus on increasing AI processing within a defined power budget. For cloud providers and businesses deploying AI at scale, throughput improvements can determine how many requests a system can handle and how efficiently infrastructure investments are used.
Maia 300 builds on Microsoft’s existing AI infrastructure
The new accelerator follows Maia 200, Microsoft’s previous-generation AI chip, which the company says is already running production workloads. These include Microsoft 365 Copilot and models developed by Microsoft’s internal AI team.
Microsoft says Maia 200 delivers up to 1.7 times more tokens per watt than leading GPU infrastructure for built-in AI features in Teams, Copilot and Cowork. The company plans to extend its use to GitHub Copilot and Microsoft Foundry by the end of 2026, according to the announcement.
The transition from Maia 200 to Maia 300 reflects Microsoft’s broader strategy of developing its own AI hardware alongside its cloud infrastructure. Specialised accelerators can be designed around the workloads a company runs most frequently, potentially reducing reliance on general-purpose GPU systems for particular tasks.
For Microsoft’s AI services, greater inference efficiency could mean improved responsiveness, increased capacity, and more economical operation at scale. However, actual results will depend on the models being used, the complexity of individual tasks and how the hardware is integrated into production systems.
The planned 2027 deployment also means that the practical impact of Maia 300 will become clearer as the hardware enters service and Microsoft shares performance results from real-world workloads. Its claimed throughput improvements indicate the company’s ambitions, but independent testing will be needed to determine how the accelerator compares with competing systems across different applications.
As demand for AI-powered software grows, Microsoft is investing in the computing infrastructure needed to support more advanced models and agent-based services. Maia 300 represents the next stage of that effort, with a design centred on inference efficiency, memory performance and the ability to scale across large data centres.
