Cloudflare has unveiled its latest 12th-generation servers, powered by AMD’s impressive 96-core EPYC 9684X Genoa-X processors. This new technology promises a significant boost in performance and efficiency, marking a major upgrade for Cloudflare’s global infrastructure. The newly introduced servers feature enhanced capabilities designed to handle more data, optimise performance, and improve energy efficiency.
At the heart of Cloudflare’s Gen 12 servers is AMD’s 9684X Genoa-X processor, which boasts 96 cores, 192 threads, and a massive 1152MB L3 cache—three times larger than that of standard Genoa processors. This substantial cache improvement enables faster data processing and reduces latency, especially for data-heavy applications. According to Cloudflare, the Genoa-X offers a 22.5% performance improvement compared to previous AMD EPYC models, making it an ideal choice for high-demand tasks.
The upgraded servers can process 145% more requests per second (RPS) and offer a 63% increase in power efficiency compared to their predecessors, the Gen 11 servers. This leap in efficiency is credited to a new thermal-mechanical design and expanded support for GPUs, which significantly enhance the performance of artificial intelligence (AI) and machine learning (ML) workloads.
Optimised memory, storage, and network
The Gen 12 servers have 384GB of DDR5-4800 memory spread across 12 channels, providing faster memory access and higher throughput. They also feature 16TB of NVMe storage, which ensures speedy data retrieval and storage capabilities. With dual 25 GbE network connectivity, these servers are well-equipped to handle a broad range of high-intensity computational tasks, further optimising performance for demanding operations.
Additionally, each server is powered by dual 800W Titanium-grade power supply units, offering improved energy efficiency across Cloudflare’s global network of data centres. The shift from the previous 1U form factor to a 2U design allows for better airflow management, improving overall efficiency. Cloudflare highlights that the new airflow design reduced fan power consumption by 150W, resulting in lower energy usage and superior performance.
The Gen 12 servers’ power consumption, which stands at 600W under typical operating conditions, is an increase from the Gen 11 servers’ 400W. However, this higher consumption is justified by the new generation’s significant performance improvements.
AI capabilities and security upgrades
Cloudflare’s Gen 12 servers are designed to excel in AI and machine learning tasks. The new servers support up to two PCIe add-in cards, allowing for scalable GPU deployment. This design ensures that Cloudflare can deploy GPUs where they are most needed, minimising latency in regions with high demand for AI processing.
In addition to the hardware upgrades, the Gen 12 servers feature enhanced security measures, including hardware root of trust (HRoT) and Data Centre Secure Control Module (DC-SCM 2.0) integration. These technologies safeguard the integrity of boot firmware and reduce the risk of firmware-based attacks, providing extra layers of security for Cloudflare’s global infrastructure.
Looking forward, Cloudflare has started testing AMD’s 5th-generation EPYC “Turin” CPUs for its next-gen servers, indicating a strong focus on future growth and technological advancement.
AI product upgrades
Alongside the server announcement, Cloudflare has significantly improved its AI developer products. Workers AI now benefits from more powerful GPUs in over 180 cities worldwide, enabling it to handle larger models like Meta’s Llama 3.1 70B and Llama 3.2. This upgrade allows for more complex AI tasks to be executed efficiently.
Cloudflare’s AI Gateway has also seen enhancements, with the introduction of persistent logs in beta. These logs enable detailed performance analysis, with features like search, tagging, and annotations, making it easier for developers to monitor and optimise their AI deployments.
Lastly, Cloudflare’s vector database, Vectorize, has reached general availability. This service now supports indexes of up to five million vectors and offers significantly reduced latency. Cloudflare has also introduced a simplified unit-based pricing structure for its AI products, making cost management easier for developers.
Cloudflare continues to expand its capabilities, pushing the boundaries of technology while focussing on efficiency, security, and AI advancements.