Nvidia shifts focus to storage as the next frontier for AI performance

Nvidia is pushing storage higher up the AI infrastructure agenda, arguing that future gains will depend not only on faster processors but also on how efficiently systems can deliver data to GPUs. The company has introduced a series of initiatives to reduce delays in moving data from storage to accelerated computing systems.

Following the Future of Memory and Storage conference, Nvidia announced that it had open-sourced its cuFile APIs and the storage software stack beneath them. It has also formally launched Storage-Next, an industry initiative involving more than 40 flash and storage vendors.

The company is also promoting SCADA, a framework that enables GPUs to take a more active role in managing storage requests.

Nvidia targets the growing gap between storage and AI workloads

Nvidia’s efforts build on technology that has been in development for several years. The company introduced cuFile in 2019 as part of its GPUDirect Storage technology, with the system becoming available in production environments from 2021. Its primary purpose is to remove the CPU from the main data path, allowing data to move directly between storage and GPU memory using Direct Memory Access.

However, while cuFile reduces the amount of data processing handled by the CPU, the processor has traditionally remained responsible for controlling storage requests. The host system determines which data to retrieve and initiates the requests, leaving the GPU to act primarily as the destination for the resulting data.

That arrangement works well when storage operations involve large blocks of data because the overhead associated with individual requests becomes relatively small. The situation changes considerably with smaller transfers. At 512 bytes, the fixed cost per request becomes much more significant, potentially leaving the CPU overwhelmed even when the underlying storage hardware still has capacity available.

SCADA is intended to address that limitation by shifting more of the storage-control process to the GPU. Rather than relying on the CPU to manage every operation, the GPU can construct and complete storage requests itself. This approach allows it to keep a large number of operations active simultaneously, similar to how GPUs already handle memory latency through massive parallelism.

The technologies are therefore designed to complement each other rather than replace one another. Nvidia sees cuFile as useful for large data transfers, while SCADA is intended to improve performance when AI workloads require many small reads.

AI inference puts new pressure on flash storage

The need for this approach is linked to the way modern AI applications access data. Enterprise SSDs have traditionally been optimised around 4KB random reads, reflecting the requirements of virtualisation and database workloads. AI inference can operate on much smaller pieces of information, including embeddings that may be only a few hundred bytes and key-value cache blocks that can remain below 1KB.

The mismatch can lead to significant read amplification. A drive designed around 4KB operations may need to perform considerably more work than is required for the useful data requested. At the scale of AI deployments, where storage can contain tens of terabytes of small objects, that overhead can determine whether flash storage can effectively act as an additional memory tier.

The key-value cache is particularly important for inference. It stores attention information for tokens a model has already processed, allowing systems to avoid repeating expensive calculations. As context windows become longer and AI agents handle thousands of conversations at the same time, these caches can quickly exceed the amount of memory available directly on GPUs.

When cache data is removed from GPU memory, systems can either recompute it or retrieve it from another memory tier. Recomputing can consume additional GPU resources, making storage-based retrieval increasingly attractive. Data can instead move through system memory and flash storage, but this requires extremely high volumes of small random reads.

Using flash as part of this hierarchy could allow each GPU to support longer contexts and more simultaneous users. That has a direct impact on the cost of operating AI models, as greater utilisation of each GPU can reduce the infrastructure costs associated with individual users.

This is why Nvidia is placing particular emphasis on 512-byte input/output operations per second rather than focusing solely on conventional storage bandwidth. For many AI inference workloads, the ability to handle enormous numbers of small requests may be more important than achieving the highest possible sequential transfer speeds.

Nvidia opens its storage technology to a wider industry

Nvidia’s decision to open-source cuFile represents a significant change for a technology layer that has historically been closely associated with its CUDA software ecosystem. The company has a strong commercial interest in making GPU-initiated storage widely available, but achieving that goal requires storage manufacturers to support the technology across drives, controllers and complete systems.

A proprietary interface would make that adoption more difficult because storage companies would have less incentive to build around technology controlled by the leading supplier of the GPUs that consume their products. Nvidia is instead attempting to encourage broader adoption by publishing the interface, opening the implementation and bringing storage companies together through Storage-Next.

The initiative now includes more than 40 flash and storage vendors. Industry participation also includes Intel, which is particularly notable because Intel supplies x86 processors used in many storage controllers. The move could ultimately reduce the amount of CPU involvement required for storage I/O, even as companies that supply those processors participate in developing the new software approach.

Google and Meta are also involved in efforts to standardise how accelerators communicate with storage. Both companies are developing their own AI accelerators, making common interfaces potentially useful beyond Nvidia’s hardware ecosystem.

A growing list of hardware partners backs Nvidia’s plans. Systems from DDN, Dell, HPE, IBM, VAST Data and WEKA are expected in the second half of 2026, while Kioxia is developing XL-Flash drives designed to support 512-byte access patterns under the Storage-Next initiative.

The company has also outlined a longer-term target of Gen7 SSDs capable of sustaining 100 million IOPS per drive. That figure is currently a design target for controller manufacturers rather than a specification available from a commercial product.

Storage-Next has been discussed publicly since GTC 2025, but Nvidia’s latest announcements give the initiative a more defined structure and a larger group of participating companies. The broader goal is to make storage a more active component of AI computing, reducing the bottleneck between increasingly powerful accelerators and the data they need to process.