Dell Storage Engines Redefine AI Inference Speed with KV Cache Offloading

Dell Technologies is transforming AI inference performance with its innovative KV Cache offloading solution, powered by PowerScale and ObjectScale storage engines. Designed to overcome GPU memory bottlenecks in large language model (LLM) workloads, Dell’s scalable solution achieves up to 19x faster Time to First Token (TTFT) compared to standard vLLM configurations. By offloading the KV Cache from GPU memory to high-performance storage, organizations can significantly enhance inference efficiency, reduce latency, and lower operational costs.

The solution integrates vLLM, LMCache, and NVIDIA NIXL (extended by Dell with an S3-over-RDMA plugin for ObjectScale), supporting both file and object storage environments. Benchmark results show Dell’s PowerScale and ObjectScale outperforming competitors such as VAST, with TTFT as low as 0.82 seconds for the Qwen3-32B model—demonstrating superior acceleration and GPU utilization.

Beyond inference optimization, Dell’s AI Data Platform (AIDP) streamlines the entire AI data lifecycle from ingestion to knowledge creation and deployment empowering organizations to operationalize AI at scale. With this storage-driven approach, Dell is setting a new standard for efficient, scalable, and sustainable AI infrastructure, enabling businesses to maximize the value of their AI investments.