Over 100 Million Windows PCs to Experience Accelerated AI Performance with NVIDIA's TensorRT-LLM for RTX GPUs

Written by Boldson on 15 November 2023. Posted in Hardware

NVIDIA is set to bring a significant acceleration to AI workloads on millions of Windows 11 PCs powered by the latest RTX GPUs. The upcoming integration of TensorRT-LLM into Windows 11 will benefit over 100 million RTX users starting November 21st.

TensorRT-LLM will enhance AI performance for RTX 40 and RTX 30 GPU owners, delivering up to 5x acceleration. Along with TensorRT-LLM, NVIDIA and Microsoft are also enhancing DirectML for popular AI models.

NVIDIA will offer an API interface to OpenAI's ChatAPI, ensuring more efficient local data access and accelerating AI processing on Windows 11 PCs.