Dell Quantized Models on Workstations Boost AI

Dell clients are eager to work with large language models (LLMs) to create cutting-edge new capabilities that will boost output, efficiency

Dell provides hardware solutions designed to accommodate AI workloads, ranging from mobile and stationary workstation PCs to servers for high-performance computing, data storage

NVIDIA RTX GPUs presently dominate the market for AI processing in PCs with specialized circuits called Tensor cores, even though AI routines can also be handled by the CPU or a new class of dedicated AI circuitry called an NPU

Parameter size of the chosen LLM is one of the most crucial factors in determining hardware requirements. Take Llama-2 LLM from Meta AI

Customers can choose from a variety of Dell Precision workstations if they wish to run the Llama-2 model Dell Quantized at 4-bit precision

The requirements increase when operating at higher precision (BF16), but Dell offers solutions that can handle any size LLM and any required level of precision

Precision workstations as a sandbox to build GenAI solutions gives developers the confidence to fine-tune an LLM