It is possible to run local LLMs on AMD GPUs by using Ollama. The most recent version of Llama 3.2
The amdgpu-install script helps you install a cohesive collection of stack components, including the ROCm Software Stack and other Radeon software for Linux components
Ollama’s broad support for AMD GPUs is evidence of how widely available executing LLMs locally is becoming
Makes the installation of the AMD GPUs stack easier by using command line arguments that let you select the following and by encapsulating the distribution-specific package installation logic
Users may run models like Llama 3.2 on their own hardware with a variety of choices, ranging from high-end AMD Instinct accelerators to consumer-grade AMD Radeon RX graphics cards
SELinux may restrict containers’ access to AMD GPU hardware in various Linux editions. To enable containers to utilize devices, execute sudo setsebool container_use_devices=1 on the host system
To restrict Ollama to use a subset of your system’s AMD GPUs, you may set HIP_VISIBLE_DEVICES to a list of GPUs separated by commas