Install Ollama and DeepseekR1 on Debian12

In this guide, we’ll cover how to install Ollama and run the DeepSeek R1 model on Debian, including the NVIDIA drivers and CUDA Toolkit needed for GPU acceleration. We’ll also look at how to configure Ollama to use one or more GPUs, and finish by installing Open WebUI for a friendly browser-based interface.

Before you start, you’ll want:

• Debian 12 (Bookworm) or later
• An NVIDIA GPU that supports CUDA (a CPU-only setup works too, just more slowly, and you can skip the driver steps)
• A working internet connection

Step 1: Install the NVIDIA Drivers and CUDA Toolkit

If you’re running CPU-only, skip straight to Step 2. Otherwise, start by updating the system and installing the build tools the NVIDIA installer needs:

sudo apt update
sudo apt install -y build-essential dkms

Next, download the latest driver for your card from NVIDIA’s driver download page, choosing your GPU and the Linux 64-bit .run file. The driver can’t install while the graphical desktop is running, so switch to a text console first (Ctrl+Alt+F2), log in, stop your display manager, and run the installer:

sudo service lightdm stop   # use gdm3 instead if you're on GNOME
cd ~/Downloads
sudo sh NVIDIA-Linux-x86_64-<version>.run

Follow the prompts, accept the licence, and install the 32-bit compatibility libraries if it asks. Once it finishes, reboot:

sudo reboot

With the driver in place, install the CUDA Toolkit. Grab the installer from the CUDA download page, selecting Debian 12 and x86_64, then run it:

sudo sh cuda_<version>_linux.run

When prompted, say yes to installing the CUDA Toolkit but no to the driver, since you’ve already installed that from the standalone package. Finally, add CUDA to your path so the tools are found, then reload your shell:

echo 'export PATH=/usr/local/cuda-<version>/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-<version>/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

Step 2: Install Ollama

Ollama doesn’t have an official Debian package, so the simplest route is the official install script, which sets it up as a systemd service for you:

sudo apt install -y curl
curl -fsSL https://ollama.com/install.sh | sh

Confirm it installed correctly by checking the version:

ollama --version

Step 3: Configure GPU Usage

Because the installer runs Ollama as a systemd service, you control which GPUs it uses by editing the service file at /etc/systemd/system/ollama.service. The key setting is CUDA_VISIBLE_DEVICES, which tells Ollama which cards it’s allowed to see.

The behaviour depends on what you set:

• CPU-only: leave the GPU variables out entirely, and Ollama falls back to the CPU.
• A single GPU: set Environment="CUDA_VISIBLE_DEVICES=0" so only the first card is used.
• Multiple GPUs: list them, for example Environment="CUDA_VISIBLE_DEVICES=0,1,2", and add Environment="OLLAMA_SCHED_SPREAD=1" if you want Ollama to spread the load across all of them rather than filling one at a time.

A complete service file for a three-GPU setup looks like this:

[Unit]
Description=Ollama Service
After=network-online.target

[Service]
Environment="CUDA_VISIBLE_DEVICES=0,1,2"
Environment="OLLAMA_SCHED_SPREAD=1"
Environment="PATH=/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin"
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3

[Install]
WantedBy=default.target

After editing the file, reload systemd and restart the service so the changes take effect:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Step 4: Pull the DeepSeek R1 Model

DeepSeek R1 comes in several sizes, and the one you can run depends on your hardware. The smallest, 1.5B, runs on CPU with about 8GB of RAM, so it’s fine for a basic machine with no GPU. The 7B and 8B models want a GPU with at least 8GB of VRAM, the 14B and 32B models need roughly 12 to 24GB, and the 70B model is firmly in high-end territory, needing around 48GB of VRAM or a multi-GPU setup.

Pick the size that fits your hardware and pull it with Ollama. The run command downloads the model on first use and then drops you straight into a chat prompt:

ollama run deepseek-r1:1.5b     # CPU-friendly
ollama run deepseek-r1:7b
ollama run deepseek-r1:8b
ollama run deepseek-r1:14b
ollama run deepseek-r1:32b
ollama run deepseek-r1:70b      # needs serious hardware

If you’re not sure where to start, pull the largest one that comfortably fits inside your VRAM, since a model that spills out of GPU memory falls back to the CPU and slows to a crawl.

Step 5: Install Open WebUI

Talking to the model on the command line is fine, but Open WebUI gives you a clean, ChatGPT-style interface in the browser. The quickest way to run it is with uv, a fast Python runner, which you can install with:

curl -LsSf https://astral.sh/uv/install.sh | sh

Then start Open WebUI, which pulls in its own dependencies and launches a local server:

uvx --python 3.11 open-webui@latest serve

Once it’s running, open your browser and go to http://localhost:8080. Open WebUI detects your local Ollama automatically, so your DeepSeek R1 model will be there in the model list, ready to chat with.

Related guides

• The Terminal Coding Agent I Use to Run My Own Servers
• Why your long-running AI request dies at 60 seconds