How to Install PyTorch with NVIDIA GPU/CUDA Acceleration Support on Debian 12

PyTorch is an open-source machine-learning (ML) framework from Facebook/Meta. It’s an alternative to TensorFlow. PyTorch is a very popular AI/ML framework and it’s getting more popular day by day.

PyTorch can accelerate the AI/ML applications using an NVIDIA GPU via the NVIDIA CUDA library natively just like TensorFlow.

In this article, we will show you how to install PyTorch with NVIDIA GPU/CUDA acceleration support on Debian 12 “Bookworm”.

Topic of Contents:

Installing the NVIDIA GPU Drivers on Debian 12
Installing NVIDIA CUDA on Debian 12
Installing Python 3 PIP and Python 3 Virtual Environment (venv) on Debian 12
Creating a Python 3 Virtual Environment for PyTorch
Upgrading Python 3 PIP to the Latest Version on the Python 3 PyTorch Virtual Environment
Installing PyTorch with NVIDIA GPU/CUDA Acceleration Support on Debian 12
Activating the PyTorch Python 3 Virtual Environment
Accessing PyTorch and Checking If NVIDIA GPU/CUDA Acceleration Is Available
Conclusion

Installing the NVIDIA GPU Drivers on Debian 12

For PyTorch NVIDIA GPU/CUDA acceleration to work, you must install the NVIDIA GPU drivers on Debian 12. If you need any assistance in installing the NVIDIA GPU drivers on your Debian 12 system, read this article.

Installing NVIDIA CUDA on Debian 12

For PyTorch NVIDIA GPU/CUDA acceleration to work on Debian 12, you must install NVIDIA CUDA on Debian 12. If you need any assistance in installing NVIDIA CUDA on your Debian 12 system, read this article.

Installing Python 3 PIP and Python 3 Virtual Environment (venv) on Debian 12

To install PyTorch on Debian 12, you need to have the Python 3 PIP and Python virtual environment (venv) installed.

First, update the APT package repository cache with the following command:

$ sudo apt update

To install Python 3 PIP and Python 3 virtual environment (venv), run the following command:

$ sudo apt install python3-pip python3-venv python3-dev

To confirm the installation, press “Y” and then press <Enter>.

Python 3 PIP and Python 3 venv are being installed. It takes a while to complete.

At this point, Python 3 PIP and Python 3 venv should be installed.

Creating a Python 3 Virtual Environment for PyTorch

The standard practice for installing the Python libraries on Debian 12 is installing them in a Python virtual environment so that they don’t interfere with the system’s Python packages/libraries.

To create a new Python 3 virtual environment for PyTorch in the “/opt/pytorch” directory, run the following command:

$ sudo python3 -m venv /opt/pytorch

Upgrading Python 3 PIP to the Latest Version on the Python 3 PyTorch Virtual Environment

To upgrade Python 3 PIP to the latest version on the Python 3 “/opt/pytorch” virtual environment, run the following command:

$ sudo /opt/pytorch/bin/pip3 install --upgrade pip

Installing PyTorch with NVIDIA GPU/CUDA Acceleration Support on Debian 12

For the PyTorch NVIDIA GPU/CUDA acceleration to work, you must install the correct version of PyTorch that supports the NVIDIA CUDA driver version that you installed on your Debian 12 system. At the time of this writing, PyTorch supports the NVIDIA CUDA driver versions 11.8 and 12.1. For updated information on the NVIDIA CUDA driver versions that PyTorch supports, check the official website of PyTorch.

To check the NVIDIA CUDA driver version that you installed on your Debian 12 system, run the following command. As you can see, we have NVIDIA CUDA version 11.8 installed on our Debian 12 system.

$ nvcc --version

To install PyTorch with NVIDIA CUDA 11.8 support on the PyTorch Python 3 virtual environment, run the following command:

$ sudo /opt/pytorch/bin/pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

To install PyTorch with NVIDIA CUDA 12.1 support on the PyTorch Python 3 virtual environment, run the following command:

$ sudo /opt/pytorch/bin/pip3 install torch torchvision torchaudio

PyTorch is being installed on the PyTorch Python 3 virtual environment. It takes a while to complete.

At this point, PyTorch should be installed on the PyTorch Python 3 virtual environment

Activating PyTorch Python 3 Virtual Environment

To activate the PyTorch Python “/opt/pytorch” virtual environment, run the following command:

$ . /opt/pytorch/bin/activate

The PyTorch Python 3 virtual environment should be activated.

Accessing PyTorch and Checking If NVIDIA GPU/CUDA Acceleration Is Available

To open the Python 3 interactive shell, run the following command:

$ python3

Python 3 interactive shell should be opened.

First, import PyTorch with the following line of code:

$ import torch

To check the version of PyTorch that you installed, run the following line of code. As you can see, we are running PyTorch 2.1.0 with NVIDIA CUDA 11.8 acceleration support (cu118).

$ torch.__version__

To check whether PyTorch is capable of using your NVIDIA GPU for NVIDIA CUDA acceleration, you can run the following line of code as well. If NVIDIA CUDA support is available, “True” will be printed.

$ torch.cuda.is_available()

If you have multiple GPUs installed on your computer, you can check the number of GPUs that PyTorch can use with the following line of code. As you can see, we have the NVIDIA GPU (RTX 4070) installed on our Debian 12 system.

$ torch.cuda.device_count()

To exit out of the Python interactive shell, run the following line of code:

$ quit()

Conclusion

In this article, we showed you how to install Python 3 PIP and Python 3 virtual environment (venv) on Debian 12. We also showed you how to create a Python 3 virtual environment for PyTorch on Debian 12 and how to install PyTorch with NVIDIA CUDA 11.8 and 12.1 acceleration support on Debian 12 as well. Finally, we showed you how to activate the PyTorch Python virtual environment and access PyTorch on Debian 12.