How to install ML software with GPU acceleration

Python

sudo apt install python3-pip
pip3 install --upgrade pip

Opencv with cuda

Install cuDNN

https://developer.nvidia.com/cudnn-downloads

Install jtop to see GPU usage in realtime

# update pip as root
sudo curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python get-pip.py
sudo python -m pip install jetson-stats

# restart needed
jtop

nvidia-smi

Install Pytorch with CUDA

see https://developer.download.nvidia.com/compute/redist/jp/v60dp/pytorch/

pip install --no-cache https://developer.download.nvidia.com/compute/redist/jp/v60dp/pytorch/torch-2.2.0a0+81ea7a4.nv24.01-cp310-cp310-linux_aarch64.whl
# pip install torchvision

See https://github.com/dusty-nv/jetson-containers/tree/master/packages/l4t/l4t-pytorch