gputil
A Python module for programmatically retrieving NVIDIA GPU status and selecting available GPUs based on memory and load.
What is gputil?
GPUtil is a Python module that provides programmatic access to NVIDIA GPU status via the `nvidia-smi` command-line tool. It helps developers identify available GPUs based on current memory and load thresholds, automating GPU selection for tasks like deep learning training and computational workloads. The module simplifies GPU resource management in multi-GPU setups without requiring external dependencies.
Developers and researchers working with NVIDIA GPUs in Python, especially those in deep learning, scientific computing, or any field requiring efficient GPU resource allocation. It is particularly useful for teams managing shared GPU servers or running multi-GPU experiments.
GPUtil offers a straightforward, dependency-free way to interact with NVIDIA GPUs, eliminating the need to manually parse `nvidia-smi` output. Its flexible filtering and ordering options make it ideal for automating GPU selection, saving time and reducing errors in GPU-intensive workflows.
Overview
A Python module for getting the GPU status from NVIDA GPUs using nvidia-smi programmically in Python
Use Cases
Best For
- Automatically selecting idle GPUs for deep learning training jobs
- Monitoring GPU utilization across multiple NVIDIA devices in real-time
- Integrating GPU availability checks into TensorFlow or Caffe pipelines
- Managing GPU resources in shared computing environments or labs
- Building custom GPU scheduling or orchestration tools
- Running GPU-dependent scripts that require specific memory/load thresholds
Not Ideal For
- Environments with AMD or non-NVIDIA GPUs, as it exclusively relies on NVIDIA's `nvidia-smi` tool.
- Production clusters requiring advanced job scheduling, resource quotas, or container orchestration, where tools like Kubernetes GPU plugins are more suitable.
- Applications needing real-time, sub-second GPU monitoring, as `nvidia-smi` sampling rates may introduce latency.
Pros & Cons
Pros
Uses only Python's standard library, making it easy to install and run without external dependencies, as shown in the README's minimal requirements.
Offers multiple ordering options (by ID, load, memory, or random) and customizable thresholds for load and memory, allowing tailored availability checks.
Includes practical integration examples for deep learning frameworks like TensorFlow and Caffe, speeding up implementation in common workflows.
Provides functions like showUtilization() and a thread-based monitor for real-time GPU status display, useful for basic monitoring needs.
Cons
Tied exclusively to NVIDIA GPUs via `nvidia-smi`, making it unusable for systems with AMD, Intel, or other GPU vendors.
Lacks advanced features such as historical data logging, predictive load balancing, or integration with modern orchestration tools, limiting its use in complex environments.
Relies on the correct installation and output parsing of `nvidia-smi`, which can fail if the NVIDIA driver is outdated or if there are system-specific issues.
Frequently Asked Questions
Related Projects
spdlogFast C++ logging library.
ctopTop-like interface for container metrics
ntopWeb-based Traffic and Security Network Traffic Monitoring
htophtop is an interactive text-mode process viewer for Unix systems. It aims to be a better 'top'.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.