TensorRT
NVIDIA's SDK for high-performance deep learning inference optimization and deployment on NVIDIA GPUs.
What is TensorRT?
TensorRT is NVIDIA's SDK for high-performance deep learning inference optimization on NVIDIA GPUs. It takes trained neural network models and optimizes them through techniques like layer fusion, precision calibration, and kernel auto-tuning to achieve low latency and high throughput for production deployment. The open-source components include plugins, parsers, and sample applications that demonstrate TensorRT's capabilities.
AI engineers and developers deploying deep learning models in production environments requiring maximum inference performance on NVIDIA GPUs, including those in data centers, edge devices, and automotive systems.
Developers choose TensorRT for its deep integration with NVIDIA hardware, delivering unmatched inference performance through hardware-aware optimizations, and its support for diverse deployment targets from cloud to embedded systems.
Overview
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
Use Cases
Best For
- Optimizing trained PyTorch or TensorFlow models for NVIDIA GPU inference
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
