Inference Optimization

Bindings for many languagesC++

A high-performance C/C++ port of OpenAI's Whisper model for efficient, cross-platform speech recognition.

Colossal-AI - An Integrated Large-scale Model Training System with Efficient Parallelization TechniquesPython

A unified deep learning system for efficient large-scale model training and inference with advanced parallelism strategies.

faster-whisperPython

A fast, memory-efficient reimplementation of OpenAI's Whisper speech-to-text model using CTranslate2.

TensorRTC++

NVIDIA's SDK for high-performance deep learning inference optimization and deployment on NVIDIA GPUs.

oneDNNC++

An open-source cross-platform performance library of basic building blocks for deep learning applications, optimized for CPUs and GPUs.