Inference Optimization

Bindings for many languagesC++

A high-performance C/C++ port of OpenAI's Whisper model for efficient, cross-platform speech recognition.

Colossal-AI - An Integrated Large-scale Model Training System with Efficient Parallelization TechniquesPython

A unified deep learning system for efficient large-scale model training and inference with advanced parallelism strategies.

faster-whisperPython

A fast, memory-efficient reimplementation of OpenAI's Whisper speech-to-text model using CTranslate2.

TensorRTC++

NVIDIA's SDK for high-performance deep learning inference optimization and deployment on NVIDIA GPUs.

oneDNNC++

An open-source cross-platform performance library of basic building blocks for deep learning applications, optimized for CPUs and GPUs.

whisper.tfliteC

Efficient inference implementation for Transformer models on edge devices, originally focused on OpenAI's Whisper speech recognition.