Inference
Showing 32 of 32 projects
A high-throughput, memory-efficient inference and serving engine for large language models (LLMs).
A high-performance C/C++ port of OpenAI's Whisper model for efficient, cross-platform speech recognition.
High-performance C/C++ port of OpenAI's Whisper for efficient, cross-platform speech recognition.
A unified deep learning system for efficient large-scale model training and inference with advanced parallelism strategies.
Cross-platform framework for building customizable on-device machine learning pipelines for live and streaming media.
A high-performance serving framework for large language models and multimodal models, delivering low-latency and high-throughput inference.
A fast, memory-efficient reimplementation of OpenAI's Whisper speech-to-text model using CTranslate2.
A high-performance neural network inference framework optimized for mobile platforms, enabling efficient AI deployment on edge devices.
An open standard format for representing machine learning models to enable interoperability between frameworks.
An exhaustive pattern matching library for TypeScript with smart type inference and expressive API.
A low-level tensor library for machine learning with integer quantization, automatic differentiation, and zero runtime allocations.
NVIDIA's SDK for high-performance deep learning inference optimization and deployment on NVIDIA GPUs.
An open-source inference serving platform for deploying AI models from multiple frameworks across cloud, data center, and edge devices.
SDKs for adding private, on-device AI features like LLM chat, speech-to-text, and text-to-speech to mobile and web apps.
A lightweight, dependency-free JavaScript library for descriptive, regression, and inference statistics.
A deep learning framework for research, development, and production with flexible Python API and C++ core.
A lightweight header-only C++ library for running Keras (TensorFlow) models without linking against TensorFlow.
A uniform interface to run deep learning models from multiple frameworks like TensorFlow, PyTorch, and Keras in C++ and Python.
A lightweight, portable pure C99 ONNX inference engine for embedded devices with hardware acceleration support.
A command-line tool for neural network inference using Unix pipeline philosophy.
A Scala toolkit for deployable probabilistic modeling using imperatively-defined factor graphs.
A Python library for statistical learning with a focus on time-dependent modeling, including point processes and generalized linear models.
A TypeScript machine learning library for the web and Node.js with a simple, consistent API.
A parallel deep learning framework written in modern Fortran for training and inference of dense, convolutional, and transformer networks.
A toolkit for developing and deploying TensorFlow Lite models on mobile and IoT devices with cross-platform support.
Deploy TensorFlow graphs for fast evaluation and export to environments without TensorFlow, using NumPy.
A Docker-based image annotation tool for bounding box labeling with auto-labeling support, designed for deep learning training.
Tensorflow bindings for the Elixir programming language, enabling machine learning inference and tensor operations.
A conceptual, declarative query language for TypeDB that enables polymorphic queries and intuitive data modeling.
A collection of Jupyter notebooks demonstrating TensorFlow Lite model quantization, conversion, and optimization techniques for deep neural networks.
A curated list of algorithms and academic papers for auditing black-box algorithms like recommendation systems and classifiers.
Saul is a declarative domain-specific language in Scala for designing flexible machine learning models with relational feature extraction.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.