Model Inference

gpt4allC++

Run large language models (LLMs) privately on everyday desktops and laptops without requiring API calls or GPUs.

Paddle - PArallel Distributed Deep LEarning by BaiduC++

An industrial deep learning framework supporting unified dynamic/static graphs, automatic parallelism, and integrated training/inference for large models.

PaddlePaddleC++

An industrial deep learning framework from China supporting unified dynamic/static graphs, automatic parallelism, and integrated training/inference for large models.

BigDLPython

An LLM acceleration library for Intel XPU (GPU, NPU, CPU) to speed up local inference and finetuning of popular models.

TensorFlowSharpC#

A .NET binding to the TensorFlow C API for running existing machine learning models in C# and F#.

KotlinDLKotlin

A high-level Deep Learning API for JVM and Android developers, written in Kotlin and inspired by Keras.

neuropodC++

A uniform interface to run deep learning models from multiple frameworks like TensorFlow, PyTorch, and Keras in C++ and Python.

hugotGo

Run ONNX transformer pipelines (like Hugging Face) natively in Go for inference and fine-tuning, with support for CPU, GPU, and TPU.

leavesGo

A pure Go library for making predictions with Gradient Boosting Regression Trees models from LightGBM, XGBoost, and scikit-learn.

CybertronGo

A pure Go package for running inference with pre-trained Transformer models from Hugging Face, enabling NLP tasks without external languages.

TensorSwiftSwift

A lightweight Swift library for tensor calculations with TensorFlow-like APIs, enabling ML model inference.

Perfect TensorFlowSwift

A TensorFlow C API wrapper enabling machine learning in server-side Swift applications.

ONNX-ScalaScala

A type-safe, functional ONNX API and backend for deep learning and classical machine learning in Scala 3.