Model Inference

gpt4allC++

Run large language models (LLMs) privately on everyday desktops and laptops without requiring API calls or GPUs.

Paddle - PArallel Distributed Deep LEarning by BaiduC++

An industrial deep learning framework supporting unified dynamic/static graphs, automatic parallelism, and integrated training/inference for large models.

PaddlePaddleC++

An industrial deep learning framework from China supporting unified dynamic/static graphs, automatic parallelism, and integrated training/inference for large models.

BigDLPython

An LLM acceleration library for Intel XPU (GPU, NPU, CPU) to speed up local inference and finetuning of popular models.

TensorFlowSharpC#

A .NET binding to the TensorFlow C API for running existing machine learning models in C# and F#.

KotlinDLKotlin

A high-level Deep Learning API for JVM and Android developers, written in Kotlin and inspired by Keras.