Llama

OllamaGo

A platform to run, manage, and serve open-source large language models (LLMs) locally or on your own infrastructure.

OllamaGo

A platform to run, manage, and serve open-source large language models locally with a simple CLI and REST API.

vllmPython

A high-throughput, memory-efficient inference and serving engine for large language models (LLMs).

unslothPython

A web UI and optimization library for running and fine-tuning open-source AI models locally with 2x faster training and 70% less VRAM.

LocalAIGo

An open-source AI engine that runs LLMs, vision, voice, and image/video models on any hardware with drop-in OpenAI API compatibility.

sglangPython

A high-performance serving framework for large language models and multimodal models, delivering low-latency and high-throughput inference.

LangChain4jJava

An open-source Java library that simplifies integrating LLMs into Java applications through a unified API and comprehensive toolbox.

MedicalGPTPython

An open-source pipeline for training medical domain GPT models using PT, SFT, RLHF, DPO, ORPO, and GRPO methods.

shimmyRust

A lightweight, single-binary Rust inference server providing 100% OpenAI-API compatible endpoints for local GGUF models.

LLamaSharpC#

A C#/.NET library for efficient local inference of LLaMA and other large language models, based on llama.cpp.

EasyLMPython

A JAX/Flax-based framework for easy and scalable pre-training, fine-tuning, evaluation, and serving of large language models.

llm-uiTypeScript

A React library for building smooth, customizable user interfaces for LLM-powered applications with streaming output.