Llm Evaluation

FastChatPython

An open platform for training, serving, and evaluating large language model based chatbots.

promptfooTypeScript

A CLI and library for evaluating, red-teaming, and comparing LLM prompts, agents, and RAGs with simple declarative configs.

OpikPython

An open-source platform for debugging, evaluating, and monitoring LLM applications, RAG systems, and agentic workflows with tracing and automated evaluations.

CometLLMPython

An open-source platform for debugging, evaluating, and monitoring LLM applications, RAG systems, and agentic workflows.

OpenAI EvalsPython

A framework and open-source registry for evaluating large language models (LLMs) and LLM systems.

TensorZeroRust

An open-source LLMOps platform unifying gateway, observability, evaluation, optimization, and experimentation for industrial-grade LLM applications.

GarakHTML

A command-line tool for red-teaming and vulnerability scanning of large language models (LLMs).

evidentlyJupyter Notebook

An open-source Python framework to evaluate, test, and monitor ML and LLM systems with 100+ built-in metrics.

agentaTypeScript

An open-source LLMOps platform for prompt management, evaluation, and observability to build reliable LLM applications faster.