ggml
A low-level tensor library for machine learning with integer quantization, automatic differentiation, and zero runtime allocations.
What is ggml?
ggml is a low-level tensor library for machine learning that provides efficient tensor operations, automatic differentiation, and integer quantization support. It is designed to run machine learning models with minimal dependencies and zero runtime memory allocations, making it suitable for resource-constrained environments.
Developers and researchers building or deploying machine learning models, especially those focused on efficient inference, quantization, or embedded systems.
Developers choose ggml for its minimal footprint, cross-platform portability, and focus on inference efficiency, particularly when working with quantized models or requiring predictable performance without third-party dependencies.
Overview
Tensor library for machine learning
Use Cases
Best For
- Running large language models (LLMs) on CPUs with quantization
- Building efficient inference engines for embedded or edge devices
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
