libonnx
A lightweight, portable pure C99 ONNX inference engine for embedded devices with hardware acceleration support.
What is libonnx?
Libonnx is a lightweight, portable inference engine for ONNX models written in pure C99. It enables running machine learning models on embedded devices and resource-constrained environments while supporting hardware acceleration through custom resolvers. The library provides a simple C API for loading models, running inference, and managing tensors without external dependencies.
Embedded systems developers and engineers who need to deploy ONNX-based machine learning models on resource-constrained devices like microcontrollers, IoT devices, or edge computing platforms.
Developers choose Libonnx for its minimal footprint, pure C99 implementation that ensures maximum portability, and hardware acceleration support that allows optimization for specific embedded hardware. Unlike heavier frameworks, it's designed specifically for embedded environments where resource efficiency is critical.
Overview
A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.
Use Cases
Best For
- Deploying ONNX models on microcontrollers and embedded systems
- Running machine learning inference on resource-constrained edge devices
- Integrating hardware accelerators with ONNX models in embedded applications
- Cross-compiling neural network inference for ARM-based embedded platforms
- Building lightweight computer vision applications for embedded devices
- Creating portable machine learning solutions without Python or heavy frameworks
Not Ideal For
- Projects requiring the latest ONNX operator versions or cutting-edge model architectures beyond opset 24
- Teams with Python-centric ML workflows seeking seamless inference integration without low-level C coding
- Applications needing out-of-the-box GPU or NPU acceleration without custom resolver implementation
- High-throughput server-side inference scenarios where advanced features like dynamic batching are essential
Pros & Cons
Pros
Implemented in pure C99 with no external dependencies, allowing it to be dropped directly into projects and compiled across diverse embedded platforms, as demonstrated by the cross-compilation example for ARM64.
Supports custom hardware accelerators through resolver arrays, enabling optimized inference for specific embedded hardware, mentioned in the context allocation function for passing resolvers.
Designed specifically for resource-constrained environments, making it ideal for microcontrollers and edge devices without the overhead of larger frameworks, as highlighted in the project description.
Provides straightforward C functions like onnx_context_alloc_from_file and onnx_run, making integration simple for C-based embedded projects, with clear code snippets in the README.
Cons
Not all ONNX operators are implemented; the README notes that some tests fail due to unimplemented operators, limiting compatibility with certain models and requiring careful model selection.
Requires explicit allocation and freeing of context and tensors using C functions like onnx_context_free, which can be error-prone and less safe compared to managed languages or higher-level frameworks.
Relies heavily on external ONNX documentation and has only a Chinese discussion post for support, indicating a smaller ecosystem and potential hurdles for troubleshooting or advanced use cases.
Frequently Asked Questions
Related Projects
PyTorch - Tensors and Dynamic neural networks in Python with strong GPU accelerationTensors and Dynamic neural networks in Python with strong GPU acceleration
kerasDeep Learning for humans
streamlitStreamlit — A faster way to build and share data apps.
gradioBuild and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.