How to run cONNXr on an Arduino or similar microcontroller?

Compile the C99 source files directly into your embedded project, as it has zero dependencies, but ensure your model uses only supported operators and data types. Refer to the out-of-the-box examples for guidance on integrating with input protocols.

cONNXr vs Microsoft ONNX Runtime: which is better for embedded devices?

cONNXr is lighter and written in C99 for extreme portability on constrained hardware, while ONNX Runtime is more feature-rich but heavier, often requiring C++ and accelerators. Choose cONNXr for bare-metal or legacy systems, and ONNX Runtime for devices with more resources.

What ONNX operators does cONNXr currently support?

It implements only a few basic operators, so check the documentation or source code for specifics; complex operators like LSTMs or attention layers are likely unsupported, limiting model selection.

How to add a custom operator implementation in cONNXr?

Use the extensible architecture to replace operators by modifying the source code, as mentioned in the README for dedicated accelerators, but this requires deep C programming knowledge and lacks a plugin system.

Is cONNXr stable enough for a commercial product?

No, the project is in early stage and described as a prototype, so it's risky for production due to potential bugs, incomplete features, and lack of long-term support guarantees.

Can cONNXr run quantized models for lower memory usage?

Limited support; the README includes a quantized MNIST model in the TODO list, but other quantized models may fail due to data type or operator limitations, so test thoroughly.

Open-Awesome

cONNXr

MITC

A pure C99 ONNX runtime with zero dependencies, designed for embedded devices and old hardware.

GitHub

219 stars39 forks0 contributors

What is cONNXr?

cONNXr is a pure C99 runtime for executing ONNX models, enabling machine learning inference on embedded devices and old hardware. It solves the problem of deploying trained models to environments that lack support for modern C++ or hardware accelerators, providing a lightweight, dependency-free solution.

Target Audience

Embedded systems engineers and developers working with resource-constrained devices, bare-metal hardware, or legacy systems that require machine learning inference capabilities.

Value Proposition

Developers choose cONNXr for its extreme portability and zero-dependency design, allowing inference on hardware where other runtimes are too heavy or incompatible, while remaining framework-agnostic via the ONNX standard.

Overview

Pure C ONNX runtime with zero dependancies for embedded devices

Use Cases

Best For

Running ML inference on microcontrollers and embedded devices
Deploying models to legacy hardware with limited C/C++ support
Bare-metal implementations requiring a lightweight runtime
Edge computing scenarios with strict resource constraints
Prototyping ML applications for embedded systems
Educational projects exploring ONNX runtime internals

Not Ideal For

Projects requiring a full set of ONNX operators for complex models like transformers or RNNs
High-performance inference on modern servers or GPUs where acceleration is needed
Production systems that demand stable, battle-tested runtimes with long-term support
Applications that rely on multi-threading or parallelism to meet real-time deadlines

Pros & Cons

Pros

Zero Dependencies

Pure C99 implementation with no external libraries allows deployment on virtually any embedded device, as highlighted by its compatibility with old hardware in the README.

Framework Agnostic

Supports models from any training framework via ONNX export, making it versatile for TensorFlow, Keras, or scikit-learn, as stated in the documentation.

Embedded-First Design

Optimized for bare-metal and resource-constrained environments, enabling ML inference where modern C++ or accelerators are unavailable, per the project's philosophy.

Extensible Architecture

Allows developers to replace operators with custom implementations for dedicated hardware accelerators, offering flexibility for specialized deployments.

Cons

Limited Operator Support

Only basic operators are implemented, so many ONNX models will fail to run, as admitted in the README under limitations.

Early Development Stage

Explicitly labeled as not production-ready and a proof-of-concept, meaning it lacks stability and comprehensive testing for real-world use.

Single-Threaded Performance

Non multi-threaded C99 code restricts inference speed on devices that could benefit from parallelism, a trade-off for portability.

DataType Incompleteness

Reference implementation focuses on float, causing issues with other data types like int16 or double, limiting model compatibility.

Frequently Asked Questions

Related Projects

PyTorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Stars101,899

Forks28,473

Last commit17 hours ago

keras

Deep Learning for humans

Streamlit — A faster way to build and share data apps.

Stars45,326

Forks4,331

Last commit22 hours ago

gradio

Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!

Stars43,191

Forks3,557

Last commit20 hours ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub