frugally-deep
A lightweight header-only C++ library for running Keras (TensorFlow) models without linking against TensorFlow.
What is frugally-deep?
frugally-deep is a lightweight, header-only C++ library designed to run trained Keras models directly in C++ applications. It solves the problem of deploying machine learning models developed in Python/Keras into production C++ systems without requiring TensorFlow as a dependency. The library reimplements a subset of TensorFlow operations needed for prediction, resulting in smaller binary sizes and efficient CPU-based inference.
C++ developers and engineers who need to integrate pre-trained Keras models into performance-critical applications, embedded systems, or environments where Python or TensorFlow dependencies are undesirable.
Developers choose frugally-deep for its minimal footprint, ease of integration, and ability to run Keras models in pure C++ without linking against TensorFlow. Its header-only nature and support for complex model architectures make it a versatile and efficient alternative for inference in resource-constrained scenarios.
Overview
A lightweight header-only library for using Keras (TensorFlow) models in C++.
Use Cases
Best For
- Deploying Keras models in embedded systems with limited resources
- Integrating neural networks into high-performance C++ applications
- Reducing binary size by avoiding TensorFlow dependencies
- Running inference on CPUs in multi-threaded C++ pipelines
- Ensuring numerical consistency between Python training and C++ deployment
- Using Keras models in environments where Python is not available
Not Ideal For
- Applications requiring GPU acceleration for model inference
- Projects using Keras models with unsupported layers like LSTM, GRU, or Conv3D
- Teams needing strict compatibility with Python/TensorFlow versions outside the tested range
- Real-time systems where single-threaded prediction latency is critical, as optimizations focus on throughput via parallelism
Pros & Cons
Pros
As a header-only library, frugally-deep can be integrated into C++ projects without compiling external dependencies, simplifying build processes and reducing deployment overhead.
It supports sequential models and complex computational graphs from Keras' functional API, including nested models and residual connections, making it versatile for various architectures beyond simple layers.
Depends only on header-only libraries like Eigen and nlohmann/json, resulting in smaller binary sizes compared to linking against TensorFlow, ideal for resource-constrained or embedded systems.
The conversion script generates and runs test cases to ensure numerical consistency between Keras and C++ predictions, reducing deployment risks and increasing confidence in model accuracy.
Cons
Several key layer types, such as LSTM, GRU, and Conv3D, are not supported, restricting its use for models that rely on these advanced architectures, as noted in the README's 'Currently not supported' list.
The library ignores GPUs and uses only one CPU core per prediction, which can be a performance bottleneck for compute-intensive models that typically benefit from GPU acceleration, despite throughput gains via parallelism.
Requires specific versions of Python, TensorFlow, and Keras (e.g., TensorFlow 2.18.0, Keras 3.8.0), which can complicate integration in environments with different or custom setups, potentially leading to compatibility issues.
The disclaimer explicitly states that the API might change in the future, posing a risk for long-term maintenance and production systems that rely on stable interfaces.
Frequently Asked Questions
Related Projects
Tensorflow - Open source software library for numerical computation using data flow graphsAn Open Source Machine Learning Framework for Everyone
PyTorch - Tensors and Dynamic neural networks in Python with strong GPU accelerationTensors and Dynamic neural networks in Python with strong GPU acceleration
MXnet - Lightweight, Portable, Flexible Distributed/Mobile Deep Learning frameworkLightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
CNTK - Microsoft Cognitive ToolkitMicrosoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.