How to set up RunAnywhere for offline speech recognition in a Flutter app?

Install the runanywhere and runanywhere_onnx packages via pub.dev, initialize the SDK, download a Whisper model (e.g., Whisper Tiny), and use the speech-to-text APIs. Refer to the Flutter quick start in the README for code examples and ensure your device meets the memory requirements.

RunAnywhere vs Google's ML Kit for on-device AI?

RunAnywhere offers a more comprehensive offline voice AI pipeline and supports custom GGUF models via llama.cpp, whereas ML Kit focuses on Google-curated models with tighter Android/iOS integration but less flexibility. Choose RunAnywhere for privacy-first, cross-platform apps needing speech synthesis and LLM capabilities without cloud fallback.

What models can I use for text generation with RunAnywhere on iOS?

You can use GGUF-format models like SmolLM2, Qwen, Llama, or Mistral via llama.cpp integration. The README lists supported models with sizes and RAM requirements, and you need to download and load them using the SDK's model management functions, as shown in the Swift example.

Is RunAnywhere suitable for building a real-time voice assistant on Android?

Yes, it supports a full STT → LLM → TTS pipeline offline on Android, but performance depends on device hardware and model size. Use lighter models like SmolLM2 for faster inference, and check the Kotlin documentation for implementation details on streaming and voice activity detection.

How much storage space do RunAnywhere models take on a device?

Model sizes vary from ~400MB for SmolLM2 to ~4GB for Mistral 7B, plus additional space for speech and vision models. The README provides a table with approximate sizes, and you should plan for progressive downloading and caching in your app's storage management.

Can I use RunAnywhere for image generation in a web app?

Yes, the Web SDK (beta) supports on-device image generation via diffusion models, as shown in the Image Generation GIF. However, it requires WebAssembly and WebGPU support in modern browsers, and models must be hosted or downloaded locally, which can impact load times.

Open-Awesome

RunAnywhere

NOASSERTIONC++cpp-desktop-v0.20.42

SDKs for adding private, on-device AI features like LLM chat, speech-to-text, and text-to-speech to mobile and web apps.

Visit Website GitHub

10.3k stars395 forks0 contributors

What is RunAnywhere?

RunAnywhere is a production-ready toolkit that provides multi-platform SDKs for integrating on-device AI capabilities into applications. It enables tasks like text generation, speech-to-text, and text-to-speech to run locally without cloud dependency, ensuring privacy, offline functionality, and low latency. The project solves the problem of cloud reliance and data privacy concerns by allowing AI features to operate entirely on the user's device.

Target Audience

Mobile and web developers building applications that require private, offline AI features, such as voice assistants, chatbots, or AI-powered tools, across platforms like iOS, Android, React Native, Flutter, and the web. It is also suitable for developers prioritizing data privacy and low-latency AI interactions.

Value Proposition

Developers choose RunAnywhere because it offers a unified, multi-platform solution for on-device AI that eliminates cloud costs, latency, and privacy risks. Its unique selling point is the fully offline voice AI pipeline that combines speech-to-text, LLM processing, and text-to-speech, along with support for various AI models and vision capabilities on supported platforms.

Overview

Production ready toolkit to run AI locally

Use Cases

Best For

Building privacy-first mobile applications that require AI features without sending data to the cloud.
Developing offline-capable voice assistants that integrate speech-to-text, LLM reasoning, and text-to-speech locally.
Creating cross-platform AI tools with SDKs for Swift, Kotlin, React Native, Flutter, and the web.
Implementing on-device text generation using models like Llama, Mistral, or Qwen via llama.cpp.
Adding speech transcription and synthesis to applications using Whisper and Piper models without internet dependency.
Building applications with vision-language models for on-device image understanding and generation on supported platforms.

Not Ideal For

Projects requiring state-of-the-art AI models with billions of parameters that exceed typical device memory limits
Teams building Android or cross-platform apps needing comprehensive vision AI features, as visual language models are unsupported on Android, React Native, and Flutter
Production applications on Web, React Native, or Flutter where stable SDKs are critical, given their beta status
Scenarios demanding seamless cloud integration for scalability or real-time updates, as the toolkit is designed for offline, on-device execution

Pros & Cons

Pros

Multi-Platform Coverage

Provides stable SDKs for Swift and Kotlin, plus beta SDKs for Web, React Native, and Flutter, enabling consistent on-device AI across iOS, Android, and cross-platform frameworks, as shown in the platform support table.

Privacy-First Design

Ensures all AI inference runs locally with no data sent to the cloud, addressing privacy concerns and enabling offline functionality, as emphasized in the philosophy and feature descriptions.

Integrated Voice Pipeline

Combines speech-to-text, LLM processing, and text-to-speech into a fully offline voice assistant, demonstrated in the Voice AI GIF and supported across all SDKs except where noted.

Flexible Model Management

Supports downloading and loading various GGUF and ONNX models (e.g., Llama, Whisper) with progress tracking, allowing customization based on device constraints, as detailed in the Supported Models section.

Cons

Platform Feature Gaps

Key capabilities like vision language models are only available on iOS and Web, not on Android, React Native, or Flutter, limiting cross-platform use cases, as indicated in the Features table.

Beta SDK Instability

Web, React Native, and Flutter SDKs are labeled as beta, which may lead to breaking changes, incomplete features (e.g., structured output missing in some), and less reliable documentation for production use.

Hardware Intensive

Requires at least 2GB of RAM and 4GB+ recommended for larger models, excluding low-end devices and increasing app size due to model downloads, as noted in the Requirements section.

Complex Initial Setup

Involves multiple steps like initializing SDKs, registering backends (e.g., LlamaCPP), and managing model downloads separately, which adds overhead compared to cloud-based APIs with simpler integrations.

Frequently Asked Questions

Related Projects

promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

🦉 Data Versioning and ML Experiments

Stars15,934

Forks1,334

Last commit3 days ago

txtai

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

Stars11,030

Forks1,095

Last commit1 day ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub

RunAnywhere

NOASSERTIONC++cpp-desktop-v0.20.42

SDKs for adding private, on-device AI features like LLM chat, speech-to-text, and text-to-speech to mobile and web apps.

Visit Website GitHub

10.3k stars395 forks0 contributors

What is RunAnywhere?

Target Audience

Value Proposition

Overview

Production ready toolkit to run AI locally

Use Cases

Best For

Building privacy-first mobile applications that require AI features without sending data to the cloud.
Developing offline-capable voice assistants that integrate speech-to-text, LLM reasoning, and text-to-speech locally.
Creating cross-platform AI tools with SDKs for Swift, Kotlin, React Native, Flutter, and the web.
Implementing on-device text generation using models like Llama, Mistral, or Qwen via llama.cpp.
Adding speech transcription and synthesis to applications using Whisper and Piper models without internet dependency.
Building applications with vision-language models for on-device image understanding and generation on supported platforms.

Not Ideal For

Projects requiring state-of-the-art AI models with billions of parameters that exceed typical device memory limits
Teams building Android or cross-platform apps needing comprehensive vision AI features, as visual language models are unsupported on Android, React Native, and Flutter
Production applications on Web, React Native, or Flutter where stable SDKs are critical, given their beta status
Scenarios demanding seamless cloud integration for scalability or real-time updates, as the toolkit is designed for offline, on-device execution

Pros & Cons

Pros

Multi-Platform Coverage

Privacy-First Design

Ensures all AI inference runs locally with no data sent to the cloud, addressing privacy concerns and enabling offline functionality, as emphasized in the philosophy and feature descriptions.

Integrated Voice Pipeline

Combines speech-to-text, LLM processing, and text-to-speech into a fully offline voice assistant, demonstrated in the Voice AI GIF and supported across all SDKs except where noted.

Flexible Model Management

Cons

Platform Feature Gaps

Key capabilities like vision language models are only available on iOS and Web, not on Android, React Native, or Flutter, limiting cross-platform use cases, as indicated in the Features table.

Beta SDK Instability

Hardware Intensive

Requires at least 2GB of RAM and 4GB+ recommended for larger models, excluding low-end devices and increasing app size due to model downloads, as noted in the Requirements section.

Complex Initial Setup