MultiGPT

MultiGPT uses llama-kotlin-android, a native Kotlin wrapper around llama.cpp compiled via Android NDK/JNI. It loads GGUF-format quantized models directly into device memory using mmap and runs inference locally on your phone's ARM CPU with NEON SIMD acceleration. The app auto-detects your device's RAM, CPU cores, and available memory, then dynamically adjusts context size, batch size, and thread count. This means the AI model runs entirely on your Android device without any internet connection or cloud servers.

MultiGPT supports running any GGUF-format model locally on Android including: Llama 3.2 (1B, 3B parameters), Mistral 7B, Phi-3 Mini (3.8B), Gemma 2B, Qwen 2.5 (0.5B-7B), TinyLlama 1.1B, and other compatible GGUF models. The app includes a built-in model catalog that fetches available models from Hugging Face. Smaller models (1-3B parameters) run smoothly on most modern Android devices with 4GB+ RAM, while 7B models require devices with 8GB+ RAM.

MultiGPT supports 6 cloud AI providers: OpenAI (GPT-4o, GPT-4 Turbo, GPT-4), Anthropic (Claude 3.5 Sonnet, Claude 3 Opus/Sonnet/Haiku), Google Gemini (Gemini 1.5 Pro, Flash, 1.0 Pro), Groq (Llama 3.1/3.2, Gemma 2), AWS Bedrock (12+ foundation models including Claude, Titan, Jurassic-2, Llama), and Ollama (any self-hosted model via server URL). Each provider is configured with your own API key or credentials, and the app dynamically fetches the latest available models from each provider.

For OpenAI, Anthropic, Google Gemini, and Groq, simply enter your API key in the provider settings. For AWS Bedrock, enter your Access Key, Secret Key, Region, and optional Session Token. For Ollama, enter your server URL (e.g., http://192.168.1.100:11434). All API keys are stored locally on your device using encrypted DataStore preferences — they are never sent to our servers. The app communicates directly with each provider's API.

It depends on which mode you use. For LOCAL AI inference with GGUF models via llama.cpp, NO internet is required — the model runs entirely on your Android device and works in airplane mode. For CLOUD providers (OpenAI, Claude, Gemini, Groq, Bedrock), you need an internet connection to reach their API servers. For Ollama, you need network access to reach your Ollama server (which can be on your local LAN). You can use local and cloud models simultaneously in the same conversation.

llama.cpp is an open-source C/C++ library that enables efficient Large Language Model (LLM) inference on consumer hardware. MultiGPT uses it via llama-kotlin-android (a Kotlin/JNI wrapper). It's ideal for Android because: it's written in pure C/C++ with minimal dependencies, supports ARM NEON SIMD instructions for fast mobile inference, works with quantized GGUF models that fit in mobile memory, uses memory-mapped (mmap) file loading for efficiency, requires no Python runtime or heavy ML frameworks, and can leverage multiple CPU cores on Android devices.

MultiGPT performs automatic RAM checks before loading any model. Requirements: TinyLlama 1.1B needs ~2GB available RAM (4GB+ total device RAM), Gemma 2B needs ~3GB available (6GB+ total), Llama 3.2 3B and Phi-3 Mini need ~4GB available (6GB+ total), and Mistral 7B or Llama 3.1 7B need ~6-8GB available (8GB+ total device RAM). The app dynamically adjusts context window and batch size based on your device's available memory for stable performance.

Yes. For local AI models via llama.cpp, all processing happens entirely on your Android device — your prompts and outputs never leave your phone. For cloud providers, your API keys communicate directly with provider APIs (OpenAI, Anthropic, Google, etc.) with no proxy servers or middleware. API keys are encrypted and stored locally via DataStore. MultiGPT has zero analytics, zero tracking, zero telemetry, and requires no user accounts. The app is open source so you can audit the code yourself.

Yes! This is MultiGPT's unique feature. Enable multiple providers in a single conversation — for example, a local Llama 3.2 3B model alongside GPT-4o and Claude. Send one message and see responses from all enabled models side-by-side. This lets you compare how a free, private, offline local model performs against paid cloud models for your specific use case.

GGUF (GPT-Generated Unified Format) is the standard file format used by llama.cpp for storing quantized AI models. MultiGPT includes a built-in model catalog that lists popular GGUF models from Hugging Face with download links. You can also manually download GGUF files from Hugging Face. Models come in various quantization levels — Q4_K_M (recommended for mobile, best balance of size and quality) and Q8_0 (higher quality, larger files). Once downloaded to your device, models work forever offline.

Ollama is a self-hosted AI server that runs on your PC, Mac, or Linux machine. In MultiGPT, you enter your Ollama server's URL (e.g., http://192.168.1.100:11434) and the app auto-discovers all models available on your server. This lets you run larger models (13B, 70B+) on powerful desktop hardware while chatting from your Android phone over your local network. It's a great middle ground between fully local on-device inference and cloud APIs.