Termux-LlamaCpp Runtime & Supervisor
Production-Grade, Prebuilt GGUF LLM Runtime, Model Manager & OpenAI-Compatible REST/SSE Supervisor for Android Termux & ARM64.
Zero-Compilation Prebuilt
ARM64 NEON & DotProd
OpenAI REST & SSE
Apache-2.0 Open Source
⚡ Instant Zero-Compilation Installation (Android Termux)
curl -sSL https://raw.githubusercontent.com/uno-km/termux-llamacpp/master/scripts/install.sh | bash1. System Architecture Overview
Termux-LlamaCpp eliminates the traditional friction of running Large Language Models on mobile hardware. By packaging pre-compiled Android Bionic ARM64 binaries alongside verified shared libraries, it bypasses the multi-gigabyte Clang compiler toolchain and guarantees instantaneous execution in under 3 seconds.
| Component Layer | Technical Implementation | Engineering Responsibility |
|---|---|---|
| Prebuilt Engine | ARMv8.2-A + FP16 + DotProd Clang Binaries | Direct native CPU tensor execution with zero compilation delay. |
| Reverse Proxy Supervisor | Python/Node.js Async Multi-Threaded Gateway | Translates OpenAI v1 REST requests to native loopback endpoints with CORS protection. |
| Model Manager | Unified Hugging Face Caching & Checksum Verifier | Resilient GGUF streaming download with SHA-256 integrity checks. |
| Security Verifier | Ed25519 Public Key Cryptographic Manifest | Fail-closed anti-downgrade policy and TOCTOU symlink traversal prevention. |
2. Key Measured Performance (Samsung Galaxy S20+ Real-Device Ground Truth)
Empirically tested on Samsung Galaxy S20+ 5G (Qualcomm Snapdragon 865, 8-Core Kryo 585 ARM64 CPU) running Android 13 Termux:
PROMPT PROCESSING SPEED
16.19 t/s
61.77 ms / token (38 tokens in 2.34s)
TOKEN GENERATION SPEED
10.23 t/s
97.75 ms / token interactive generation
COLD MODEL LOAD TIME
~1.8 sec
Direct memory load (1.92 GiB 3B Q4_K_M)
INSTALLATION DURATION
< 3.0 sec
Zero-compilation atomic binary extraction
3. Supported Ecosystem Integrations
Termux-LlamaCpp serves as the primary high-throughput LLM backbone across the entire AMEVA on-device ecosystem:
- Termux-AIChain: Direct local agent DAG task execution via loopback HTTP adapter.
- Termux-Playwright: Web crawling agent intelligence and automated browser DOM synthesis.
- Termux-STT: Multi-speaker audio transcription refinement and instant meeting note summarization.