AMEVA-Runtime
Next-Gen Unified On-Device Hardware Orchestration & 6-Modality Acceleration Runtime for Mobile & Edge (with 1:1 Modality Library Isolation & Adreno OpenCL Smart Routing)
Install the official package directly into your runtime:
pip install ameva-runtime
# or: npm install @ameva/runtimeThe Engineering Challenge
Deploying generative multi-modal AI on mobile hardware encounters severe vendor-specific compiler bugs and driver anomalies. Qualcomm Adreno GPUs suffer from JIT compiler crashes during specialization constant dispatch, while ARM Mali drivers encounter compute shader integer truncation infinite loops and headless fence deadlocks without active display swapchains.
The Architectural Breakthrough
AMEVA Runtime and AMUDA (AMEVA-Compute) form a unified on-device hardware orchestration and AI acceleration engine engineered for mobile and edge systems. It dynamically inspects SoC topology and driver environments, routing compute graphs between Qualcomm Adreno, ARM Mali OpenCL, and ARM Cortex CPU-NEON.
Key Capabilities & Built-in Hardening
SmartRouter Silicon Topology Dispatch
Analyzes device nodes (/dev/kgsl-3d0, /dev/mali0) and cgroups to dispatch between Qualcomm Adreno and ARM Mali.
Native Adreno 830 Full Offloading
Runs all LLM layers on the GPU of Snapdragon 8 Elite through the bundled ggml-ameva backend: Qwen2.5-0.5B prompts at 1010-1081 tokens/sec (CPU 30-58) and generation at 36-77 tokens/sec (CPU 16-49) with the CPU's text (measured with v2.8.0 while the device ran other work).
Mali Valhall MatMul Quirk Engine
On Mali the GPU route runs on ggml-ameva with kernel layouts measured per device and weight shape. Galaxy A35 (Mali-G68): Qwen2.5-0.5B prompts at 190 tokens/sec (CPU 109-110) and generation at 21.5 tokens/sec (CPU 35-37) with the CPU's text; Llama-3.2-1B and Gemma-2-2B run as well (v2.8.0 release notes).
Whisper STT 2.26x Hardware Acceleration
Accelerates Whisper Large-v3-Turbo on mobile GPU (360s vs 816s CPU, 56% time reduction) while unburdening CPU.
Z-Image Turbo 6.0B DiT Vulkan Compute
Dispatches SOTA Diffusion Transformer compute with Flash Attention and VAE tiling directly on mobile Qualcomm Adreno and ARM Mali GPUs.
Two-Track Enterprise Supply Chain Architecture
Decouples proprietary source code into private repositories while providing verified release binaries via public registry for tokenless 1-Click provisioning.
Unified Multi-Modality Architecture
Drives GGUF LLM, Whisper STT, LLaVA Vision, Stable Diffusion / DiT, Piper TTS, and BitNet from a single runtime.
Supported Compute Kernels & Operations
| Subsystem Category | Operations & Kernels | Status |
|---|---|---|
| Galaxy S25 (Qualcomm Adreno 830) | Vulkan through ggml-ameva, all LLM layers on the GPU: prompt 1010-1081 t/s (CPU 30-58), generation 36-77 t/s (CPU 16-49) for Qwen2.5-0.5B | Production |
| Galaxy A35 (ARM Mali-G68) | Vulkan through ggml-ameva: prompt 190 t/s (CPU 109-110), generation 21.5 t/s (CPU 35-37) for Qwen2.5-0.5B; Whisper Large-v3-Turbo (2.26x) | Production |
| Mobile DiT Diffusion (Z-Image Turbo 6.0B) | Vulkan DiT Pipeline, Flash Attention (--diffusion-fa), VAE Tiling | Production (v2.7.4) |
| Two-Track Public Registry | Anonymous 1-Click Auto-Provisioning via uno-km/ameva-runtime-releases | Active |
| Multi-Modality Engines | GGUF LLM, Whisper STT, LLaVA Vision, Stable Diffusion / DiT, Piper TTS, BitNet | Production |
Canonical Usage Example
import ameva_runtime as ameva
from ameva_runtime import vulkan
# 1. Execute LLM inference directly with optimal on-device hardware dispatch
result = ameva.run(
model="qwen2.5-0.5b",
prompt="Space in Korean is:",
max_tokens=32
)
print(f"Generated text: {result.text}")
print(f"Hardware backend: {result.backend_used} ({result.tokens_per_second:.2f} t/s)")
# 2. Hardware diagnostic inspection via Vulkan engine
doc = vulkan.Doctor()
report = doc.run_self_test(verbose=False)
print(f"GPU Target: {report.device_name} (Passed: {report.passed_stages}/{report.total_stages})")