Termux-LlamaCpp - AMEVA Official Docs

Quickstart & Practical Recipes

Step-by-step practical guides for running local LLM inference, deploying the OpenAI-compatible server, and connecting client applications.

Recipe 1: Interactive CLI Prompt Inference

Execute zero-latency single prompt generation with explicit token budgets directly from Termux:

CLI Execution

# Download curated lightweight model
termux-llama download qwen2.5-1.5b-instruct

# Run interactive CLI with 4 threads and direct memory mapping
termux-llama run qwen2.5-1.5b-instruct -p "Explain quantum computing in 3 sentences." --temp 0.2 --max-tokens 128

Recipe 2: Starting the OpenAI-Compatible Supervisor Server

Launch the multi-threaded reverse proxy supervisor exposing standard OpenAI endpoints on loopback http://127.0.0.1:8080:

Server Launch

termux-llama serve Llama-3.2-3B-Instruct-Q4_K_M.gguf --port 8080 --ctx 2048 --threads 4

Recipe 3: OpenAI REST & SSE Streaming Curl Requests

Interact with the server using standard HTTP requests:

REST API Requests

# 1. Health & Model Readiness Check
curl -s http://127.0.0.1:8080/health

# 2. List Available Models
curl -s http://127.0.0.1:8080/v1/models

# 3. Non-Streaming Chat Completion
curl -X POST http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Llama-3.2-3B-Instruct-Q4_K_M.gguf",
    "messages": [{"role": "user", "content": "Hello!"}],
    "temperature": 0.0,
    "max_tokens": 16
  }'

# 4. Real-Time Server-Sent Events (SSE) Streaming
curl -N -X POST http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Llama-3.2-3B-Instruct-Q4_K_M.gguf",
    "messages": [{"role": "user", "content": "Stream a short greeting."}],
    "stream": true
  }'

Recipe 4: Python SDK Integration

Python (3.9+)

from termux_llamacpp import LlamaRuntime

runtime = LlamaRuntime()
server = runtime.serve(
    model="Llama-3.2-3B-Instruct-Q4_K_M.gguf",
    port=8080,
    ctx_size=2048,
    threads=4
)

print(f"OpenAI server ready at: {server.endpoint}")

Recipe 5: Node.js / TypeScript SDK Integration

TypeScript / JavaScript (ESM)

import { LlamaRuntime } from "termux-llamacpp";

const runtime = new LlamaRuntime();
const server = await runtime.serve({
  model: "Llama-3.2-3B-Instruct-Q4_K_M.gguf",
  port: 8080,
  ctxSize: 2048,
  threads: 4
});

console.log(`Server listening on ${server.endpoint}`);