termux-train

On-Device Tiny Model & Small LLM Training Manual

Comprehensive engineering guide for training RoPE Transformers, Whisper LoRA, and DocFold models on mobile hardware.

1. Mobile Hardware Constraints & LMK Defense

Android OS triggers the Low Memory Killer (LMK) when free memory falls below 300MB. termux-train avoids this via:

• Zero-Dependency Core: Pure Python autograd consuming < 30MB base RAM.
• Streaming MMap Datasets: Direct page-cache disk streaming without heap bloat.
• SafeTensors Binary Format: Zero-copy tensor serialization.

2. Recipe 1: Tiny Transformer Language Model (RoPE + KV Cache)

from termux_train import Tensor, nn, optim, set_backend
from termux_train.tokenization import CharTokenizer

set_backend("auto")
tokenizer = CharTokenizer()
tokenizer.build_vocab(["to be or not to be that is the question"])

model = nn.TinyTransformerLM(
    vocab_size=tokenizer.vocab_size,
    d_model=64,
    num_heads=4,
    d_ff=128,
    num_layers=2,
    pos_type="rope",
    tie_weights=True
)

optimizer = optim.AdamW(model.parameters(), lr=0.005)
# Train and generate with KV cache
generated = model.generate(tokenizer.encode("to be"), max_new_tokens=20)

3. Recipe 2: Tiny Whisper Speech-to-Text LoRA Fine-Tuning

# Freeze base model and train LoRA adapters only (<30KB)
trainable_params = nn.adapter_parameters(model)
optimizer = optim.AdamW(trainable_params, lr=0.02)
# Save adapter
checkpoint.save_lora_adapter(model, "whisper_lora.safetensors")
# Merge for zero-overhead inference
nn.merge_lora_adapters(model)

4. Recommended Hyperparameter Matrix by Device RAM