Advanced Parameters & Tuning
Kernel-level tuning, buffer pool sizing, and thread configuration
Hardware Tuning & GPU Slicing Flags
Granular parameters to fine-tune execution on Qualcomm Adreno and ARM Mali GPUs:
| Flag | Type | Default | Recommendation | Architectural Function |
|---|---|---|---|---|
--vocab-slice |
Integer | 0 (disabled) |
4096 on Mali, 8192 on Adreno |
Partitions the final vocabulary projection matrix into sequential slices, avoiding large temporary logit tensor allocations. |
--chunk-layers |
Integer | 0 (disabled) |
2 on Mali, 4 on Adreno |
Dispatches $N$ transformer layers per command buffer, relinquishing GPU execution context to reset mobile OS watchdog timers. |
--stream-layers |
Boolean | false |
Enabled on ≤8GB devices | Streams inactive layer weights from host RAM to GPU VRAM buffers on demand, keeping GPU resident memory minimal. |
--guard-band |
Integer (MB) | 300 |
300 - 500 |
System RAM threshold preserved at all times against Android Low Memory Killer (LMK). |
--virtual-ram-pool |
String | "" |
Fleet node IP list | Aggregates physical RAM across mobile devices into a unified 44GB virtual memory pool. |