Advanced Parameters & Tuning

Complete technical control over 12 local server hardware flags and 8 full-spectrum sampling parameters

Directly control process initialization flags when binding to llama-server or BitNet.cpp:

Parameter	Type	Default	Valid Range	Technical Function & Performance Impact
`threads`	`int`	`CPU-1`	`1 ~ 16`	Number of dedicated CPU threads for BLAS/NEON tensor computation.
`n_ctx`	`int`	`2048`	`512 ~ 32768`	Total token capacity allocated for the model context window.
`n_batch`	`int`	`512`	`32 ~ 2048`	Prompt evaluation batch size. Higher values increase prompt processing speed.
`n_ubatch`	`int`	`256`	`16 ~ 512`	Micro-batch size for memory-constrained mobile devices.
`n_gpu_layers`	`int`	`0`	`0 ~ 99`	Number of model layers offloaded to Vulkan / OpenCL / GPU compute.
`flash_attn`	`bool`	`False`	`True / False`	Flash Attention kernel acceleration toggle (`-fa`). Reduces memory bandwidth.
`cache_type_k`	`str`	`"f16"`	`"f16", "q8_0", "q4_0"`	Key cache quantization format. `q8_0` saves 50% RAM; `q4_0` saves 75% RAM.
`cache_type_v`	`str`	`"f16"`	`"f16", "q8_0", "q4_0"`	Value cache quantization format.
`mlock`	`bool`	`False`	`True / False`	Lock model weights in RAM to prevent Android OS paging / disk swapping.
`cont_batching`	`bool`	`True`	`True / False`	Continuous batching support for multi-turn conversations.
`rope_freq_scale`	`float`	`None`	`0.1 ~ 1.0`	Linear RoPE context extension factor for extended context windows.
`port`	`int`	`8080`	`1024 ~ 65535`	Local TCP port for the model server.

Parameter	Type	Default	Valid Range	Technical Description
`temperature`	`float`	`0.7`	`0.0 ~ 2.0`	Nucleus generation randomness. Set `0.0` for deterministic code and JSON parsing.
`top_p`	`float`	`0.95`	`0.0 ~ 1.0`	Cumulative probability cutoff threshold for candidate token filtering.
`top_k`	`int`	`40`	`1 ~ 100`	Integer limit on candidate token selection pool.
`min_p`	`float`	`0.05`	`0.0 ~ 1.0`	Minimum relative probability cutoff to eliminate low-rank hallucinations.
`repeat_penalty`	`float`	`1.1`	`1.0 ~ 2.0`	Frequency penalty scale to avoid infinite token repetition loops.
`stop`	`List[str]`	`None`	`List[str]`	Generation termination sequence delimiters.
`seed`	`int`	`None`	`int`	Random seed for exact deterministic generation reproducibility.
`grammar`	`str`	`None`	`str`	GBNF or Regex structural constraint schema for forced JSON output.