Mobile Memory Management & INT8 Quantization
Guidelines for MMap disk streaming, SafeTensors zero-copy serialization, and LMK defense.
1. SafeTensors Zero-Copy Binary Architecture
Traditional PyTorch Python pickling creates duplicate memory copies. SafeTensors maps contiguous binary buffers directly to memory:
from termux_train import checkpoint
# Save model parameters to SafeTensors
checkpoint.save_safetensors(model.state_dict(), "model.safetensors")
# Load with metadata validation
tensors, metadata = checkpoint.load_safetensors("model.safetensors")2. INT8 AbsMax Weight Quantization (75% RAM Reduction)
from termux_train import nn
# Convert FP32 linear layer to zero-allocation INT8 layer
q_linear = nn.quantize_linear_int8(model.fc1)
# Inference runs via (x @ W_int8) * scale
y = q_linear(x)3. MMap Streaming Token Dataset
from termux_train.data import MMapTokenDataset
# Stream multi-gigabyte corpus from disk
dataset = MMapTokenDataset.create_from_tokens(tokens=token_list, filepath="dataset.bin", seq_len=64)
x, target = dataset[0] # Instant zero-allocation slice