6. FlashAttention-2 vs Standard Attention Profiler
Profiles on-chip SRAM tiling and evaluates real-time VRAM scaling from O(N^2) to O(N) across varying sequence lengths.
Transformer Attention Parameters
SRAM Tiling Formula:
Zero intermediate $N imes N$ matrix materialized in global VRAM.
Q_block @ K_block.T -> Online Softmax -> @ V_blockZero intermediate $N imes N$ matrix materialized in global VRAM.
VRAM Memory Materialization (Bytes)
WebGPU Kernel Dispatch Latency (ms)
Select parameters and click Profile to execute live attention comparison.