Mechanistic Interpretability of Transformer MLPs Through Exact Soft‑Gate Decomposition

Transformer feed-forward (MLP) blocks perform nonlinear transformations, but their internal structure is often difficult to interpret using current methods. This paper presents an exact soft-gate decomposition that extracts the effective weight matrix for each MLP block at varying inputs. This approach generalises exact piecewise-affine decomposition from ReLU networks to Gaussian Error Linear Unit (GELU) and Sigmoid Linear Unit (SiLU) activations, covering both standard and gated MLP architectures. Verification across five models, including BERT, mBART, and GPT-2 for language tasks, ViT for vision tasks, and the gated SiLU architecture TinyLlama, confirms the accuracy of this method at floating-point arithmetic precision. The application of singular value decomposition to the effective weight matrix yields six structural metrics that characterise each block based on the dimensionality, intensity, and spectral concentration of its transformation. When applied to BERT, mBART, and GPT-2, and alongside ablation studies that provide causal ground truth, this analysis indicates that MLP computation is organised differently across encoder-only, encoder-decoder, and decoder-only models. Specifically, BERT blocks vary by transformation dimensionality, mBART blocks by transformation intensity, and GPT-2 blocks by intensity, which shows detrimental excess in the later layers. These structural differences, revealed only through the exact effective weight matrix, explain why scalar importance measures often fail to generalise across different architectures