Transformer-squared does this through singular-value decomposition (SVD), a linear algebra trick that breaks down a matrix into three other matrices that reveal its inner structure and geometry.