Estimating Neural Representation Geometry in Large Networks

Quantifying the structure and similarity of neural representations is essential for better understanding and training of large neural-network models. However, conventional analysis based on covariance matrices yields biased estimators under realistic constraints of finite data. In this talk, I present recent advances that address these limitations. A dynamic-programming-based estimator is introduced for computing the spectral moments of kernel integral operators from finite measurement matrices, even when both inputs and features are subsampled. This method is also applied to improving the accuracy of the Centered Kernel Alignment (CKA) metric used to compare neural representations across models. Our results highlight the importance of debiased statistical estimators and principled, scalable tools for neural-representation analysis.

Speaker

Daniel Dongyuel Lee is the Tisch University Professor of Electrical and Computer Engineering at Cornell Tech and Cornell Engineering.