Functional Modularity from Activation Patterns Only

We introduce a framework for discovering and inducing functional modularity in neural networks through unsupervised analysis of activation patterns alone, requiring neither task labels nor architectural constraints. Unlike existing approaches that rely on supervised auxiliary tasks or modify the training objective, our method maintains complete separation between task learning and modularity discovery by projecting computational units into a learned embedding space through contrastive learning. This geometric organization reveals how neurons self-organize into functional ensembles during training, enabling task-agnostic measurement of modular structure. We further demonstrate that this learned space can optionally guide gradient flow to actively encourage specialization without degrading task performance, eliminating the allegedly present accuracy-modularity trade-off. Through systematic Bayesian architecture search across MNIST and Fashion-MNIST, we show that our framework consistently identifies networks exhibiting both competitive accuracy and measurable functional modularity. Our approach discovers emergent structure rather than imposing it, providing interpretable insights into how neural networks decompose computational problems into specialized subnetworks.