Understanding Neural Networks with Network Science

Neural networks are networks, yet we rarely study them using the tools developed to understand complex systems. In this talk, I will use network science to examine how task difficulty shapes a model’s internal structure, when different graph neural network architectures work best, and whether a model trained on a small graph can transfer to a much larger one. First, I will represent the layers of multilayer perceptrons as signed, weighted bipartite graphs and probe them through pruning, binarization, noise injection, and network randomization. By comparing models trained on tasks of different difficulty, I will show how task complexity shapes the structure and robustness of learned representations. In many cases, the signs of the weights matter more than their exact magnitudes. I will then turn to graph neural networks. HypBench uses models from network geometry to generate graphs with controlled topologies, node features, and correlations between them. This framework lets us identify the network regimes in which different GNN architectures perform best. Finally, I will examine how GNNs behave under geometric renormalization (GR). GR produces smaller graphs that preserve key geometric and structural properties of the original network. A GNN can be trained on one of these coarse-grained replicas and applied directly to the full graph without retraining. Despite the change in scale, the model retains much of its predictive performance, while its learned representations and predictive trajectories remain aligned across graph resolutions.