Mathematical Theory Reveals Hidden Structure in Symmetry-Based Neural Networks