The authors developed a computational model of a recurrent network, termed the harmonic oscillator recurrent network (HORN), which enables the control of oscillatory dynamics. When functioning in an oscillatory state, HORNs surpass non-oscillatory recurrent networks in learning speed, resistance to noise, and efficiency in parameter use. Additionally, they closely mirror the behavior of biological neural systems, implying that natural neural networks may leverage oscillatory dynamics for computation. The wave-based interference patterns produced by these networks facilitate a holistic and highly parallel representation of both spatial and temporal relationships among stimulus features. A thorough analysis of HORNs' dynamics revealed a powerful computational mechanism rooted in the distinct properties of coupled oscillator networks. This mechanism utilizes wave interference and superposition to represent and process stimuli. Remarkably, without requiring precise parameter adjustments for different experiments, HORNs exhibited dynamic characteristics similar to those observed in the cerebral cortex, suggesting that natural neural networks may also exploit this computational strategy. To explore this further, they integrated additional biologically inspired features into HORNs and discovered that these enhancements typically improved task performance without increasing the number of trainable parameters. Within HORNs, individual nodes convert any input into oscillations, allowing them to extract features through resonance and regulate gain based on frequency. As a whole, the network generates transient stimulus representations characterized by wave interference patterns. Performance tests on conventional pattern recognition benchmarks demonstrated that gradient-based learning can take advantage of this expanded dynamic range, leading to significantly improved outcomes compared to RNNs without oscillatory nodes. Enforcing oscillatory activity in network nodes acts as an inductive bias, enhancing the model’s expressive capacity. This oscillatory bias has also been shown to boost performance in spiking neural networks by allowing subthreshold membrane potentials to oscillate. These findings point to a universal computational principle based on coupled oscillators, enabling wave-based representations applicable to both large neuronal populations and individual neurons. They trained on geometrically structured stimuli and HORNs developed localized connectivity patterns. The strong performance of HORNs can be attributed to several factors, primarily the network nodes’ natural tendency to engage in oscillations. Their simulations indicate that complex, transient, and stimulus-specific synchronization patterns that emerge during learning enhance information processing, highlighting the critical role of oscillatory properties in network nodes. This supports the idea that oscillations and synchrony, widely observed in neuronal systems, are not merely incidental but functionally significant. Simulations involving geometrically structured visual inputs produced results similar to those obtained with time-series data that lack explicit geometric organization. This suggests that the identified computational principles are effective in processing both spatial and temporal relationships among input signals using a unified representational format. During learning, nodes activated by semantically related features increase their mutual coupling, and during recall, they self-organize into synchronized stimulus-specific assemblies with enhanced joint responses. This behavior mirrors the activity observed in the visual cortex, where neurons tuned to related features exhibit similar dynamic associations, reinforcing the binding by synchrony (BBS) hypothesis. HORNs, by leveraging the resonance properties of coupled oscillators, replicate this essential feature of cortical networks. Moreover, their spontaneous activity resembles that of natural cortical networks, where stimulus presentation reduces variance and aligns network dynamics with stimulus-specific substates. These substates exist within the broader space of spontaneous activity and emerge through comparisons of sensory inputs with stored priors. Consequently, spontaneous activity can be interpreted as a composite of learned stimulus-specific representations. |
Last modified on 07-Feb-25 |