Neuroscience-inspired Artificial Intelligence technologies

When researchers use the term “neuroscience”, they include all fields which are involved with the analysis of the brain, the behaviors derived from its working, cognitive processes and psychology. Likewise, when researchers use the term “artificial intelligence” they mean research, trained algorithms aimed at building intelligent systems and technology. When discussing AI research, two fields are pivotal and are worth mentioning: deep learning and reinforcement learning, which both stem from neuroscientific ideas. Overall, many concepts that are typical of AI are rooted in neuroscience. The key factor is that neuroscience is now able and will be more and more able to support future AI research with the help of the global scientific community. Deep learning procedures have a long history, as the origins of advances in neural networks date back to the 1940s.

Precisely, during those years, discoveries regarding neural computation began with the establishment and construction of artificial neural networks that could solve logical functions (McCulloch and Pitts, 1943). Afterwards, supervisory feedback was implemented, in addition to environmental statistics in an unsupervised fashion (Rosenblatt, 1958 & Hebb, 1949). These operations paved the way to modern research on deep learning (Schmidhuber, 2014). This pioneering work led to the development of the backpropagation algorithm, which allowed learning to occur in networks featured with multiple layers (Rumelhart et al, 1985; Werbos, 1974). Even though the beginning of the articulation of AI models started mostly from a symbolic approach, the deriving inflexibility became an obstacle.

In response to this event researchers headed towards a different direction. In order to solve complex, real-world problems, researchers worked on parallel distributed processing (PDP), and stated that human cognition and behavior emerge from dynamic interaction within networks in order to minimize error and maximize reward. This project started off as a temporary solution to small-scale problems, but later it showed striking success in a great plethora of human behaviors, which is, surely, the goal of neuroscience studies (Hinton et al., 1986). Capitalizing on the PDP movement’s appeal to biological computation, current state-of-the-art convolutional neural networks (CNNs) incorporate several canonical hallmarks of neural computation, in conjunction with non-linear transduction, divisive normalization, and maximum-based pooling of inputs (Yamins and DiCarlo, 2016).

These computations and reckonings were directly inspired by single-cell recordings from the mammalian visual cortex which revealed how visual inputs are filtered and pooled in cells belonging to the cerebral area V1 (Hubel and Wiesel, 1959). With the passage of time, deep learning evolved out of PDP research to emerge as a key field within AI, as demonstrated by the development of deep belief networks (Hinton et al., 2006) and the creation of great datasets derived by research on human language (Deng et al., 2009). During this period, it continued to draw ideas from neuroscience. Not only this scientific field was essential in terms of deep learning, it was also instrumental in erecting a second pillar of modern AI, which is: reinforcement learning. Specifically, RL methods address the problem of how to maximize future reward by mapping states in the environment to actions and are among the most widely used tools in AI research (Sutton and Barto, 1998).

Although the scientific community has mixed opinions regarding its usage, RL methods were originally inspired by research into animal learning. Of particular value was a phenomenon called “second-order conditioning”, where affective significance is conferred on a conditioned stimulus (CS) through association with another CS rather than directly via association with the unconditioned stimulus (Sutton and Barto, 1981). On this matter, and as in the case of deep learning, investigations initially inspired by observations from neuroscience led to further developments that have importantly shaped the direction of AI research. One significant example concerns recent AI work on attention. Up until lately, most CNN models worked directly on entire images or entire video frames, with equal priority given to all image pixels at the earliest stage of processing.

However, the primate visual system works differently. Rather than processing all inputs in parallel, visual attention shifts tactically and advantageously among locations and objects, with representation operations taking turns (Koch and Ullman, 1985). Consequently, attentional mechanisms have been a source of inspiration for AI architectures that take “glimpses” of the input image at each step, update internal state representations, and then select the next location to sample and examine (Larochelle and Hinton, 2010). Furthermore, in some architectures, attentional mechanisms have been used to select information to be read out from the internal memory of the network.

This has contributed to success in machine translation (Bahdanau et al., 2014) and led to important advances on memory and reasoning tasks (Graves et al., 2016). During training, many systems learn how to generate images that mimic the structure of primary images. In other words, an increasing numerosity of generative AI models are able to create new images based on previously trained data. In addition, they’re also able to generate never-seen-before images based on a brief description voluntarily provided by the user. The successful integration of reinforcement learning with deep learning in another recent breakthrough that is worth mentioning. For example, the deep Q-network (DQN) exhibits expert play on Atari 2600 video games by learning to transform a vector of image pixels into a policy for selecting actions (e.g., joystick movements).

In this regard, one key element in DQN is “experience replay”, where the network stores a subset of the training data in an instance-based way, and then “replays” it offline, learning anew from successes or failures that have happened in the past. For sure, experience replay (like what happens in real-life situations) is critical to maximizing data efficiency, and to get better results. Behaviors that are successful are going to be reiterated & replicated, while the ones that lead to negative consequences will be gradually eliminated. Experience replay was heavily inspired by theories that seek to understand how the multiple memory systems in the mammalian brain might interact. The scientific community agrees on the existence of complementary learning systems located in the hippocampus and neocortex (Kumaran et al., 2016). Precisely, the hippocampus acts to encode novel information after a single exposure, but this information is gradually consolidated to the neocortex in sleep or resting windows of time that are interleaved with periods of activity.

This consolidation is accompanied by replay in the hippocampus and neocortex, which is observed as a reinstatement of the structures patterns of neural activities that accompanied the learning event (O’Neill et al., 2010). Recent AI research has developed architectures that implement episodic control (Blundell et al, 2016). These networks store specific experiences (for example rewards that derive from the Atari game) and select new actions based on the similarity between the current situation input and the previous events into account. As predicted from several neuroscientific discoveries (Lengyel and Dayan, 2007), artificial agents employing episodic control show striking gains in performance over deep RL networks, early on during learning (Blundell et al., 2016). Moreover, they are able to achieve success on tasks that depend heavily on one-shot, one-time learning, where typical deep RL systems fail. The theme of learning is a deeply complex one, especially when reiterating how both biological and artificial agents must have a capacity for continual learning, that is, an ability to master new tasks without forgetting how to perform previous ones (Thrun and Mitchell, 1995). While humans and non-human animals seem relatively able to learn without forgetting, neural artificial networks fall for a phenomenon called “catastrophic forgetting”, which for example happens as well in artificial language models (McClelland et al, 1995).

Given the importance of continual learning, this liability of neural networks remains a significant challenge for the development of related AI systems. This is already considered feasible, as showcased by the implementation of a form of “elastic” weight consolidation (EWC) (Kirkpatrick et al, 2017), which acts by slowing down learning in a subset of network weights identified as crucial to previous tasks, thereby anchoring these criteria to previously encountered solutions. This is deeply relevant, as it allows several tasks to be learned without an increase in network capacity, with weights that are balanced between tasks. In this way, the EWC algorithm allows deep RL networks to support continual learning on a larger scale. Encouragingly, recent AI research has developed networks that “learn to learn”, acquiring knowledge on new frontiers by leveraging prior experience with related problems, to support one-shot concept learning (Santoro et al., 2016, Vinyals et al., 2016) and accelerating learning in RL tasks (Wang et al., 2016).

Once again, this new horizon of AI discoveries builds on concepts from neuroscience: “learning to learn” was first illustrated in studies of animal learning (Harlow, 1949), and has subsequently been studied in developmental psychology (Adolph, 2005; Kemp et al., 2010, Smith, 1995). Regarding future directions, it would be astonishingly interesting to dive deeper into psychological processes such as altered states of consciousness, mania and metacognition. Certainly, there is a “vicious circle” characterizing the interaction between Neuroscience and Artificial Intelligence. The former has led to incredible contributions to the latter and when hypothesizing about the future exchanges between the two fields, the principal hope is that there will be a continuous flow of collaborations between researchers. In fact, the identification of a common language (Marblestone et al., 2016) between the two areas of expertise will generate the flourishing of new theoretical and practical insights which will hopefully prompt accelerated scientific results, crucial - for example - in the medical field.

Thank you for reading this far,

Giulia Fosci 23/09/2026

Sources: Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In ICLR 2015. https://arxiv.org/abs/1409.0473 Blundell, C., Uria, B., Li, Y., et al. (2016). Model-free episodic control. arXiv. https://arxiv.org/abs/1606.04460 Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition (pp. 248–255). IEEE. https://doi.org/10.1109/CVPR.2009.5206848 Graves, A., Wayne, G., & Danihelka, I. (2014). Neural Turing Machines. arXiv. https://arxiv.org/abs/1410.5401 Harlow, H. F. (1949). The formation of learning sets. Psychological Review, 56(1), 51–65. Hebb, D. O. (1949). The organization of behavior: A neuropsychological theory. Wiley. Hinton, G. E., McClelland, J. L., & Rumelhart, D. E. (1986). Distributed representations. In D. E. Rumelhart, J. L. McClelland, & the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 1, pp. 77–109). MIT Press. Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527 Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurones in the cat’s striate cortex. The Journal of Physiology, 148(3), 574–591. https://doi.org/10.1113/jphysiol.1959.sp006308 Kemp, C., Goodman, N. D., & Tenenbaum, J. B. (2010). Learning to learn causal models. Cognitive Science, 34(7), 1185–1243. Kirkpatrick, J., et al. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13), 3521–3526. https://doi.org/10.1073/pnas.1611835114 Koch, C., & Ullman, S. (1985). Shifts in selective visual attention: Towards the underlying neural circuitry. Human Neurobiology, 4(4), 219–227. Kumaran, D., Hassabis, D., & McClelland, J. L. (2016). What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends in Cognitive Sciences, 20(7), 512–534. https://doi.org/10.1016/j.tics.2016.05.004 Larochelle, H., & Hinton, G. E. (2010). Learning to combine foveal glimpses with a third-order Boltzmann machine. In Advances in Neural Information Processing Systems (Vol. 23). Lengyel, M., & Dayan, P. (2007). Hippocampal contributions to control: The third way. In Advances in Neural Information Processing Systems (Vol. 20). Marblestone, A. H., Wayne, G., & Kording, K. P. (2016). Toward an integration of deep learning and neuroscience. Frontiers in Computational Neuroscience, 10, Article 94. https://doi.org/10.3389/fncom.2016.00094 McClelland, J. L., McNaughton, B. L., & O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex. Psychological Review, 102(3), 419–457. https://doi.org/10.1037/0033-295X.102.3.419 McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5, 115–133. https://doi.org/10.1007/BF02478259 O’Neill, J., Pleydell-Bouverie, B., Dupret, D., & Csicsvari, J. (2010). Play it again: Reactivation of waking experience and memory. Trends in Neurosciences, 33(5), 220–229. https://doi.org/10.1016/j.tins.2010.01.006 Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386–408. https://doi.org/10.1037/h0042519 Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536. https://doi.org/10.1038/323533a0 Santoro, A., Bartunov, S., Botvinick, M., et al. (2016). Meta-learning with memory-augmented neural networks. In Proceedings of the 33rd International Conference on Machine Learning (pp. 1842–1850). Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85–117. https://doi.org/10.1016/j.neunet.2014.09.003 Sutton, R. S., & Barto, A. G. (1981). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88(2), 135–170. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. MIT Press. Thrun, S., & Mitchell, T. M. (1995). Lifelong robot learning. Robotics and Autonomous Systems, 15(1–2), 25–46. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., & colleagues. (2016). Matching networks for one shot learning. In Advances in Neural Information Processing Systems (Vol. 29). Wang, J. X., et al. (2016). Learning to reinforcement learning. arXiv. https://arxiv.org/abs/1611.05763 Werbos, P. J. (1974). Beyond regression: New tools for prediction and analysis in the behavioral sciences (Doctoral dissertation, Harvard University). Yamins, D. L. K., & DiCarlo, J. J. (2016). Using goal-driven deep learning models to understand sensory cortex. Nature Neuroscience, 19, 356–365. https://doi.org/10.1038/nn.4244