Publications

Active inference is a first principle account of how autonomous agents operate in dynamic, non-stationary environments. This problem is also considered in reinforcement learning (RL), but limited work exists on comparing the two approaches on the same discrete-state environments. In this paper, we provide: 1) an accessible overview of the discrete-state formulation of active inference, highlighting natural behaviors in active inference that are generally engineered in RL; 2) an explicit discrete-state comparison between active inference and RL on an OpenAI gym baseline. We begin by providing a condensed overview of the active inference literature, in particular viewing the various natural behaviors of active inference agents through the lens of RL. We show that by operating in a pure belief-based setting, active inference agents can carry out epistemic exploration, and account for uncertainty about their environment in a Bayes-optimal fashion. Furthermore, we show that the reliance on an explicit reward signal in RL is removed in active inference, where reward can simply be treated as another observation; even in the total absence of rewards, agent behaviors are learned through preference learning. We make these properties explicit by showing two scenarios in which active inference agents can infer behaviors in reward-free environments compared to both Q-learning and Bayesian model-based RL agents; by placing zero prior preferences over rewards and by learning the prior preferences over the observations corresponding to reward. We conclude by noting that this formalism can be applied to more complex settings if appropriate generative models can be formulated. In short, we aim to demystify the behavior of active inference agents by presenting an accessible discrete state-space and time formulation, and demonstrate these behaviors in a OpenAI gym environment, alongside RL agents.

It is posited that cognitive and affective dysfunction in patients with major depression disorder (MDD) may be caused by dysfunctional signal propagation in the brain. By leveraging dynamic causal modeling, we investigated whole-brain directed signal propagation (effective connectivity) among distributed large-scale brain networks in patients with MDD. The results revealed the existence of two mutual inhibitory systems: the anterior default mode network, auditory network, sensorimotor network, salience network and visual networks formed an “emotional” brain, while the posterior default mode network, central executive networks, cerebellum and dorsal attention network formed a “rational brain”. These two networks exhibited excitatory intra-system connectivity and inhibitory inter-system connectivity. Patients were characterized by potentiated intra-system connections within the “emotional brain”, as well as over-inhibition of the “rational brain” by the “emotional brain”. The hierarchical architecture of the whole-brain effective connectivity networks was then analyzed using a PageRank algorithm which revealed a shift of the controlling role of the “rational brain” to the “emotional brain” in the patients. Finally, baseline effective connectivity patterns were predictive of antidepressant treatment outcomes. These findings inform basic organization of distributed large-scale brain networks and furnish a better characterization of the neural mechanisms of depression, which may facilitate effective treatment.

Abstract Using neuroimaging and electrophysiological data to infer neural parameter estimations from theoretical circuits requires solving the inverse problem. Here, we provide a new Julia language package designed to i) compose complex dynamical models in a simple and modular way with ModelingToolkit.jl, ii) implement parameter fitting based on spectral dynamic causal modeling (sDCM) using the Laplace approximation, analogous to MATLAB implementation in SPM, and iii) leverage Julia’s unique strengths to increase accuracy and speed by employing Automatic Differentiation during the fitting procedure. To illustrate the utility of our flexible modular approach, we provide a method to improve correction for fMRI scanner field strengths (1.5T, 3T, 7T) when fitting models to real data.

Identifying a coupled dynamical system out of many plausible candidates, each of which could serve as the underlying generator of some observed measurements, is a profoundly ill posed problem that commonly arises when modelling real world phenomena. In this review, we detail a set of statistical procedures for inferring the structure of nonlinear coupled dynamical systems (structure learning), which has proved useful in neuroscience research. A key focus here is the comparison of competing models of (ie, hypotheses about) network architectures and implicit coupling functions in terms of their Bayesian model evidence. These methods are collectively referred to as dynamical casual modelling (DCM). We focus on a relatively new approach that is proving remarkably useful; namely, Bayesian model reduction (BMR), which enables rapid evaluation and comparison of models that differ in their network architecture. We illustrate the usefulness of these techniques through modelling neurovascular coupling (cellular pathways linking neuronal and vascular systems), whose function is an active focus of research in neurobiology and the imaging of coupled neuronal systems.

AUTISM is a biological disorder which affects social cognition, and understanding brain abnormalities of the former will elucidate the brain basis of the latter. We report structural MRI data on 15 high-functioning individuals with autistic disorder. A voxel-based whole brain analysis identified grey matter differences in an amygdala centered system relative to 15 age- and IQ-matched controls. Decreases of grey matter were found in anterior parts of this system (right paracingulate sulcus, left inferior frontal gyrus). Increases were found in posterior parts (amygdala/peri-amygdaloid cortex, middle temporal gyrus, inferior temporal gyrus), and in regions of the cerebellum. These structures are implicated in social cognition by animal, imaging and histopathological studies. This study therefore provides converging evidence of the physiological basis of social cognition.

Epileptic seizure activity shows complicated dynamics in both space and time. To understand the evolution and propagation of seizures spatially extended sets of data need to be analysed. We have previously described an efficient filtering scheme using variational Laplace that can be used in the Dynamic Causal Modelling (DCM) framework [Friston, 2003] to estimate the temporal dynamics of seizures recorded using either invasive or non-invasive electrical recordings (EEG/ECoG). Spatiotemporal dynamics are modelled using a partial differential equation -- in contrast to the ordinary differential equation used in our previous work on temporal estimation of seizure dynamics [Cooray, 2016]. We provide the requisite theoretical background for the method and test the ensuing scheme on simulated seizure activity data and empirical invasive ECoG data. The method provides a framework to assimilate the spatial and temporal dynamics of seizure activity, an aspect of great physiological and clinical importance.

Active inference is an account of cognition and behavior in complex systems which brings together action, perception, and learning under the theoretical mantle of Bayesian inference. Active inference has seen growing applications in academic research, especially in fields that seek to model human or animal behavior. While in recent years, some of the code arising from the active inference literature has been written in open source languages like Python and Julia, to-date, the most popular software for simulating active inference agents is the DEM toolbox of SPM, a MATLAB library originally developed for the statistical analysis and modelling of neuroimaging data. Increasing interest in active inference, manifested both in terms of sheer number as well as diversifying applications across scientific disciplines, has thus created a need for generic, widely-available, and user-friendly code for simulating active inference in open-source scientific computing languages like Python. The Python package we present here, pymdp (see https://github.com/infer-actively/pymdp), represents a significant step in this direction: namely, we provide the first open-source package for simulating active inference with partially-observable Markov Decision Processes or POMDPs. We review the package's structure and explain its advantages like modular design and customizability, while providing in-text code blocks along the way to demonstrate how it can be used to build and run active inference processes with ease. We developed pymdp to increase the accessibility and exposure of the active inference framework to researchers, engineers, and developers with diverse disciplinary backgrounds. In the spirit of open-source software, we also hope that it spurs new innovation, development, and collaboration in the growing active inference community.

Abstract Two important theories in cognitive neuroscience are predictive coding and the global workspace theory. A key research task is to understand how these two theories relate to one another, and particularly, how the brain transitions from a predictive early state to the eventual engagement of a brain-scale state (the global workspace). To address this question, we present a source-localisation of EEG responses evoked by the local-global task – an experimental paradigm that engages a predictive hierarchy, which encompasses the global workspace. The results of our source reconstruction suggest three-phases of processing. The first phase involves the sensory (here auditory) regions of the superior temporal lobe and predicts sensory regularities over a short timeframe (as per the local effect). The third phase is brain-scale, involving inferior frontal, as well as inferior and superior parietal regions; consistent with a global neuronal workspace (as per the global effect). Crucially, our analysis suggests that there is an intermediate (second) phase, involving modulatory interactions between inferior frontal and superior temporal regions. Furthermore, sedation with propofol reduces modulatory interactions in the second phase. This selective effect is consistent with a predictive coding explanation of sedation, with propofol acting on descending predictions of the precision of prediction errors; thereby constraining access to the global neuronal workspace.

Abstract Learning new motor skills relies on neural plasticity within motor and limbic systems. This study uniquely combined diffusion tensor imaging and multiparametric mapping MRI to detail these neuroplasticity processes. We recruited 18 healthy male participants who underwent 960 min of training on a computer-based motion game, while 14 were scanned without training. Diffusion tensor imaging, which quantifies tissue microstructure by measuring the capacity for, and directionality of, water diffusion, revealed mostly linear changes in white matter across the corticospinal-cerebellar-thalamo-hippocampal circuit. These changes related to performance and reflected different responses to upper- and lower-limb training in brain areas with known somatotopic representations. Conversely, quantitative MRI metrics, sensitive to myelination and iron content, demonstrated mostly quadratic changes in gray matter related to performance and reflecting somatotopic representations within the same brain areas. Furthermore, while myelin and iron-sensitive multiparametric mapping MRI was able to describe time lags between different cortical brain systems, diffusion tensor imaging detected time lags within the white matter of the motor systems. These findings suggest that motor skill learning involves distinct phases of white and gray matter plasticity across the sensorimotor network, with the unique combination of diffusion tensor imaging and multiparametric mapping MRI providing complementary insights into the underlying neuroplastic responses.

People track facial expression dynamics with ease to accurately perceive distinct emotions. Although the superior temporal sulcus (STS) appears to possess mechanisms for perceiving changeable facial attributes such as expressions, the nature of the underlying neural computations is not known. Motivated by novel theoretical accounts, we hypothesized that visual and motor areas represent expressions as anticipated motion trajectories. Using magnetoencephalography, we show predictable transitions between fearful and neutral expressions (compared with scrambled and static presentations) heighten activity in visual cortex as quickly as 165 ms poststimulus onset and later (237 ms) engage fusiform gyrus, STS and premotor areas. Consistent with proposed models of biological motion representation, we suggest that visual areas predictively represent coherent facial trajectories. We show that such representations bias emotion perception of subsequent static faces, suggesting that facial movements elicit predictions that bias perception. Our findings reveal critical processes evoked in the perception of dynamic stimuli such as facial expressions, which can endow perception with temporal continuity.