Publications

Exploration and preference satisfaction trade-off in reward-free learning

Biological agents have meaningful interactions with their environment despite the absence of immediate reward signals. In such instances, the agent can learn preferred modes of behaviour that lead to predictable states -- necessary for survival. In this paper, we pursue the notion that this learnt behaviour can be a consequence of reward-free preference learning that ensures an appropriate trade-off between exploration and preference satisfaction. For this, we introduce a model-based Bayesian agent equipped with a preference learning mechanism (pepper) using conjugate priors. These conjugate priors are used to augment the expected free energy planner for learning preferences over states (or outcomes) across time. Importantly, our approach enables the agent to learn preferences that encourage adaptive behaviour at test time. We illustrate this in the OpenAI Gym FrozenLake and the 3D mini-world environments -- with and without volatility. Given a constant environment, these agents learn confident (i.e., precise) preferences and act to satisfy them. Conversely, in a volatile setting, perpetual preference uncertainty maintains exploratory behaviour. Our experiments suggest that learnable (reward-free) preferences entail a trade-off between exploration and preference satisfaction. Pepper offers a straightforward framework suitable for designing adaptive agents when reward functions cannot be predefined as in real environments.

Extracting novel information from neuroimaging data using neural fields

We showcase three case studies that illustrate how neural fields can be useful in the analysis of neuroimaging data. In particular, we argue that neural fields allow one to: (i) compare evidences for alternative hypotheses regarding neurobiological determinants of stimulus-specific response variability; (ii) make inferences about between subject variability in cortical function and microstructure using non-invasive data and (iii) estimate spatial parameters describing cortical sources, even without spatially resolved data.

Bayesian Mechanics for Stationary Processes.

This paper develops a Bayesian mechanics for adaptive systems. Firstly, we model the interface between a system and its environment with a Markov blanket. This affords conditions under which states internal to the blanket encode information about external states. Second, we introduce dynamics and represent adaptive systems as Markov blankets at steady-state. This allows us to identify a wide class of systems whose internal states appear to infer external states, consistent with variational inference in Bayesian statistics and theoretical neuroscience. Finally, we partition the blanket into sensory and active states. It follows that active states can be seen as performing active inference and well-known forms of stochastic control (such as PID control), which are prominent formulations of adaptive behaviour in theoretical biology and engineering.

Pathfinding: a neurodynamical account of intuition

The brain network for emotional body language reading: Combined structural and effective connectivity

Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

In this chapter, we identify fundamental geometric structures that underlie the problems of sampling, optimisation, inference and adaptive decision-making. Based on this identification, we derive algorithms that exploit these geometric structures to solve these problems efficiently. We show that a wide range of geometric theories emerge naturally in these fields, ranging from measure-preserving processes, information divergences, Poisson geometry, and geometric integration. Specifically, we explain how \emph{(i)} leveraging the symplectic geometry of Hamiltonian systems enable us to construct (accelerated) sampling and optimisation methods, \emph{(ii)} the theory of Hilbertian subspaces and Stein operators provides a general methodology to obtain robust estimators, \emph{(iii)} preserving the information geometry of decision-making yields adaptive agents that perform active inference. Throughout, we emphasise the rich connections between these fields; e.g., inference draws on sampling and optimisation, and adaptive decision-making assesses decisions by inferring their counterfactual consequences. Our exposition provides a conceptual overview of underlying ideas, rather than a technical discussion, which can be found in the references herein.

PsPM-SCRV4: Skin conductance responses in fear conditioning with visual CS and auditory US

This dataset includes skin conductance response (SCR) measurements, CS and US information, keypress responses and keypress response times for each of 32 healthy unmedicated participants (16 males and 16 females aged 22.4+/-4.5 years) participating in a classical (Pavlovian) discriminant delay fear conditioning task. CS is a visual stimulus with variation in position on screen and color. Us is a 1s long white noise burst at 95dB presented over headphones. SOA between the CS and US is varied between participants to be 4, 10, or 16 s. The ITI is selected randomly on each trial from 14, 19, or 23 s

Loss of sensory attenuation in patients with functional (psychogenic) movement disorders

Functional movement disorders require attention to manifest yet patients report the abnormal movement to be out of their control. In this study we explore the phenomenon of sensory attenuation, a measure of the sense of agency for movement, in this group of patients by using a force matching task. Fourteen patients and 14 healthy control subjects were presented with forces varying from 1 to 3 N on the index finger of their left hand. Participants were required to match these forces; either by pressing directly on their own finger or by operating a robot that pressed on their finger. As expected, we found that healthy control subjects consistently overestimated the force required when pressing directly on their own finger than when operating a robot. However, patients did not, indicating a significant loss of sensory attenuation in this group of patients. These data are important because they demonstrate that a fundamental component of normal voluntary movement is impaired in patients with functional movement disorders. The loss of sensory attenuation has been correlated with the loss of sense of agency, and may help to explain why patients report that they do not experience the abnormal movement as voluntary.

Neurovascular coupling: insights from multi-modal dynamic causal modelling of fMRI and MEG

This technical note presents a framework for investigating the underlying mechanisms of neurovascular coupling in the human brain using multi-modal magnetoencephalography (MEG) and functional magnetic resonance (fMRI) neuroimaging data. This amounts to estimating the evidence for several biologically informed models of neurovascular coupling using variational Bayesian methods and selecting the most plausible explanation using Bayesian model comparison. First, fMRI data is used to localise active neuronal sources. The coordinates of neuronal sources are then used as priors in the specification of a DCM for MEG, in order to estimate the underlying generators of the electrophysiological responses. The ensuing estimates of neuronal parameters are used to generate neuronal drive functions, which model the pre or post synaptic responses to each experimental condition in the fMRI paradigm. These functions form the input to a model of neurovascular coupling, the parameters of which are estimated from the fMRI data. This establishes a Bayesian fusion technique that characterises the BOLD response - asking, for example, whether instantaneous or delayed pre or post synaptic signals mediate haemodynamic responses. Bayesian model comparison is used to identify the most plausible hypotheses about the causes of the multimodal data. We illustrate this procedure by comparing a set of models of a single-subject auditory fMRI and MEG dataset. Our exemplar analysis suggests that the origin of the BOLD signal is mediated instantaneously by intrinsic neuronal dynamics and that neurovascular coupling mechanisms are region-specific. The code and example dataset associated with this technical note are available through the statistical parametric mapping (SPM) software package.

Trust as Extended Control: Active Inference and User Feedback During Human-Robot Collaboration

To interact seamlessly with robots, users must infer the causes of a robot's behavior and be confident about that inference. Hence, trust is a necessary condition for human-robot collaboration (HRC). Despite its crucial role, it is largely unknown how trust emerges, develops, and supports human interactions with nonhuman artefacts. Here, we review the literature on trust, human-robot interaction, human-robot collaboration, and human interaction at large. Early models of trust suggest that trust entails a trade-off between benevolence and competence, while studies of human-to-human interaction emphasize the role of shared behavior and mutual knowledge in the gradual building of trust. We then introduce a model of trust as an agent's best explanation for reliable sensory exchange with an extended motor plant or partner. This model is based on the cognitive neuroscience of active inference and suggests that, in the context of HRC, trust can be cast in terms of virtual control over an artificial agent. In this setting, interactive feedback becomes a necessary component of the trustor's perception-action cycle. The resulting model has important implications for understanding human-robot interaction and collaboration, as it allows the traditional determinants of human trust to be defined in terms of active inference, information exchange and empowerment. Furthermore, this model suggests that boredom and surprise may be used as markers for under and over-reliance on the system. Finally, we examine the role of shared behavior in the genesis of trust, especially in the context of dyadic collaboration, suggesting important consequences for the acceptability and design of human-robot collaborative systems.

A Response to Our Theatre Critics

Cortical beta oscillations reflect the contextual gating of visual action feedback

In sensorimotor integration, the brain needs to decide how its predictions should accommodate novel evidence by 'gating' sensory data depending on the current context. Here, we examined the oscillatory correlates of this process by recording magnetoencephalography (MEG) data during a new task requiring action under intersensory conflict. We used virtual reality to decouple visual (virtual) and proprioceptive (real) hand postures during a task in which the phase of grasping movements tracked a target (in either modality). Thus, we rendered visual information either task-relevant or a (to-be-ignored) distractor. Under visuo-proprioceptive incongruence, occipital beta power decreased (relative to congruence) when vision was task-relevant but increased when it had to be ignored. Dynamic causal modeling (DCM) revealed that this interaction was best explained by diametrical, task-dependent changes in visual gain. These results suggest a crucial role for beta oscillations in the contextual gating (i.e., gain or precision control) of visual vs proprioceptive action feedback, depending on current behavioral demands.

The Variational Principles of Cognition

Precision and False Perceptual Inference

Accurate perceptual inference fundamentally depends upon accurate beliefs about the reliability of sensory data. In this paper, we describe a Bayes optimal and biologically plausible scheme that refines these beliefs through a gradient descent on variational free energy. To illustrate this, we simulate belief updating during visual foraging and show that changes in estimated sensory precision (i.e. confidence in visual data) are highly sensitive to prior beliefs about the contents of a visual scene. In brief, confident prior beliefs induce an increase in estimated precision when consistent with sensory evidence, but a decrease when they conflict. Prior beliefs held with low confidence are rapidly updated to posterior beliefs, determined by sensory data. These induce much smaller changes in beliefs about sensory precision. We argue that pathologies of scene construction may be due to abnormal priors, and show that these can induce a reduction in estimated sensory precision. Having previously associated this precision with cholinergic signalling, we note that several neurodegenerative conditions are associated with visual disturbances and cholinergic deficits; notably, the synucleinopathies. On relating the message passing in our model to the functional anatomy of the ventral visual stream, we find that simulated neuronal loss in temporal lobe regions induces confident, inaccurate, empirical prior beliefs at lower levels in the visual hierarchy. This provides a plausible, if speculative, computational mechanism for the loss of cholinergic signalling and the visual disturbances associated with temporal lobe Lewy body pathology. This may be seen as an illustration of the sorts of hypotheses that may be expressed within this computational framework.

The Free Energy Principle drives neuromorphic development

We show how any system with morphological degrees of freedom and locally limited free energy will, under the constraints of the free energy principle, evolve toward a neuromorphic morphology that supports hierarchical computations in which each level of the hierarchy enacts a coarse-graining of its inputs, and dually a fine-graining of its outputs. Such hierarchies occur throughout biology, from the architectures of intracellular signal transduction pathways to the large-scale organization of perception and action cycles in the mammalian brain. Formally, the close formal connections between cone-cocone diagrams (CCCD) as models of quantum reference frames on the one hand, and between CCCDs and topological quantum field theories on the other, allow the representation of such computations in the fully-general quantum-computational framework of topological quantum neural networks.

Control Flow in Active Inference Systems—Part I: Classical and Quantum Formulations of Active Inference

Living systems face both environmental complexity and limited access to free-energy resources. Survival under these conditions requires a control system that can activate, or deploy, available perception and action resources in a context specific way. In this Part I, we introduce the free-energy principle (FEP) and the idea of active inference as Bayesian prediction-error minimization, and show how the control problem arises in active inference systems. We then review classical and quantum formulations of the FEP, with the former being the classical limit of the latter. In the accompanying Part II, we show that when systems are described as executing active inference driven by the FEP, their control flow systems can always be represented as tensor networks (TNs). We show how TNs as control systems can be implemented within the general framework of quantum topological neural networks, and discuss the implications of these results for modeling biological systems at multiple scales.

Speed-Dependent Responses in V5: A Replication Study

Design and analysis of fMRI studies with neurologically impaired patients

Abstract Functional neuroimaging can be used to characterize two types of abnormality in patients with neurological deficits: abnormal functional segregation and abnormal functional integration. In this paper we consider the factors that influence the experimental design, analysis, and interpretation of such studies. With respect to experimental design, we emphasize that: 1) task selection is constrained to tasks the patient is able to perform correctly, and 2) the most sensitive designs entail presenting stimuli of the same type close together. In terms of data preprocessing, prior to statistical analysis, we note that structural pathology may call for constraints on nonlinear transformations, used by spatial normalization, to prevent distortion of intact tissue. This means that one may have to increase spatial smoothing to reduce the impact of inaccurate normalization. Important issues in statistical modeling concern the first level of analysis (estimation of activation within subject), which has to distinguish correct from incorrect responses. At the second level (between subjects), inference should be based on between‐subjects variance. Provided that these and other constraints are met, deficits in functional segregation are indicated when activation in one or a set of regions is higher or lower in patients relative to control subjects. In contrast, deficits in functional integration are implied when the influence of one brain region on another is stronger or weaker in patients relative to control subjects. J. Magn. Reson. Imaging 2006. © 2006 Wiley‐Liss, Inc.