Publications

The anatomy of choice: dopamine and decision-making

This paper considers goal-directed decision-making in terms of embodied or active inference. We associate bounded rationality with approximate Bayesian inference that optimizes a free energy bound on model evidence. Several constructs such as expected utility, exploration or novelty bonuses, softmax choice rules and optimism bias emerge as natural consequences of free energy minimization. Previous accounts of active inference have focused on predictive coding. In this paper, we consider variational Bayes as a scheme that the brain might use for approximate Bayesian inference. This scheme provides formal constraints on the computational anatomy of inference and action, which appear to be remarkably consistent with neuroanatomy. Active inference contextualizes optimal decision theory within embodied inference, where goals become prior beliefs. For example, expected utility theory emerges as a special case of free energy minimization, where the sensitivity or inverse temperature (associated with softmax functions and quantal response equilibria) has a unique and Bayes-optimal solution. Crucially, this sensitivity corresponds to the precision of beliefs about behaviour. The changes in precision during variational updates are remarkably reminiscent of empirical dopaminergic responses-and they may provide a new perspective on the role of dopamine in assimilating reward prediction errors to optimize decision-making.

What is value—accumulated reward or evidence?

Why are you reading this abstract? In some sense, your answer will cast the exercise as valuable-but what is value? In what follows, we suggest that value is evidence or, more exactly, log Bayesian evidence. This implies that a sufficient explanation for valuable behavior is the accumulation of evidence for internal models of our world. This contrasts with normative models of optimal control and reinforcement learning, which assume the existence of a value function that explains behavior, where (somewhat tautologically) behavior maximizes value. In this paper, we consider an alternative formulation-active inference-that replaces policies in normative models with prior beliefs about the (future) states agents should occupy. This enables optimal behavior to be cast purely in terms of inference: where agents sample their sensorium to maximize the evidence for their generative model of hidden states in the world, and minimize their uncertainty about those states. Crucially, this formulation resolves the tautology inherent in normative models and allows one to consider how prior beliefs are themselves optimized in a hierarchical setting. We illustrate these points by showing that any optimal policy can be specified with prior beliefs in the context of Bayesian inference. We then show how these prior beliefs are themselves prescribed by an imperative to minimize uncertainty. This formulation explains the saccadic eye movements required to read this text and defines the value of the visual sensations you are soliciting.

PsPM-trSP1: SCR, and heart beat measurement in response to aversive/neutral IAPS pictures while subjected to auditory distractors

This dataset includes skin conductance response (SCR) and pulse time stamp (HB) measurements for each of 60 healthy unmedicated participants (30 males and 30 females aged 23.7 +/- 4.8 years) in response to the 45 most arousing negative, and 45 least arousing neutral IAPS pictures, each presented for 1 s each, while listening to regular or random distractor sounds, as described in Bach et al. (2015). ITI was selected randomly on each trial from 7.65 s, 9 s, or 10.35 s. The experiment was preceded by a 2-minute resting period and divided into 3 blocks, separated by resting periods. Each resting period begins and ends with an event marker in the psychophysiological recordings.

Gradient-free MCMC methods for dynamic causal modelling

In this technical note we compare the performance of four gradient-free MCMC samplers (random walk Metropolis sampling, slice-sampling, adaptive MCMC sampling and population-based MCMC sampling with tempering) in terms of the number of independent samples they can produce per unit computational time. For the Bayesian inversion of a single-node neural mass model, both adaptive and population-based samplers are more efficient compared with random walk Metropolis sampler or slice-sampling; yet adaptive MCMC sampling is more promising in terms of compute time. Slice-sampling yields the highest number of independent samples from the target density - albeit at almost 1000% increase in computational time, in comparison to the most efficient algorithm (i.e., the adaptive MCMC sampler).

The fantastic organ

The Age of Insight by Eric Kandel is a brutal and beautiful book. Its pages burgeon with beautiful images and beautiful ideas. The ideas are pursued with relentless honesty and diligence for more than 600 pages. There are parts of this book that should only be read in private. I know this after a slightly fraught flight to London, trying to read the book while covering its pictures of masturbating women with my boarding card—I did not want to offend the sensibilities of an Indian gentleman, who showed a polite interest in my reading material. I am not quite sure why I did this, perhaps because I am English; or perhaps because I found the pictures curiously arousing—despite their innocent titles {e.g. ‘Seated Woman in Armchair’ by Gustav Klimt [(1862–1918): c.1913], pencil and white chalk}. Happily, the book explained why I found them so alluring. THE AGE OF INSIGHT: THE QUEST TO UNDERSTAND THE UNCONSCIOUS IN ART, MIND, AND BRAIN, FROM VIENNA 1900 TO THE PRESENT By Eric R. Kandel 2012 New York: Random House ISBN: 978-1-4000-6871-5 Price: $40.00 Kandel’s treatment is a book of two parts. The first deals with innovative thinking and insights in science and art in turn of the century Vienna. The second overviews the neurobiology of perception and emotion, bringing the reader back to deep questions about perspective taking and neuroaesthetics. The premise that underpins both parts is that the brain is an inference machine, generating hypotheses and fantasies that are tested against sensory data. Put simply, the brain is—literally—a fantastic organ (fantastic: from Greek phantastikos , able to create mental images, from phantazesthai ). For me, the story starts with Hermann von Helmholtz (1821–94) and the notion of unconscious inference (Helmholtz, 1866/1962). Kandel places this story in the context of history and art, in an …

Abnormal Effective Connectivity in the Brain is Involved in Auditory Verbal Hallucinations in Schizophrenia

DCM, Conductance Based Models and Clinical Applications

Variational Bayes under the Laplace approximation

Imaging neuroscience: Principles or maps?

This article reviews some recent trends in imaging neuroscience. A distinction is made between making maps of functional responses in the brain and discerning the rules or principles that underlie their organization. After considering developments in the characterization of brain imaging data, several examples are presented that highlight the context-sensitive nature of neuronal responses that we measure. These contexts can be endogenous and physiological, reflecting the fact that each cortical area, or neuronal population, expresses its dynamics in the context of interactions with other areas. Conversely, these contexts can be experimental or psychological and can have a profound effect on the regional effects elicited. In this review we consider experimental designs and analytic strategies that go beyond cognitive subtraction and speculate on how functional imaging can be used to address both the details and principles underlying functional integration and specialization in the brain.

The Relationship Between Synchronization Among Neuronal Populations and Their Mean Activity Levels

In the past decade the importance of synchronized dynamics in the brain has emerged from both empirical and theoretical perspectives. Fast dynamic synchronous interactions of an oscillatory or nonoscillatory nature may constitute a form of temporal coding that underlies feature binding and perceptual synthesis. The relationship between synchronization among neuronal populations and the population firing rates addresses two important issues: the distinction between rate coding and synchronization coding models of neuronal interactions and the degree to which empirical measurements of population activity, such as those employed by neuroimaging, are sensitive to changes in synchronization. We examined the relationship between mean population activity and synchronization using biologically plausible simulations. In this article, we focus on continuous stationary dynamics. (In a companion article, Chawla (forth-coming), we address the same issue using stimulus-evoked transients.) By manipulating parameters such as extrinsic input, intrinsic noise, synaptic efficacy, density of extrinsic connections, the voltage-sensitive nature of postsynaptic mechanisms, the number of neurons, and the laminar structure within the populations, we were able to introduce variations in both mean activity and synchronization under a variety of simulated neuronal architectures. Analyses of the simulated spike trains and local field potentials showed that in nearly every domain of the model's parameter space, mean activity and synchronization were tightly coupled. This coupling appears to be mediated by an increase in synchronous gain when effective membrane time constants are lowered by increased activity. These observations show that under the assumptions implicit in our models, rate coding and synchrony coding in neural systems with reciprocal interconnections are two perspectives on the same underlying dynamic. This suggests that in the absence of specific mechanisms decoupling changes in synchronization from firing levels, indexes of brain activity that are based purely on synaptic activity (e.g., functional magnetic resonance imaging) may also be sensitive to changes in synchronous coupling.

Active inference and epistemic value

We offer a formal treatment of choice behavior based on the premise that agents minimize the expected free energy of future outcomes. Crucially, the negative free energy or quality of a policy can be decomposed into extrinsic and epistemic (or intrinsic) value. Minimizing expected free energy is therefore equivalent to maximizing extrinsic value or expected utility (defined in terms of prior preferences or goals), while maximizing information gain or intrinsic value (or reducing uncertainty about the causes of valuable outcomes). The resulting scheme resolves the exploration-exploitation dilemma: Epistemic value is maximized until there is no further information gain, after which exploitation is assured through maximization of extrinsic value. This is formally consistent with the Infomax principle, generalizing formulations of active vision based upon salience (Bayesian surprise) and optimal decisions based on expected utility and risk-sensitive (Kullback-Leibler) control. Furthermore, as with previous active inference formulations of discrete (Markovian) problems, ad hoc softmax parameters become the expected (Bayes-optimal) precision of beliefs about, or confidence in, policies. This article focuses on the basic theory, illustrating the ideas with simulations. A key aspect of these simulations is the similarity between precision updates and dopaminergic discharges observed in conditioning paradigms.

Psychosis and the Experience of Self: Brain Systems Underlying Self‐Monitoring

EEG and MEG Data Analysis in SPM8

SPM is a free and open source software written in MATLAB (The MathWorks, Inc.). In addition to standard M/EEG preprocessing, we presently offer three main analysis tools: (i) statistical analysis of scalp-maps, time-frequency images, and volumetric 3D source reconstruction images based on the general linear model, with correction for multiple comparisons using random field theory; (ii) Bayesian M/EEG source reconstruction, including support for group studies, simultaneous EEG and MEG, and fMRI priors; (iii) dynamic causal modelling (DCM), an approach combining neural modelling with data analysis for which there are several variants dealing with evoked responses, steady state responses (power spectra and cross-spectra), induced responses, and phase coupling. SPM8 is integrated with the FieldTrip toolbox , making it possible for users to combine a variety of standard analysis methods with new schemes implemented in SPM and build custom analysis tools using powerful graphical user interface (GUI) and batching tools.

Thalamocortical dynamics underlying spontaneous transitions in beta power in Parkinsonism

Parkinson's disease (PD) is a neurodegenerative condition in which aberrant oscillatory synchronization of neuronal activity at beta frequencies (15-35 Hz) across the cortico-basal ganglia-thalamocortical circuit is associated with debilitating motor symptoms, such as bradykinesia and rigidity. Mounting evidence suggests that the magnitude of beta synchrony in the parkinsonian state fluctuates over time, but the mechanisms by which thalamocortical circuitry regulates the dynamic properties of cortical beta in PD are poorly understood. Using the recently developed generic Dynamic Causal Modelling (DCM) framework, we recursively optimized a set of plausible models of the thalamocortical circuit (n = 144) to infer the neural mechanisms that best explain the transitions between low and high beta power states observed in recordings of field potentials made in the motor cortex of anesthetized Parkinsonian rats. Bayesian model comparison suggests that upregulation of cortical rhythmic activity in the beta-frequency band results from changes in the coupling strength both between and within the thalamus and motor cortex. Specifically, our model indicates that high levels of cortical beta synchrony are mainly achieved by a delayed (extrinsic) input from thalamic relay cells to deep pyramidal cells and a fast (intrinsic) input from middle pyramidal cells to superficial pyramidal cells. From a clinical perspective, our study provides insights into potential therapeutic strategies that could be utilized to modulate the network mechanisms responsible for the enhancement of cortical beta in PD. Specifically, we speculate that cortical stimulation aimed to reduce the enhanced excitatory inputs to either the superficial or deep pyramidal cells could be a potential non-invasive therapeutic strategy for PD.

‘Dark matter’, second waves and epidemiological modelling

Recent reports using conventional Susceptible, Exposed, Infected and Removed models suggest that the next wave of the COVID-19 pandemic in the UK could overwhelm health services, with fatalities exceeding the first wave. We used Bayesian model comparison to revisit these conclusions, allowing for heterogeneity of exposure, susceptibility and transmission. We used dynamic causal modelling to estimate the evidence for alternative models of daily cases and deaths from the USA, the UK, Brazil, Italy, France, Spain, Mexico, Belgium, Germany and Canada over the period 25 January 2020 to 15 June 2020. These data were used to estimate the proportions of people (i) not exposed to the virus, (ii) not susceptible to infection when exposed and (iii) not infectious when susceptible to infection. Bayesian model comparison furnished overwhelming evidence for heterogeneity of exposure, susceptibility and transmission. Furthermore, both lockdown and the build-up of population immunity contributed to viral transmission in all but one country. Small variations in heterogeneity were sufficient to explain large differences in mortality rates. The best model of UK data predicts a second surge of fatalities will be much less than the first peak. The size of the second wave depends sensitively on the loss of immunity and the efficacy of Find-Test-Trace-Isolate-Support programmes. In summary, accounting for heterogeneity of exposure, susceptibility and transmission suggests that the next wave of the SARS-CoV-2 pandemic will be much smaller than conventional models predict, with less economic and health disruption. This heterogeneity means that seroprevalence underestimates effective herd immunity and, crucially, the potential of public health programmes.

Predictive coding: A free-energy formulation

Statistical parametric maps in functional imaging: A general linear approach

Statistical parametric maps are spatially extended statistical processes that are used to test hypotheses about regionally specific effects in neuroimaging data. The most established sorts of statistical parametric maps (e.g., Friston et al. [1991]: J Cereb Blood Flow Metab 11:690–699; Worsley et al. [1992]: J Cereb Blood Flow Metab 12:900–918) are based on linear models, for example ANCOVA, correlation coefficients and t tests. In the sense that these examples are all special cases of the general linear model it should be possible to implement them (and many others) within a unified framework. We present here a general approach that accomodates most forms of experimental layout and ensuing analysis (designed experiments with fixed effects for factors, covariates and interaction of factors). This approach brings together two well established bodies of theory (the general linear model and the theory of Gaussian fields) to provide a complete and simple framework for the analysis of imaging data. The importance of this framework is twofold: (i) Conceptual and mathematical simplicity, in that the same small number of operational equations is used irrespective of the complexity of the experiment or nature of the statistical model and (ii) the generality of the framework provides for great latitude in experimental design and analysis. © 1995 Wiley‐Liss, Inc.

A variational approach to scripts

This paper proposes a formal reconstruction of the script construct by leveraging the activeinference framework, a behavioural modelling framework that casts action, perception,emotions, and attention as processes of (Bayesian or variational) inference. We propose afirst principles account of the script construct that integrates its different uses in thebehavioural and social sciences. We begin by reviewing the recent literature that uses thescript construct. We then examine the main mathematical and computational features ofactive inference. Finally, we leverage the resources of active inference to offer a formalmodel of scripts. Our integrative model accounts for the dual nature of scripts (as internal,psychological schema used by agents to make sense of event types and as constitutivebehavioural categories that make up the social order) and also for the stronger and weakerconceptions of the construct (which do and do not relate to explicit action sequences,respectively).