This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain.
Social neuroscience has often been criticized for approaching the investigation of the neural processes that enable social interaction and cognition from a passive, detached, third-person perspective, without involving any real-time social interaction. With the emergence of second-person neuroscience, investigators have uncovered the unique complexity of neural-activation patterns in actual, real-time interaction. Social cognition that occurs during social interaction is fundamentally different from that unfolding during social observation. However, it remains unclear how the neural correlates of social interaction are to be interpreted. Here, we leverage the active-inference framework to shed light on the mechanisms at play during social interaction in second-person neuroscience studies. Specifically, we show how counterfactually rich mutual predictions, real-time bodily adaptation, and policy selection explain activation in components of the default mode, salience, and frontoparietal networks of the brain, as well as in the basal ganglia. We further argue that these processes constitute the crucial neural processes that underwrite bona fide social interaction. By placing the experimental approach of second-person neuroscience on the theoretical foundation of the active-inference framework, we inform the field of social neuroscience about the mechanisms of real-life interactions. We thereby contribute to the theoretical foundations of empirical second-person neuroscience.
A weak version of the life-mind continuity thesis entails that every living system also has a basic mind (with a non-representational form of intentionality). The strong version entails that the same concepts that are sufficient to explain basic minds (with non-representational states) are also central to understanding non-basic minds (with representational states). We argue that recent work on the free energy principle supports the following claims with respect to the life-mind continuity thesis: (i) there is a strong continuity between life and mind; (ii) all living systems can be described as if they had representational states; (iii) the ’as-if representationality’ entailed by the free energy principle is central to understanding both basic forms of intentionality and intentionality in non-basic minds. In addition to this, we argue that the free energy principle also renders realism about computation and representation compatible with a strong life-mind continuity thesis (although the free energy principle does not entail computational and representational realism). In particular, we show how representationality proper can be grounded in ’as-if representationality’.
Event Abstract Back to Event A predictive coding account of MMN and brain plasticity Marta Garrido1*, J M Kilner2, S J Kiebel2, K E Stephan2, 3, Torsten Baldeweg4 and K J Friston2 1 Department of Psychology, University California Los Angeles, United States 2 Wellcome Trust Centre for Neuroimaging, University College London, United Kingdom 3 Institute for Empirical Research in Economics, University of Zurich, Switzerland 4 Developmental Cognitive Neuroscience, Institute of Child Health, University College London, United Kingdom Predictive coding models state that the brain perceives and makes inferences about the world by recursively updating predictions about sensory input. Thus, perception could result from comparing bottom-up input from the environment with top-down predictions. Predictive coding as a model of cortical organization and function has been used to frame the mismatch negativity (MMN), and supporting empirical evidence has been furnished by dynamic causal modelling (DCM), a novel tool for connectivity analysis of neuroimaging data. In the light of this framework, the generation of the MMN, an event-related response to unpredictable events, reflects prediction error, which occurs whenever the current input does not match a previously learnt rule. In brief, this talk will discuss how MMN can be framed within a predictive coding scheme, and show the usefulness of DCM in investigating the underlying cortical mechanisms of responses to unpredictable auditory events. Moreover, alternative candidate models that map onto mechanistic hypotheses for MMN generation will be discussed. These models correspond to alternative hierarchical cortical networks, which can be statistically evaluated within the Bayesian framework of DCM. Finally, it will be shown how tone repetition can induce connectivity changes over time (or plasticity), both between distant cortical areas and within an area belonging to a cortical network. This suggests that learning an auditory perceptual model from the environment is associated with repetition-dependent plasticity in the human brain. Conference: MMN 09 Fifth Conference on Mismatch Negativity (MMN) and its Clinical and Scientific Applications, Budapest, Hungary, 4 Apr - 7 Apr, 2009. Presentation Type: Oral Presentation Topic: Symposium 2: Predictive models within and of MMN Citation: Garrido M, Kilner J, Kiebel S, Stephan K, Baldeweg T and Friston K (2009). A predictive coding account of MMN and brain plasticity. Conference Abstract: MMN 09 Fifth Conference on Mismatch Negativity (MMN) and its Clinical and Scientific Applications. doi: 10.3389/conf.neuro.09.2009.05.040 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 23 Mar 2009; Published Online: 23 Mar 2009. * Correspondence: Marta Garrido, Department of Psychology, University California Los Angeles, Los Angeles, United States, migarrido@ucla.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Marta Garrido J M Kilner S J Kiebel K E Stephan Torsten Baldeweg K J Friston Google Marta Garrido J M Kilner S J Kiebel K E Stephan Torsten Baldeweg K J Friston Google Scholar Marta Garrido J M Kilner S J Kiebel K E Stephan Torsten Baldeweg K J Friston PubMed Marta Garrido J M Kilner S J Kiebel K E Stephan Torsten Baldeweg K J Friston Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.
The locus coeruleus (LC) in the pons is the major source of noradrenaline (NA) in the brain. Two modes of LC firing have been associated with distinct cognitive states: changes in tonic rates of firing are correlated with global levels of arousal and behavioural flexibility, whilst phasic LC responses are evoked by salient stimuli. Here, we unify these two modes of firing by modelling the response of the LC as a correlate of a prediction error when inferring states for action planning under Active Inference (AI). We simulate a classic Go/No-go reward learning task and a three-arm 'explore/exploit' task and show that, if LC activity is considered to reflect the magnitude of high level 'state-action' prediction errors, then both tonic and phasic modes of firing are emergent features of belief updating. We also demonstrate that when contingencies change, AI agents can update their internal models more quickly by feeding back this state-action prediction error–reflected in LC firing and noradrenaline release–to optimise learning rate, enabling large adjustments over short timescales. We propose that such prediction errors are mediated by cortico-LC connections, whilst ascending input from LC to cortex modulates belief updating in anterior cingulate cortex (ACC). In short, we characterise the LC/ NA system within a general theory of brain function. In doing so, we show that contrasting, behaviour-dependent firing patterns are an emergent property of the LC that translates state-action prediction errors into an optimal balance between plasticity and stability.
