Deep Active Inference and Scene Construction

Abstract Adaptive agents must act in intrinsically uncertain environments with complex latent structure. Here, we elaborate a model of visual foraging – in a hierarchical context – wherein agents infer a higher-order visual pattern (a ‘scene’) by sequentially sampling ambiguous cues. Inspired by previous models of scene construction – that cast perception and action as consequences of approximate Bayesian inference – we use active inference to simulate decisions of agents categorizing a scene in a hierarchically-structured setting. Under active inference, agents develop probabilistic beliefs about their environment, while actively sampling it to maximise the evidence for their internal generative model. This approximate evidence maximization (i.e. self-evidencing) comprises drives to both maximise rewards and resolve uncertainty about hidden states. This is realised via minimization of a free energy functional of posterior beliefs about both the world as well as the actions used to sample or perturb it, corresponding to perception and action, respectively. We show that active inference, in the context of hierarchical scene construction, gives rise to many empirical evidence accumulation phenomena, such as noise-sensitive reaction times and epistemic saccades. We explain these behaviours in terms of the principled drives that constitute the expected free energy , the key quantity for evaluating policies under active inference. In addition, we report novel behaviours exhibited by these active inference agents that furnish new predictions for research on evidence accumulation and perceptual decision-making. We discuss the implications of this hierarchical active inference scheme for tasks that require planned sequences of information-gathering actions to infer compositional latent structure (such as visual scene construction and sentence comprehension). Finally, we propose experiments to contextualise active inference in relation to other formulations of evidence accumulation (e.g. drift-diffusion models) in tasks that require planning in uncertain environments with higher-order structure.