active-control.txt

This document discusses the relationship between the frameworks of Active Inference (AIF) and Control as Inference (CAI) for adaptive action selection. 
1. **Formalisms of AIF and CAI**: Both AIF and CAI can be formalized in the context of a partially observed Markov Decision Process (POMDP), where the goal is to infer the posterior distribution over latent variables like states and actions given observations.
2. **Control as Inference**: CAI introduces an additional "optimality" variable to encode the notion of value, where the goal is to recover the posterior over states and actions given the belief that the agent will observe itself being optimal. The variational bound can be decomposed into terms related to extrinsic value, state divergence, action divergence, and observation ambiguity.
3. **Active Inference**: Unlike CAI, AIF does not introduce additional variables to incorporate value, but instead assumes the generative model is intrinsically biased towards valuable states or observations. The expected free energy functional in AIF requires maximizing both extrinsic and intrinsic value.
4. **Encoding Value**: The key distinction between AIF and CAI is how they encode value into the generative model, with AIF using biased priors and CAI using exogenous optimality likelihoods. This leads to differences in the objectives and exploratory behavior of the two frameworks.

This document provides a formal comparison between the frameworks of Active Inference and Control as Inference, highlighting their similarities and key differences in how they encode the notion of "value" into their respective generative models.