Grounding Large Language Models in DSGE Simulators for Policy Generation and Forecasting
This paper tackles the problem that LLM-generated economic policy can sound plausible yet be dynamically inconsistent. It grounds an instruction-tuned language model inside six DSGE simulators via a common RL interface, using PPO and GRPO to learn multi-turn policy actions with delayed rewards. The core idea is to replace linguistic plausibility with simulated economic consequences, testing credit assignment, semantic-shift conditioning, and cross-simulator transfer. Results matter for any setting where language agents must make sequential decisions with long-horizon structural feedback.
This manuscript is best understood as a detailed methodology preprint: the interface design, ablation protocol, and simulator diagnostics are thoughtfully pre-registered, but the confirmatory policy-learning experiments are entirely absent. Every primary comparison cell in Table 4 and Section 8 is marked TBD and explicitly disclaimed as "not numerical claims," leaving all six hypotheses untested and the title's promise of empirical grounding unfulfilled.
The simulator wrapper is genuinely well engineered, exposing shock abstraction, named parameter updates, deep state cloning, and validated irreversible transitions across six distinct structural models. The authors also deserve credit for an unusually careful ablation design that refuses to conflate algorithmic, reward, semantic, and transfer effects.
The paper's most serious limitation is that no confirmatory learning results are present. As the authors note themselves, "Tables for the confirmatory policy-learning runs use TBD cells until their logged artifacts are inserted; these cells are not numerical claims," which leaves the central PPO–GRPO comparison and every downstream hypothesis entirely unsubstantiated.\n\n A second concern is that the reward-horizon calibration (Table 6) shows plausible interventions often worsen short-horizon loss, creating a difficult optimization landscape the authors call a "reward-horizon inversion" but cannot yet show either algorithm escapes. Additionally, the authors concede that "without task-specific SFT, on-policy RL can refine only behaviors explored by the instruction model," raising the possibility that observed limitations would be coverage problems rather than algorithmic ones.
The cited evidence for PPO's superiority in multi-turn settings comes from Wang and Ammanabrolu (2025), and the authors correctly caution that these TextWorld figures are "not from our DSGE environment." I fetched the cited source and confirmed it is a genuine multi-turn RL study that sets out to "compare biased (PPO, GRPO) and unbiased (RLOO) policy gradient RL algorithms," lending external credibility to the framing. Within the paper itself, however, the only concrete empirical content is simulator verification (Table 3) and a manual action grid (Table 6); the primary claims about credit assignment, semantic conditioning, and transfer remain unevaluated.
Reproducibility infrastructure is exemplary in design: Appendix A demands git commits, YAML hashes, checkpoints, per-token log probabilities, and counterfactual losses, while the repository URL is disclosed in Section 10. Yet because the confirmatory runs are incomplete, the promised "rollout-accounting tests" have not been executed, making the reproducibility of main claims purely prospective. Hardware constraints are honestly reported, but the need to serialize rollouts because "PPO's separate critic is the tighter memory constraint" may limit experimental scale.
Large language models can produce economic policy responses that sound reasonable, but this does not show that their actions are consistent with economic dynamics. We test this by placing an instruction-tuned language model inside six Snowdrop-backed dynamic stochastic general equilibrium (DSGE) simulators. At each turn, the model observes the economy and a change in economic discourse, selects a bounded policy action, and receives the next simulated state and an economic reward. We implement a common Python interface for repeated rollouts, persistent shocks, state cloning, and rolling-horizon simulation. This setting creates a long-horizon credit-assignment problem. Policy effects may appear several quarters after an action is taken. PPO has a learned value function that can propagate delayed reward to earlier tokens through generalized advantage estimation. GRPO has no learned value function and instead assigns a group-relative advantage from complete rollout returns. It therefore cannot distinguish which earlier turn caused the outcome; if every rollout receives the same return, the normalized advantage is zero. We use PPO as the primary method and GRPO as a matched critic-free baseline. The experiments also test directional semantic signals, reward horizon, trajectory warm starts, cross-simulator transfer, and historically anchored pandemic and monetary-policy shocks. The objective is to judge policy actions by their simulated economic consequences rather than by plausible language alone.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.