Soft MPCritic: Value-Based Reinforcement Learning for Non-Differentiable Predictive Control
Reinforcement learning (RL) and model predictive control (MPC) offer complementary strengths, yet existing MPC-RL frameworks commonly rely on gradient-based optimization and therefore cannot handle the non-differentiable dynamics and objectives common in RL. We propose Soft MPCritic, an MPC-RL framework that learns a soft value function and uses sampling-based planning both for online control and for generating value targets. Specifically, Soft MPCritic instantiates MPC with model predictive path integral control (MPPI) and learns a terminal Q-function through fitted value iteration. Because the value targets are computed by the same planner used for control, the learned value function is...
| Authors | Thomas Banker, Nathan P. Lawrence, Ali Mesbah |
|---|---|
| Primary category | cs.LG |
| Published | 1 April 2026 |
| Last updated at source | 12 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.