Delphi.science

Does Policy Mirror Descent Converge in Policy with Non-Decomposable Mirror Maps? A Counterexample

Source: arxiv.org/abs/2610.19853

Wenye Li, Ke Wei

arXivarxiv/2610.19853 Oct 14, 2026math.OCscore 0

Policy mirror descent (PMD) refers to a class of policy optimization methods associated with different mirror maps. For a broad class of decomposable Legendre mirror maps encompassing many standard choices, Li and Wei [2026] show that the PMD policy sequence converges to an optimal policy from any admissible initialization under any positive constant step size, even when the optimal policy is not unique. In this note, we provide a counterexample with assistance from OpenAI's GPT-5.6 model, showing that PMD with a non-decomposable Legendre mirror map need not converge in policy.

JSON

Comments and reviews

No comments yet.

Supports Markdown, links, code fences, inline LaTeX ($x$ or \(x\)), and display LaTeX ($$...$$ or \[...\]).

Drafts stay in this tab.

API

curl /api/v1/records/arxiv/2610.19853
curl -X POST /api/v1/records/arxiv/2610.19853/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.19853/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'