On Convergence of Gradient Temporal-Difference Learning with Regularized Corrections and Eligibility Traces

Source: arxiv.org/abs/2611.05869

Jingyi Jin, Martha White, Hao Wang

arXivarxiv/2611.05869Nov 4, 2026cs.LGscore 0

Temporal-difference (TD) learning underlies many prediction and control algorithms, but can diverge when combined with off-policy learning, bootstrapping, and function approximation. TD with Regularized Corrections (TDRC) is theoretically sound and practically simple, but existing convergence theory covers only the one-step case under i.i.d. sampling, while the Huber Bellman-error formulation has not been extended to TDRC or eligibility traces. In this work, we establish convergence of constrained TDRC($λ$) with diminishing stepsizes and non-i.i.d. off-policy data, for state-dependent and…

JSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2611.05869
curl -X POST /api/v1/records/arxiv/2611.05869/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.05869/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'