2018-05-16: Helle Hvid Hansen (TU Delft)

Tue, 14 Oct 2025 16:51:42 +0000

Helle Hvid Hansen (TU Delft)

Long-Term Values in Markov Decision Processes, (Co)Algebraically

Abstract

In this talk, we study Markov decision processes (MDPs) with the discounted sums criterion from the perspective of coalgebra and algebra. Probabilistic systems, similar to MDPs but without rewards, have been extensively studied, also coalgebraically, from the perspective of program semantics. Here, we focus on the role of MDPs as models in optimal planning, where the reward structure is central. Our main contributions are: (i) a coinductive explanation of policy improvement using a new proof principle, based on Banach's Fixpoint Theorem, that we call contraction coinduction, and (ii) showing that the long-term value function of a policy can be obtained via a generalized notion of corecursive algebra, which takes boundedness into account.

This is joint work with Frank Feys (Delft) and Larry Moss (Indiana).

2018-05-16 11:00:00 UTC
LT1310