Partially observable Markov decision processes with uniformly distributed signal processes

Abstract

A Partially observed Markov decision process (P.O.M.D.P.) is a sequential decision problem where information concerning parameters of interest is incomplete, and possible actions include sampling, surveying, or otherwise collecting additional information. Such problems can theoretically be solved as dynamics programs, but the relevant state space is
infinite which inhibits algorithmic solution. We formulate a (P.O.M.D.P.) with a continous signal space and a method to convert a problem with uniformly distributed signal processes. We discussed how to solve (P.O.M.D.P.) problems with continous signal processes. However, in order to obtain a value function which is close to the optimal value function, we might need to construct a step function with large number of signals.