Abstract: A discussion is made of the relationship between the unobservable status and sample value of Markov decision processes. With a definition of error-avoiding characteristic function introduced, a hidden Markov decision model is built by using the structure of hidden Markov processes and Markov decision models. The status estimation and the stage income function are also discussed.