See: Stochastic Process, Unobserved Variable, Observed Variable, Partially Observable Markov Decision Process.