A Minimum Discrimination Information Approach for Hidden Markov Modeling

A new iterative approach for hidden Markov modeling of information sources which aims at minimizing the discrimination information (or the cross-entropy) between the source and the model is proposed. This approach does not require the commonly used assumption that the source to be modeled is a hidden Markov process. The algorithm is started from the model estimated by the traditional maximum likelihood approach and alternatively decreases the discrimination information over all probability distributions of the source which agree with the given measurements and all models. This results in a "generalized Baum algorithm" for hidden Markov modeling. The iterative procedure is shown to be a descent algorithm for the discrimination information measure and its local convergence is proved.