A New Outlook of Shannon's Information Measures.

We formalize the work of Reza, Csiszar, and Korner on the underlying mathematical structure of Shannon's Information measures. Let X sub i, i=1,..., n be discrete random variables. By regarding random variables as set variables, let OMEGA = UNION of i X sub i be the universal set and F be the sigma-field generated by {X sub i}. We show that Shannon's information measures constitute a unique measure on F. To be precise, the Shannon information measure (i.e., Shannon's information measures as a whole) is a measure on F. This point of view, which we believe is of fundamental importance, has apparently been overlooked in the past by information theorists. As a consequence we introduce the I-Diagram, which is a geometrical representation of the relationship among the information measures.