Kullback-Leibler Distance and Statatistical Model

Proceedings of the International Conference on Modern research in Engineering,Technology & Science

Year: 2019

DOI: https://www.doi.org/10.33422/icmets.2019.03.203

[Fulltext PDF]

Kullback-Leibler Distance and Statatistical Model

Tomohiro Washino and Tadashi Takahashi

 

ABSTRACT: 

Kullback-Leibler distance is discovered in statistical mechanics of the 19th century and is a concept called the relative entropy. In the 20th century, it came to be known that it is important quantity in statistics and learning theory. We cannot only define it in Euclidean space, but also can expand the concept in general probability distribution. Kullback-Leibler distance is not only important as the concept, but also derive an algorithm. Kullback-Leibler distance is not only an important concept, but also it can derive the algorithm. As for the information science meaning of Kullback-Leibler distance and the relations with the mathematic properties, there exist many unknown structures. We constituted some examples about the pole using a computer algebra system and explored the properties.

Keywords: Kullback-Leibler distance, Bayesian statistics, Some properties.