Asymptotic normality of the posterior in relative entropy

Bertrand S. Clarke

doi:10.1109/18.746784

Asymptotic normality of the posterior in relative entropy

Bertrand S. Clarke

Research output: Contribution to journal › Article › peer-review

10 Scopus citations

Abstract

We show that the relative entropy between a posterior density formed from a smooth likelihood and prior and a limiting normal form tends to zero in the independent and identically distributed case. The mode of convergence is in probability and in mean. Applications to codelengths in stochastic complexity and to sample size selection are briefly discussed.

Original language	English (US)
Pages (from-to)	165-176
Number of pages	12
Journal	IEEE Transactions on Information Theory
Volume	45
Issue number	1
DOIs	https://doi.org/10.1109/18.746784
State	Published - 1999
Externally published	Yes

Keywords

Asymptotic normality
Posterior density
Relative entropy

ASJC Scopus subject areas

Information Systems
Computer Science Applications
Library and Information Sciences

Access to Document

10.1109/18.746784

Cite this

@article{43c36fd690d14b87b66420fc69b1283f,

title = "Asymptotic normality of the posterior in relative entropy",

abstract = "We show that the relative entropy between a posterior density formed from a smooth likelihood and prior and a limiting normal form tends to zero in the independent and identically distributed case. The mode of convergence is in probability and in mean. Applications to codelengths in stochastic complexity and to sample size selection are briefly discussed.",

keywords = "Asymptotic normality, Posterior density, Relative entropy",

author = "Clarke, {Bertrand S.}",

note = "Funding Information: Manuscript received July 26, 1995; revised May 22, 1998. This work was supported in part by the NSERC under Operating Grant 5-54891. The author is with the Department of Statistics, University of British Columbia, Vancouver, BC, Canada V6T 1Z2. Communicated by A. R. Barron, Associate Editor for Nonparametric Estimation, Classification, and Neural Networks. Publisher Item Identifier S 0018-9448(99)00073-5.",

year = "1999",

doi = "10.1109/18.746784",

language = "English (US)",

volume = "45",

pages = "165--176",

journal = "IRE Professional Group on Information Theory",

issn = "0018-9448",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "1",

}

TY - JOUR

T1 - Asymptotic normality of the posterior in relative entropy

AU - Clarke, Bertrand S.

N1 - Funding Information: Manuscript received July 26, 1995; revised May 22, 1998. This work was supported in part by the NSERC under Operating Grant 5-54891. The author is with the Department of Statistics, University of British Columbia, Vancouver, BC, Canada V6T 1Z2. Communicated by A. R. Barron, Associate Editor for Nonparametric Estimation, Classification, and Neural Networks. Publisher Item Identifier S 0018-9448(99)00073-5.

PY - 1999

Y1 - 1999

N2 - We show that the relative entropy between a posterior density formed from a smooth likelihood and prior and a limiting normal form tends to zero in the independent and identically distributed case. The mode of convergence is in probability and in mean. Applications to codelengths in stochastic complexity and to sample size selection are briefly discussed.

AB - We show that the relative entropy between a posterior density formed from a smooth likelihood and prior and a limiting normal form tends to zero in the independent and identically distributed case. The mode of convergence is in probability and in mean. Applications to codelengths in stochastic complexity and to sample size selection are briefly discussed.

KW - Asymptotic normality

KW - Posterior density

KW - Relative entropy

UR - http://www.scopus.com/inward/record.url?scp=0032636553&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032636553&partnerID=8YFLogxK

U2 - 10.1109/18.746784

DO - 10.1109/18.746784

M3 - Article

AN - SCOPUS:0032636553

SN - 0018-9448

VL - 45

SP - 165

EP - 176

JO - IRE Professional Group on Information Theory

JF - IRE Professional Group on Information Theory

IS - 1

ER -

Asymptotic normality of the posterior in relative entropy

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this