A Bayesian approach to cluster validation

Hoyt A. Koepke, Bertrand Clarke

Research output: Contribution to conferencePaperpeer-review


In this paper, we propose a novel approach to validating clusterings. We treat a given clustering as a baseline and define a collection of perturbations of it that give possibly different assignment of points to clusters. If these are indexed by a hyperparameter, integrating with respect to a prior gives an averaged assignment matrix. This matrix can be visualized as a heat map, allowing clusterings and their stability properties to be readily seen. The difference between an averaged assignment matrix and the baseline gives a measure of the stability of the baseline. This approach motivates a general and computationally fast algorithm for evaluating the stability of distance-based and exponential-model type clusterings, including k-means. In addition, these criteria can be used to choose the optimal number of clusters. Our method compares favorably with data based perturbation procedures, such as subsampling, in some conditions such as small sample size. In addition, there is evidence that our method performs better relative to subsampling methods on some problems.

Original languageEnglish (US)
StatePublished - 2008
Event10th International Symposium on Artificial Intelligence and Mathematics, ISAIM 2008 - Fort Lauderdale, FL, United States
Duration: Jan 2 2008Jan 4 2008


Conference10th International Symposium on Artificial Intelligence and Mathematics, ISAIM 2008
Country/TerritoryUnited States
CityFort Lauderdale, FL

ASJC Scopus subject areas

  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'A Bayesian approach to cluster validation'. Together they form a unique fingerprint.

Cite this