Prequential analysis of complex data with adaptive model reselection

Jennifer Clarke, Bertrand Clarke

Research output: Contribution to journalArticle

6 Scopus citations

Abstract

In Prequential analysis, an inference method is viewed as a forecasting system, and the quality of the inference method is based on the quality of its predictions. This is an alternative approach to more traditional statistical methods that focus on the inference of parameters of the data generating distribution. In this paper, we introduce adaptive combined average predictors (ACAPs) for the Prequential analysis of complex data. That is, we use convex combinations of two different model averages to form a predictor at each time step in a sequence. A novel feature of our strategy is that the models in each average are re-chosen adaptively at each time step. To assess the complexity of a given data set, we introduce measures of data complexity for continuous response data. We validate our measures in several simulated contexts prior to using them in real data examples. The performance of ACAPs is compared with the performances of predictors based on stacking or likelihood weighted averaging in several model classes and in both simulated and real data sets. Our results suggest that ACAPs achieve a better trade off between model list bias and model list variability in cases where the data is very complex. This implies that the choices of model class and averaging method should be guided by a concept of complexity matching, i.e. the analysis of a complex data set may require a more complex model class and averaging strategy than the analysis of a simpler data set. We propose that complexity matching is akin to a bias-variance tradeoff in statistical modeling.

Original languageEnglish (US)
Pages (from-to)274-290
Number of pages17
JournalStatistical Analysis and Data Mining
Volume2
Issue number4
DOIs
StatePublished - Nov 2009

Keywords

  • Bayes model averaging
  • Complexity
  • Model selection
  • Model uncertainty
  • Predictive optimality
  • Prequential analysis
  • Stacking

ASJC Scopus subject areas

  • Analysis
  • Information Systems
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Prequential analysis of complex data with adaptive model reselection'. Together they form a unique fingerprint.

  • Cite this