Entering the black box of neural networks: A descriptive study of clinical variables predicting community-acquired pneumonia

Paul S. Heckerling, B. S. Gerber, T. G. Tape, R. S. Wigton

Research output: Contribution to journalArticlepeer-review

23 Scopus citations


Objectives: Artificial neural networks have proved to be accurate predictive instruments in several medical domains, but have been criticized for failing to specify the information upon which their predictions are based. We used methods of relevance analysis and sensitivity analysis to determine the most important predictor variables for a validated neural network for community-acquired pneumonia. Methods: We studied a feed-forward, back-propagation neural network trained to predict pneumonia among patients presenting to an emergency department with fever or respiratory complaints. We used the methods of full retraining, weight elimination, constant substitution, linear substitution, and data permutation to identify a consensus set of important demographic, symptom, sign, and comorbidity predictors that influenced network output for pneumonia. We compared predictors identified by these methods to those identified by a weight propagation analysis based an the matrices of the network, and by logistic regression. Results: Predictors identified by these methods were clinically plausible, and were concordant with those identified by weight analysis, and by logistic regression using the same data. The methods were highly correlated in network error, and led to variable sets with errors below bootstrap 95% confidence intervals for networks with similar numbers of inputs. Scores for variable relevance tended to be higher with methods that precluded network retraining (weight elimination) or that permuted variable values (data permutation), compared with methods that permitted retraining (full retraining) or that approximated its effects (constant and linear substitution). Conclusion: Methods of relevance analysis and sensitivity analysis are useful for identifying important predictor variables used by artificial neural networks.

Original languageEnglish (US)
Pages (from-to)287-296
Number of pages10
JournalMethods of Information in Medicine
Issue number3
StatePublished - 2003


  • Analysis
  • Diagnosis computer-assisted
  • Neural networks (computer)
  • Pneumonia
  • Sensitivity

ASJC Scopus subject areas

  • Health Informatics
  • Advanced and Specialized Nursing
  • Health Information Management


Dive into the research topics of 'Entering the black box of neural networks: A descriptive study of clinical variables predicting community-acquired pneumonia'. Together they form a unique fingerprint.

Cite this