Impact Analysis of Stacked Machine Learning Algorithms Based Feature Selections for Deep Learning Algorithm Applied to Regression Analysis

Shrirang Ambaji Kulkarni, Varadraj P. Gurupur, Christian King

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Ensemble learning algorithms have proved to be one of the best machine learning algorithms towards optimal performances in terms of regression and classification tasks for a variety of applications. When applied for small or medium structured datasets, eXtreme Gradient Boosting (XGBoost) has emerged as a popular ensemble learning technique based on its performance merits. In recent years Light Gradient Boosting Machine (LightGBM) has emerged as a promising ensemble strategy that is competing with XGBoost in terms of performance. Also, Lasso Regression has proven capabilities in terms of feature selections and applications for small datasets. This paper illustrates experimentation performed on a diabetes dataset where the authors tested the hypothesis that feature selection has relatively no impact on the performances of Deep Learning Algorithms as they have built-in capabilities in terms of layers to perform feature selection on their own. Therefore, the hypothesis tested stacking using Deep Learning - Multi-Layer Perceptron (DMLP) with optimal algorithms like XGBoost, LightGBM, and Lasso Regression. In the present work, DMLP with all feature variables (DMLP-ALL) outscored DMLP with stacked selected features (DMLP-MS) by 8.78 % in terms of R2. Also, DMLP-ALL outperformed the benchmarked algorithm Automated Machine Learning (AML) by 10.25% in terms of R2. The validation of the proposed stacking models by applying a moderate-sized dataset provides promising results for deep learning models stacked with a powerful Level-0 learner.

Original languageEnglish (US)
Title of host publicationSoutheastCon 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages269-275
Number of pages7
ISBN (Electronic)9781665406529
DOIs
StatePublished - 2022
Externally publishedYes
EventSoutheastCon 2022 - Mobile, United States
Duration: Mar 26 2022Apr 3 2022

Publication series

NameConference Proceedings - IEEE SOUTHEASTCON
Volume2022-March
ISSN (Print)0734-7502

Conference

ConferenceSoutheastCon 2022
Country/TerritoryUnited States
CityMobile
Period3/26/224/3/22

Keywords

  • Deep learning Algorithms
  • Machine learning algorithms
  • Regression metrics
  • Stacking models

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software
  • Electrical and Electronic Engineering
  • Control and Systems Engineering
  • Signal Processing

Fingerprint

Dive into the research topics of 'Impact Analysis of Stacked Machine Learning Algorithms Based Feature Selections for Deep Learning Algorithm Applied to Regression Analysis'. Together they form a unique fingerprint.

Cite this