Predicting amazon spot prices with LSTM networks

Matt Baughman, Christian Haas, Rich Wolski, Ian Foster, Kyle Chard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Scopus citations

Abstract

Amazon spot instances provide preemptable computing capacity at a cost that is often significantly lower than comparable on-demand or reserved instances. Spot instances are charged at the current spot price: a fluctuating market price based on supply and demand for spot instance capacity. However, spot instances are inherently volatile, the spot price changes over time, and instances can be revoked by Amazon with as little as two minutes’ warning. Given the potential discount—up to 90% in some cases—there has been significant interest in the scientific cloud computing community to leverage spot instances for workloads that are either fault-tolerant or not time-sensitive. However, cost-effective use of spot instances requires accurate prediction of spot prices in the future. We explore here the use of long/short-term memory (LSTM) recurrent neural networks for spot price prediction. We describe our model and compare it against a baseline ARIMA model using historical spot pricing data. Our results show that our LSTM approach can reduce training error by as much as 95%.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th Workshop on Scientific Cloud Computing, ScienceCloud 2018 - Co-located with HPDC 2018
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450358637
DOIs
StatePublished - Jun 11 2018
Event9th Workshop on Scientific Cloud Computing, ScienceCloud 2018 - Tempe, United States
Duration: Jun 11 2018 → …

Publication series

NameProceedings of the 9th Workshop on Scientific Cloud Computing, ScienceCloud 2018 - Co-located with HPDC 2018

Conference

Conference9th Workshop on Scientific Cloud Computing, ScienceCloud 2018
CountryUnited States
CityTempe
Period6/11/18 → …

Keywords

  • Amazon spot instances
  • LSTM networks
  • Spot price prediction

ASJC Scopus subject areas

  • Computer Science Applications
  • Software
  • Computational Theory and Mathematics

Fingerprint Dive into the research topics of 'Predicting amazon spot prices with LSTM networks'. Together they form a unique fingerprint.

Cite this