TY - GEN
T1 - Predicting amazon spot prices with LSTM networks
AU - Baughman, Matt
AU - Haas, Christian
AU - Wolski, Rich
AU - Foster, Ian
AU - Chard, Kyle
N1 - Publisher Copyright:
© 2018 Copyright held by the owner/author(s).
PY - 2018/6/11
Y1 - 2018/6/11
N2 - Amazon spot instances provide preemptable computing capacity at a cost that is often significantly lower than comparable on-demand or reserved instances. Spot instances are charged at the current spot price: a fluctuating market price based on supply and demand for spot instance capacity. However, spot instances are inherently volatile, the spot price changes over time, and instances can be revoked by Amazon with as little as two minutes’ warning. Given the potential discount—up to 90% in some cases—there has been significant interest in the scientific cloud computing community to leverage spot instances for workloads that are either fault-tolerant or not time-sensitive. However, cost-effective use of spot instances requires accurate prediction of spot prices in the future. We explore here the use of long/short-term memory (LSTM) recurrent neural networks for spot price prediction. We describe our model and compare it against a baseline ARIMA model using historical spot pricing data. Our results show that our LSTM approach can reduce training error by as much as 95%.
AB - Amazon spot instances provide preemptable computing capacity at a cost that is often significantly lower than comparable on-demand or reserved instances. Spot instances are charged at the current spot price: a fluctuating market price based on supply and demand for spot instance capacity. However, spot instances are inherently volatile, the spot price changes over time, and instances can be revoked by Amazon with as little as two minutes’ warning. Given the potential discount—up to 90% in some cases—there has been significant interest in the scientific cloud computing community to leverage spot instances for workloads that are either fault-tolerant or not time-sensitive. However, cost-effective use of spot instances requires accurate prediction of spot prices in the future. We explore here the use of long/short-term memory (LSTM) recurrent neural networks for spot price prediction. We describe our model and compare it against a baseline ARIMA model using historical spot pricing data. Our results show that our LSTM approach can reduce training error by as much as 95%.
KW - Amazon spot instances
KW - LSTM networks
KW - Spot price prediction
UR - http://www.scopus.com/inward/record.url?scp=85050077734&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85050077734&partnerID=8YFLogxK
U2 - 10.1145/3217880.3217881
DO - 10.1145/3217880.3217881
M3 - Conference contribution
AN - SCOPUS:85050077734
T3 - Proceedings of the 9th Workshop on Scientific Cloud Computing, ScienceCloud 2018 - Co-located with HPDC 2018
BT - Proceedings of the 9th Workshop on Scientific Cloud Computing, ScienceCloud 2018 - Co-located with HPDC 2018
PB - Association for Computing Machinery, Inc
T2 - 9th Workshop on Scientific Cloud Computing, ScienceCloud 2018
Y2 - 11 June 2018
ER -