Learning Acoustic Word Embeddings with Dynamic Time Warping Triplet Networks

Denis Shitov, Elena Pirogova, Tadeusz A. Wysocki, Margaret Lech

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


In the last years, acoustic word embeddings (AWEs) have gained significant interest in the research community. It applies specifically to the application of acoustic embeddings in the Query-by-Example Spoken Term Detection (QbE-STD) search and related word discrimination tasks. It has been shown that AWEs learned for the word or phone classification in one or several languages can outperform approaches that use dynamic time warping (DTW). In this paper, a new method of learning AWEs in the DTW framework is proposed. It employs a multitask triplet neural network to generate the AWEs. The triplet network learns acoustic representations of words through a comparison of DTW distances. In addition, a multitask objective, including a conventional word classification component, and a triplet loss component is proposed. The triplet loss component applies the DTW distance for the word discrimination task. The multitask objective ensures that the embeddings can be used with DTW directly. Experimental validation shows that the proposed approach is well-suited, but not necessarily restricted to the QbE-STD search. A comparison with several baseline methods shows that the new method leads to a significant improvement of the results on the word discrimination task. An evaluation of the word clustering in the learned embedding space is presented.

Original languageEnglish (US)
Article number9104974
Pages (from-to)103327-103338
Number of pages12
JournalIEEE Access
StatePublished - 2020


  • Acoustic word embedding
  • dynamic time warping
  • query-by-example
  • triplet network

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering


Dive into the research topics of 'Learning Acoustic Word Embeddings with Dynamic Time Warping Triplet Networks'. Together they form a unique fingerprint.

Cite this