Word Recognition from Continuous Articulatory Movement Time-Series Data using Symbolic Representations

Jun Wang, Arvind Balasubramanian, Luis Mojica De La Vega, Jordan R. Green, Ashok Samal, Balakrishnan Prabhakaran

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Scopus citations

Abstract

Although still in experimental stage, articulation-based silent speech interfaces may have significant potential for facilitating oral communication in persons with voice and speech problems. An articulation-based silent speech interface converts articulatory movement information to audible words. The complexity of speech production mechanism (e.g., coarticulation) makes the conversion a formidable problem. In this paper, we reported a novel, real-time algorithm for recognizing words from continuous articulatory movements. This approach differed from prior work in that (1) it focused on word-level, rather than phoneme-level; (2) online segmentation and recognition were conducted at the same time; and (3) a symbolic representation (SAX) was used for data reduction in the original articulatory movement timeseries. A data set of 5,900 isolated word samples of tongue and lip movements was collected using electromagnetic articulograph from eleven English speakers. The average speaker-dependent recognition accuracy was up to 80.00%, with an average latency of 302 miliseconds for each word prediction. The results demonstrated the effectiveness of our approach and its potential for building a real-time articulationbased silent speech interface for clinical applications. The across-speaker variation of the recognition accuracy was discussed.

Original languageEnglish (US)
Title of host publicationSLPAT 2013 - 4th Workshop on Speech and Language Processing for Assistive Technologies, SLPAT 2013, Workshop Proceedings
PublisherAssociation for Computational Linguistics (ACL)
Pages119-127
Number of pages9
ISBN (Electronic)9781937284930
StatePublished - 2013
Event4th Workshop on Speech and Language Processing for Assistive Technologies, SLPAT 2013 - Grenoble, France
Duration: Aug 21 2013Aug 22 2013

Publication series

NameSLPAT 2013 - 4th Workshop on Speech and Language Processing for Assistive Technologies, SLPAT 2013, Workshop Proceedings

Conference

Conference4th Workshop on Speech and Language Processing for Assistive Technologies, SLPAT 2013
Country/TerritoryFrance
CityGrenoble
Period8/21/138/22/13

Keywords

  • Laryngectomy
  • SAX
  • Silent speech recognition
  • Support vector machine
  • Time-series

ASJC Scopus subject areas

  • Language and Linguistics
  • Computer Science Applications
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'Word Recognition from Continuous Articulatory Movement Time-Series Data using Symbolic Representations'. Together they form a unique fingerprint.

Cite this