MyBlog

Navigating the Evolution of Automatic Speech Recognition (ASR)

Automatic Speech Recognition (ASR) is a rapidly advancing technology that holds immense relevance in today’s digital age. As a voice-based technology it is a beneficial way to make Human Computer Interaction more accessible. However, the inner workings of ASR can be complex, so we aim to demystify the concept of ASR and present it in a simple and understandable manner while giving an overview of the different time periods and what led to the transitions between them.
Naghmana Asif
Naghmana Asif

Deep Learning

Deep Speech emerged as a significant advancement within the realm of end-to-end ASR. This architecture simplified ASR system development for diverse use cases, circumventing the complexity associated with traditional methods. Deep Speech’s architecture does not require hand-designed components to address background noise, reverberation, or speaker variation. Instead, it directly learns a function that is robust to such effects. Furthermore, it eliminates the need for a phoneme dictionary, revolutionizing the conventional approach to ASR. Which leads to the question: How can such a “simple” architecture exist and perform ASR without requiring the basic units of speech (phonemes)? Before diving into that however, it is important to mention that the architecture alone was not enough to compete with the complex HCLG systems, it required significantly more data to reach competitive levels.

Connectionist Temporal Classification (CTC) loss is the objective function which needs to be optimized when dealing with a setup as is present for Deep Speech. So, how is CTC helping us answer our questions to some degree? In ASR, two sequences are at play, and if the system learns to map one sequence to the other, it has learned to solve the ASR problem. For HCLG, the sequences were those of the input speech and the sequence of phonemes. The CTC loss is designed in a way which makes it possible for the second sequence to be those of characters/words/sub-words. A graph similar to HCLG is constructed, but with character states and a few special states, and as the CTC loss is optimized thesystem is able to assign the correct states to different sections of the speech input. This signifies that the model learns to map speech to characters/words/subwords. In an HCLG system the sequential nature of speech is modeled by the HCLG graph structure, however Recurrent Neural Networks (RNNs) can do the same, but much more competently because of their ability to remember the history of a sequence infinitely (in theory). In comparison, HMMs remember the information of only the previous state. Deep Speech used this architecture in combination with standard deep feed-forward blocks and fed it a lot of training data, while leaving the alignment to the CTC loss.

The Evolution of Automatic Speech Recognition

In the ever-evolving landscape of technology, the journey of Automatic Speech Recognition (ASR) has been nothing short of remarkable. Over the years, numerous approaches have been explored and refined, each contributing to the advancement of this pivotal technology. In this blog post, we delve into the fascinating evolution of ASR, shedding light on the key milestones and transitions that have shaped its trajectory.

Statistical Approach

Hidden Markov Models (HMMs) were one of the earliest fundamental approaches introduced to address speech recognition. This approach was encompassed within the HCLG (HMM – Context – Lexicon – Grammar) framework, which decomposed the problem into distinct components, each addressing specific aspects. However, with advancements in hardware and the flourishing domain of Big Data, the field of ASR transitioned into the end-to-end era, where individual components were consolidated into a singular Deep Learning model.

End-to-end Approach

Neural networks have revolutionized speech recognition, marking a significant milestone in technological advancement. By leveraging large datasets and powerful computational capabilities (e.g. GPUs), they have significantly impacted applications like voice assistants, or transcription services. Systems can now handle a wide range of languages, accents, and speech variations more effectively and accurately, often rivaling human-level accuracy. Notably, the year 2017 heralded a transformative era in Deep Learning with the introduction of the Transformer architecture, which swiftly found its application in ASR. However, as we navigate through the landscape of ASR, we must ponder: are Transformers the ultimate solution, or are there yet uncharted territories awaiting exploration? In the ensuing sections of this blog post, we will embark on a journey into the intricacies of ASR, unraveling its inner workings and exploring the possibilities that lie ahead.

Unveiling the Spectrum of Automatic Speech Recognition: From HMMs to Whisperand Deep Learning Techniques

Automatic Speech Recognition (ASR) involves a series of complex steps that enable machines to convert spoken language into written text. To understand the fundamental mechanisms of ASR, let’s delve into key components, focusing on two main approaches: Hidden Markov Models (HMMs) and Deep Learning.

Hidden Markov Models (HMMs)

Speech recognition systems comprise two primary components: the Acoustic Model (AM) and the Language Model (LM). Initially, this division was finely delineated within the HCLG (HMM – Context – Lexicon – Grammar) framework. HCLG is a graph constructed by composing a Grammar, Lexicon, Context and an HMM Weighted Finite State Transducers (WFSTs). In this graph, vertices depict symbols which together transform into a sentence once the graph has been traversed. The edges which connect the vertices have penalties or costs which are incurred upon getting opted for. When solved for the least costly path, the output is the best transcript for the input audio.

The Acoustic Model (AM) of the HCLG system is comprised of HMM, Context, and Lexicon WFSTs, collectively modeling speech as a sequence of phonemes. The Grammar WFST hosts the Language Model (LM), depicting the likelihood of word sequences. Notably, AM and LM training can be conducted separately andcombined using WFST math, rendering this setup highly convenient and potent in low-resource ASR environments.

Conclusion

The evolution of Automatic Speech Recognition (ASR) technology from its nascent stages to the present era represents a remarkable journey marked by transformative advancements. From simple systems relying on statistical approaches to sophisticated neural networks and transformers, ASR has undergone a profound transformation, driven by the quest for seamless human-computer interaction.

The journey underscores the relentless pursuit of innovation and improvement in ASR technology, with current Deep Learning-based systems standing as a testament to the remarkable progress achieved. These advanced ASR systems excel at adapting to diverse settings and linguistic contexts, paving the way for enhanced accessibility and usability across various applications.

At Fraunhofer IAIS and Lamarr Institute, we are committed to addressing the challenges inherent in ASR technology while continuously enhancing model capabilities. Through rigorous research and development efforts, we strive to overcome obstacles such as robustness, out-of-vocabulary words, and hallucinations, enabling the seamless integration of ASR into daily lives. Our aim is to empower individuals with greater accessibility through voice technology, promising a future where ASR serves as a ubiquitous and indispensable tool for communication and interaction.