Audio Signal Analysis and Speech Recognition

There will be presented basic formats of digital audio signal representation and signal processing methods to improve the quality and reconstruction of useful signals. Methods of blind separation and extraction of source signals from their mixtures and methods of locating sound sources in the vicinity of the observer will be discussed. A model of language phonetics in the form of phonemes and three-phonemes will be introduced. Typical acoustic models based on MFCC and LPC features will be presented. The role of HMM and N-grams models in the field of stochastic acoustic-phonetic modeling and recognition of sentence words and phrases will be discussed. The main methods of speaker recognition will be discussed.

The lecture is conducted in a traditional form with the use of electronic materials. 18 hours of the lecture part are planned as eight 2-hour meetings and two 1-hour tests.

Exercises are dedicated for solving algorithmic and computational tasks by the tutor together with the students. The tasks are a practical illustration of the issues discussed in the lecture. 12 hours of exercises are planned as six 2-hour meetings. Students activity during classes is assessed, including solving the previously given tasks.

The laboratory consists in designing programming solutions to selected issues with the use of tools and program libraries installed on computers in the laboratory. It is planned to work in 2-person groups. Laboratory part consists of 7 classes, 2 hours each, devoted to separate issues presented during the lecture and exercises, and in one 1-hour introductory class. The degree of implementation of the assigned laboratory tasks is assessed.

Content:

Lecture (9 x 2h):
1. Introduction – digital audio representation
2. Audio signal preprocessing – filtering and transformation
3. Analysis of audio mixtures
4. Acoustic-phonetic speech modelling
5. Automatic speech recognition
6. Neural speech recognition
7. Speaker modelling and recognition
8. Speaker/speech classification

Midterm and final tests

Exercises (6 x 2h):
1. Audio representation and processing
2. Transforms and filtering of audio and mixtures
3. Speech features
4. Classic ASR
5. Neural ASR
6. Speaker recognition/classification

Laboratory: (1h + 7 x 2h)
1. Introduction (1h)
2. Audio processing and transformation
3. Audio mixtures
4. Speech features
5. Speech recognition
6. Neural models in speech modelling and recognition
7. Speaker modeling and recognition
8. Speech/speaker classification

1. W. Kasprzak: ASAR. E-skrypt do wykładu i ćwiczeń, Politechnika Warszawska, 2022-2023.
2. L. R. Rabiner and R. W. Schafer: Introduction to Digital Speech Processing. Foundations and Trends in Signal Processing. vol. 1, no. 1-2 (2007), pp. 1-194, NOW - the essence of knowledge, Boston - Delft. (Roz. 1-6, 9)
3. Man-Wai Mak and Jen-Tzung Chien: Machine Learning for Speaker Recognition, INTERSPEECH 2016 Tutorial.
4. Uday Kamath, John Liu, James Whitaker: Deep Learning for NLP and Speech Recognition. Springer Nature Switzerland AG, 2019 (Roz. 3, 8, 12)
5. Aurelien Geron: Hands on machine learning with Scikit-Learn, Keras and TensorFlow. 2nd ed., O'Reilly Media Inc., Sebastopol, CA, 2019.
6. W. Kasprzak: Adaptive computation methods in digital image sequence analysis. (Rozdziały 5-9). Prace Naukowe - Elektronika, Nr. 127 (2000), Oficyna Wydawnicza PW, Warszawa.
7. G.E. Hinton et al.: Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process. Mag., 29(6), 82–97, 2014