Image and Speech Recognition

Lecture 30 h

Exercises 15 h

Laboratory (project) 15 h

3

Structure

The course consists of lecture, exercises and project (lab). The lecture is accompanied by computational tasks, solved during exercises, illustrating particular presented algorithms and methods. A project means the design and implementation of a computer program.

Course objectives

To study basic methods and algorithms in digital image- and speech-analysis. After completing this course students will be able to design

image and speech analysis programs dealing with pattern (image or speech) processing, pattern segmentation and object (or spoken word) recognition.

Prerequisities

• Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program, preferably in one of the languages: C/C++, Java, C, Python or Matlab.

• Familiarity with the basic mathematical analysis, linear algebra and probability theory.

In the first part elements of the pattern recognition theory are introduced. Among them are: distance metrics, adaptive data transformations - PCA, LDA, ICA, state estimation and regression, pattern clustering and classification (SVM, Bayes, k-NN, MLP, DNN).

The image analysis part covers image processing, image segmentation and object recognition solutions. Deep learning techniques for image analysis are introduced.

The area of speech recognition starts with basic audio signal processing in the time and frequency domain. Standard speech features are introduced (MFCC, LPC, PLP). The phonetic model of speech is explained. Classic and deep learning architectures for speech recognition are discussed.

Basic

[1] W. Kasprzak: Image and Speech Recognition. Lecture notes, Warsaw University of Technology, 2011-2022.

[2] A. Wilkowski: Neural Networks and Machine Learning in image recognition. Lecture notes, WUT, 2022.

Readings

[1] R. Duda, P. Hart, D. Stork: Pattern Classification. 2nd edition, John Wiley, 2001. (Chapters: 2, 3, 4, 10)

[2] Christopher M. Bishop: Pattern Recognition and Machine Learning. Springer Science-Business media, 2006. (Chapters: 1-5, 9, 12-14)

[3] R. C. Gonzales, Woods: Digital Image Processing. Prentice Hall, 2008 (3d edition), 2018 (4th edition). (Chapters: 3-6, 9-13)

[4] L. R. Rabiner and R. W. Schafer: Introduction to Digital Speech Processing.

Foundations and Trends in Signal Processing. NOW - the essence of knowledge, 2007, Boston - Delft. (Chapters 1-6, 9)

[5] U. Kamath, J. Liu, J. Whitaker: Deep Learning for NLP and Speech Recognition. Springer Nature Switzerland AG, 2019 (Chapters 3, 8, 12)

[6] Charu C. Aggarwal: Neural Networks and Deep Learning. A Textbook. Springer International Publishing, 2018

After completing this course students will be able to design image and speech analysis programs dealing with pattern (image or speech) processing, pattern segmentation and object (or spoken word) recognition.

Students are collecting assessment points. They come from a continuous assessment in the lecture time:

• two written tests (midterm and final) both for 0-30 pts.;

• exercise attendance and activity 0-10 pts.;

• a homework (project), evaluated in the project part (0-30 pts.).

There is an obligatory attendance of exercises and an optional attendance of the lecture. The Pass mark for this course will be set at 51 pts.

Credits will be awarded to candidates who pass this course.

The goal of each project work, is to design a particular pattern analysis system and to implement it as a program application in a programming language (C++, Java, Matlab, C#, Python are prefered). The pattern analysis system performs an image or speech recognition task.

Programming tools

[1] Open Source Computer Vision library - diverse image processing and analysis algorithms in C++ and Python interface. https://docs.opencv.org/master/

[2] Aurelien Geron: Hands on machine learning with Scikit-Learn, Keras and TensorFlow. 2nd ed., O'Reilly Media Inc., Sebastopol, CA, 2019.

[3] MATLAB: Image Processing Toolbox, Computer Vision Toolbox.

[4] Librosa. Audio and music processing in Python. https://librosa.org/

[5] HTK speech recognition toolkit. https://htk.eng.cam.ac.uk/