Engineering 8801/9881 Pattern Recognition, Spring 2006, MUN

General Course Information

Instructor: Dr. Charles Robertson

Office hours: EN-3026, Tuesdays and Thursdays from 8:30 - 9:30 PM, or by appointment

Textbook: Pattern Classification, 2nd Edition, by Duda, Hart, and Stork

Course syllabus

Lectures

Lecture 19 - PDF Notes (full) - Feature Extraction Example

Lecture 18 - PDF Notes (with blanks) - PDF Notes (full) - Feature Extraction

Lecture 17 - PDF Notes (with blanks) - PDF Notes (full) - Feature Selection

Lecture 16 - no electronic notes - Working on example given in lecture 15.

Lecture 15 - PDF Notes (with blanks) - PDF Notes (full) - Clustering

Lecture 14 - PDF Notes (with blanks) - PDF Notes (full) - Clustering

Lecture 13 - PDF Notes (with blanks) - PDF Notes (full) - Linear Discriminants

Lecture 12 - PDF Notes (with blanks) - PDF Notes (full) - Linear Discriminants

Lecture 11 - PDF Notes (with blanks) - (no 'full' PDF) - Bayesian Belief Nets

Lecture 10 - PDF Notes (with blanks) - PDF Notes (full) - Bayesian Belief Nets

Lecture 9 - No electronic notes - Handouts will be given in class.

Lecture 8 - PDF Notes (with blanks) - PDF Notes (full) - Bayes estimation (learning). Density estimation.

Lecture 7 - PDF Notes (with blanks) - PDF Notes (full) - Probabilistic Classification. Maximum Likelihood parameter estimation. Bias and Biased Estimates.

Lecture 6 - PDF Notes (with blanks) - PDF Notes (full) - Bayesian Classification. Maximum A Posterior Classifier. Loss Function. Conditional Risk.

Lecture 5 - PDF Notes - MICD. Distance based classification example.

Lecture 4 - PDF Notes - Equivariance feature weighting. Orthonormal whitening. MICD.

Lecture 3 - PDF Notes - Transformation of random variables. Distance based classification.

Lecture 2 - No electronic notes - Mathematical foundations - univariate normal distribution, multivariate normal distribution.

Lecture 1 - PDF Notes - Review of course syllabus. Introduction to pattern recognition, including industrial inspection example from chapter 1 of textbook. Detailed look at the three key steps of pattern recognition (sensing, feature extraction, and classification).

Practice Problems

Midterm 2005 - PDF

Final 2005 - PDF

Assignments

Assignment 5 - PDF - Posted on EN-3026 door for pick-up.

Assignment 4 - PDF

Assignment 3 - PDF

Assignment 2 - PDF

Assignment 1 - PDF

9881 Project

Purpose

To demonstrate a thorough understanding of a topic in the field of pattern recognition.

Overview

Students in 9881 must pick a problem where some area of pattern recognition can be used to solve it. You must research the topic, create an application, give a short presentation, and submit written reports. Implementation of pattern recognition algorithm(s) are necessary. Use Matlab, C++, or Java as your programming language, or check with me for suitability of other languages.

Deliverables

May 9 - Topics due. Submit via email your topics, which should be 2 or 3 sentences describing the problem you will be investigating.

May 18 - Abstract due. Submit via email a detailed description of your project, including the problem and anticipated pattern recognition technologies you will employ. Include a list of references (note that Wikipedia is not a valid reference).

June 20 - Progress report due. Submit via email. This report should be about 3 pages.

July 28 - Final reports due. Submit via email in PDF or Word format, as well as a physical copy. You will also have to submit code and executable applications.

Ideas for Topics

Image pattern analysis is a rich field, and there are lots of interesting project possibilities. Techniques: edge detection, texture classifications. Areas: character recognition (handwritten, printed, from scanned documents), document layout processing, photographs - counting number of people present, facial recognition, medical images, industrial inspection, affect of noise on images, finding objects in SAR images, recognizing sketches.

Other application areas: audio, video, biomedical, mechanical, geological, biometrics (use of biomedicine for security and identification), intrusion detection for networked computers, music, word-analysis (spoken, written), speech processing, speaker recognition, meteorological, web understanding.

Some possible approaches to solve the problem: artifical neural networks, decision trees, principal component analysis, discriminants, hidden Markov models, Bayesian belief networks, support vector machines, fuzzy classification, genetic algorithms, unsupervised learning (clustering).

Final Report Guidelines

Submit the final bound copy of your report to Box 55 on Friday, July 28th. It does not need to be formally bound, but please, do not simply staple the report.

The following are general guidelines for the final report. You are allowed to take some liberties with the format, but I am basically expecting a well-written, graduate-level report where you present the work that you did over the past semester. At the end of the report, I must have a clear understanding of what you accomplished. You should have taken some time to understand a particular pattern recognition problem, to implement in software a solution (in part or full), and then conducted some experiments to determine how well your solution worked. You should write about 30 to 40 pages, plus appendices.

Sections to include

  1. Title Page
  2. Statement of Academic Honesty (see below)
  3. Abstract (no longer than 1 page)
  4. Introduction (Objectives, Motivation, etc.)
  5. Background
  6. Experiment (or Methodology)
  7. Results
  8. Discussion
  9. Conclusion
  10. References

Statement of Academic Honesty

The first page after the title page should be a page with the following statement: "I vouch that the material contained within this report is my own, except where noted and attributed to the original source." Include your name and the date, and sign the submitted paper copy.

Other Comments

Be careful with your language - except in the Discussion and Conclusion sections, you should not use the personal pronoun 'I'. I prefer that you write with active language, but in any case, you should take care to use good grammar and proper spelling.

Write the abstract last - simply because the abstract is a summary of the report, and is much easier to write after the rest of the report is done. Choose a font size and line spacing that will allow me to easily read and add comments.

References

Journal of Machine Learning Research Homepage - All papers are freely downloadable.

Check the site ftp://ftp.wiley.com/public/sci_tech_med/pattern for diagrams that you might find informative.

A good article by Jain, Duin, and Mao: Statistical Pattern Recognition: A Review (you might need to be on the MUN campus to get access to the article)

Other sites

IEEE Transactions on Pattern Analysis and Machine Intelligence

PatRec Course at McGill - there is a good list of links under the course material section

Pattern Recognition course page at Concordia - check out the slides

A Matlab tutorial

Another Matlab tutorial