University of Arizona, Department of Computer Science

CSc 630

Advanced Topics in Software Systems (Spring 2003)
Multimedia data mining and retrieval

Time and Place MW 3:00-4:15, Gould-Simpson 701
Areas Emphasis will be placed on the interaction of image data with computer vision and machine learning. Thus this course will have a significant computer vision component, and should be considered by anyone interested in computer vision, machine learning, or artificial intelligence in general. Those interested in databases and human computer interfaces may also wish to consider this course.
Description Large multimedia data sets are becoming increasingly available, and they offer great opportunity, but our ability to exploit them is minimal. This course will focus on current approaches for searching, browsing, and mining various types of data such as text, web links, images, sound, video, and scientific collections. The focus will be on applying methods from machine learning and computer vision to these problems. In addition, computer vision will be studied as a data mining problem.
Pre requisite None other than the normal qualifications required to take graduate level computer science courses. However, note that some of the material is quite mathematical, and students should be prepared to struggle with it. More specifically, a side affect of this course is that a number of generally useful mathematical methods will be learnt/reviewed including:

  • Basic probability
  • Basic information theory
  • Principal component analysis
  • K-means (and other) clustering
  • Multi-dimensional scaling
  • Expectation-maximization
  • Instructor Kobus Barnard
    Office Hours MW, 1:30-2:30, 4:30-5:00, GS 730
    Text There are no required texts. The material is mostly available on-line.
    Format This course is research focused. Students will be expected to read a number of current papers, present and lead the discussion for several of them, and do a project. Research oriented projects will be strongly encouraged. Due to the inter-disciplinary nature of the course there should be plenty of scope to integrate projects with any of a number of research projects. Specifically projects emphasizing computer vision, machine learning, databases, and human computer interaction are all possible. Group projects will be encouraged. A short presentation of each project will be required.

    Before the day when specific papers will be presented, students should E-mail the instructor a short message containing feedback on the reading material. These can be summaries, comments, notes, questions for discussion, etc. Any format which helps you to think about the material is acceptable. (For some background material, answers to a few questions may be required instead).

    Each student will take the class for one lecture. The reading for that session should be presented with material from other sources as necessary. For example, if your paper is on some standard mathematical method. it would be of interest to do a quick survey of any on-line code that is available on the web. Within a few days of the presentation, the presenter should E-mail the instructor PDF slides for the session. The presenter should try to keep the sessions interactive, and leave plenty of time for group discussion (which is lead by the presenter). Thus the formal presentation need not exceed 40 minutes. If we decide as a class that it is easiest to cooperate on the printing of materials, then it will be up to the proposed presenter to distribute those materials at least one week in advance.

    Students will be expected to participate in the course in a number of other ways. By their nature the following activities are not required from every student for every meeting.

    Any comments/suggestions that you have on your peers' presentations should E-mailed to the instructor. I will distill and extend them, and pass them onto the appropriate parties.

    Participation in class discussion will be expected.

    Any other activities that help the aims of this class and/or the research done as part of your peers' projects can count towards participation. An example might be filling out a web survey which collects data on how well results fulfill queries.


  • Feedback on papers E-mailed to the instructor (15%)
  • Participation (15%)
  • Presentations (20%)
  • Project (50%)
  • Topics

    Project Milestones

    February 21: By this date you should have either proposed a project OR contacted me to discuss some of the options that may suit you.

    March 17: By this date you should have proposed a project.

    You are strongly encouraged to do a project which is useful to someone's research program--preferably your own! If you are actively doing research (say as part of a PhD or Master's thesis), you should be thinking of how some of the ideas and techniques that we have been discussing might apply to it. I will also be providing a number of project possibilities which may be of interest.

    Class Schedule

    On Jan 15 there will be an introductory meeting led by the instructor. For most subsequent meetings the class will be led by the presenter for that day. Students should sign up to lead the discussion for one of the days as soon as possible. If you need to change later, then arrange a swap with someone, and let the instructor know.

    Depending on the number of students presentating, some of the slots may be taken up by the instructor, guest lecturers, or project presentations. The list of papers will be under construction for much of the term, and may be modified to suit student interest or other reasons. However, every reasonable attempt will be made to restrict changes to slots at least two weeks in the future.

    A few papers are not available on-line. They will be made available in class by the instructor. Extra copies will be put outside the instructor's office.

    Note that a few of these links will only work from UA CS machines, as these papers are not available to the general public. (A few are semi-priveleged documents, and the others are available to the UA community through the UA library subscription to on-line journals. Most journal articles of interest to us are available in this way. If you are not already familiar with this service, I suggest that you start taking advantage of it.)

    Date Presenter Paper Slides
    Jan 22
    Tim Campbell Introduction to probability AND information theory handout Slides
    Jan 27
    Jason Addison PCA handout. If you are already comfortable with PCA, then I suggest you read this as well. Slides
    Jan 29
    Haiyan Qiao Indexing by latent semantic analysis (Deerwester et al.) Slides
    Feb 03
    Scott Morris A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models (Blimes).

    More sources of EM information provided by Scott

    Link to original version of Applet that was presented in class
    Overview with interesting examples
    Another intro to EM, explained using "lower bound"
    Least squares as MLE
    Feb 05
    Matt Johnson Unsupervised learning by proabablistic latent semantic analysis (Hofmann). For extra interest see this report. Slides
    Feb 10
    Sriraman Madapu The Mathematics of Statistical Machine Translation: Parameter Estimation (Brown et al.) Slides
    Feb 12
    Miriam Miklofsky Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary (Dugulu et al.). Additional information is available here. Slides
    Feb 17
    Ranjini Swamina The Earth Mover's Distance as a Metric for Image Retrieval (Rubner et al.) AND A Metric for Distributions with Applications to Image Databases (Rubner et al.) Slides
    Feb 19
    Manigantan Sethuraman Learning the semantics of words and pictures (Barnard et al.) AND Clustering Art (Barnard et al.) Slides
    Feb 24
    YongJun Cho Nonlinear dimensionality reduction by locally linear embedding (Roweis and Saul). Slides
    Feb 26
    GROUP A global geometric framework for nonlinear dimensionality reduction (Tenenbaum et al.)
    Mar 03
    GROUP Analysis of user need in image archives (Armitage and Enser) AND End-User Searching Challenges Indexing Practices in the Digital Newspaper Photo Archive (Makkulu and Sormunen).
    Mar 05
    Kobus Barnard Face recongnition using eigenfaces (Turk and Pentland) AND Trainable videorealistic speech animation (Ezzat et al.)
    Mar 10
    Nikhil Vasudev Blobworld: Color- and Texture-Based Image Segmentation Using EM and Its Application to Image Querying and Classification (Carson et al.) Slides
    Mar 12
    Sripriya Koma The Bayesian Image Retrieval System, PicHunter: Theory, Implementation and Psychological Experiments (Cox et al.)
    Mar 24
    GROUP A Tutorial on Support Vector Machines for Pattern Recognition (Burges). For extra interest, especially for SVM projects, have a look at The Relevance Vector Machine (Tipping).
    Mar 26
    Lei Zhu The PageRank Citation Ranking: Bringing Order to the Web (Page et al.) AND Authoritative sources in a hyperlinked environment (Kleinberg). If you are particularly interested in this topic take a look at this paper.
    Mar 31
    GROUP Clustering Methods for Collaborative Filtering (Lyle H. Unger and Dean P. Foster)
    Apr 02
    GROUP An Algorithm for Automated Rating of Reviewers (Tracy Riggs and Robert Wilensky).
    Apr 07
    GROUP Robust Hyperlinks: Cheap, Everywhere, Now (Thomas A. Phelps and Robert Wilensky) AND Robust Intra-document Locations (Thomas A. Phelps and Robert Wilensky )
    Apr 09
    GROUP Mining association rules between sets of items in large databases (Agrawal et al.) AND Temporal association rules: A survey (Lopez)
    Apr 14
    GROUP Shape matching and object recognition using shape context (Belongie et al.)
    Apr 16
    GROUP Shape matching continued.
    Apr 21
    GROUP Name-It: Naming and Detecting Faces in News Video (Satoh et al.)
    Apr 23
    GROUP Search the audio, browse the video--a generic paradigm for video collections (Amir et al.) [ I don't know of an electronic source, copies will provided in class and extra ones will be available outside my office. ]
    Apr 28
    GROUP An introduction to hidden Markov models (/ and Juang) [ I don't know of an electronic source, copies will provided in class and extra ones will be available outside my office. ]
    Apr 30

    May 05
    GROUP Project presentations
    May 07
    GROUP Project presentations