Unless otherwise specified, questions have unit value. The total value of the
assignments from each week will vary substantively.
Recall that assignments are graded rather loosely on effort, and that 3/4 of
the total marks (1/2 for ugrads) over all assignments over all weeks
represents 100%. This policy is in place partly to allow for error in the
grading approach which, by necessity, is somewhat subjective, and needs to be
done somewhat superficially. It is recommended (and requested) that you try to
overshoot the 3/4 requirement, rather than worry about the details of how the
grading is done.
Problems denoted EXTRA can be substituted for other problems, or done in
addition, but they do not count towards the computation of the 3/4
requirement. They may be discussed in class depending on time and interest.
They are problems that I think might be useful, and likely be assigned if we
had more time per chapter.
Sometimes you will explicitly have to choose some of your own problems. Even
when this is not the case, you can substitute some problems in the book if
they appear more helpful to you. For now, limit the number of substitutions to
50% of what you hand in. This parameter may be increased or decreased as we go
on.
You are encouraged to discuss the problems with your peers, but I would like
individual final submissions demonstrating effort and understanding of what
was done. If you end up working closely with someone on a problem set, make a
note on your submission saying who it was.
Since this is graduate level research course that is graded predominately on
effort, I am confident that there will not be any problems with academic
honesty. However, do note that non-negligible deviations are often
surprisingly easy to spot, and can be verified by discussing the submitted
solutions with the student.
-----------------------------------------------------------------------------
Problems for Week 2, due Thursday, September 11.
Total value is 10.
-----------------------------------------------------------------------------
1. Problem 2.1 in book.
2. Explain in your own words the essence of "conjugate priors". Give some
examples.
ZERO VALUE (don't hand in anything).
*. Make sure you understand Figure 2.7 and Equation 2.52. (Personally, I am
happy jumping from 2.44 to proposing a coordinate transformation based on
the eigenvalue decomposition of the covariance matrix as reflected in
2.52; the interveening steps did not do too much for me).
3. (Linear algebra warm-up).
a) If the eigenvector decomposition of a matrix S is
given by S = U L U', derive the eigenvector decomposition of inv(S).
b) Go from 2.48 backwards to 2.45
QUADRUPLE VALUE
4. The data file
linked here
comes from a 2D multivariate Gaussian. The following is likley best done
using either Matlab, C/C++, R, or Mathematica. Since you do NOT have to
hand in code, I am not particular regrading your choice of tools. If you
have absolutely no opinion, I suggest using Matlab. If you want to try
Matlab, but have no prior experience, then you will have to spend some time
getting familiar with it, which is likely time well spent. Some information
about getting started is
linked here.
Specific commands that might prove useful in this assignment include:
HELP
LOAD -ASCII
PLOT
HIST, HIST3
EIG
a) Plot the data. Either a 2D scatter plot or 2D histogram (3D plot) is
fine. Include the plot in your writeup.
b) Compute the sample mean and covariance (of the data). Report your
values.
c) Subtract the sample mean from the data. Now consider the points where
the X ordinate is in the interval (-2.1,-1.9). Provide a plot for a
histogram of the Y values with x in inside that interval. Similarly, for
X inside (2.9,3,1).
d) From the above, make a case regrading the independence (or lack thereof)
of X and Y.
e) Construct a linear transformation of the data based on the eigenvalue
decomposition of the covariance matrix that maps the data towards a
Gaussian distribution where X and Y are independent. Provide the
transformation, AND a plot of the transformed data.
5. Explain why a function with an exponent of the form given by 2.71 is
Normal, regardless of what the constant is. (This justifies some of the
bits that follow in the text---I think that the author could have added
one or two more sentences here).
6. Problem 2.57
7. Evaluate the two components in the vector equation 2.228 in the Guassian
case, and argue that this is what you expected the answer to be. (For
example, you could give pointers to previous results in the book derived
differently).
ZERO VALUE (don't hand in anything)
*. If you did not get to section 2.4.3, read the first half of it, so you
know what is meant by non-informative prior and improper prior.
ZERO VALUE (don't hand in anything)
*. If you did not get to section 2.5, read it, and make sure that you
understand what non-parametric methods are about, particularly kernel
methods and nearest neighbor methods.
EXTRA
8. Derive 2.76. (This is similar to, but not much harder than problem 2.24,
but is more satisfying than just checking that the expression is correct).
EXTRA
9. Problem 2.40. Make sure that your solution is consistent with 2.141 and
2.142.