Instructions

Please make sure your final output file is a pdf document. You can submit handwritten solutions for non-programming exercises or type them using R Markdown, LaTeX or any other word processor. All programming exercises MUST be done in R, typed up clearly and with all code attached. Submissions should be made on gradescope: go to Assignments \(\rightarrow\) Homework 5.

Questions

  1. Simulate data assuming \(\boldsymbol{y_i} = (y_{i1},y_{i2})^T \sim \mathcal{N}_2(\boldsymbol{\theta}, \Sigma)\), \(i = 1, \ldots, 100\), with \(\boldsymbol{\theta} = (0,0)^T\) and \(\Sigma\) chosen so that the marginal variances are one and \(\rho = 0.8\).
    • Part (a): Estimate the MLE of \((\boldsymbol{\theta}, \Sigma)\). Make two contour plots, one using the true values of \((\boldsymbol{\theta}, \Sigma)\), and the other using the MLEs. Comment on the differences, if any.
    • Part (b): Assuming independent normal & inverse-Wishart priors for \(\boldsymbol{\theta}\) and \(\Sigma\), that is, \(\pi(\boldsymbol{\theta}, \Sigma) = \pi(\boldsymbol{\theta}) \pi(\Sigma)\), run Gibbs sampler (hyperparameters up to you but you must justify your choices) to generate posterior samples for \((\boldsymbol{\theta}, \Sigma)\).
    • Part (c): Compare Bayes estimates of \((\boldsymbol{\theta}, \Sigma)\) to MLE and truth. Comment on the similarities and differences. How much influence do you think your prior had on the Bayes estimates in particular?

    • Part (d): Given \(y_{i2}\) values for 50 new (test) subjects, describe in as much detail as possible, how you would use the Gibbs sampler to predict new \(y_{i1}\) values, given the \(y_{i2}\) values, from the “conditional posterior predictive distribution” of \((y_{i1} | y_{i2})\).

      You are not expected to derive the maximum likelihood estimators for the two parameters. If you don’t know what the MLEs are, you can look up the forms (just the forms!) online.

  2. Hoff problem 7.4.

    You can find the data mentioned in the question here: http://www2.stat.duke.edu/~pdh10/FCBS/Exercises/.

    For 7.4(d), part (ii) is NOT required but feel free to attempt it for practice; you are only required to submit answers to parts (i) and (iii). For 7.4(d)(i), there is no need to read or complete the entire Exercise 7.1, simply take the Jeffrey’s prior (same as the prior on the class slides) and find the corresponding full conditionals.

Grading

20 points.