Please make sure your final output file is a pdf document. You can submit handwritten solutions for non-programming exercises or type them using R Markdown, LaTeX or any other word processor. All programming exercises MUST be done in R, typed up clearly and with all code attached. Submissions should be made on gradescope: go to Assignments \(\rightarrow\) Homework 3.
Hoff problem 5.2.
Suppose we have \(n\) independent observations \(Y = (y_1,y_2,\ldots,y_n)\), where each \(y_i \sim \mathcal{N}(\theta, \sigma^2)\), and the parameters \(\theta\) and \(\sigma^2\) are unknown. Jeffreys’ prior for \(\theta\) and \(\sigma^2\) (jointly) is \[\pi(\theta,\sigma^2) \propto (\sigma^2)^{-3/2}.\] Derive the posterior under this prior and state whether it is proper. What happens when \(n=1\) versus \(n>1\)?
You can either integrate directly to confirm it is proper (you probably shouldn’t), or try to put it into a form of a distribution or combination of distributions you can try to recognize (just like we have been doing in class). Also, do this in terms of the variance not the precision, i.e., keep the normal likelihood in terms of the variance before combining it with the prior.
Find the Jeffreys’ prior density for \(\theta\). Recall that for single parameter models, the Jeffreys’ prior is \(\pi(\theta) \propto \sqrt{\mathcal{I}(\theta)}\), where \(\mathcal{I}(\theta)\) is the Fisher information for \(\theta\). Use one of the two definitions of Fisher information on the slides to find \(\mathcal{I}(\theta)\), then set \(\pi(\theta) \propto \sqrt{\mathcal{I}(\theta)}\).
What values of \(a\) and \(b\) for the gamma density \(\textrm{Ga}(a,b)\) will result in a close match to the Jeffreys’ density you found?
Suppose we have count data \(y_i\) with \((i = 1,\ldots, n)\), where \(x_i = 0\) for control subjects and \(x_i = 1\) for treated subjects. Consider the following model for the data: \[y_i \sim \textrm{Poisson}(\lambda \gamma^{x_i}),\] where \(\lambda = \mathbb{E}[y_i | x_i = 0]\) and \(\gamma\) is a multiplicative change in the mean in the treated group. Choose gamma priors for the parameters \(\lambda\) and \(\gamma\), \[\lambda \sim \textrm{Ga}(1,1),\ \ \ \gamma \sim \textrm{Ga}(1,1).\] Is the joint posterior for \(\lambda\) and \(\gamma\) conjugate? Why or why not?
Just something to keep in mind: you will write your own sampler for this question in the next homework.
20 points.