I have been struggling on a homework question where I have to compute the posterior density of a distribution. While I can compute the posterior, I believe I made a mistake because the area under the posterior density tends to infinity.
I have spent hours double checking my work and have had several friends also come up with the same answer, so I was hoping that someone could take a look at the work and either spot the mistake or explain what is going on.
The setup is as follows. Let:
- $y = (y_1,\ldots,y_N)$
- $p_x(x) = \exp(-x)$
- $p_{y_i|x} = xy_i^{x-1} = x\exp((x-1)\log{y_i})$ for $y_i \in (0,1)$ and $x>0$
Given this information, we can calculate $p_{x|y}(x|y)$ using Bayes' Rule as follows:
$$\begin{align} p_{x|y}(x|y) &\propto p_{y|x}(y|x)p_x(x) \\ &\propto \prod_{i=1}^N p_{y_i|x}(y_i|x)p_x(x)\\ &\propto \prod_{i=1}^N x\exp((x-1)\log{y_i})\exp(-x)\\ &\propto x^N\exp((x-1)\sum_{i=1}^N\log{y_i})\exp(-Nx)\\ &\propto x^N\exp(x(\sum_{i=1}^N\log{y_i}-N))\exp(\sum_{i=1}^N\log{y_i})\\ \end{align}$$
Letting $S=\sum_{i=1}^N\log{y_i}$, this results in:
$$\begin{align} p_{x|y}(x|y) = \frac{1}{Z}x^N\exp(x(S-N)\exp(-S) \end{align}$$
Where $Z$ denotes a normalization constant that can be calculated as:
$$\begin{align} Z &= \int_{x>0}p_{y|x}(y|x)p_x(x)dx \\ &= \int_{x>0}x^N\exp(x(S-N))\exp(-S)dx \\ &= \exp(-S)\frac{1}{N-S}\int_{x>0}x^N(N-S)\exp(-x(N-S))\exp(-S)dx \\ &= \exp(-S)\frac{1}{N-S}\mathbb{E}[X^N] \\ &= \exp(-S)\frac{1}{N-S}\frac{N!}{(N-S)^N}\\ &= \exp(-S)\frac{N!}{(N-S)^{N+1}}\\ \end{align}$$
Note that I have evaluated the above integral using the $N$th moment of an exponential random variable.
Putting everything together, this gives me:
$$\begin{align} p_{x|y}(x|y) &= \frac{x^N\exp(x(S-N))\exp(-S)}{\exp(-S)\frac{N!}{(N-S)^{N+1}}}\\ &= \frac{(N-S)^{N+1}}{N!} x^N\exp(x(S-N))\\ \end{align}$$
Unfortunately, this density approaches infinity as $x \rightarrow \infty$, and so it does not normalize to 1.
