1.

a)

See Question 8 of Chapter 3.

b)

First, note that

\[\begin{multline} S_{n}^{2}=\frac{1}{n-1}\sum_{i}\left(X_{i}-\bar{X}\right)^{2}=\frac{1}{n-1}\sum_{i}\left(X_{i}^{2}-2X_{i}\bar{X}+\bar{X}^{2}\right)\\ =\frac{1}{n-1}\sum_{i}X_{i}^{2}-\frac{n}{n-1}\bar{X}^{2}=c_{n}\frac{1}{n}\sum_{i}X_{i}^{2}-d_{n}\bar{X}^{2} \end{multline}\]

where \(c_{n}\rightarrow1\) and \(d_{n}\rightarrow1\). By the WLLN, \(n^{-1}\sum_{i}X_{i}^{2}\) and \(\bar{X}^{2}\) converge, in probability, to \(\mathbb{E}[X_{1}^{2}]\) and \(\mu^{2}\). By Theorem 5.5 (d), \(c_{n}n^{-1} \sum_{i}X_{i}^{2}\) and \(d_{n}\bar{X}^{2}\) converge, in probability, to the same quantities. Lastly, by Theorem 5.5 (a), \(S_{n}^{2}\) converges, in probability, to \(\mathbb{E}[X_{1}^{2}]-\mu^{2}=\sigma^{2}\).

2.

Suppose \(X_{n}\) converges to \(b\) in quadratic mean. By Jensen’s inequality,

\[\mathbb{E}\left[X_{n}-b\right]^{2}\leq\mathbb{E}\left[\left|X_{n}-b\right|\right]^{2}\leq\mathbb{E}\left[\left(X_{n}-b\right)^{2}\right]\rightarrow0\]

Therefore, \(\mathbb{E}X_{n}\rightarrow b\).

Next, note that

\[\mathbb{E}\left[\left(X_{n}-b\right)^{2}\right]=\mathbb{E}\left[X_{n}^{2}\right]-2b\mathbb{E}\left[X_{n}\right]+b^{2}=\mathbb{V}(X_{n})+\mathbb{E}[X_{n}]^{2}-2b\mathbb{E}\left[X_{n}\right]+b^{2}\]

Taking limits of both sides reveals \(\lim_{n}\mathbb{V}(X_{n})=0\). As for the converse, we can apply the limits \(\lim_{n}\mathbb{E}[X_{n}]=b\) and \(\lim_{n}\mathbb{V}(X_{n})=0\) directly to the equation above.

3.

First, note that

\[\begin{multline} \mathbb{E}\left[\left(\bar{X}-\mu\right)^{2}\right]=\mathbb{E}\left[\bar{X}^{2}-2\mu\bar{X}+\mu^{2}\right]=\mathbb{E}\left[\bar{X}^{2}\right]-\mu^{2}\\ =\frac{1}{n^{2}}\mathbb{E}\left[\sum_{i}X_{i}^{2}+\sum_{i,j\colon i\neq j}X_{i}X_{j}\right]-\mu^{2}=\frac{1}{n}\mathbb{E}\left[X_{1}^{2}\right]+\frac{n-1}{n}\mathbb{E}\left[X_{1}X_{2}\right]-\mu^{2}. \end{multline}\]

Taking the limit,

\[\mathbb{E}\left[\left(\bar{X}-\mu\right)^{2}\right]\rightarrow\mathbb{E}\left[X_{1}X_{2}\right]-\mu^{2}=\mathbb{E}\left[X_{1}\right]\mathbb{E}\left[X_{2}\right]-\mu^{2}=0.\]

4.

Let \(\epsilon>0\). For \(n\) sufficiently large,

\[\mathbb{P}(\left|X_{n}-0\right|>\epsilon)=\mathbb{P}(X_{n}>\epsilon)=\mathbb{P}(X_{n}=n)=1/n^{2}\rightarrow0\]

and hence \(X_{n}\) converges in probability. However,

\[\mathbb{E}\left[\left(X_{n}-0\right)^{2}\right]=\mathbb{E}\left[X_{n}^{2}\right]\ge\mathbb{E}\left[X_{n}^{2}I_{\{X_{n}=n\}}\right]=n^{2}\mathbb{P}(X_{n}=n)=1\]

and hence \(X_{n}\) does not converge in quadratic mean.

5.

It is sufficient to prove the second claim since convergence in quadratic mean implies convergence in probability. First, note that

\[\left(\frac{1}{n}\sum_{i}X_{i}^{2}-p\right)^{2}=\frac{1}{n^{2}}\sum_{i}X_{i}^{4}+\frac{1}{n^{2}}\sum_{i,j\colon i\neq j}X_{i}^{2}X_{j}^{2}-\frac{2}{n}p\sum_{i}X_{i}^{2}+p^{2}.\]

Taking expectations, and using the fact that \(X_i^k = X_i\) and \(\mathbb{E} X_i = p\),

\[\begin{multline} \mathbb{E}\left[\left(\frac{1}{n}\sum_{i}X_{i}^{2}-p\right)^{2}\right]=\frac{1}{n}\mathbb{E}\left[X_{1}^{4}\right]+\frac{n-1}{n}\mathbb{E}\left[X_{1}^{2}\right]\mathbb{E}\left[X_{2}^{2}\right]-2p\mathbb{E}\left[X_{1}^{2}\right]+p^{2}\\ =\frac{1}{n}p+\frac{n-1}{n}p^{2}-p^{2}\rightarrow p^{2}-p^{2}=0. \end{multline}\]

6.

Letting \(F\) denote the CDF of a standard normal distribution, by the CLT,

\[\begin{multline} \mathbb{P}\left(\frac{X_{1}+\cdots+X_{100}}{100}\geq68\right)\\ =\mathbb{P}\left(\frac{\sqrt{100}}{2.6}\left(\frac{X_{1}+\cdots+X_{100}}{100}-68\right)\geq0\right)\approx 1-F(0)=0.5. \end{multline}\]

7.

Let \(f>0\) be a function and \(\epsilon>0\) be a constant. Then,

\[\mathbb{P}(\left|f(n)X_{n}-0\right|>\epsilon)=\mathbb{P}(X_{n}>\epsilon/f(n))\leq\mathbb{P}(X_{n}\neq0)=1-\exp(-1/n)\rightarrow0.\]

It follows that \(f(n) X_n\) converges to zero in probability. Take \(f = 1\) for Part (a) and \(f(n) = n\) for (b).

8.

Letting \(F\) denote the CDF of a standard normal distribution, by the CLT,

\[\begin{multline} \mathbb{P}(Y<90)=\mathbb{P}(X_{1}+\cdots+X_{100}<90)\\ =\mathbb{P}\left(\frac{\sqrt{100}}{1}\left(\frac{X_{1}+\cdots+X_{100}}{100}-1\right)<-1\right)\approx F(-1) \end{multline}\]

9.

Let \(\epsilon>0\). Then,

\[\mathbb{P}(\left|X_{n}-X\right|>\epsilon)\leq\mathbb{P}(X_{n}\neq X)=1/n\rightarrow0.\]

Therefore, \(X_{n}\) converges in probability (and hence in distribution) to \(X\). On the other hand,

\[\begin{multline} \mathbb{E}\left[\left(X-X_{n}\right)^{2}\right]=\mathbb{E}\left[\left(X-e^{n}\right)^{2}I_{\{X_{n}\neq X\}}\right]\\ =\mathbb{E}\left[1-2Xe^{n}+e^{2n}\right]\mathbb{P}(X_{n}\neq X)=\frac{1+e^{2n}}{n}\rightarrow\infty. \end{multline}\]

10.

Since \(1\leq x^{k}/t^{k}\) whenever \(x\geq t>0\), it follows that

\[\mathbb{P}(Z>t)=\frac{1}{\sqrt{2\pi}}\int_{t}^{\infty}e^{-x^{2}/2}dx\leq\frac{1}{t^{k}}\frac{1}{\sqrt{2\pi}}\int_{t}^{\infty}|x|^{k}e^{-x^{2}/2}dx\leq\frac{\mathbb{E}\left[\left|Z\right|^{k}I_{\{Z>t\}}\right]}{t^{k}}.\]

Therefore,

\[\mathbb{P}(|Z|>t)=2\mathbb{P}(Z>t)\leq\frac{\mathbb{E}\left[\left|Z\right|^{k}\left(I_{\{Z>t\}}+I_{\{Z<-t\}}\right)\right]}{t^{k}}\leq\frac{\mathbb{E}\left[\left|Z\right|^{k}\right]}{t^{k}}.\]

11.

First, note that \(X\) is almost surely zero. Let \(\epsilon>0\) and \(Z\) be a standard normal random variable. Then,

\[\mathbb{P}(\left|X_{n}-X\right|>\epsilon)=\mathbb{P}(\left|X_{n}\right|>\epsilon)=\mathbb{P}(\left|Z\right|>\epsilon\sqrt{n})\leq\frac{\mathbb{E}\left[Z^{2}\right]}{\epsilon^{2}n}=\frac{1}{\epsilon^{2}n}\rightarrow0.\]

Therefore, \(X_{n}\) converges in probability (and hence in distribution) to zero.

12.

Let \(F\) be the CDF of an integer valued random variable \(K\). Let \(k\) be an integer. It follows that \(F(k)=F(k+c)\) for all \(0\leq c<1\). We use this observation multiple times below.

To prove the forward direction, suppose \(X_{n}\rightsquigarrow X\). By definition, \(F_{X_{n}}\rightarrow F_{X}\) at all points of continuity of \(F_{X}\). Therefore,

\[\begin{multline} \mathbb{P}(X_{n}=k)=F_{X_{n}}(k+1/2)-F_{X_{n}}(k-1/2)\rightarrow F_{X}(k+1/2)-F_{X}(k-1/2)\\=\mathbb{P}(X=k). \end{multline}\]

To prove the reverse direction, suppose \(\mathbb{P}(X_{n}=k)\rightarrow\mathbb{P}(X=k)\) for all integers \(k\). Let \(j\) be an integer and note that

\[F_{X_{n}}(j)=\sum_{k\leq j}\mathbb{P}(X_{n}=k)\rightarrow\sum_{k\leq j}\mathbb{P}(X=k)=F_{X}(j)\]

and hence \(X_{n}\rightsquigarrow X\) as desired.

13.

First, note that

\[F_{X_{n}}(x)=\mathbb{P}(\min\left\{ Z_{1},\ldots,Z_{n}\right\} \leq x/n)=1-\mathbb{P}(Z_{1}\geq x/n)^{n}.\]

If \(x\leq0\), it follows that \(F_{X_{n}}(x)=0\). Otherwise,

\[\begin{multline} \mathbb{P}(Z_{1}\geq x/n)^{n}=\left(1-\mathbb{P}(Z_{1}\leq x/n)\right)^{n}=\left(1-\int_{0}^{x/n}f(z)dz\right)^{n}\\ =\left(1-f(c_{n})\frac{x}{n}\right)^{n}=\left(e^{-f(c_{n})x/n}+O(n^{-2})\right)^{n}\rightarrow e^{-\lambda x}. \end{multline}\]

Therefore, \(F_{X_{n}}(x)\rightarrow(1-e^{-\lambda x})I_{(0,\infty)}(x)\) and hence \(X_{n}\) converges in distribution to an \(\operatorname{Exp}(\lambda)\) random variable.

14.

By the CLT

\[\frac{\sqrt{n}}{\sigma}\left(\bar{X}-\mu\right)=\frac{\sqrt{n}}{1/\sqrt{12}}\left(\bar{X}-\frac{1}{2}\right)\rightsquigarrow N(0,1).\]

Let \(g(x)=x^{2}\) so that \(g^{\prime}(x)=2x\). By the delta method,

\[\frac{\sqrt{n}}{\left|g^{\prime}(\mu)\right|\sigma}\left(g(\bar{X})-g(\mu)\right)=\frac{\sqrt{n}}{1/\sqrt{12}}\left(Y_{n}-\frac{1}{4}\right)\rightsquigarrow N(0,1).\]

15.

Define \(g:\mathbb{R}^{2}\rightarrow\mathbb{R}\) by \(g(x)=x_{1}/x_{2}\). Then, \(\nabla g(x)=(1/x_{2},-x_{1}/x_{2}^{2})^{\intercal}\). Define \(\nabla_{\mu}=\nabla g(\mu)\) for brevity. By the multivariate delta method,

\[\sqrt{n}\left(Y_{n}-\frac{\mu_{1}}{\mu_{2}}\right)\rightsquigarrow N(0,\nabla_{\mu}^{\intercal}\Sigma\nabla_{\mu})=N(0,\Sigma_{11}/\mu_{2}^{2}-2\Sigma_{12}x_{1}/x_{2}^{3}+\Sigma_{22}x_{1}^{2}/x_{2}^{4}).\]

16.

Let \(X_{n},X,Y\sim N(0,1)\) be IID with \(X_{n}=Y_{n}\). Trivially, \(X_{n}\rightsquigarrow X\) and \(Y_{n}\rightsquigarrow Y\). However, \(\mathbb{V}(X_{n}+Y_{n})=\mathbb{V}(2X_{n})=4\) while \(\mathbb{V}(X+Y)=2\) and hence \(X_{n}+Y_{n}\) does not converge in distribution to \(X+Y\).