Acknowledgements: Thanks to Ben S. for correcting some mistakes.

1.

a)

See Question 8 of Chapter 3.

b)

First, note that

\[\begin{multline} S_{n}^{2}=\frac{1}{n-1}\sum_{i}\left(X_{i}-\bar{X}\right)^{2}=\frac{1}{n-1}\sum_{i}\left(X_{i}^{2}-2X_{i}\bar{X}+\bar{X}^{2}\right)\\ =\frac{1}{n-1}\sum_{i}X_{i}^{2}-\frac{n}{n-1}\bar{X}^{2}=c_{n}\frac{1}{n}\sum_{i}X_{i}^{2}-d_{n}\bar{X}^{2} \end{multline}\]

where \(c_{n}\rightarrow1\) and \(d_{n}\rightarrow1\). By the WLLN, \(n^{-1}\sum_{i}X_{i}^{2}\) and \(\bar{X}^{2}\) converge, in probability, to \(\mathbb{E}[X_{1}^{2}]\) and \(\mu^{2}\). By Theorem 5.5 (d), \(c_{n}n^{-1} \sum_{i}X_{i}^{2}\) and \(d_{n}\bar{X}^{2}\) converge, in probability, to the same quantities. Lastly, by Theorem 5.5 (a), \(S_{n}^{2}\) converges, in probability, to \(\mathbb{E}[X_{1}^{2}]-\mu^{2}=\sigma^{2}\).

2.

Suppose \(X_{n}\) converges to \(b\) in quadratic mean. By Jensen’s inequality,

\[\mathbb{E}\left[X_{n}-b\right]^{2} \leq\mathbb{E}\left[\left|X_{n}-b\right|\right]^{2} \leq\mathbb{E}[(X_{n}-b)^2] \rightarrow0\]

Therefore, \(\mathbb{E}X_{n}\rightarrow b\). Next, note that

\[\mathbb{E}[(X_{n}-b)^2] =\mathbb{E}\left[X_{n}^{2}\right]-2b\mathbb{E}\left[X_{n}\right]+b^{2} =\mathbb{V}(X_{n})+\mathbb{E}[X_{n}]^{2}-2b\mathbb{E}\left[X_{n}\right]+b^{2}\]

Taking limits of both sides reveals \(\lim_{n}\mathbb{V}(X_{n})=0\). As for the converse, we can apply the limits \(\lim_{n}\mathbb{E}[X_{n}]=b\) and \(\lim_{n}\mathbb{V}(X_{n})=0\) directly to the equation above.

3.

Since the expectation of \(\bar{X}\) is \(\mu\) and the variance of \(\bar{X}\) converges to zero, the desired result is obtained by an application of our findings in Problem 2.

Alternatively, taking a more direct approach, note that

\[\begin{multline} \mathbb{E}\left[\left(\bar{X}-\mu\right)^{2}\right]=\mathbb{E}\left[\bar{X}^{2}-2\mu\bar{X}+\mu^{2}\right]=\mathbb{E}\left[\bar{X}^{2}\right]-\mu^{2}\\ =\frac{1}{n^{2}}\mathbb{E}\left[\sum_{i}X_{i}^{2}+\sum_{i,j\colon i\neq j}X_{i}X_{j}\right]-\mu^{2}=\frac{1}{n}\mathbb{E}\left[X_{1}^{2}\right]+\frac{n-1}{n}\mathbb{E}\left[X_{1}X_{2}\right]-\mu^{2}. \end{multline}\]

Taking the limit,

\[\mathbb{E}[(\bar{X}-\mu)^{2}] \rightarrow\mathbb{E}\left[X_{1}X_{2}\right]-\mu^{2} =\mathbb{E}\left[X_{1}\right]\mathbb{E}\left[X_{2}\right]-\mu^{2} =0.\]

4.

Let \(\epsilon>0\). For \(n\) sufficiently large,

\[\mathbb{P}(\left|X_{n}-0\right|>\epsilon)=\mathbb{P}(X_{n}>\epsilon)=\mathbb{P}(X_{n}=n)=1/n^{2}\rightarrow0\]

and hence \(X_{n}\) converges in probability. However,

\[\mathbb{E}\left[\left(X_{n}-0\right)^{2}\right]=\mathbb{E}\left[X_{n}^{2}\right]\ge\mathbb{E}\left[X_{n}^{2}I_{\{X_{n}=n\}}\right]=n^{2}\mathbb{P}(X_{n}=n)=1\]

and hence \(X_{n}\) does not converge in quadratic mean.

5.

It is sufficient to prove the second claim since convergence in quadratic mean implies convergence in probability. Similarly to Problem 3, we can define \(Y_i = X_i^2\) and apply our findings in Problem 2 to \(\bar{Y}\).

Alternatively, taking a more direct approach, note that

\[\left(\frac{1}{n}\sum_{i}X_{i}^{2}-p\right)^{2}=\frac{1}{n^{2}}\sum_{i}X_{i}^{4}+\frac{1}{n^{2}}\sum_{i,j\colon i\neq j}X_{i}^{2}X_{j}^{2}-\frac{2}{n}p\sum_{i}X_{i}^{2}+p^{2}.\]

Taking expectations, and using the fact that \(X_i^k = X_i\) and \(\mathbb{E} X_i = p\),

\[\begin{multline} \mathbb{E}\left[\left(\frac{1}{n}\sum_{i}X_{i}^{2}-p\right)^{2}\right]=\frac{1}{n}\mathbb{E}\left[X_{1}^{4}\right]+\frac{n-1}{n}\mathbb{E}\left[X_{1}^{2}\right]\mathbb{E}\left[X_{2}^{2}\right]-2p\mathbb{E}\left[X_{1}^{2}\right]+p^{2}\\ =\frac{1}{n}p+\frac{n-1}{n}p^{2}-p^{2}\rightarrow p^{2}-p^{2}=0. \end{multline}\]

6.

Letting \(F\) denote the CDF of a standard normal distribution, by the CLT,

\[\begin{multline} \mathbb{P}\left(\frac{X_{1}+\cdots+X_{100}}{100}\geq68\right)\\ =\mathbb{P}\left(\frac{\sqrt{100}}{2.6}\left(\frac{X_{1}+\cdots+X_{100}}{100}-68\right)\geq0\right)\approx 1-F(0)=0.5. \end{multline}\]

7.

Let \(f>0\) be a function and \(\epsilon>0\) be a constant. Then,

\[\mathbb{P}(\left|f(n)X_{n}-0\right|>\epsilon)=\mathbb{P}(X_{n}>\epsilon/f(n))\leq\mathbb{P}(X_{n}\neq0)=1-\exp(-1/n)\rightarrow0.\]

It follows that \(f(n) X_n\) converges to zero in probability. Take \(f = 1\) for Part (a) and \(f(n) = n\) for (b).

8.

Letting \(F\) denote the CDF of a standard normal distribution, by the CLT,

\[\begin{multline} \mathbb{P}(Y<90)=\mathbb{P}(X_{1}+\cdots+X_{100}<90)\\ =\mathbb{P}\left(\frac{\sqrt{100}}{1}\left(\frac{X_{1}+\cdots+X_{100}}{100}-1\right)<-1\right)\approx F(-1) \end{multline}\]

9.

Let \(\epsilon>0\). Then,

\[\mathbb{P}(\left|X_{n}-X\right|>\epsilon)\leq\mathbb{P}(X_{n}\neq X)=1/n\rightarrow0.\]

Therefore, \(X_{n}\) converges in probability (and hence in distribution) to \(X\). On the other hand,

\[\begin{multline} \mathbb{E}\left[\left(X-X_{n}\right)^{2}\right]=\mathbb{E}\left[\left(X-e^{n}\right)^{2}I_{\{X_{n}\neq X\}}\right]\\ =\mathbb{E}\left[1-2Xe^{n}+e^{2n}\right]\mathbb{P}(X_{n}\neq X)=\frac{1+e^{2n}}{n}\rightarrow\infty. \end{multline}\]

10.

Since \(1\leq x^{k}/t^{k}\) whenever \(x\geq t>0\), it follows that

\[\mathbb{P}(Z>t) =\mathbb{E}\left[I_{\{Z > t\}}\right] \leq\mathbb{E}\left[I_{\{Z > t\}} \left(\frac{Z}{t}\right)^k \right] \leq\frac{\mathbb{E}\left[I_{\{Z > t\}} \left|Z\right|^k \right]}{t^k}\]

Therefore, since the distribution is symmetric,

\[\mathbb{P}(|Z|>t) =2\mathbb{P}(Z>t) \leq\frac{\mathbb{E}\left[\left|Z\right|^{k}\left(I_{\{Z>t\}}+I_{\{Z<-t\}}\right)\right]}{t^{k}} \leq\frac{\mathbb{E}\left|Z\right|^{k}}{t^{k}}.\]

Note that we only used symmetry in establishing the above and hence the result is more general than the problem description implies.

11.

First, note that \(X\) is almost surely zero. Let \(\epsilon>0\) and \(Z\) be a standard normal random variable. Then,

\[\mathbb{P}(\left|X_{n}-X\right|>\epsilon)=\mathbb{P}(\left|X_{n}\right|>\epsilon)=\mathbb{P}(\left|Z\right|>\epsilon\sqrt{n})\leq\frac{\mathbb{E}\left[Z^{2}\right]}{\epsilon^{2}n}=\frac{1}{\epsilon^{2}n}\rightarrow0.\]

Therefore, \(X_{n}\) converges in probability (and hence in distribution) to zero.

12.

Let \(F\) be the CDF of an integer valued random variable \(K\). Let \(k\) be an integer. It follows that \(F(k)=F(k+c)\) for all \(0\leq c<1\). We use this observation multiple times below.

To prove the forward direction, suppose \(X_{n}\rightsquigarrow X\). By definition, \(F_{X_{n}}\rightarrow F_{X}\) at all points of continuity of \(F_{X}\). Therefore,

\[\begin{multline} \mathbb{P}(X_{n}=k)=F_{X_{n}}(k+1/2)-F_{X_{n}}(k-1/2)\rightarrow F_{X}(k+1/2)-F_{X}(k-1/2)\\=\mathbb{P}(X=k). \end{multline}\]

To prove the reverse direction, suppose \(\mathbb{P}(X_{n}=k)\rightarrow\mathbb{P}(X=k)\) for all integers \(k\). Let \(j\) be an integer and note that

\[F_{X_{n}}(j)=\sum_{k\leq j}\mathbb{P}(X_{n}=k)\rightarrow\sum_{k\leq j}\mathbb{P}(X=k)=F_{X}(j)\]

and hence \(X_{n}\rightsquigarrow X\) as desired.

13.

First, note that

\[F_{X_{n}}(x)=\mathbb{P}(\min\left\{ Z_{1},\ldots,Z_{n}\right\} \leq x/n)=1-\mathbb{P}(Z_{1}\geq x/n)^{n}.\]

If \(x\leq0\), it follows that \(F_{X_{n}}(x)=0\). Otherwise,

\[\begin{multline} \mathbb{P}(Z_{1}\geq x/n)^{n}=\left(1-\mathbb{P}(Z_{1}\leq x/n)\right)^{n}=\left(1-\int_{0}^{x/n}f(z)dz\right)^{n}\\ =\left(1-f(c_{n})\frac{x}{n}\right)^{n}=\left(e^{-f(c_{n})x/n}+O(n^{-2})\right)^{n}\rightarrow e^{-\lambda x}. \end{multline}\]

Therefore, \(F_{X_{n}}(x)\rightarrow(1-e^{-\lambda x})I_{(0,\infty)}(x)\) and hence \(X_{n}\) converges in distribution to an \(\operatorname{Exp}(\lambda)\) random variable.

14.

By the CLT

\[\frac{\sqrt{n}}{\sigma}\left(\bar{X}-\mu\right)=\frac{\sqrt{n}}{1/\sqrt{12}}\left(\bar{X}-\frac{1}{2}\right)\rightsquigarrow N(0,1).\]

Let \(g(x)=x^{2}\) so that \(g^{\prime}(x)=2x\). By the delta method,

\[\frac{\sqrt{n}}{\left|g^{\prime}(\mu)\right|\sigma}\left(g(\bar{X})-g(\mu)\right)=\frac{\sqrt{n}}{1/\sqrt{12}}\left(Y_{n}-\frac{1}{4}\right)\rightsquigarrow N(0,1).\]

15.

Define \(g:\mathbb{R}^{2}\rightarrow\mathbb{R}\) by \(g(x)=x_{1}/x_{2}\). Then, \(\nabla g(x)=(1/x_{2},-x_{1}/x_{2}^{2})^{\intercal}\). Define \(\nabla_{\mu}=\nabla g(\mu)\) for brevity. By the multivariate delta method,

\[\sqrt{n}\left(Y_{n}-\frac{\mu_{1}}{\mu_{2}}\right)\rightsquigarrow N(0,\nabla_{\mu}^{\intercal}\Sigma\nabla_{\mu})=N(0,\Sigma_{11}/\mu_{2}^{2}-2\Sigma_{12}\mu_{1}/\mu_{2}^{3}+\Sigma_{22}\mu_{1}^{2}/\mu_{2}^{4}).\]

16.

Let \(X_{n},X,Y\sim N(0,1)\) be IID with \(X_{n}=Y_{n}\). Trivially, \(X_{n}\rightsquigarrow X\) and \(Y_{n}\rightsquigarrow Y\). However, \(\mathbb{V}(X_{n}+Y_{n})=\mathbb{V}(2X_{n})=4\) while \(\mathbb{V}(X+Y)=2\) and hence \(X_{n}+Y_{n}\) does not converge in distribution to \(X+Y\).