Wikimaths

A collaborative site on maths but not only!

Outils pour utilisateurs

Outils du site


world:useful-bounds

$$ \newcommand{\arginf}{\mathrm{arginf}} \newcommand{\argmin}{\mathrm{argmin}} \newcommand{\argmax}{\mathrm{argmax}} \newcommand{\asconv}[1]{\stackrel{#1-a.s.}{\rightarrow}} \newcommand{\Aset}{\mathsf{A}} \newcommand{\b}[1]{{\mathbf{#1}}} \newcommand{\ball}[1]{\mathsf{B}(#1)} \newcommand{\bproof}{\textbf{Proof :}\quad} \newcommand{\bmuf}[2]{b_{#1,#2}} \newcommand{\card}{\mathrm{card}} \newcommand{\chunk}[3]{{#1}_{#2:#3}} \newcommand{\convprob}[1]{\stackrel{#1-\text{prob}}{\rightarrow}} \newcommand{\Cov}{\mathbb{C}\mathrm{ov}} \newcommand{\CPE}[2]{\PE\lr{#1| #2}} \renewcommand{\det}{\mathrm{det}} \newcommand{\dimlabel}{\mathsf{m}} \newcommand{\dimU}{\mathsf{q}} \newcommand{\dimX}{\mathsf{d}} \newcommand{\dimY}{\mathsf{p}} \newcommand{\dlim}{\Rightarrow} \newcommand{\e}[1]{{\left\lfloor #1 \right\rfloor}} \newcommand{\eproof}{\quad \Box} \newcommand{\eremark}{</WRAP>} \newcommand{\eqdef}{:=} \newcommand{\eqlaw}{\stackrel{\mathcal{L}}{=}} \newcommand{\eqsp}{\;} \newcommand{\Eset}{ {\mathsf E}} \newcommand{\esssup}{\mathrm{essup}} \newcommand{\fr}[1]{{\left\langle #1 \right\rangle}} \newcommand{\falph}{f} \renewcommand{\geq}{\geqslant} \newcommand{\hchi}{\hat \chi} \newcommand{\Id}{\mathrm{Id}} \newcommand{\img}{\text{Im}} \newcommand{\indi}[1]{\mathbf{1}_{#1}} \newcommand{\indiacc}[1]{\mathbf{1}_{\{#1\}}} \newcommand{\indin}[1]{\mathbf{1}\{#1\}} \newcommand{\itemm}{\quad \quad \blacktriangleright \;} \newcommand{\ker}{\text{Ker}} \newcommand{\klbck}[2]{\mathrm{K}\lr{#1||#2}} \newcommand{\law}{\mathcal{L}} \newcommand{\labelinit}{\pi} \newcommand{\labelkernel}{Q} \renewcommand{\leq}{\leqslant} \newcommand{\lone}{\mathsf{L}_1} \newcommand{\lrav}[1]{\left|#1 \right|} \newcommand{\lr}[1]{\left(#1 \right)} \newcommand{\lrb}[1]{\left[#1 \right]} \newcommand{\lrc}[1]{\left\{#1 \right\}} \newcommand{\lrcb}[1]{\left\{#1 \right\}} \newcommand{\ltwo}[1]{\PE^{1/2}\lrb{\lrcb{#1}^2}} \newcommand{\Ltwo}{\mathrm{L}^2} \newcommand{\mc}[1]{\mathcal{#1}} \newcommand{\mcbb}{\mathcal B} \newcommand{\mcf}{\mathcal{F}} \newcommand{\meas}[1]{\mathrm{M}_{#1}} \newcommand{\norm}[1]{\left\|#1\right\|} \newcommand{\normmat}[1]{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert #1 \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} \newcommand{\nset}{\mathbb N} \newcommand{\one}{\mathsf{1}} \newcommand{\PE}{\mathbb E} \newcommand{\PP}{\mathbb P} \newcommand{\Psif}{\Psi_f} \newcommand{\pscal}[2]{\langle #1,#2\rangle} \newcommand{\pscal}[2]{\langle #1,#2\rangle} \newcommand{\psconv}{\stackrel{\PP-a.s.}{\rightarrow}} \newcommand{\qset}{\mathbb Q} \newcommand{\rmd}{\mathrm d} \newcommand{\rme}{\mathrm e} \newcommand{\rmi}{\mathrm i} \newcommand{\Rset}{\mathbb{R}} \newcommand{\rset}{\mathbb{R}} \newcommand{\rti}{\sigma} \newcommand{\section}[1]{==== #1 ====} \newcommand{\seq}[2]{\lrc{#1\eqsp: \eqsp #2}} \newcommand{\set}[2]{\lrc{#1\eqsp: \eqsp #2}} \newcommand{\sg}{\mathrm{sgn}} \newcommand{\supnorm}[1]{\left\|#1\right\|_{\infty}} \newcommand{\thv}{{\theta_\star}} \newcommand{\tmu}{ {\tilde{\mu}}} \newcommand{\Tset}{ {\mathsf{T}}} \newcommand{\Tsigma}{ {\mathcal{T}}} \newcommand{\ttheta}{{\tilde \theta}} \newcommand{\tv}[1]{\left\|#1\right\|_{\mathrm{TV}}} \newcommand{\unif}{\mathrm{Unif}} \newcommand{\Xset}{{\mathsf X}} \newcommand{\Xsigma}{\mathcal X} \newcommand{\Yset}{{\mathsf Y}} \newcommand{\Ysigma}{\mathcal Y} \newcommand{\Var}{\mathbb{V}\mathrm{ar}} \newcommand{\zset}{\mathbb{Z}} \newcommand{\Zset}{\mathsf{Z}} $$

2017/10/07 23:39 · douc

Bounds on the tail of the normal distribution.

If $X_1$ follows a standard normal distribution, then $$ \PP(X_1>x)=\int_x^\infty \frac{1}{\sqrt{2\pi}} e^{-t^2/2} dt \leq \int_x^\infty \frac{t}{x} \frac{1}{\sqrt{2\pi}} e^{-t^2/2} dt =\frac{e^{-x^2/2}}{x\sqrt{2\pi}} $$

Bounds on the tail of a max distribution

Assume that $(X_i)$ are iid. Denote $M_n=\max_{i\in [1:n]} X_{i}$. $$ \PP(M_n>x)\leq \sum_{i=1}^n \PP(X_i>x) \leq n \PP(X_1>x) $$ In the case where the distribution of $X_1$ is standard normal, then \begin{equation} \label{eq:bound:max} \PP(M_n>x) \leq n \frac{e^{-x^2/2}}{x\sqrt{2\pi}} \end{equation} The bound is not bad in $x$ but not very nice in $n$.

Bounds on the moments of a max distribution

Let $(X_i)$ be iid standard gaussian random variables. Then, by Jensen's inequality, for all $\lambda \in (0,1/2)$, $$ \rme^{\lambda \PE[M_n^2]} \leq \PE[\rme^{\lambda M_n^2}] \leq n \PE[\rme^{\lambda X^2}]=n \int \frac{\rme^{-\frac{1-2\lambda}{2}x^2}}{\sqrt{2\pi}} \rmd x=n \lr{1-2\lambda}^{-1/2} $$ Taking the $\log$ and dividing by $\lambda$, we get: $$ \PE[M_n^2] \leq \frac{\log n-2^{-1}\log(1-2\lambda)}{\lambda} $$ Choosing $\lambda$ such that $\log n=-2^{-1}\log(1-2\lambda)$ yields for $n \geq 2$, $$ \PE[M_n^2] \leq \frac{4\log n}{1-n^{-2}}= 4 \log n +\frac{4 \log n}{n^2-1} \leq 4\log n +1 $$ With a similar argument, we can show that $$ \PE[M_n] \leq \sqrt{2 \log n} $$ Finally, a Markov inequality yields for all $x>0$ $$ \PP(M_n > x) \leq \frac{\sqrt{2 \log n}}{x} $$ which is better than the previous bound \eqref{eq:bound:max} wrt $n$ but dramatic wrt $x$…

Another inequality which can be useful for max distribution

Let $Z$ be a non-negative random variable on a probability space $(\Omega,\mcf,\PP)$ and assume that there exists a constant $c> 1$ such that for all $t\geq 0$, $$ \PP(Z>t)\leq c\rme^{-2nt^2} $$ Then, $$ \PE[Z] \leq \sqrt{ \frac{\log c +1}{2n} } $$

$\bproof$ \begin{align*} \PE[Z^2]&=\int_0^\infty \PP(Z>\sqrt{t})\rmd t \leq \int_0^\infty 1\wedge \lr{c\rme^{-2nt} } \rmd t =\frac{\log c}{2n} + \int_{\frac{\log c}{2n}} ^\infty c\rme^{-2nt} \rmd t =\frac{\log c}{2n} +\frac{1}{2n} \end{align*} The proof is completed by noting that $\PE[Z] \leq \sqrt{\PE[Z^2]}$ $\eproof$

Some comments on the approach

It may seem a bit convoluted to bound $\PE[Z]$ using a bound of $\PE[Z^2]$. I tried using a direct proof. \begin{align*} \PE[Z]&=\int_0^\infty \PP(Z>t)\rmd t \leq \int_0^\infty 1\wedge \lr{c\rme^{-2nt^2} } \rmd t =\sqrt{\frac{\log c}{2n}} + \int_{\sqrt{\frac{\log c}{2n}}} ^\infty c\rme^{-2nt^2} \rmd t \\ &\leq \sqrt{\frac{\log c}{2n}} +\int_{\sqrt{\frac{\log c}{2n}}} ^\infty \frac{t}{\sqrt{\frac{\log c}{2n}}} c\rme^{-2nt^2} \rmd t = \sqrt{\frac{\log c}{2n}} +\lrb{\frac{c\rme^{-2nt^2}}{-4n\sqrt{\frac{\log c}{2n}}} }_{\sqrt{\frac{\log c}{2n}}}^\infty \\ &=\frac{1}{\sqrt{2n}} \lr{\sqrt{\log c}+ \frac{1}{2 \sqrt{\log c}}} \end{align*} The bound is less sharp because on the second line, we only apply a rough bound on the survival function of a Gaussian distribution. And not surprisingly, the resulting bound is less sharp that the previous one because: $\sqrt{a+1} \leq \sqrt{a}+\frac{1}{2\sqrt{a}}$.

Kolmogorov inequality

Let \((M_k)_{k\in\nset}\) be a square integrable \((\mcf_k)_{k\in\nset}\)-martingale. Then, \begin{equation} \PP\lr{\sup_{k=1}^n |M_k| \geq \epsilon} \leq \frac{\PE[M_n^2]}{\epsilon^2} \end{equation}

$\bproof$

Let \(\sigma=\inf\set{k\geq 1}{|M_k| \geq \epsilon}\) with the convention that $\inf \emptyset=\infty$. Then, \begin{equation} \PP\lr{\sup_{k=1}^n |M_k| \geq \epsilon} =\PP(\sigma \leq n)=\PP\lr{|M_{\sigma \wedge n}| \geq \epsilon} \leq \frac{\PE[M^2_{\sigma \wedge n}]}{\epsilon^2} \label{eq:kolm:one} \end{equation} We first rewrite the rhs using that \((M_{\sigma \wedge k})_{k\in\nset}\) is also a \((\mcf_k)_{k\in\nset}\)-martingale. To see this last property, write \(M_{\sigma \wedge k}=\indiacc{k\leq \sigma} M_{k}+\indiacc{k> \sigma}M_\sigma\), which implies \begin{equation*} \PE\lrb{M_{\sigma \wedge k}|\mcf_{k-1}}=\indiacc{k\leq \sigma}\underbrace{\PE\lrb{M_{k}|\mcf_{k-1}}}_{M_{k-1}}+\indiacc{k> \sigma} M_\sigma=M_{\sigma \wedge (k-1)} \end{equation*} Now, the rhs of \eqref{eq:kolm:one} can be written using \begin{align*} {\PE[M^2_{\sigma \wedge n}]}&= {\PE[M_1^2]+\sum_{k=1}^{n-1} \PE\lrb{\lr{M_{\sigma \wedge (k+1)} - M_{\sigma \wedge k} }^2}}\\ &={\PE[M_1^2]+\sum_{k=1}^{n-1} \PE\lrb{\lr{M_{k+1} - M_{k} }^2 \indiacc{\sigma > k}}}\\ &\leq {\PE[M_1^2]+\sum_{k=1}^{n-1} \PE\lrb{\lr{M_{k+1} - M_{k} }^2 }}={\PE[M_n^2]} \end{align*} $\eproof$

world/useful-bounds.txt · Dernière modification: 2020/11/15 00:27 par rdouc