常見機率分布

若 A1,A2,A3 互相獨立，則 A1 與 A2∪A3 互相獨立。

證明

P(A1(A2∪A3))=P(A1A2∪A1A3)

=P(A1A2)+P(A1A3)−P(A1A2A3)

=P(A1)P(A2)+P(A1)P(A3)−P(A1)(A2A3)

=P(A1)[P(A2)+P(A3)−P(A2A3)]

=P(A1)P(A2∪A3)

Discrete Distributions

Binomial Distribution

Bin(k;n,θ)=(nk)θk(1−θ)n−k,k=0,...,n

Bernoulli Distribution

Ber(x;θ)=θk(1−θ)1−k,k=0,1

Multinomial Distribution

Mu(x|n,θ)=(nx1,⋯,xK)∏Kj=1θxjj,x=⎛⎝⎜⎜x1⋮xK⎞⎠⎟⎟

Multinoulli Distribution

Mu(x|1,θ)=∏Kj=1θxjj

Poisson Distribution

Poi(k|λ)=e−λλkk!

Empirical Distribution / Measure

Pemp(A)=1|D|∑x∈DΠ(x∈A)

Continuous Distributions

Gaussian Distribution

N(x;μ,σ2)=12πσ2−−−−√e−12σ2(x−μ)2

Φ(x;μ,σ2)=∫x−∞N(t;μ,σ2)dt

Φ(x;μ,σ2)=12[1+erf(z/2√)],where

z=x−μσ

erf(x)=2π√∫x0e−t2dt

Student t Distribution

T(t;μ,σ2,n)=1nπ−−−√Γ((n+1)/2)Γ(n/2)[1+1n(x−μσ)2]−n+12

F Distribution

U∼χ2(n1),V∼χ2(n2)

F=U/n1V/n2∼F(n1,n2):

f(x;n1,n2)=1B(x;n12,n22)(n1n2)n12xn12−1(1+n1n2x)n1+n22Π(x≥0),where

B(a,b)=Γ(a)Γ(b)Γ(a+b)

Laplace Distribution

Lap(x;μ,b)=12bexp(−|x−μ|b)

Gamma Distribution

Ga(x;shape=α,rate=β)=βαΓ(α)xα−1e−βxΠ(x≥0)

Γ(x)=∫+∞0tx−1e−tdt

Exponential Distribution

Expon(x;λ)=Ga(x;1,λ)=λe−λxΠ(x≥0)

Erlang Distribution

Erlang(x;λ)=Ga(x;2,λ)

Chi-squared Distribution

χ2(x;n)=Ga(x;n2,12)=2−n2Γ(n2)xn2−1e−12xΠ(x≥0)

Inverse Gamma Distribution

If X∼Ga(x;shape=α,rate=β),then1X∼IG(x;shape=α,scale=β):

IG(x;shape=α,scale=β)=βαΓ(α)x−(α+1)e−β/xΠ(x>0)

Beta Distribution

Beta(x;a,b)=1B(a,b)xa−1(1−x)b−1,where

B(a,b)=Γ(a)Γ(b)Γ(a+b)

Pareto Distribution

Pareto(x;k,m)=kmkx−(k+1)Π(x≥m)

Multivariate Gaussian

N(x;μ,Σ)=1(2π)D/2|Σ|1/2exp[−12(x−μ)⊺Σ−1(x−μ)]

Multivariate Student t Distribution

T((x;μ,Σ,n)=|Σ|−1/2(nπ)D/2Γ((n+D)/2)Γ(n/2)[1+1n(x−μ)⊺Σ−1(x−μ)]−n+D2

Σ:Scalematrix

Dirichlet Distribution

Dir(x;α)=1B(α)∏Kk=1xαk−1kΠ(x∈SK)

SK={x:0≤xk≤1,∑Kk=1xk=1}

Multivariate change of variables

fy(y)=fx(x)|det(∂x∂y)|