Probability 6 -- Exponential Family Distribution
Is there a general form that summaries as many distributions as possible? Maybe exponential family is what you want.
The exponential family is a set of distributions that can be represented by a certain form. The probability density function of a \(k\) dimensional random vector \(\mathbf{x} = (x_1, x_2, ..., x_k)^{\intercal}\) with respect to a \(d\) dimensional vector parameter \(\boldsymbol{\theta} = (\theta_1, \theta_2, ..., \theta_d)^{\intercal}\) can be directly written in the form \[ f_X(\mathbf{x} \mid \boldsymbol{\theta}) = h(\mathbf{x}) \exp(\boldsymbol{\eta(\theta)} \cdot \mathbf{T}(\mathbf{x}) - A(\mathbf{x})). \] Here \(\boldsymbol{\eta(\theta)} = (\eta_1(\boldsymbol{\theta}), \eta_2(\boldsymbol{\theta}), ..., \eta_s(\boldsymbol{\theta}))^T\) is a \(s\) dimensional natural parameter defined on \(\boldsymbol{\theta},\) \(\mathbf{T}(\mathbf{x})\) is a \(s\) dimensional vector of sufficient statistic and \(A(\boldsymbol{\theta})\) is a log-partition function.
Below are some examples of selecting different parameters in exponential family to get distributions. [[TODO]]