Multivariate Distributions
Multivariate distributions are the distributions whose variate forms are Multivariate
(i.e each sample is a vector). Abstract types for multivariate distributions:
const MultivariateDistribution{S<:ValueSupport} = Distribution{Multivariate,S}
const DiscreteMultivariateDistribution = Distribution{Multivariate, Discrete}
const ContinuousMultivariateDistribution = Distribution{Multivariate, Continuous}
Common Interface
The methods listed below are implemented for each multivariate distribution, which provides a consistent interface to work with multivariate distributions.
Computation of statistics
#
Base.length
— Method
length(d::MultivariateDistribution) -> Int
Return the sample dimension of distribution d
.
#
Base.size
— Method
size(d::MultivariateDistribution)
Return the sample size of distribution d
, i.e (length(d),)
.
#
Base.eltype
— Method
eltype(::Type{Sampleable})
The default element type of a sample. This is the type of elements of the samples generated by the rand
method. However, one can provide an array of different element types to store the samples using rand!
.
#
Statistics.mean
— Method
mean(d::MultivariateDistribution)
Compute the mean vector of distribution d
.
#
Statistics.var
— Method
var(d::MultivariateDistribution)
Compute the vector of element-wise variances for distribution d
.
#
Statistics.cov
— Method
cov(d::MultivariateDistribution)
Compute the covariance matrix for distribution d
. (cor
is provided based on cov
).
#
Statistics.cor
— Method
cor(d::MultivariateDistribution)
Computes the correlation matrix for distribution d
.
#
StatsBase.entropy
— Method
entropy(d::MultivariateDistribution)
Compute the entropy value of distribution d
.
#
StatsBase.entropy
— Method
entropy(d::MultivariateDistribution, b::Real)
Compute the entropy value of distribution , w.r.t. a given base.
Probability evaluation
#
Distributions.insupport
— Method
insupport(d::MultivariateDistribution, x::AbstractArray)
If is a vector, it returns whether x is within the support of . If is a matrix, it returns whether every column in is within the support of .
#
StatsAPI.loglikelihood
— Method
loglikelihood(d::Distribution{ArrayLikeVariate{N}}, x) where {N}
The log-likelihood of distribution d
with respect to all variate(s) contained in x
.
Here, x
can be any output of rand(d, dims...)
and rand!(d, x)
. For instance, x
can be
-
an array of dimension
N
withsize(x) == size(d)
, -
an array of dimension
N + 1
withsize(x)[1:N] == size(d)
, or -
an array of arrays
xi
of dimensionN
withsize(xi) == size(d)
.
For multivariate distributions, the pdf value is usually very small or large, and therefore direct evaluation of the pdf may cause numerical problems. It is generally advisable to perform probability computation in log scale. |
Sampling
#
Base.rand
— Method
rand(::AbstractRNG, ::Sampleable)
Samples from the sampler and returns the result.
#
Random.rand!
— Method
rand!(::AbstractRNG, ::Sampleable, ::AbstractArray)
Samples in-place from the sampler and stores the result in the provided array.
In addition to these common methods, each multivariate distribution has its special methods, as introduced below. |
Distributions
#
Distributions.Multinomial
— Type
The Multinomial distribution generalizes the binomial distribution. Consider n independent draws from a Categorical distribution over a finite set of size k, and let where represents the number of times the element occurs, then the distribution of is a multinomial distribution. Each sample of a multinomial distribution is a k-dimensional integer vector that sums to n.
The probability mass function is given by
Multinomial(n, p) # Multinomial distribution for n trials with probability vector p
Multinomial(n, k) # Multinomial distribution for n trials with equal probabilities
# over 1:k
#
Distributions.AbstractMvNormal
— Type
The Multivariate normal distribution is a multidimensional generalization of the normal distribution. The probability density function of a d-dimensional multivariate normal distribution with mean vector and covariance matrix is:
We realize that the mean vector and the covariance often have special forms in practice, which can be exploited to simplify the computation. For example, the mean vector is sometimes just a zero vector, while the covariance matrix can be a diagonal matrix or even in the form of . To take advantage of such special cases, we introduce a parametric type MvNormal
, defined as below, which allows users to specify the special structure of the mean and covariance.
struct MvNormal{T<:Real,Cov<:AbstractPDMat,Mean<:AbstractVector} <: AbstractMvNormal
μ::Mean
Σ::Cov
end
Here, the mean vector can be an instance of any AbstractVector
. The covariance can be of any subtype of AbstractPDMat
. Particularly, one can use PDMat
for full covariance, PDiagMat
for diagonal covariance, and ScalMat
for the isotropic covariance — those in the form of . (See the Julia package PDMats for details).
We also define a set of aliases for the types using different combinations of mean vectors and covariance:
const IsoNormal = MvNormal{Float64, ScalMat{Float64}, Vector{Float64}}
const DiagNormal = MvNormal{Float64, PDiagMat{Float64,Vector{Float64}}, Vector{Float64}}
const FullNormal = MvNormal{Float64, PDMat{Float64,Matrix{Float64}}, Vector{Float64}}
const ZeroMeanIsoNormal{Axes} = MvNormal{Float64, ScalMat{Float64}, Zeros{Float64,1,Axes}}
const ZeroMeanDiagNormal{Axes} = MvNormal{Float64, PDiagMat{Float64,Vector{Float64}}, Zeros{Float64,1,Axes}}
const ZeroMeanFullNormal{Axes} = MvNormal{Float64, PDMat{Float64,Matrix{Float64}}, Zeros{Float64,1,Axes}}
Multivariate normal distributions support affine transformations:
d = MvNormal(μ, Σ)
c + B * d # == MvNormal(B * μ + c, B * Σ * B')
dot(b, d) # == Normal(dot(b, μ), b' * Σ * b)
#
Distributions.MvNormal
— Type
MvNormal
Generally, users don’t have to worry about these internal details.
We provide a common constructor MvNormal
, which will construct a distribution of appropriate type depending on the input arguments.
#
Distributions.MvNormalCanon
— Type
MvNormalCanon
The multivariate normal distribution is an exponential family distribution, with two canonical parameters: the potential vector and the precision matrix . The relation between these parameters and the conventional representation (i.e. the one using mean and covariance ) is:
The canonical parameterization is widely used in Bayesian analysis. We provide a type MvNormalCanon
, which is also a subtype of AbstractMvNormal
to represent a multivariate normal distribution using canonical parameters. Particularly, MvNormalCanon
is defined as:
struct MvNormalCanon{T<:Real,P<:AbstractPDMat,V<:AbstractVector} <: AbstractMvNormal
μ::V # the mean vector
h::V # potential vector, i.e. inv(Σ) * μ
J::P # precision matrix, i.e. inv(Σ)
end
We also define aliases for common specializations of this parametric type:
const FullNormalCanon = MvNormalCanon{Float64, PDMat{Float64,Matrix{Float64}}, Vector{Float64}}
const DiagNormalCanon = MvNormalCanon{Float64, PDiagMat{Float64,Vector{Float64}}, Vector{Float64}}
const IsoNormalCanon = MvNormalCanon{Float64, ScalMat{Float64}, Vector{Float64}}
const ZeroMeanFullNormalCanon{Axes} = MvNormalCanon{Float64, PDMat{Float64,Matrix{Float64}}, Zeros{Float64,1,Axes}}
const ZeroMeanDiagNormalCanon{Axes} = MvNormalCanon{Float64, PDiagMat{Float64,Vector{Float64}}, Zeros{Float64,1,Axes}}
const ZeroMeanIsoNormalCanon{Axes} = MvNormalCanon{Float64, ScalMat{Float64}, Zeros{Float64,1,Axes}}
MvNormalCanon share the same set of methods as MvNormal .
|
#
Distributions.MvLogNormal
— Type
MvLogNormal(d::MvNormal)
The Multivariate lognormal distribution is a multidimensional generalization of the lognormal distribution.
If has a multivariate normal distribution then has a multivariate lognormal distribution.
Mean vector and covariance matrix of the underlying normal distribution are known as the location and scale parameters of the corresponding lognormal distribution.
#
Distributions.Dirichlet
— Type
Dirichlet
The Dirichlet distribution is often used as the conjugate prior for Categorical or Multinomial distributions. The probability density function of a Dirichlet distribution with parameter is:
# Let alpha be a vector
Dirichlet(alpha) # Dirichlet distribution with parameter vector alpha
# Let a be a positive scalar
Dirichlet(k, a) # Dirichlet distribution with parameter a * ones(k)
#
Distributions.Product
— Type
Product <: MultivariateDistribution
An N dimensional MultivariateDistribution
constructed from a vector of N independent UnivariateDistribution
s.
Product(Uniform.(rand(10), 1)) # A 10-dimensional Product from 10 independent `Uniform` distributions.
Addition Methods
AbstractMvNormal
In addition to the methods listed in the common interface above, we also provide the following methods for all multivariate distributions under the base type AbstractMvNormal
:
#
Distributions.invcov
— Method
invcov(d::AbstractMvNormal)
Return the inversed covariance matrix of d.
#
Distributions.logdetcov
— Method
logdetcov(d::AbstractMvNormal)
Return the log-determinant value of the covariance matrix.
#
Distributions.sqmahal
— Method
sqmahal(d, x)
Return the squared Mahalanobis distance from x to the center of d, w.r.t. the covariance. When x is a vector, it returns a scalar value. When x is a matrix, it returns a vector of length size(x,2).
sqmahal!(r, d, x)
with write the results to a pre-allocated array r
.
#
Base.rand
— Method
rand(::AbstractRNG, ::Distributions.AbstractMvNormal)
Sample a random vector from the provided multi-variate normal distribution.
#
Base.minimum
— Method
minimum(d::Distribution)
Return the minimum of the support of d
.
#
Base.maximum
— Method
maximum(d::Distribution)
Return the maximum of the support of d
.
#
Base.extrema
— Method
extrema(d::Distribution)
Return the minimum and maximum of the support of d
as a 2-tuple.
MvLogNormal
In addition to the methods listed in the common interface above, we also provide the following methods:
#
Distributions.location
— Method
location(d::MvLogNormal)
Return the location vector of the distribution (the mean of the underlying normal distribution).
#
Distributions.scale
— Method
scale(d::MvLogNormal)
Return the scale matrix of the distribution (the covariance matrix of the underlying normal distribution).
#
Statistics.median
— Method
median(d::MvLogNormal)
Return the median vector of the lognormal distribution. which is strictly smaller than the mean.
#
StatsBase.mode
— Method
mode(d::MvLogNormal)
Return the mode vector of the lognormal distribution, which is strictly smaller than the mean and median.
It can be necessary to calculate the parameters of the lognormal (location vector and scale matrix) from a given covariance and mean, median or mode. To that end, the following functions are provided.
#
Distributions.location
— Method
location{D<:AbstractMvLogNormal}(::Type{D},s::Symbol,m::AbstractVector,S::AbstractMatrix)
Calculate the location vector (the mean of the underlying normal distribution).
-
If
s == :meancov
, then m is taken as the mean, and S the covariance matrix of a lognormal distribution. -
If
s == :mean | :median | :mode
, then m is taken as the mean, median or mode of the lognormal respectively, and S is interpreted as the scale matrix (the covariance of the underlying normal distribution).
It is not possible to analytically calculate the location vector from e.g., median + covariance, or from mode + covariance.
#
Distributions.location!
— Method
location!{D<:AbstractMvLogNormal}(::Type{D},s::Symbol,m::AbstractVector,S::AbstractMatrix,μ::AbstractVector)
Calculate the location vector (as above) and store the result in
#
Distributions.scale
— Method
scale{D<:AbstractMvLogNormal}(::Type{D},s::Symbol,m::AbstractVector,S::AbstractMatrix)
Calculate the scale parameter, as defined for the location parameter above.
#
Distributions.scale!
— Method
scale!{D<:AbstractMvLogNormal}(::Type{D},s::Symbol,m::AbstractVector,S::AbstractMatrix,Σ::AbstractMatrix)
Calculate the scale parameter, as defined for the location parameter above and store the result in Σ
.
#
StatsAPI.params
— Method
params{D<:AbstractMvLogNormal}(::Type{D},m::AbstractVector,S::AbstractMatrix)
Return (scale,location) for a given mean and covariance
Product distributions
#
Distributions.product_distribution
— Function
product_distribution(dists::AbstractArray{<:Distribution{<:ArrayLikeVariate{M}},N})
Create a distribution of M + N
-dimensional arrays as a product distribution of independent M
-dimensional distributions by stacking them.
The function falls back to constructing a ProductDistribution
distribution but specialized methods can be defined.
product_distribution(dists::AbstractVector{<:Normal})
Create a multivariate normal distribution by stacking the univariate normal distributions.
The resulting distribution of type MvNormal
has a diagonal covariance matrix.
Using product_distribution
is advised to construct product distributions. For some distributions, it constructs a special multivariate type.