Dependent Dirichlet process

In the mathematical theory of probability, the dependent Dirichlet process (DDP) provides a non-parametric prior over evolving mixture models. A construction of the DDP built on a Poisson point process. The concept is named after Peter Gustav Lejeune Dirichlet. In many applications we want to model a collection of distributions such as the one used to represent temporal and spatial stochastic processes. The Dirichlet process assumes that observations are exchangeable and therefore the data points have no inherent ordering that influences their labeling. This assumption is invalid for modelling temporal and spatial processes in which the order of data points plays a critical role in creating meaningful clusters.

Dependent Dirichlet process

The dependent Dirichlet process (DDP) originally formulated by MacEachern led to the development of the DDP mixture model (DDPMM) which generalizes DPMM by including birth, death and transition processes for the clusters in the model. In addition, a low-variance approximations to DDPMM have been derived leading to a dynamic clustering algorithm.

Under time-varying setting, it is natural to introduce different DP priors for different time steps. The generative model can be written as follows:

[math]\displaystyle{ D_t \sim \operatorname{DP}(\alpha, H_t) }[/math]

[math]\displaystyle{ \theta_{t,i} \mid D_t \sim D_t ~~~ \text{ for }i=1,\ldots,n_t,~t=0,\ldots,T }[/math]

[math]\displaystyle{ X_{t:i} \mid \theta_{t,i} \sim F(\theta_{t:i}) ~~~ \text{ for } i=1,\ldots, n_t,~t=0, \ldots,T }[/math]

A Poisson-based construction of DDP exploits the connection between Poisson and Dirichlet processes. In particular, by applying operations that preserve complete randomness to the underlying Poisson processes: superposition, subsampling and point transition, a new Poisson and therefore a new Dirichlet process is produced.

References

• S. N. MacEachern, "Dependent Nonparametric Processes", in Proceedings of the Bayesian Statistical Science Section, 1999