Required libraries¶
In [1]:
!date
import warnings
warnings.simplefilter('ignore')
In [2]:
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
from model_selection.bayes_clustering import GMM_BC
Generate Toy Data¶
In [3]:
D = 2
K = 3
np.random.seed(10)
mus = np.random.normal(scale=3, size=(K, D))
covariances = np.empty((K, D, D))
for k in range(K):
covariances[k] = np.eye(D)
In [4]:
N_samples = [50]*K
plt.figure()
data = []
for mu, cov, n in zip(mus, covariances, N_samples):
nxy = np.random.multivariate_normal(mu, cov, size=n).astype("float32")
data.append(nxy)
plt.scatter(nxy[:,0], nxy[:,1])
data = np.concatenate(data)
Determine K using this Module¶
Model Definition¶
For now, this module provides the GMM(Gaussian Mixture Model) only.
The model setup is as follows:
Likelihood
\begin{align*} p(x\mid w,\mu,\Sigma) = \sum_{k=1}^{K} w_k\ {\rm Normal}(x\mid\mu_k, \Sigma_k), \quad \text{where} \sum_{k=1}^{K}w_k = 1 \end{align*}Initial Priors \begin{align*} \mu_k &\sim_{i.i.d} {\rm Normal}(\mathbb{0}, \mathbb{I}) \quad \text{for}\ k=1,\cdots,K\\ \Sigma_k &\sim_{i.i.d} {\rm LKJCholeskyCov}(\eta=2, sd) \quad \text{where}\ sd \sim {\rm HalfCauchy}(\beta = 2.5)\quad \text{for}\ k=1,\cdots,K\\ w &\sim {\rm Dirichlet}(\alpha = \underbrace{(1,1,\cdots,1)}_{K})\\ \end{align*}
We use NUTS as MCMC sampling method.
In [5]:
# K candidates you want to calculate the Bayes IC
K_cand = np.arange(2, 5+1)
# Create instance and Run MCMC sampling over K_cand
gmm = GMM_BC(K_cand, n_samples=2000)
gmm.fit(data)
In [6]:
# calculating the bayesian IC
# as a default, we use WAIC(smaller is better)
gmm.compare(gmm.result)
Out[6]: