Introduction
Methods
Dirichlet process Gaussian mixture models
Blocked Gibbs sampling scheme to fit DPGMM
Data preprocessing: turning big into small
Big data implementation of DPGMM
Results and discussion
Effect of quantization on posterior inference
The data
Analysis and results
C. No | C. Size | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 16.3 | 1.1 |
1.2
| 1.8 | 1.6 | 2.5 | 2.2 | 0.6 | 2.0 | 3.9 | 3.3 | 3.5 | 6.8 | 8.5 |
2 | 13.5 | 0.1 |
0.0
| 0.1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.2 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
3 | 13.3 | 3.1 |
2.0
| 2.8 | 2.2 | 6.6 | 4.5 | 2.6 | 1.1 | 3.4 | 6.3 | 20.3 | 16.9 | 14.1 |
4 | 12.8 | 9.3 |
10.0
| 19.7 | 16.8 | 14.9 | 32.8 | 14.6 | 17.5 | 19.8 | 50.1 | 31.3 | 18.4 | 26.0 |
5 | 9.7 | 19.6 |
16.4
| 15.9 | 10.8 | 11.6 | 14.6 | 7.4 | 7.6 | 9.7 | 5.1 | 17.5 | 12.2 | 13.0 |
6 | 7.0 | 18.6 |
13.4
| 11.8 | 7.9 | 8.3 | 12.0 | 13.8 | 12.4 | 4.1 | 0.7 | 1.8 | 3.6 | 3.6 |
7 | 6.2 | 0.9 |
1.9
| 3.0 | 19.6 | 3.3 | 9.5 | 34.5 | 27.8 | 25.0 | 22.7 | 5.8 | 16.7 | 12.5 |
8 | 6.0 | 8.3 |
9.0
| 9.0 | 4.4 | 3.3 | 5.9 | 4.0 | 6.3 | 3.1 | 1.4 | 6.5 | 4.0 | 3.9 |
9 | 5.5 | 22.0 |
31.8
| 24.3 | 11.7 | 24.8 | 7.6 | 13.0 | 13.9 | 11.5 | 3.4 | 5.6 | 8.5 | 6.2 |
10 | 3.5 | 0.1 |
0.1
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.0 |
11 | 2.2 | 6.6 |
6.5
| 5.4 | 20.3 | 9.9 | 6.4 | 2.8 | 9.0 | 13.6 | 1.8 | 3.1 | 6.1 | 5.8 |
12 | 1.9 | 0.8 |
0.3
| 0.3 | 0.3 | 0.8 | 1.7 | 0.4 | 0.5 | 0.8 | 0.8 | 1.3 | 2.4 | 2.5 |
13 | 0.9 | 4.8 |
5.5
| 1.7 | 0.9 | 0.8 | 0.0 | 2.2 | 0.4 | 0.5 | 0.1 | 0.4 | 0.2 | 1.1 |
14 | 0.4 | 2.9 |
1.2
| 2.4 | 1.9 | 7.4 | 0.8 | 3.0 | 1.1 | 1.6 | 2.5 | 1.8 | 0.2 | 1.2 |
15 | 0.2 | 1.6 |
0.3
| 0.5 | 0.0 | 1.7 | 0.0 | 0.0 | 0.0 | 0.2 | 0.6 | 0.5 | 0.0 | 0.3 |
16 | 0.2 | 0.3 |
0.1
| 0.4 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.4 | 0.5 |
17 | 0.1 | 0.0 |
0.1
| 0.0 | 0.0 | 0.0 | 0.8 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.2 |
18 | 0.1 | 0.0 |
0.0
| 0.5 | 1.6 | 4.1 | 0.8 | 1.2 | 0.0 | 1.3 | 0.1 | 0.7 | 3.5 | 0.3 |
19 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.2 | 1.5 | 1.1 | 0.0 | 0.0 | 0.2 |
20 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.0 | 0.1 | 0.0 | 0.0 | 0.1 |
21 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
22 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
23 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.3 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
24 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
25 | 0.0 | 0.0 |
0.0
| 0.1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
26 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Total incursions | 1788 |
690
| 923 | 316 | 121 | 361 | 502 | 856 | 613 | 4220 | 2436 | 1075 | 1206 |
C. No | C. Size | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 |
2007
| 2008 | 2009 | 2010 | 2011 | 2012 | 2013 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 16.4 | 37.8 | 28.6 | 34.7 | 18.7 | 27.3 | 18.8 |
14.0
| 18.3 | 21.9 | 5.4 | 9.9 | 14.3 | 14.4 |
2 | 15.6 | 11.8 | 15.7 | 17.1 | 27.8 | 26.4 | 26.3 |
20.6
| 16.3 | 26.5 | 26.6 | 15.3 | 25.3 | 26.6 |
3 | 14.2 | 4.4 | 5.7 | 5.2 | 5.7 | 6.6 | 2.2 |
0.2
| 1.2 | 1.6 | 2.7 | 4.4 | 8.8 | 8.7 |
4 | 13.2 | 0.3 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 |
5 | 7.2 | 2.2 | 5.8 | 6.1 | 19.3 | 2.5 | 8.4 |
39.9
| 30.8 | 9.0 | 52.2 | 53.5 | 19.0 | 25.4 |
6 | 6.4 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.2 |
7 | 5.7 | 15.3 | 18.0 | 15.9 | 9.8 | 18.2 | 19.0 |
16.4
| 16.6 | 25.2 | 7.3 | 9.8 | 18.8 | 13.1 |
8 | 5.3 | 0.2 | 0.3 | 0.3 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.1 | 0.4 | 0.9 |
9 | 4.1 | 23.0 | 19.0 | 14.3 | 7.0 | 6.6 | 9.8 |
1.6
| 5.8 | 6.5 | 0.9 | 1.4 | 2.0 | 4.2 |
10 | 3.7 | 0.2 | 0.1 | 1.0 | 1.3 | 0.8 | 2.8 |
0.0
| 0.2 | 0.3 | 0.1 | 0.2 | 0.3 | 0.1 |
11 | 3.5 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
12 | 2.4 | 3.8 | 4.1 | 4.4 | 10.4 | 9.1 | 5.9 |
5.6
| 10.6 | 3.6 | 1.5 | 2.5 | 6.0 | 2.9 |
13 | 1.8 | 0.3 | 0.6 | 0.3 | 0.0 | 2.5 | 3.9 |
0.2
| 0.0 | 2.3 | 0.1 | 1.2 | 2.3 | 0.9 |
14 | 0.3 | 0.3 | 0.3 | 0.0 | 0.0 | 0.0 | 1.4 |
1.4
| 0.1 | 0.3 | 1.4 | 0.7 | 1.4 | 0.7 |
15 | 0.2 | 0.5 | 1.5 | 0.4 | 0.0 | 0.0 | 1.4 |
0.2
| 0.1 | 1.5 | 1.1 | 1.0 | 1.2 | 1.8 |
16 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 1.1 | 0.6 | 0.0 | 0.0 | 0.0 |
17 | 0.0 | 0.0 | 0.3 | 0.2 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
18 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
19 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
20 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Total incursions | 1788 | 690 | 923 | 316 | 121 | 361 |
502
| 856 | 613 | 4220 | 2436 | 1075 | 1206 |
C. No | C. Size | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 |
2010
| 2011 | 2012 | 2013 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 21.9 | 11.6 | 12.8 | 17.0 | 25.3 | 16.5 | 26.9 | 25.1 | 15.6 | 19.6 |
67.2
| 40.5 | 25.7 | 26.0 |
2 | 21.9 | 1.7 | 2.0 | 2.5 | 4.4 | 0.8 | 2.2 | 0.0 | 0.5 | 0.3 |
0.5
| 0.1 | 2.0 | 1.5 |
3 | 11.8 | 29.4 | 24.4 | 24.2 | 27.2 | 25.6 | 19.3 | 14.2 | 13.9 | 14.2 |
7.2
| 9.9 | 12.9 | 13.5 |
4 | 11.2 | 2.0 | 2.5 | 3.9 | 3.5 | 8.3 | 5.9 | 2.0 | 3.2 | 11.1 |
0.6
| 0.9 | 2.5 | 4.6 |
5 | 10.8 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
6 | 10.7 | 30.6 | 25.1 | 27.1 | 15.2 | 8.3 | 23.0 | 15.8 | 16.3 | 10.6 |
3.9
| 6.9 | 13.5 | 10.7 |
7 | 5.1 | 9.4 | 13.5 | 10.5 | 12.3 | 25.6 | 13.2 | 23.8 | 18.4 | 34.7 |
18.9
| 34.7 | 30.2 | 28.7 |
8 | 1.5 | 7.0 | 9.6 | 6.9 | 7.3 | 3.3 | 1.1 | 1.6 | 3.7 | 0.8 |
0.0
| 0.6 | 1.6 | 1.2 |
9 | 1.1 | 2.9 | 4.1 | 1.6 | 3.2 | 3.3 | 2.0 | 0.6 | 2.1 | 2.3 |
0.7
| 1.7 | 1.6 | 1.6 |
10 | 0.9 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
11 | 0.7 | 2.9 | 2.8 | 3.6 | 1.3 | 0.0 | 2.8 | 3.0 | 1.2 | 0.7 |
0.6
| 0.9 | 2.7 | 2.5 |
12 | 0.6 | 0.0 | 0.0 | 0.2 | 0.0 | 0.0 | 0.0 | 0.2 | 1.4 | 0.2 |
0.0
| 0.0 | 0.0 | 0.0 |
13 | 0.5 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
14 | 0.4 | 1.2 | 1.5 | 1.8 | 0.0 | 7.4 | 1.1 | 10.0 | 22.6 | 5.2 |
0.3
| 3.2 | 7.0 | 9.4 |
15 | 0.3 | 0.4 | 0.1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.2 | 0.0 |
0.0
| 0.0 | 0.0 | 0.1 |
16 | 0.2 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.2 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
17 | 0.2 | 0.9 | 1.2 | 0.4 | 0.3 | 0.0 | 2.5 | 0.0 | 0.2 | 0.2 |
0.1
| 0.5 | 0.3 | 0.2 |
18 | 0.1 | 0.0 | 0.0 | 0.1 | 0.0 | 0.0 | 0.0 | 2.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
19 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.2 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
20 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.4 | 0.5 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
21 | 0.0 | 0.0 | 0.4 | 0.1 | 0.0 | 0.8 | 0.0 | 0.2 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
22 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
23 | 0.0 | 0.1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.1 | 0.1 |
24 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
25 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
26 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
27 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
0.0
| 0.0 | 0.0 | 0.0 |
Total incursions | 1788 | 690 | 923 | 316 | 121 | 361 | 502 | 856 | 613 |
4220
| 2436 | 1075 | 1206 |