1 Introduction
Visualization type | References | Data type | Layout |
---|---|---|---|
Parallel coordinates plot | Abi Akle et al. (2019) Hofer et al. (2018) Netzel et al. (2017) Johansson and Forsell (2016) Kanjanabose et al. (2015) Wang et al. (2015) Harrison et al. (2014) Heer and Shneiderman (2012) Heinrich et al. (2012) Claessen and van Wijk (2011) Lehmann et al. (2010) Henley et al. (2007) Keim (2002) Shneiderman (1996) Inselberg and Dimsdale (1990) | Multi-attribute | Cartesian-coordinates based visualization |
Sunburst visualization | Hofer et al. (2018) Harrison et al. (2014) Rodden (2014) Kim and Draper (2014) Mansmann et al. (2012) Diehl et al. (2010) Stab et al. (2010) Draper et al. (2009) Keim et al. (2006) Stasko and Zhang (2000) | Hierarchical | Polar-coordinates based visualization |
Polar coordinates plota | Perkhofer et al. (2019a) Liu et al. (2017) Albo et al. (2016) Harrison et al. (2014) Claessen and van Wijk (2011) Draper et al. (2009) Elmqvist et al. (2007) | Multi-attribute | Polar-coordinates based visualization |
Treemap visualization | Perkhofer et al. (2019a) Wang et al. (2015) Kim and Draper (2014) Bostock et al. (2011) Keim et al. (2006) Songer et al. (2004) Bruls et al. (2000) Johnson et al. (1991) | Hierarchical | Cartesian-coordinates based visualization |
Sankey visualization | Hofer et al. (2018) Chou et al. (2016) Rodden (2014) Riehmann et al. (2005) | Hierarchical | Cartesian-coordinates based visualization |
Heatmap visualization | Perkhofer et al. (2019a) Barter and Yu (2018) Perrot et al. (2017) Severino (2015) | Multi-attribute | Cartesian-coordinates based visualization |
2 Theoretical background and hypotheses
2.1 Encode: choosing the visual representation and design
2.1.1 Classification and description of frequently used Big Data visualizations
- lines, which are parallel to each other suggest a positive correlation,
- lines crossing in an X-shape suggest a negative correlation, and
- lines crossing randomly, show no particular relationship.
2.1.2 Possible factors influencing usability of Big Data visualizations
- H1a: The basic layout influences usability of a visualization.
- H1b: Cartesian-coordinate based visualization types outperform polar-coordinate based visualization types.
- H2a: The underlying dataset influences the usability of a visualization.
- H2b: Hierarchy based visualizations types outperform multi-attribute based visualization types.
- H3a: The task type influences the usability of a visualization.
- H3b: Users will perform better with a multi-attribute visualization than with a hierarchy-based visualization when confronted with the task type identify.
- H3c: Users will perform better with a hierarchy-based visualization than with a multi-attribute visualization when confronted with the task type summarize.
- H4: Previous experience/usage of the different visualization types positively effects usability.
2.2 Manipulate: using interaction to manipulate existing elements
- H5a: Interaction influences the usability of a visualization.
- H5b: Users will perform better with a highly interactive visualization than with a mostly static one.
3 Study design
3.1 Data sample
3.2 Manipulation of the independent variables
Task type | Proxy of task used for this task classification |
---|---|
Identify | Wine&Co9 sells fewer than 3.000 bottles of white wine Options: ● This statement is correct. ● This statement is incorrect. ● This statement cannot be answered |
Compare | Wine&Co11 sells more red wine than Schenkenfelder 2 Options: ● This statement is correct. ● This statement is incorrect. ● This statement cannot be answered |
Summarize | Overall, more white wine is sourced from North/and South America than from Europe Options: ● This statement is correct. ● This statement is incorrect. ● This statement cannot be answered |
3.3 Dependent variable usability
3.4 Control variable
Visualization type | Static (%) | Interactive (%) | |
---|---|---|---|
Sunburst visualization | Not at all | 46 | 60 |
Occasionally | 40 | 31 | |
Monthly | 8 | 5 | |
Weekly | 0 | 4 | |
Daily | 6 | 0 | |
Sankey visualization | Not at all | 51 | 51 |
Occasionally | 43 | 31 | |
Monthly | 4 | 10 | |
Weekly | 2 | 8 | |
Daily | 0 | 0 | |
Parallel coordinates plot | Not at all | 48 | 46 |
Occasionally | 40 | 38 | |
Monthly | 10 | 4 | |
Weekly | 2 | 10 | |
Daily | 0 | 2 | |
Polar coordinates plot | Not at all | 63 | 53 |
Occasionally | 29 | 27 | |
Monthly | 8 | 14 | |
Weekly | 0 | 4 | |
Daily | 0 | 2 |
3.5 Procedure
3.6 Participants
Visualization type | N | Task assessments | Gender (female %) | Age | Completed degree | Working experiencea | Visual analyticsb |
---|---|---|---|---|---|---|---|
Sunburst | 52 | 624 | 52% | 37.9 | Bachelor 83% Master 11% Doctorate 2% N/A 4% | Yes 94% No 2% N/A 4% | No exp. 21% Slightly 38% Moderately 39% Extremely 2% |
Sankey | 49 | 588 | 45% | 36.1 | Bachelor 86% Master 4% Doctorate 2% N/A 8% | Yes 88% No 6% N/A 6% | No exp. 12% Slightly 41% Moderately 45% Extremely 2% |
Polar coordinates plot | 49 | 588 | 40% | 39.3 | Bachelor 82% Master 16% Doctorate 0% N/A 2% | Yes 96% No 0% N/A 4% | No exp. 16% Slightly 43% Moderately 39% Extremely 2% |
Parallel coordinates plot | 48 | 576 | 35% | 37.1 | Bachelor 75% Master 19% Doctorate 0% N/A 6% | Yes 94% No 2% N/A 4% | No exp. 29% Slightly 36% Moderately 33% Extremely 2% |
4 Results
4.1 Results of interaction technique (per visualization type)
4.1.1 Descriptive statistics
Means | Effectivity (in %) | Efficiency (in s) | Satisfaction (1–5) | |||
---|---|---|---|---|---|---|
Static | Interactive | Static | Interactive | Static | Interactive | |
Sunburst | 68.7 | 67.4 | 86.2 | 52.6 | 2.0 | 2.7 |
Sankey | 72.7 | 72.2 | 82.7 | 55.4 | 2.5 | 3.2 |
Parallel coordinates | 72.3 | 75.4 | 83.3 | 48.5 | 1.8 | 3.0 |
Polar coordinates | 78.9 | 68.9 | 143.0 | 90.1 | 1.1 | 2.6 |
Ø | 73.4 | 71.0 | 99.8 | 62.3 | 1.8 | 2.9 |
Ø Cartesian-based | 72.5 | 73.8 | 83.03 | 51.97 | 2.1 | 3.1 |
Ø Polar-based | 74.2 | 70.0 | 116.86 | 63.23 | 1.5 | 2.7 |
∆ Layout | − 1.7 | 3.8 | − 33.83 | − 11.26 | 0.6 | 0.4 |
Ø Hierarchy based | 70.8 | 69.8 | 84.37 | 54.04 | 2.3 | 3.0 |
Ø Multi-attribute based | 75.7 | 75.7 | 114.13 | 59.56 | 1.4 | 2.9 |
∆ Dataset | − 4.9 | − 5.9 | − 29.76 | − 5.52 | 0.9 | 0.1 |
4.1.2 MANCOVA for evaluating effectivity, efficiency, and satisfaction
Effect | Test | Value | F | Sig. | Partial eta squared |
---|---|---|---|---|---|
Usage | Pillai’s Trace | 0.023 | 16.011 | 0.000 | 0.023 |
Wilks’ Lambda | 0.977 | 16.011 | 0.000 | 0.023 | |
VisType | Pillai’s Trace | 0.272 | 66.589 | 0.000 | 0.091 |
Wilks’ Lambda | 0.735 | 72.944 | 0.000 | 0.098 | |
Interaction | Pillai’s Trace | 0.285 | 266.168 | 0.000 | 0.285 |
Wilks’ Lambda | 0.715 | 266.168 | 0.000 | 0.285 | |
VisType x Interaction | Pillai’s Trace | 0.039 | 9.000 | 0.000 | 0.013 |
Wilks’ Lambda | 0.961 | 9.000 | 0.000 | 0.013 |
Effect | Dependent variable | Type III sum of squares | F | Sig. | Partial eta squared |
---|---|---|---|---|---|
Usage | RA_z | 0.746 | 0.747 | 0.387 | 0.000 |
RT_z | 0.180 | 0.214 | 0.644 | 0.000 | |
SAT_z | 0.816 | 134.610 | 0.000 | 0.063 | |
VisType | RA_z | 5.421 | 1.811 | 0.143 | 0.003 |
RT_z | 303.862 | 143.305 | 0.000 | 0.177 | |
SAT_z | 163.566 | 80.236 | 0.000 | 0.107 | |
Interaction | RA_z | 1.022 | 1.025 | 0.311 | 0.001 |
RT_z | 218.534 | 309.189 | 0.000 | 0.134 | |
SAT_z | 306.526 | 451.092 | 0.000 | 0.184 | |
VisType x Interaction | RA_z | 6.019 | 2.011 | 0.110 | 0.003 |
RT_z | 16.899 | 7.970 | 0.000 | 0.012 | |
SAT_z | 32.665 | 16.024 | 0.000 | 0.023 | |
R Squared | RA_z | 0.007 | |||
RT_z | 0.278 | ||||
SAT_z | 0.323 |
Dependent variable | Significant pairs only | Mean difference | Sig. |
---|---|---|---|
Efficiency RT_z | Static—interactive | − 0.662 | 0.000 |
Satisfaction SAT_z | Static—interactive | − 0.784 | 0.000 |
Dependent variable | Significant pairs only | Mean difference | Sig. |
---|---|---|---|
Effectivity RA_z | Hierarchy—multi-attribute | − 0.079 | 0.077 |
Efficiency RT_z | Sunburst—polar coordinates | 0.848 | 0.000 |
Sankey—polar coordinates | 0.869 | 0.000 | |
Parallel coordinates—polar coordinates | 0.911 | 0.000 | |
Polar—Cartesian | − 0.499 | 0.000 | |
Hierarchy—multi-attribute | 0.419 | 0.000 | |
Satisfaction SAT_z | Sunburst—Sankey | − 0.398 | 0.000 |
Sunburst—polar coordinates | 0.393 | 0.000 | |
Sankey—parallel coordinates | 0.407 | 0.000 | |
Sankey—polar coordinates | 0.384 | 0.000 | |
Parallel coordinates—polar coordinates | 0.384 | 0.000 | |
Polar—Cartesian | − 0.406 | 0.000 | |
Hierarchy—multi-attribute | 0.417 | 0.000 |
4.1.3 ANCOVA for evaluating our sum score for usability
Dependent variable | Type III sum of squares | F | Sig. | Partial eta squared |
---|---|---|---|---|
Usage | 472.451 | 24.472 | 0.000 | 0.012 |
VisType | 2228.604 | 38.480 | 0.000 | 0.055 |
Interaction | 3295.185 | 170.686 | 0.000 | 0.079 |
VisType × Interaction | 247.064 | 4.266 | 0.005 | 0.006 |
R squared | 0.147 |
Dependent variable | Significant pairs only | Mean difference | Sig. |
---|---|---|---|
Usability | Static—interactive | − 2.570 | 0.000 |
Dependent variable | Significant pairs only | Mean difference | Sig. |
---|---|---|---|
Usability | Sunburst—Sankey | − 1.207 | 0.020 |
Sunburst—parallel coordinates | − 1.133 | 0.037 | |
Sankey—polar coordinates | 2.203 | 0.000 | |
Parallel coordinates—polar coordinates | 2.129 | 0.000 | |
Polar—Cartesian | − 1.741 | 0.000 | |
Hierarchy—multi-attribute | 1.229 | 0.000 |
4.2 Results on task type (per interactive visualization type)
4.2.1 Descriptive statistics
Means | Effectivity (in %) | Efficiency (in s) | SATa | ||||
---|---|---|---|---|---|---|---|
Identify | Compare | Summarize | Identify | Compare | Summarize | ||
Sunburst | 51.1 | 77.2 | 80.4 | 49.0 | 55.5 | 54.0 | 2.7 |
Sankey | 57.1 | 74.4 | 84.7 | 53.2 | 52.7 | 60.2 | 3.2 |
Parallel Coord. | 81.0 | 67.9 | 77.4 | 50.1 | 46.8 | 48.7 | 3.0 |
Polar Coord. | 76.7 | 61.1 | 68.9 | 90.4 | 92.2 | 87.7 | 2.6 |
Ø | 66.3 | 70.2 | 77.4 | 60.9 | 62.1 | 64.2 | 2.9 |
Ø Hierarchy | 54.0 | 75.8 | 83.2 | 51.0 | 54.1 | 58.0 | 2.9 |
Ø Multi-attribute | 78.7 | 67.9 | 77.4 | 70.9 | 46.8 | 48.7 | 2.9 |
∆ Dataset | − 24.7 | 7.9 | 5.8 | − 19.9 | 7.3 | 9.3 | – |
4.2.2 MANCOVA for evaluating effectivity, efficiency and satisfaction
Effect | Test | Value | F | Sig. | Partial eta squared |
---|---|---|---|---|---|
Usage | Pillai’s Trace | 0.070 | 24.71 | 0.000 | 0.070 |
Wilks’ Lambda | 0.970 | 24.71 | 0.000 | 0.070 | |
VisType | Pillai’s Trace | 0.298 | 36.48 | 0.000 | 0.099 |
Wilks’ Lambda | 0.710 | 40.58 | 0.000 | 0.108 | |
TaskType | Pillai’s Trace | 0.011 | 1.82 | 0.091 | 0.005 |
Wilks’ Lambda | 0.989 | 1.82 | 0.091 | 0.005 | |
VisType x TaskType | Pillai’s Trace | 0.043 | 2.40 | 0.001 | 0.014 |
Wilks’ Lambda | 0.957 | 2.41 | 0.001 | 0.014 |
Effect | Dependent variable | Type III sum of squares | F | Sig. | Partial eta squared |
---|---|---|---|---|---|
Usage | RA_z | 2.081 | 2.114 | 0.146 | 0.002 |
RT_z | 0.105 | 0.377 | 0.539 | 0.000 | |
SAT_z | 46.534 | 70.307 | 0.000 | 0.066 | |
VisType | RA_z | 3.712 | 1.237 | 0.288 | 0.004 |
RT_z | 92.759 | 30.920 | 0.000 | 0.251 | |
SAT_z | 36.058 | 23.019 | 0.000 | 0.052 | |
TaskType | RA_z | 10.275 | 5.138 | 0.006 | 0.010 |
RT_z | 0.207 | 0.140 | 0.690 | 0.001 | |
SAT_z | 0.020 | 0.010 | 0.985 | 0.000 | |
VisType x TaskType | RA_z | 36.847 | 6.236 | 0.000 | 0.036 |
RT_z | 1.842 | 1.101 | 0.360 | 0.007 | |
SAT_z | 0.448 | 0.113 | 0.995 | 0.001 | |
R Squared | RA_z | 0.051 | |||
RT_z | 0.257 | ||||
SAT_z | 0.119 |
Dependent variable | Significant pairs only | Mean difference | Sig. |
---|---|---|---|
Effectivity RA_z | Identify—summarize | − 0.253 | 0.004 |
4.2.3 ANCOVA for evaluating our sum score for usability
Dependent variable | Type III sum of squares | F | Sig. | Partial eta squared |
---|---|---|---|---|
Usage | 153.233 | 7.981 | 0.005 | 0.008 |
VisType | 853.638 | 14.820 | 0.000 | 0.043 |
TaskType | 133.150 | 3.467 | 0.032 | 0.007 |
VisType x TaskType | 443.878 | 3.853 | 0.001 | 0.023 |
R Squared | 0.080 |
Dependent variable | Significant pairs only | Mean difference | Sig. |
---|---|---|---|
Usability_Sum | Identify—summarize | − 0.915 | 0.027 |
5 Conclusion and future work
5.1 Discussion and implications
Hypothesis | Dependent variable | Results |
---|---|---|
H1a: The basic layout influences usability of a visualization | Usability | True |
H1b: Cartesian-coordinate based visualization types outperform polar-coordinate based visualization types | Usability Effectivity Efficiency Satisfaction | True No significance True True |
H2a: The underlying dataset influences the usability of a visualization | Usability | Partly True |
H2b: Hierarchy based visualizations types outperform multi-attribute based visualization types | Usability Effectivity Efficiency Satisfaction | Partly True False True True |
H3a: The task type influences the usability of a visualization | Usability Effectivity Efficiency Satisfaction | True True Truea No significance |
H3b: Users will perform better with a multi-attribute visualization than with a hierarchy-based visualization when confronted with the task type identify | Effectivity | True |
H3c: Users will perform better with a hierarchy-based visualization than with a multi-attribute visualization when confronted with the task type summarize | Effectivity | No significance |
H4: Previous experience/usage of the different visualization types positively effects usability | Usability Effectivity Efficiency Satisfaction | True No significance No significance True |
H5a: Interaction influences the usability of a visualization | Usability | True |
H5b: Users will perform better with a highly interactive visualization than with a mostly static one | Usability Effectivity Efficiency Satisfaction | True No significance True True |