1 Introduction
2 Background
2.1 Haptic feedback
2.2 Auditory feedback
2.3 Cross-modal perception and interaction
3 Method
3.1 Experimental setup
3.2 Pilot experiments
3.3 Experimental design
3.3.1 Participants
3.3.2 Compliance with ethical standards
3.3.3 Procedure
-
Strategies used when aiming towards the target
-
Perceived effects of auditory- versus haptic feedback
Event | Auditory feedback |
---|---|
Movement sonification
| |
Aiming movement 1) | velZ \(\rightarrow \) amplitude |
posX \(\rightarrow \) stereo panning | |
posY \(\rightarrow \) frequency | |
posZ \(\rightarrow \) rubbing force | |
Aiming movement 2) | velZ\(\rightarrow \) amplitude |
posX \(\rightarrow \) stereo panning | |
posY \(\rightarrow \) comb-filter delay time | |
posZ\(\rightarrow \) comb-filter feedback gain | |
Ball bouncing on: |
Impact sounds
|
Floor 1) | Heavy wooden impact sound |
Floor 2) | Light impact sound |
Wall 1), 2) | Metallic impact sound |
Earcons and auditory icons
| |
Scoring a goal 1), 2) | MIDI sequence, increasing pitch |
Missing the goal 1), 2) | MIDI sequence, decreasing pitch |
Grasping the ball 1), 2) | Band-pass-filtered noise with |
Sweeping f0 frequency \(+\) click |
3.3.4 Auditory feedback
3.3.5 Haptic feedback
3.4 Quantiative data
3.4.1 Performance measures
-
Error rate The total number of failed attempts to score a goal for 15 successful hits, i.e. the number of misses for a total of 15 goals.
-
Task duration The time, in seconds, required to successfully score 15 times.
3.4.2 Perceived intuitiveness
3.5 Qualitative data
3.6 Hypotheses
4 Results
friedman.test
function provided in R [40]. The Friedman test is a multisample extension of the sign test. If the Friedman test was significant, we proceeded with post-hoc analysis using multiple sign tests for two-sample paired data to compare specific feedback conditions (using SIGN.test
from the BSDA
package in R). Paired sign tests were used since the assumption of symmetry required for the paired Wilcoxon signed rank test was violated. Instead of using the very conservative Bonferroni correction of family-wise error for pairwise comparisons, we report estimates based on confidence intervals for the median of the difference (along with uncorrected p values). This procedure was used to avoid interpreting results solely based on null-hypothesis significance testing, since confidence intervals are much more informative than p values as they indicate the extent of uncertainty, as suggested by Cumming [7].4.1 Performance measures
4.1.1 Error rate
VH
|
\(AVH^1\)
|
\(AVH^2\)
|
V
|
\(AV^1\)
| |
---|---|---|---|---|---|
\(AVH^1\)
| .332 | ||||
[\(-\) 1.000, 3.000] | |||||
\(AVH^2\)
| 1.000 | .302 | |||
[\(-\) 4.708, 3.708] | [\(-\) 5.415, .708] | ||||
V
| .144 | .013 | .238 | ||
[\(-\) 15.491, .708] | [\(-\) 17.368, \(-\) 1.000] | [\(-\) 21.613, 1.000] | |||
\(AV^1\)
| 1.000 | .144 | .804 | .035 | |
[\(-\) 4.123, 1.708] | [\(-\) 7.538, .708] | [\(-\) 3.000, 2.415] | [.000, 10.830] | ||
\(AV^2\)
| 1.000 | .481 | .629 | .096 | .629 |
[\(-\) 8.123, 3.708] | [\(-\)8.708, 1.000] | [\(-\) 4.000, 3.123] | [\(-\) 0.415, 10.123] | [\(-\) 6.123, 4.706] |
4.1.2 Task duration
4.2 Perceived intuitiveness
Cond. |
VH
|
\(AVH^1\)
|
\(AVH^2\)
|
V
|
\(AV^1\)
|
\(AV^2\)
|
---|---|---|---|---|---|---|
Err. | 3.389 | 2.611 | 3.333 | 4.638 | 3.611 | 3.417 |
VH |
\(AVH^1\)
|
\(AVH^2\)
| V |
\(AV^1\)
| |
---|---|---|---|---|---|
\(AVH^1\)
| .167 [\(-\) .518, .018] | ||||
\(AVH^2\)
| .167 [\(-\) .282, .088] | 1.000 [\(-\) .241, .205] | |||
V
| .031 [.010, .708] | .019 [.154, .969] | .064 [.028, .867] | ||
\(AV^1\)
| .115 [\(-\) .438, .065] | .815 [\(-\) .087, .314] | .263 [\(-\) .130, .089] | .004 [\(-\) .810, \(-\) .102] | |
\(AV^2\)
| 1.000 [\(-\) .237, .310] | .210 [.000, .389] | .263 [\(-\) .099, .469] | .041 [\(-\) .679, \(-\) .103] | .359 [\(-\) .093, .413] |
5 Interview results
5.1 Effects of auditory feedback
Cond. |
VH
|
\(AVH^1\)
|
\(AVH^2\)
|
V
|
\(AV^1\)
|
\(AV^2\)
|
---|---|---|---|---|---|---|
Int. | 3.200 | 4.175 | 4.175 | 2.175 | 3.950 | 3.325 |