Skills, availability, willingness, expected participation and burden of sharing visual data within the frame of web surveys

There is not much evidence related to the skills of sharing visual data in the frame of web surveys. Bosch et al. (2019) found that around 25% of respondents did not know how to upload an image within a web survey for both in the moment and already stored images. Furthermore, in the study of Ilic et al. (2022), 12–15% of participants reported technical difficulties in uploading images taken in the moment. Regarding screenshots, Ohme et al. (2020) found that 61.5% of those who finished the survey did not successfully upload a screenshot, part of them sharing incomplete and imprecise content.

However, the lack of evidence addressing whether respondents have or not the skills to create and share photos, videos and screenshots from their smartphone and/or PC, prevents to know the extent to which respondents can be requested for visual data, especially considering the different ways to create and/or store them.

Regarding data produced during the survey by the device’s camera, the availability depends mainly on the possibility for respondents to create the visual data from the place where they answer the survey. The main limitation in this regard may be related to not being at the location from which the respondent is required to provide visual data. However, since surveys are mostly completed from home, even if respondents answer from smartphones (Revilla et al. 2016), the availability to produce visual data of things within the home may be high if respondents consider that their context allows them to do so. For instance, if they are asked for a picture of the TV or the last electricity bill, being at the house makes it more likely to take such picture or look for the bill (either paper or electronic). Furthermore, the device used to answer the survey might also affect the availability. In particular, in the case of PCs, the possibility to produce visual data in the moment might be lower.

The second kind of data produced during the survey are screenshots. Screenshots can be collected from most of the devices from which the survey can be answered, covering any topic that respondents can access through these devices. The availability may be limited by the size of the device screen, since they only cover the content visible in the screen. Moreover, some mobile apps restrict the use of screenshots by default for security reasons (e.g., some banks apps). However, data accessed through the browser and other apps can be extracted using this method, allowing to capture screenshots of almost everything viewed on mobile devices and PCs.

Regarding visual data already stored, its availability depends mainly on how respondents store them. There are plenty of options to store data: they can be saved automatically in smartphone galleries or clouds, or purposely stored in hard disk drives or PCs. Therefore, the limitations have to do with the chosen storing support, which may vary for the different types of visual data (e.g., photos taken with the smartphone camera are stored on the device itself, while photos taken with a digital camera are on a hard disk drive).

Moreover, within each option, the specific organization implemented by the individual might vary. For instance, some people might create sub-folders within their smartphones, whereas others might keep everything in the same folder. Then, if respondents are asked for a photo of something that happened months ago, they might have problems finding it, especially if they have a lot of visual data in their gallery.

Finally, people may store visual data useful to answer some topics but not others. Individuals mostly photograph other persons, food, pets, events (as weddings, concerts, and birthdays), cars and landscapes (Perry 2015). Also, nearly 15% of pictures are of practical things, such as receipts or shopping lists (Chennapragada 2018). Thus, although we expect a wide range of visual data to be available (especially in smartphones), we still need to learn more about the visual data that respondents store.

Previous studies about willingness have focused mainly on images taken by the device camera. In the United Kingdom, 65% of smartphone respondents in the Understanding Society Innovation Panel stated being willing to use their smartphone camera to take pictures or scan codes for a survey (Wenz et al. 2019). In Spain, 49.6% of respondents in the Netquest opt-in online panel declared that they would take photos with their smartphone and send them (Revilla et al. 2019). In the Netherlands, 38.2% of the LISS panel respondents stated being willing to share a photo of their house, 23.6% a video of the surroundings and 17.7% a selfie (Struminskaya et al. 2021).

The diversity of results does not allow to reach conclusive statements regarding willingness. Moreover, videos and screenshots have not been studied. Thus, further research is needed.

Different studies have asked respondents to share visual data created within the frame of web surveys. First, in the Understanding Society Spending Study sample (subsample of the Innovation Panel) from the 2,112 persons invited to download an app to scan and send receipts, only 10,2% did so at least once a week over the course of the five-week fieldwork (Jäckle et al. 2019).

Bosch et al. (2019), using the Netquest opt-in panel, requested millennials for images taken in the moment. In Spain, 48.6% of respondents sent an image while 24.2% skipped the question. The rest said they did not know how to share them. In Mexico, 57.6% sent images and 16.6% skipped the question.

In the Respondi opt-in online panel in Germany, higher item non-response was found for those asked to provide images taken in the moment than for those asked to answer by text (Bosch et al. 2022). Also, showing a motivational message increases the likelihood of the respondents to share an image.

In the Netherlands, two groups of respondents in the LISS panel were asked to answer sharing a photo or typing in a text, respectively, while a third one could choose the answering method (57% chose to share a photo). Between the two image groups there were no compliance differences: giving a choice did not affect the task completion. However, the completion rates for the text condition were significantly superior than those of the image groups (Ilic et al. 2022).

Regarding screenshots taken in the moment, one study in the Netherlands asked respondents of an online panel to send screenshots of the iOS Screen Time function included in iPhones. Only 11.6% of the sample successfully shared a screenshot with the information asked (Ohme et al. 2020). Interestingly, in another study in the United States asking this same task in several waves, 78% of the participants of the first wave successfully uploaded screenshots by the fourth wave (Sewall et al. 2022).

Finally, Bosch et al. (2019, 2022) also asked for already stored images. The former found that 54.7% of their sample of millennials in Spain sent an image whereas 21.8% skipped the task, while in Mexico 62.5% complied with the task and 15.2% decided not to answer. The latter found that between 48.4 and 74.9% of the respondents included in their experimental groups complied with sending an image (versus 97.8% to 99.1% in the text-based groups).

Respondents’ burden can be assessed in an objective (e.g., time and/or resources expended to provide answers) or subjective way (e.g., how respondents perceive the survey in terms of time, difficulty and stress). Burden can be affected by factors such as the survey length, the effort and capabilities required, and the respondents' motivation (Read 2019).

Regarding the burden associated to sending visual data, we expect it to be particularly related to the respondents’ skills and the availability of visual data. For instance, when asked for already stored data, depending on how respondents store their files and how many files they have stored, finding visual data may be perceived as easy/quick or long/complex. When asked for data captured during the survey using the device camera, the burden may vary depending on their smartphone usage competence and the conditions in which they answer the survey (e.g., place, others present, etc.).

Overall, Bosch et al. (2022) found that respondents providing images took more time to answer than those typing text, both when answering from PCs and smartphones, but little is known about the perceived burden itself.

The questionnaire included a maximum of 71 questions.³ After a few sociodemographic and contextual questions, respondents were asked if they would accept to participate in a survey if they could only complete it from a PC (yes/no; compulsory question) or only from a smartphone (yes/no; compulsory question). Respondents answering “yes” for both devices were presented with a set of similar questions about each device. The order of the sections was randomized: some respondents started with the questions about PC, others with the ones about smartphone. Respondents answering “yes” to only one of the two devices were shown only the questions about such device. Respondents answering “no” for both devices were filtered out. For the sake of simplicity, we will call those individuals answering the block of questions about smartphone “smartphone respondents”, and those who answer the block about PC “PC respondents”. In neither case the denomination refers to the device respondents used to answer the survey.

The main part of the questionnaire covered the four dimensions studied in the research questions (skills, availability, willingness and burden). They were presented in the following order so that the questions flowed intuitively and to reduce order effect:

Data were collected through the Netquest opt-in online panel in Spain (www.​netquest.​com) where participants get rewards each time they participate in a survey.

The target population included all people aged 18 years or older living in Spain who had access to the internet. Quotas for age, gender and education were used to get a sample similar on these variables to the overall adult internet population living in Spain, based on the estimations made by the National Statistics Institute of Spain.⁶ The survey was accessible from any device, and the layout was optimized for mobile devices. Respondents could continue without providing an answer to most questions.⁷

Data collection took place in May 2021. 1,581 individuals received the invitation to participate, while 1,376 started the survey. Of those starting the survey, 1,296 (94.2%) provided informed consent and were presented with the first survey question. 421 respondents (30.6%) were filtered out because quotas were full and 5 (0.4%) due to not passing basic anti-fraud checks. 4 (0.3%) abandoned the survey before the end and 9 (0.7%) stated they would not participate in a survey that had to be answered exclusively from a smartphone or from a PC. This left 857 respondents (62.3% of the ones who started) for the analyses. 796 (92.9%) answered the smartphone block, 691 (80.6%) the PC block, and 630 (73.5%) both. 50.5% of the participants were female. The mean age was 45.8 years. 34.9% had a higher degree. 68.6% answered using smartphones, 28.7% using PCs and 2.7% using other devices. The median completion time was 7.1 min.

The analyses were performed using R 4.0. The script is available at https://​osf.​io/​vfxbg.

Regarding visual data already stored, we report the percentages of availability for each topic (“yes”), making a distinction between the type of device (smartphone vs. PC) and kind of visual data (image vs. video). The proportions consider all respondents who saw this question (n = 796 for smartphone and n = 691 for PC).

The significance of the differences (at the 5% level) for the first four research questions was tested using Z-test when proportions were compared among different groups (e.g., those answering from home compared to those answering from their workplace) and McNemar’s test for comparisons within the same group (e.g., smartphone respondents) or between mostly similar ones (e.g., between devices, with 630 respondents in common). T-test for paired samples was used in the case of means.

Regarding RQ6, we performed regression analyses to assess whether and to what extent each of the dependent variables (levels of skills, availability, willingness, expected participation and burden as defined previously) is impacted by the following respondents' characteristics: gender (1 = women), age (numerical), level of education (using primary education or less as reference category against, independently, secondary education and tertiary education), and experience as a Netquest panelist (logarithm of the number of surveys completed in the last three months). We selected these variables because previous studies found they sometimes impact the willingness to perform innovative tasks during a survey (Revilla et al. 2019; Wenz et al. 2019).

Logistic regressions were used for skills, availability and expected participation (1 = have skills/is available/is expected to participate). For willingness, we also included the category "2 = it depends on the file". Therefore, we used a multinomial model. For the burden, we performed linear regressions with the perceived burden of answering  conventional survey questions and performing visual data-related tasks in the 0 to 4 scale as dependent variables, considering only respondents who answered all the burden questions. When an analysis had few cases (e.g., those answering from their workplace), regressions were not performed. Respondents who saw a question and did not answer it, were excluded from the regression analyses. For the sake of simplicity, we present the mean and standard deviation of the regression coefficients per dimension, which allows assessing the average magnitude of the coefficients and if the general effect was positive or negative. The full results of the 74 regressions performed (one for each type of skill, availability, willingness, expected participation, and burden, for both PC and smartphone), including the value of each coefficient and their significance, are available at https://​osf.​io/​kygp2.

To answer RQ1, Table 1 shows the percentage of respondents stating that they have the skills to create and share different types of visual data using their smartphone and/or PC.

First, almost all participants know how to take a photo (99.0%) and make a video (98.0%) using their smartphone. Second, the proportions of respondents knowing how to create screenshots and send already stored files with their smartphone are significantly lower (respectively 91.8% and 93.1%), although at least 9 out of 10 know how to perform them. All skills related to the smartphone are highly spread, even if there are differences among them.

Finally, significantly lower proportions of respondents declared knowing how to send an already stored file (86.8%) and create a screenshot (64.8%) using the PC than the smartphone. The latter one is the skill with the lowest presence among respondents.

As for RQ3, Table 4 presents the proportion of respondents who stated that they would be willing to share different types of visual data during a survey (“yes”), or that “it depends”.

First, the absolute willingness (“yes”) ranges from 37.2% (videos already stored in smartphone) to 62.1% (screenshots in smartphones). Lower figures are found among visual data already stored, whereas there are higher proportions of respondents stating absolute willingness for visual data produced during the survey. For screenshots and photos created with the smartphone the absolute willingness is over 60%, even when respondents were presented with the option “it depends”. Thus, these two methods are perceived positively from most respondents. Regarding visual data already stored, larger values of relative willingness (“it depends”) are found, especially among “smartphone respondents”.

Second, even if visual data produced during the survey have higher levels of “Yes” answers, the willingness to share screenshots (the only in-the-moment task studied for both devices) is significantly higher among “smartphone” than “PC respondents”.

Finally, more than 90% of respondents would be willing (always or in some conditions) to take a photo, screenshot, and send a stored image with their smartphone, whereas lower proportions would be willing to perform the tasks for videos and PCs. However, it is worth mentioning that all activities show a willingness higher than 80% when considering both “yes” and “it depends”.

To answer RQ5, Table 7 presents the mean perceived burden of answering different types of survey questions and performing visual data-related tasks measured using a scale from 0 (“No effort at all”) to 4 (“A huge effort”).

First, all questions and tasks present a very low perception of burden (max = 0.75). Second, even if the level of effort is perceived as small, open narrative questions have the highest perception of burden compared to visual data-related tasks in both devices. Third, making a video and taking a screenshot using the smartphone represent less effort than answering 5 and 10 radio button questions, whereas taking photos is also perceived as less burdensome than answering one radio button question. In that sense, performing visual data-related tasks would be perceived as less burdensome than answering conventional questions in a survey. When it comes to PCs, producing screenshots has a lower perception of burden than answering 10 radio button questions.

Lastly, the perception of effort associated with sending visual data already stored is higher than the one related to visual data created in the moment. Nevertheless, sharing a file is still perceived as less burdensome than responding 10 radio button questions or an open narrative question.

To answer our last research question, the results of the 74 regressions performed, including the value of each coefficient and their significance, are available at https://​osf.​io/​kygp2. As a way to summarize these results, Table 8 presents the average mean of the regression coefficients per each of the dimensions (skills, availability, willingness: yes, willingness: it depends, expected participation, and burden).⁸

First, gender impacts negatively in almost every dimension (the only exception is “willingness: it depends”), showing that women would have an overall more negative disposition than men, although they might find survey questions and visual data-related tasks less burdensome. However, the effects are significant in only few cases.

Second, age has a negative effect in all the dimensions and in both devices. Even if this is good news for burden (a negative sign means less burden), the mean coefficient in that dimension is too low ( − 0.001 for smartphone, and − 0.002 for PC) to suggest that this impact is relevant. Additionally, gender and age are the only variables negatively affecting the expected participation in both devices, but specially in smartphones: in these type of devices, gender and age are the two variables with more negative significant effects.

Third, middle and high education have overall positive effects in both devices compared to lower education. The effect of middle education is especially remarkable for participation from the PC, since their effect is always positive and significant for expected participation.

Higher participation in Netquest’s surveys during the last 3 months positively impacts almost all the dimensions in both devices. As expected since respondents already participate in a panel, the variable also increases the likelihood of the expected participation (skills, availability and willingness together) in smartphones and PCs (with more significant effects in smartphones).

Different aspects must be considered by researchers when deciding if it can be beneficial for their research and for their participants to ask for visual data in the frame of web surveys. In this paper, we focus on four dimensions that we considered particularly relevant (and their combination): skills, availability, willingness and burden.

Second, most respondents were in a situation to create visual data in the moment, particularly for things inside the house (RQ2). This is highly relevant considering that 83% of participants answered from their house. For already stored visual data (also RQ2), it is more likely to have visual data available when considering images (vs. videos), smartphones (vs. PCs), and landscapes and activities (vs. food and products).

Third, we found high levels of willingness to share visual data (RQ3). Indeed, even when respondents were proposed an “it depends” option, around 60% answered “yes”, showing an unconditional willingness to provide visual data. This is in line with what was previously found in the United Kingdom (Wenz et al. 2019). Nevertheless, previous research did not consider the conditional willingness (“it depends”). When taking this “it depends” option into account, our study hints to a higher disposition of respondents to share visual data in the frame of web surveys, with at least 8 in 10 participants who would eventually be willing to share visual data from smartphones and/or PCs, than previous studies (Revilla et al. 2019; Struminskaya et al. 2021). The highest willingness relates to sharing photos and screenshots created with the smartphone during the survey, and images already stored (using the same device).

Fourth, when analyzing the expected participation (RQ4), we found higher proportions of respondents who have the skills, have data available and are willing to share, for visual data produced during the survey (around 40–55%). For visual data already stored, these proportions sometimes get really small (minimum = 14.0% for videos of food in PCs). If we consider that respondents would participate if they had the skills, had data available and were willing to, our results combining these three dimensions should be in line with previous studies about actual participation. We do find similar results to Bosch et al. (2019) for visual data created during the survey (around 50%). However, for already stored visual data our findings show lower figures. The difference could be related to the fact that we asked for different categories of images (food, nature, activities and products vs. image of something that made respondents laugh in Bosch et al. 2019). Overall, even if there are differences across types of visual data and topics, our results suggest that asking for visual data might be a good option to consider for researchers since most respondents have the skills and willingness to share them. Availability might be more problematic, depending on the type of visual data of interest (especially low for videos already stored).

Finally, regarding the impact of sociodemographic variables (RQ6), older and female respondents were less likely to eventually participate in a survey requiring for visual data. On the contrary, people more educated and with more recent participations in the Netquest panel are expected to present higher participation.

First, we used only one (opt-in) panel in one country (Spain). Thus, robustness checks when analyzing different types of samples or countries are needed, and results from this paper cannot be extended to panels with different characteristics (such as a probability-based panel). Further, we used an opt-in panel whose respondents already voluntarily participate in surveys and use a PC and/or smartphone to do so. Thus, the levels of skills, willingness and/or availability presented in this study could be higher than those of the general population, who are likely to be less familiar with these devices and less willing to participate in a survey than panel members. In any case, the results presented in this study still offer valuable insights, particularly for researchers who also plan to use opt-in panels. Moreover, we relied on respondents’ reports to measure skills, availability, willingness and burden. However, such reports might contain errors (e.g., social desirability could push respondents to answer that they know how to make a screenshot when, actually, they do not know how to do it).

Also, analyzing skills, availability and willingness together works as a proxy of the expected participation, but there may still be other factors influencing the actual participation. For instance, respondents may state knowing how to take a photo, being in a situation to take a selfie and be willing to share visual data in a survey, but if asked specifically for a selfie they might have different concerns that prevent them from completing the task, such as the privacy or how they look this day. Finally, even if respondents stated overall low levels of burden, we believe that researchers should be cautious about this result, since respondents might have underestimated the real burden as they were not asked to actually perform the actions and subsequently evaluate the burden. Moreover, respondents were asked for the burden of doing some visual data-related tasks in general, but not within the frame of a web survey. Thus, different perceptions of burden might arise when asking for that specific type of tasks. Finally, there is still no evidence regarding the burden when asking for several pieces of visual data together.

Based on our results, we propose some recommendations. First, even if the level of skills is overall high, there are still participants who do not have them. In specific tasks (mainly screenshots from PCs), this represents an important proportion. Thus, we recommend providing guidance on how to perform the tasks before individuals are asked to do so. In particular, if screenshots from the PC are going to be requested, an explanation seems necessary. This could be in the form of a text, video or image displayed just before presenting the task.

Second, availability seems to be the most limiting factor, especially for already stored data. Moreover, the storage of visual data varies across topics and respondents. Thus, if researchers are interested in visual data that can hardly be produced during the survey itself, participants may be contacted first, explaining which kind of visual data the researchers would like from them and asking them to start collecting it. Then, respondents can be provided with a survey link allowing them to get in and out to submit the visual data whenever it is more convenient for them. So, for instance, if researchers are interested in getting videos of food individuals have eaten in restaurants, in view of the low availability, we recommend researchers to contact the respondents previously asking them to do videos of the plates they will consume in such establishments, in order to be able to share them with the researchers later on. Also, geolocalization information could be used to send reminders every time the participant goes to a restaurant (Ochoa 2022).

Overall, researchers should carefully evaluate when to ask for visual data, and should do so only when the expected benefits for both the researchers and the participants overweight the expected drawbacks. Moreover, other aspects (besides the ones studied in this paper), should also be considered when deciding on the use of visual data. In particular, data quality needs to be assessed: when measuring some concepts, visual data are expected to provide more accurate and complete insights compared to conventional questions or even to allow measuring new concepts, but empirical evidence is needed to confirm such expectations. Also, the quality of visual data depends on both the inputs sent by respondents and the subsequent classification researchers do of the data (e.g., manual or automatic coding) (Iglesias et al. 2022). Furthermore, even if the quality is increased, researchers also need to assess if the gain in quality is worth the possible extra costs. In addition, ethical considerations should also be part of the decision process. For instance, it is more likely to share information inadvertently when sharing visual data than when answering conventional survey questions (Revilla 2022) (e.g., if researchers ask for pictures of plants and the ID card with personal data is next to the plant, the respondent might share this personal information without being conscious of it). All these aspects would require further research. Finally, researchers should consider the age and level of education of their objective sample when deciding. Indeed, older and lower educated populations are less likely to send visual data in a survey, either due to lack of skills, availability and/or willingness. Then, the decision of using visual data might be different when targeting these groups.