Although it was reported in 2012 that 89% of the world’s population had access to piped water, it is estimated that at least one billion people receive this water for fewer than 24 h per day. Intermittency places a variety of burdens upon households, including inadequate quantities of supply at the household level, unpredictability of water utilities in making water available, and a disproportionate time burden on poorer households. For many intermittent water systems, the availability of water is controlled by valvemen who turn access on/off to various portions of their service area. Using this information, NextDrop sends notifications via mobile phones to customers as to when water is likely to be available. Although a pilot of NextDrop was successfully implemented in Hubli-Dharwad in India, NextDrop faced significant challenges when expanding to Bangalore. This case study investigates how a breakdown in the information pipeline, as well as corresponding human factors, prevented adoption of NextDrop in Bangalore. Specifically, randomized controlled trials found that valvemen sent reports of their activities to NextDrop only 70% of the time. Even when NextDrop passed messages onto customers, only 38% of customers reported receiving notifications, primarily because either the household “waiters” for water, usually women, did not have daytime access to the mobile phone registered with NextDrop or the notifications are buried under the many other solicitations and informational messages regularly received via SMS. Valvemen were further studied through observation and semi-structured interviews to understand their incentives for complying with NextDrop.
Notes
Tanu Kumar and Christopher Hyun are co-first authors of this chapter.
17.1 Introduction
In the summer of 2007, graduate student Emily Kumpel, from the University of California, Berkeley (UC Berkeley), attempted to collect household water quality samples in Hubli-Dharwad, India. As she went from home to home, she found piped water service to be intermittent and unpredictable; even those who had lived in the city for years could not tell her when the water would turn on. She contacted the local water utility for more accurate information, but they could only give her a rough estimate. “They told me that water would arrive a certain number of times in a week, but not exactly when,” Kumpel recalls. She realized then that significant numbers of households with piped network connections experienced uncertainty about when and if water would arrive.1
Around the globe, public services such as water and electricity are often not delivered continuously. Even though 89% of the world’s population had access to piped water by 2012, Kaminsky and Kumpel (2018) estimate that at least one billion people around the world receive their piped water for fewer than 24 h a day. Service intermittency is often a consequence of inadequate supply for the entire service area.
Advertisement
Intermittency places many burdens upon households. First, it often leads to inadequate quantities of supply at the household level. A few hours of piped water supply, after all, may be insufficient for an urban household’s cooking, cleaning, and bathing needs over the day or the week. Furthermore, intermittent water supply imposes burdens because it often arrives at unpredictable times. Water utilities are often unable to follow schedules for many reasons, including a lack of internal coordination, poor infrastructure, or problems with intermittent electricity supplies. Households are left not knowing exactly when their water supply will begin or end.
Water supply unpredictability can be particularly burdensome for poorer households. Many middle- and upper-class households can afford purchased water from tankers or automatically filling tanks to store the water whenever it arrives. For households that cannot regularly afford tanker water or do not have load-bearing roofs to support tanks, the only option is to wait for water and to fill household storage containers manually. For them, waiting for water could mean forgone wages or missed social events. These costs fall primarily upon women, who are most likely to be at home during the day.
Emily Kumpel, with a few colleagues, founded a social enterprise named NextDrop, which sent SMS (text message) notifications to urban households about water arrival times. The premise was that, since it is costly to address the underlying causes of water intermittency in piped systems, an information-based service could be a low-cost solution to ease the burden of water unpredictability. NextDrop rolled out and scaled up in the state of Karnataka in two urban areas of varying sizes, Hubli-Dharwad and Bangalore.
We studied NextDrop’s services and found that while its pilot in the twin cities of Hubli-Dharwad was promising, the services rolled out on a larger scale in Bangalore did not have a measurable impact on many dimensions of household welfare. Why not?
Advertisement
NextDrop’s system depended on water utility employees to provide timely information about water arrival to disseminate to the company’s customers (see Fig. 17.1). These “valvemen” opened and closed water valves throughout the city, channeling water to particular segments of the piped network in rotation. NextDrop found it difficult to incentivize the valvemen to send the company regular and accurate data in metropolitan Bangalore. NextDrop’s system was also not fully compatible with the existing social dynamics between households, valvemen, and utility supervisors. Moreover, even when information was accurately disseminated, customers often did not receive it; in many homes, for instance, the women who were responsible for waiting for and collecting water were not in possession of the household mobile phone during the day.
Fig. 17.1
NextDrop’s phone-based water supply notification system depended on notifications from water valvemen. (Adapted from NextDrop)
×
NextDrop’s problematic rollout in Bangalore illustrates that the preferences and priorities of employees and customers are vital human components of technological innovations. They must be understood before and during the initial rollout of an innovation, and they can vary in unexpected ways across locations. It is essential to consider the possibility of this variation when scaling up an innovation from pilot locations.
In this chapter, we analyze NextDrop’s evolution and develop this argument as follows: first, we provide an overview of the NextDrop system in its final form. We then go back in time and describe the system’s development during a class on Information and Communications Technology for Development (ICTD) at UC Berkeley in 2009. We move on to discuss the company’s initial, and apparently successful, pilot in Hubli-Dharwad, India. We follow the company’s expansion to Bangalore with a pair of companion studies that we (the authors) conducted during 2014 and 2015. Drawing on insights from these studies, we show that the intervention had little impact on Bangalore’s households, in part because key human elements in the system differed significantly and unexpectedly from the pilot location. We conclude with some takeaways for development engineers who wish to better understand and manage the human aspects of technological innovations.
17.2 The NextDrop System: An Overview
NextDrop’s founders noticed that households faced major difficulties dealing with intermittent water supply and decided to develop a technological system that would provide them with advance warning of water arrival times. This was easier said than done, as water utilities delivering intermittent water supply tend not to have sensors throughout their piped network. In systems without such sensors, the water utility may not have any centralized information about where water is flowing throughout its pipes, especially at the household level. Though Bangalore’s water utility released its schedule for piped water supply for residents to see in newspapers and local utility offices, responses from our 2014–2015 household survey revealed that this information was generally inaccurate in our study area (see Fig. 17.2). This shows that residents would have required a more accurate source of water supply information if they wanted to know when their water would turn on.
Fig. 17.2
Scheduled versus reported interval between supply days in Bangalore utility’s E3 subdivision. (From Kumar et al., 2022)
×
After a fair amount of research, NextDrop’s founders discovered that the valvemen who turned the water on and off were the best source for information on water arrival times. They developed a system through which they asked the valvemen to call a toll-free number whenever they opened a water valve, closed a valve, or did not open a valve when scheduled to do so (i.e., a supply cancellation). NextDrop developed proprietary software and then used it to send text or voice notifications with expected water arrival times to participating households that it had located in specific valve areas based on their GPS coordinates. Because it takes time for water to flow into a valve area and fully pressurize any specific portion of the network, notifications would arrive before water reached household taps, giving residents some time to get ready to receive water.
NextDrop relied on the valvemen’s knowledge of the piped network in order to accurately map which valves served which households. It collected GPS coordinates for households and, with the valvemen’s help, created maps of the valve areas, which Indian utilities typically do not possess. Each polygon in Fig. 17.2 is an example of one of Bangalore’s thousands of valve areas; each valveman was responsible for several at a time. NextDrop could then identify which households fell in which valve areas based on household GPS coordinates.
Thus, we can see that the valvemen had unique knowledge of the day-to-day workings of the urban water system. They possessed information that was not available to even the chief engineers of the utility. As a result, their cooperation was essential to the success of NextDrop’s innovation.
17.3 From Idea to Pilot to Scale-Up
17.3.1 Idea Formation at UC Berkeley, USA
After that summer in India, Kumpel, then a PhD student in the Civil and Environmental Engineering Department, took Tapan Parikh’s project-based class at UC Berkeley, “Social Enterprise for International Development.” Kumpel worked with a handful of other students—Ari Olmos, Thejovardhana Kote, Matt Gedigian, and Niranjan Krishnamurthi—to develop a technology that sent households text messages about their water supply status. The team settled on calling the system “NextDrop,” inspired by an app called “NextBus.” The team won a prize for the best project in the class. They eventually submitted an $8000 USD proposal to the Big Ideas Contest at UC Berkeley, an initiative providing funding along with training and contacts to help support student-led innovations. All this led Kumpel to do initial groundwork for a pilot in Hubli-Dharwad.2
17.3.2 Pilot in Hubli-Dharwad, India
The twin cities’ area of Hubli-Dharwad is the second most populous urban center in the state of Karnataka, with a population of just under a million people by the time NextDrop started. Rapid population growth had put pressure on the municipal water distribution system operated by the Karnataka Urban Water Supply and Drainage Board. Most residents received piped municipal water intermittently, from every other day to once a week, with a median of every 5 days (Ray et al., 2018).3
By the time NextDrop began in Hubli-Dharwad, over 50 valvemen managed more than 800 valves, supplying water to various neighborhoods at different times across the twin cities. The utility announced water schedules in local newspapers, but they were frequently out of sync with actual water delivery times. Utility workers coordinated water delivery with one another mainly through ad hoc mobile phone calls and field visits. This, along with electricity outages and pipe breaks, led to not only intermittent but also unpredictable water supply timings. Households did not know when water would turn on and for how long the supply would last. For some (mostly higher-income) residents with large storage tanks, this may have not been a major inconvenience. However, water supply intermittency coupled with unpredictability can lead to high wait times, missed work or social events, and water-related stress for low-income residents with little water storage capacity (Kumpel et al., 2012).
In 2010, Kumpel spent almost a year in Hubli-Dharwad with the NextDrop team, setting up the first iteration of their innovation. Through her dissertation research, she had already established a solid working relationship with the chief engineer of the water board, who wanted to both address the problem of intermittent water supply and implement a novel mobile phone system. Ari Olmos and Anu Sridharan, a UC Berkeley undergraduate at the time, traveled to India to pilot the system. In 2012, after much discussion and with Sridharan as chief executive officer, NextDrop became a for-profit company, giving the organization greater opportunities for funding and tendering by the government. NextDrop won several awards, including the Clinton Global Initiative Award for Outstanding Commitment, the GSMA mWomen BOP App Challenge, and the Knight Foundation News Challenge.
NextDrop’s first system was primarily manual and relied on households to report when they received their water supply. In this first iteration of the product, Kumpel worked with the Deshpande Foundation to collect mobile numbers from households, requesting customers to call in when they received water. Whenever NextDrop received a call from any household about water supply, they would use a spreadsheet to keep track and send messages to the households in the same valve areas. In later iterations, the valvemen would send notices via NextDrop’s interactive voice response system when they were about to adjust a water valve, indicating (1) “advance notice” an hour or less before opening a valve, (2) “valve opening,” (3) “valve closing,” or (4) “last-minute changes” when water supply was interrupted. NextDrop would then send notifications to customers’ mobile phones via voice or text messages based on household GPS coordinates. As the municipality did not supply funding or administrative capacity for their system, NextDrop recruited valvemen directly, who, for the most part, seemed content to contribute to an innovation that could potentially reduce the time they spent on fielding complaint calls from households.
NextDrop also implemented various incentives to encourage valvemen to use their system. The more often a valveman called NextDrop, the more they accumulated points for a list of items they could win. This worked so well that valvemen worked together to combine their points for large ticket items, such as televisions. NextDrop also recognized “Valvemen of the Quarter.” During this pilot stage, NextDrop staff members worked closely with the valvemen, being considerate of their personal needs and providing a sense of camaraderie with NextDrop. Trust was built. According to NextDrop team members, this worked well for the valvemen and the company alike.4
In 2012, after a month of close monitoring, NextDrop found the valvemen’s data to be relatively accurate for “opening” and “advance” notifications (Kumpel et al., 2012). They also found that advance notices reached customers 20 to 40 min before they received water supply, close to NextDrop’s ideal 30-minute target. Customers reported being able to go home or arrange water collection by family or neighbors in time for the start of the water supply due to NextDrop’s mobile messages. By 2012, NextDrop had over 7000 customers across 80 valve areas in Hubli-Dharwad. They offered new customers a free month of service and slowly rolled out a payment system with text messages for 10 INR ($0.20 USD) per month or voice messages for 15 INR ($0.30 USD) per month. Most of NextDrop’s customers were middle- to high-income and opted for the text messages; however, the company hoped eventually to include more low-income customers who might have preferred voice messages due to lower literacy levels.
Though NextDrop collected fees from their customers in Hubli-Dharwad, these funds were not sufficient to cover costs. In order to find new funding streams and reap economies of scale, they decided to expand to a new city. They chose the state capital, Bangalore, a city over eight times the size of Hubli-Dharwad. As excitement and potential funding for the Bangalore rollout grew, NextDrop diverted most of its resources and personnel to the new site, eventually leaving operations in Hubli-Dharwad to run primarily on autopilot.
17.3.3 Scale-Up in Bangalore, India
Growing from 5.8 million people in 2001 to 8.6 million in 2011, Bangalore is considered the Silicon Valley of India. Though known in international water policy circles for providing extensive water coverage and achieving highly successful fee collection, piped water delivered by the Bangalore Water Supply and Sewerage Board (BWSSB) is still, like in Hubli-Dharwad, intermittent (McKenzie & Ray, 2009). The utility’s public website displays a water arrival timetable showing that neighborhoods can expect their water at varying intervals across a range of time—from once a week to everyday. For some locations, specific water arrival times are not given; for others, specific times are given but not specific days, leaving supply unpredictable for households.5
NextDrop initiated its Bangalore rollout by signing a memorandum of understanding (MOU) with the BWSSB in 2013. This agreement did not provide any funding from the utility, but it did provide NextDrop with some authority over the valvemen: they and their supervisors now had to comply with NextDrop’s system. At the time, the BWSSB divided the city into 32 subdivisions. NextDrop initially rolled out in “Subdivision A,” a pseudonym we use to protect the identities of the valvemen among whom we conducted our ethnographic study. At the time, Subdivision A had five service stations with two to four valvemen at each station. Each valveman was assigned 20 or more valve areas, and each valve area covered anywhere from 20 to 200 households.
From the start, NextDrop staff knew that the valvemen in Bangalore might be reluctant to comply with their notification regime. During its pilot in Hubli-Dharwad, NextDrop had garnered a workable level of cooperation from the valvemen. The company did not keep detailed data on incentive-specific performance but believed that their combination of individual and social incentives in Hubli-Dharwad had been effective. Scaling up this highly personalized incentive system to the megacity of Bangalore did not seem feasible, however, so NextDrop relied on the BWSSB’s official hierarchy to encourage valvemen to submit the required data. In other words, the valvemen’s supervisors were required to ensure that they sent notifications to NextDrop under the MOU. In effect, the Bangalore rollout substituted the reliance on individual incentives and personal goodwill for reliance on the utility’s organizational structure—arguably a more scalable proposition.
At the same time, NextDrop continued to make changes to its technology, led by their chief engineer, Devin Miller. In Hubli-Dharwad, the messaging system was in Kannada, the primary language of the state. Bangalore is much more cosmopolitan, requiring multiple languages, such as Tamil and Telugu, in addition to Kannada, so these languages were added. NextDrop also created an online customer registration system, converted their spatial data to Google Maps, and added more specific information to customer notifications (e.g., power outages as a reason for supply delays).
17.4 Research on the Scale-Up
In 2014, our team of researchers from UC Berkeley led by Professors Alison Post (Political Science) and Isha Ray (Energy and Resources Group) began an evaluation of NextDrop’s innovation. Post researches urban water politics and service delivery in developing countries. Ray researches water and sanitation in developing countries and was one of the original authors for studies on the World Bank’s continuous water supply demonstration in Hubli-Dharwad (Ray et al., 2018). Our evaluation consisted of two studies. The first was a randomized controlled trial to assess the effects of NextDrop’s innovation on household welfare. The second was a mixed-method study aimed at understanding the NextDrop system from the water valvemen’s perspectives. PhD students Tanu Kumar (Political Science) and Christopher Hyun (Energy and Resources Group) joined the team to work on the household evaluation and valvemen studies, respectively.6
We conducted the household impact evaluation from June 2014 to November 2015. This evaluation focused on three main sets of outcomes at the household level: the costs of unpredictability, psychological effects, and political attitudes toward the government and the water utility. The costs of unpredictability included any costs associated with waiting for water, including the time spent, missed events (such as family gatherings) or missed work, missed or delayed religious observances (such as the need to wash in fresh water before Friday prayers for many Muslim families), and purchase of substitutes for piped water. Psychological effects included reported worry or stress about water arrival times or water running out. Political attitudes referred to citizens’ perceptions of the BWSSB as a water provider.
We measured the effects of NextDrop’s innovation on these outcomes for 3000 households using two waves of survey data on either end of the randomized allocation of NextDrop’s services in the BWSSB’s E3 subdivision. Overall, we detected null effects on all outcomes—except for stress related to household water management, for which we detected modest beneficial effects. We believe that these null-to-modest effects can largely be explained by the failure of accurate information about water arrivals to reach customers.
17.4.1.1 Site Selection
Our study site, E3, is a BWSSB subdivision near Bangalore’s eastern periphery. To minimize interfering with NextDrop’s operations and to keep subdivision-level characteristics constant across households, we decided to conduct the evaluation within a single subdivision. We selected E3, in consultation with NextDrop and the BWSSB, for two reasons. First, it was not among the subdivisions scheduled for immediate expansion of NextDrop’s services, making possible the selection of control group households who would not receive NextDrop’s services during the study period. Second, field visits and census-level data revealed that this was a subdivision with many low-income settlements and relatively intermittent water supply, suggesting this was an area where NextDrop’s services might be particularly impactful.
17.4.1.2 Sample Selection and Randomization
Within areas receiving piped water in E3, we surveyed households and randomly enrolled them for NextDrop’s notifications (the treatment condition) or not (the control condition). A concern common to randomized studies of information interventions such as this one is that treatment households might share information with control households. To reduce this threat to causal inference, we used cluster randomization and assigned households to treatment or control status in groups determined by their geographic location. These groups were separated by at least two streets to prevent individuals in our treatment and control groups from sharing information. Additionally, clusters were usually in different valve areas, meaning that it was unlikely for individuals across clusters to share information because water arrival times are relevant only to individuals within the same valve areas.
This cluster randomization was also stratified, or blocked, by neighborhood socioeconomic characteristics. Blocking increased the statistical precision of the study and allowed us to look for subgroup effects among the low-income households for which we predicted the effects of the information would be largest. Based on extensive site surveys, we designated blocks to be either low-income or mixed-income. Each block comprised four clusters (following Imbens, 2011) that we expected to be similar in terms of socioeconomic variables and the underlying water infrastructure. We randomly assigned two clusters in each block to receive treatment and two to the control condition.
Overall, the study included 3000 households, with the sample size calculated to detect a 30- to 45-min average reduction in the time spent waiting for water. The total sample had 10 low-income and 20 mixed-income blocks (Fig. 17.3), with 120 total clusters of 25 households each.
Fig. 17.3
The BWSSB E3 subdivision boundary is shown in blue. Areas receiving piped water supply are denoted in yellow. Green and purple polygons denote low-income clusters (treatment and control); black and red polygons denote mixed-income clusters (treatment and control). There are four clusters per block. (Adapted from Kumar et al. 2018)
×
17.4.1.3 Two-Wave Survey and Enrollment into NextDrop Services
In early 2015, we worked with a Bangalore-based survey firm to conduct a baseline survey of the 3000 study households. Enumerators contacted every third household in a cluster until a quota of 25 households was reached. They then administered a survey that we designed to obtain measures of our main outcomes and other variables of interest. Enumerators surveyed the individuals who reported managing and storing water for the household. In line with the expectation that women are generally responsible for managing water, 80% of our respondents were women.
The enumerators next offered all households the opportunity to enroll in NextDrop services by submitting their mobile phone numbers. They could sign up for text or voicemail notifications in English, Kannada, Telugu, or Tamil. Because surveys were conducted on tablets that collected GPS coordinates with 5-meter precision, NextDrop was able to correctly identify the valve areas in which households lived. NextDrop then enrolled the households in our treatment group clusters following the completion of the baseline survey. In October and November 2015, about 4 months after NextDrop had begun sending messages to customers in E3, we surveyed the study households once more. NextDrop and the research team estimated that 4 months was enough time for enrolled households in our treatment group to adapt to using the notification system and for potential impacts of the service to occur. The control group was enrolled after the completion of the study.
17.4.1.4 Results
Overall, we were unable to detect effects of NextDrop’s intervention across our study population for any of our outcome variables, aside from those related to worry or stress. Even though our sample size would have enabled us to detect a decrease in wait times for water as small as 9 min, we observed no effects on time spent waiting for water, the likelihood of missing work or community events while waiting, or using substitutes for piped water. We similarly failed to detect effects for households living in low-income clusters or those we identified as NextDrop’s target customers, namely, low-income households without automatically filling water tanks. We detected no effects on attitudes toward the utility, but this could be because of already positive attitudes at baseline.
Our baseline survey confirmed that E3 was the kind of site in which the notifications “should” have been impactful; our data indicated that over 85% of residents received water services just once or twice a week and 28% did not possess an automatically filling water tank. About 43% reported that they simply learned that water had arrived when it began to come out of their taps, rather than knowing when to expect it. Respondents spent roughly 1 h per supply day waiting for water, which they agreed led to missed work and community or family events. As we discuss below, we believe these null effects are a result of few customers receiving accurate and timely information about their water supply.
17.4.1.5 An “Information Pipeline” to Diagnose the Null Results
We created a framework consisting of each of the steps in the process of producing and disseminating information to understand our null results (Fig. 17.4). First, someone must collect the relevant information in full. Second, someone must analyze the information or compile it into a usable format. Third, someone must disseminate the information. There could be poorly aligned incentives or technical difficulties preventing any of these first three steps from happening. Organizations often cannot control those charged with executing their assigned tasks, and employees can exercise significant autonomy.
Fig. 17.4
Information pipeline: from collection to action. (Adapted from Kumar et al., 2018)
×
Fourth, this information must actually reach the intended recipients. Messages may go to the wrong person because phone numbers change or because the intended recipient does not have control over the household mobile phone. Fifth, these recipients must actually register that they received a message. Information may be sent in the wrong language, or go unnoticed if a recipient is inundated with messages, or be so useless that the recipient stops paying attention. Finally, the information must be accurate. Inaccuracies could reflect deliberate efforts to conceal information, carelessness, or some inability to measure the relevant information.
We identified several “leaks” in this information pipeline for NextDrop’s model in Bangalore. First, valvemen had to submit information by calling NextDrop’s automated interactive voice response system to log valve opening times. Yet logs of valvemen notifications to NextDrop show that they sent reports to NextDrop only 70% of the time.
Furthermore, even while NextDrop passed on all of these messages to customers, only 38% of treatment group members reported receiving notifications. We argue that these low levels of information receipt can be explained by two factors. Even among the 854 households that were regularly sent notifications, 207 household “waiters” for water, usually women, did not have daytime access to the mobile phone registered with NextDrop; a male member of the family usually took the phone to work. Second, many respondents may simply have not noticed NextDrop’s texts or voicemails, which may have become buried under the many solicitation and informational messages regularly received via SMS.
Finally, only 289 of our 1193 treatment group respondents reported that the information received in NextDrop’s notifications was either always or usually accurate. Moreover, when we compared household survey responses about the last day they had received water to the valveman reports, we found that over one-third of households reported receiving water on a different day than that reported by the valveman. In fact, 62% of households received “advance” notifications after their water had already arrived. Figure 17.5 shows each of the steps preventing information from reaching our treatment groups. We can identify two “human” factors that are particularly important here. The first is intra-household dynamics that prevent the individual responsible for waiting for water from accessing the household mobile phone. The second is the valvemen’s lack of cooperation in sending accurate information to NextDrop to be disseminated. Our second study explores the extent to which, and why, this latter dynamic was occurring.
Fig. 17.5
NextDrop’s leaky information pipeline and treatment group attrition. (Adapted from Kumar et al., 2018)
×
17.4.2 Study 2: Valvemen’s Role and “Compliance” in NextDrop’s System8
Why were the valvemen not sending notifications when they were supposed to, and why were these notifications often inaccurate? In a parallel study in a different part of Bangalore, we examined factors that could contribute to the valvemens’ willingness to comply with NextDrop’s system. They were not sending notifications to NextDrop, as they were expected to do. So why did the valvemen not comply? We framed our second study in terms of three levels of factors that may have contributed to valvemen compliance: organization-, community-, and individual-level factors. In Fig. 17.6, we measured “compliance” by comparing actual to expected notifications for each valveman in Subdivision A, using NextDrop’s database. Valvemen had low to moderate levels of compliance overall, but we also observed that the ratio of actual to expected valve opening notifications varied within the same subdivision (between 0.4 and 0.8) and even varied within the same service station (or team). This revealed that, when controlling for management, valvemen still complied at different rates, indicating that the characteristics of individual valvemen or of the different community neighborhoods within which they worked may explain why they did not comply with NextDrop.
Fig. 17.6
Notification compliance: actual/expected valve opening reports per valvemen, August to December 2014, Subdivision A. (Adapted from Hyun et al., 2018)
×
17.4.2.1 Study Design to Understand Compliance Levels
Our field observations focused on Subdivision A because NextDrop’s relationships with the valvemen were strong there, and it was relatively distant from our household evaluation study area in E3. We used an ethnographic approach to understand why valvemen complied with NextDrop’s system at low rates. Our methods included interviews with valvemen, community residents, and staff members of the water utility and NextDrop. Out of the subdivision’s 17 valvemen, we chose 9 for close observation, representing a variety of compliance levels. We followed the valvemen on their daily rounds, on breaks, and even into their homes, taking extensive notes and geolocated photographs of the neighborhoods that they serviced. Throughout our interviews and observations, we thought of the valvemen as “street-level bureaucrats” (following Maynard-Moody & Musheno, 2000, 2012) and encouraged them to share their personal narratives about their own jobs and their relationship with NextDrop.
We also created a dataset on the characteristics of the valvemen and the valve areas in which they worked. Through semi-structured interviews, we collected data on the number and gender of the valvemen’s children, their wives’ employment type, the vehicles they used for work, their ages, and the numbers of valves for which they were responsible. We also visited all 233 valve areas serviced by the 9 valvemen and collected data on the socioeconomic status (SES) of the neighborhoods, water infrastructure, and street activity, with most observations taking place around the same time during weekdays. We observed that “low” SES valve areas had narrow roadways, few trees, high noise levels, high numbers of people on the streets, and high domestic activity (e.g., cooking or washing dishes on the streets). “High” SES valve areas had fewer people on the streets and more water storage tanks, which implied less work for the valvemen. We ran linear regressions to establish whether or not our ethnographic observations were supported by correlations within our broader dataset.
17.4.2.2 Findings that Explained Overall Low Compliance
We found three explanations for low compliance with NextDrop’s system: (1) the valvemen’s views of their own jobs, (2) their overworked days, and (3) the insider knowledge valvemen had that gave them some level of job security.
First, in the water utility’s view, the valvemen’s job description was simple: listen to utility supervisors and adjust water valves as needed. However, valvemen perceived their job as responding to “the public” rather than merely to the utility’s hierarchy. Their overriding sentiment was: “My main work is working with the public.” The pressure from the public was so strong that another valveman claimed: “When I work I forget about my family and friends. These people are my family and friends.”
In public administration literature, this phenomenon is discussed as the tendency of street-level workers to “cope toward” citizens, seeing themselves as working for ordinary citizens rather than just for the government (Tummers et al., 2015; Maynard-Moody & Musheno, 2000). We saw this when observing what valvemen do when water supply was not on time—the very situation in which NextDrop’s text messages would be needed. If water arrived late, the conventional utility practice dictated that the valve area would be given less time, so as not to hold up water supply for subsequent areas. Valvemen, however, would not always follow this rule: “If I’m supposed to give them an hour of water, and due to power cuts they only get a half hour, then I will give them another half hour.” Furthermore, residents would often complain to valvemen, and residents were not asking for NextDrop’s text messages; they wanted water. We observed that valvemen routinely prioritized their time to meet residents’ actual requests over sending notifications to NextDrop.
Second, though it took valvemen less than a minute to send a notification to NextDrop, even this small act was too much when they felt overworked. Many of them worked non-stop, constantly negotiating with residents, utility supervisors, and politicians on the phone and in person while moving from valve to valve. As a consequence of their low wages, they also made time for other income-generating activities, such as side plumbing jobs. If their wives had jobs outside of the home, the valvemen would also tend to domestic work. Since they knew the water system well, they would also be called in to assist with pipe breaks at night. Though some valvemen appreciated NextDrop’s service, others found it annoying: it is “not helpful for valvemen”; “It’s just an additional job.” During these constant negotiations, moonlighting work, and moving from neighborhood to neighborhood, we rarely saw valvemen sending notifications to NextDrop—even though they knew this was why we were observing them. Many were just overwhelmed.
Finally, as mentioned earlier, valvemen had privileged knowledge of the water infrastructure and therefore did not take threats of dismissal from the utility seriously. NextDrop relied on their MOU with the water utility to get the valvemen to submit notifications, but the valvemen’s tacit knowledge of the city’s water system was highly valuable, and the utility would have been hard pressed to fire a valveman for not complying with NextDrop’s system. On-the-ground understanding of the valve areas is passed on from valveman to valveman, bypassing the head engineers. When we questioned why one valveman spoke rudely to NextDrop’s staff and even to his own utility supervisor, he was unconcerned: “I don’t worry about being fired.”
NextDrop had initially implemented positive incentives in Bangalore, as they had done in Hubli-Dharwad. They offered the valvemen prizes, such as phones and free mobile talk time. NextDrop could afford these prizes, but they were not motivating enough for valvemen who lived in a city with a higher cost of living than Hubli-Dharwad. For the Bangalore valvemen, the incentives did not match the extra work they had to do for the company. NextDrop also celebrated the “best” valveman, but this resulted in skepticism about the company’s ranking system and led to social divisions. When NextDrop eventually secured an MOU with the water utility, the company was relieved; instead of expensive and ineffective incentives, they thought they could rely on the utility’s hierarchy to keep their system going. However, as we saw, this was not enough to get valvemen to comply.
17.4.2.3 Findings that Explain Variation in Compliance
As explained above, overall compliance with NextDrop was low; however, we also uncovered explanations for why compliance varied across individual valvemen through our interviews and observations of the valvemen at work. We observed that valve area characteristics and a valveman’s family circumstances could explain differences in compliance levels.
There were often stark community-level differences between middle-class and low-income valve areas, which influenced how the valvemen worked in each area. First, valve areas differed in terms of infrastructure. Middle-class areas had wide, paved roads, making it easier for valvemen to navigate. On the other hand, low-income areas had narrower and sometimes unpaved roads, with chickens and dogs that had to be avoided. Middle-class homes had more water storage tanks with maintenance holes outside of the home, while valveman often had to enter lower-income homes to check water pressure in their faucets, leading to greater inconvenience for both the households and the valvemen.
Second, valve areas differed in terms of face-to-face interactions with residents. In low-income areas with less household water storage capacity, residents stayed near their homes to wait for water. Sometimes residents stood outdoors with buckets, dirty laundry, and dishes ready as soon as they saw the valveman in their neighborhood. Such residents often confronted valvemen with complaints. According to the valvemen, this was a major difference across valve areas: “The higher class people call our superiors and the superiors tell the valvemen the problem. The lower class people come to me directly, and I have to explain to them directly....I lose a lot of time talking to people.” This pushed NextDrop notifications to the bottom of the work list in low-income valve areas.
Along with these differences in community-level characteristics, the valvemen’s individual characteristics affected levels of compliance, particularly in relation to their personal finance and family circumstances. More children indicated greater financial need, and this was especially apparent for girl children. Daughters come with the extra financial burden of (future) marriage dowry expenses, and this was on the minds of less compliant valvemen, who had to borrow money for dowries: “We ask our relatives for help—if you help us now, we’ll help you when your daughters get married.” These valvemen had side jobs, such as driving and plumbing, which diverted them from sending notifications to NextDrop. If their wives worked outside the home in inflexible jobs, such as babysitting and dishwashing, the valvemen sometimes went home in the middle of the work day to take care of a sick child or do other work (e.g., bring in the family’s hanging laundry before it rained). A wife’s occupation, then, indicated financial need and could mean more domestic responsibilities for the valvemen.
Our regression analysis supported our ethnographic observations of the valvemen’s individual characteristics. We saw significant correlations between the number of children and levels of individual compliance, even in our small sample size. One additional child was associated with a 7% decrease in compliance. If that child was a girl, there was an 11% decrease. We also saw decreases in compliance related to the job type of a valvemen’s wife. Though we created a dataset for 233 valve area communities, we could shadow only 9 valvemen, so we do not claim causality in this case. However, we see these results as hypotheses to pursue further explorations of frontline workers’ compliance with new workplace expectations.9
While we conducted our research in 2015, NextDrop made plans to roll out in different cities in order to expand their operations. During our study period, NextDrop visited other smaller cities in the state of Karnataka, as well as the cities of Varanasi and New Delhi. This was all done while operations in Hubli-Dharwad were slowing down and the rollout in Bangalore was still incomplete. Unilever decided to fund a new project in nearby Mysore, a city slightly smaller than Hubli-Dharwad.
To improve their data collection, NextDrop experimented with using smartphones in Mysore, creating a smartphone app that would collect data from the turning of a valve key—the large, metal T-shaped tool that valvemen use to turn water valves on. Valvemen would stick the smartphone to the valve key using Velcro, and, by employing the GPS and motion sensors already in the phone, algorithms would determine if a specific valve was opening or closing. This was an innovative idea but difficult to implement in practice. Threading on the valves was inconsistent and so were valve angles, smartphone placement on the key, and network connection.
Ultimately, it became an uphill battle for NextDrop to find venture capital for a valvemen-based call-in system. Raising investor funding on this idea became unsustainable, and our studies showed that households were not benefiting from the intervention, due in large part to dependence on the valvemen. NextDrop needed to reconsider their business model and technological approach. This, combined with struggles with personnel on the team, led NextDrop’s leaders to downsize the company.
In 2016, the NextDrop team went from 20 to 5 employees. In Hubli-Dharwad, they let the system continue to run without oversight until 2019, when they officially shut it down. In Bangalore, NextDrop agreed to map out the rest of the city and register all of the remaining valvemen into the NextDrop system for the BWSSB. NextDrop shifted their maps to the BWSSB’s servers even though valvemen were no longer sending in data. In 2017, NextDrop submitted a proposal to the BWSSB for a new system with sensors in major valve areas. Engineers at NextDrop had considered the use of a sensor-based system since day 1; however, it was cost-prohibitive. As of early 2020, the BWSSB had not floated a tender for the new project nor had any funding materialized from the utility. In the meantime, the heads of NextDrop, Sridharan and Miller, moved on to different water-related projects in India.
17.6 Key Takeaways for Development Engineers
In this chapter, we traced the evolution of a technological innovation designed to reduce the burden of unpredictable water supply, which affects millions of households around the globe. Climate change and the growth of cities will only exacerbate the problem, making it even more critical to study and mitigate such burdens. While NextDrop’s water notification system proved unsuccessful at scale, it provides key lessons for development engineers, especially those working in the water or Information and Communications Technology (ICT) sectors.
Our main takeaways are best stated by NextDrop’s own leaders: “We were asking the wrong question. We were asking, ‘Can this [technology] work?’, when we needed to ask, ‘Is it worth it?’” says Sridharan, who, as of early 2020, continued as the CEO of NextDrop. “Worth” emphasizes the value of their innovation to people. In other words, even the most efficient technology must respond to felt needs in order to be successfully deployed at scale. There are needs across the human ecosystem of a technological intervention. Below we summarize three types of human needs to consider: (1) customer and end user needs, (2) needs of human intermediaries involved in implementation, and (3) changing needs across contexts during scale-up.
In our NextDrop study, we found that not everyone who actually received water supply messages on their phones acknowledged that they received them, suggesting that a number of customers felt no need to read NextDrop’s messages. Devin Miller, head engineer at NextDrop, emphasizes the importance of market research, saying “Spend a whole year on market research if you need to! If you talk to a hundred customers to understand the problem, and they all say they want it, then you’re good. But if they don’t understand or don’t say they want it, then you’re going to have a hard time.”
Technological interventions that aim to help women in particular will benefit from market research on gender roles or household bargaining structures. In many low- and middle-income countries, women may have different roles of domestic labor, access to resources, and less decision-making power than men. In our study, for example, we found that women tended to be responsible for waiting for water, but they did not have regular access to the phone to which water supply notifications were sent. Roessler et al. (2020) similarly find that, in a smartphone intervention for low-income women in Tanzania, over 40% of the intended beneficiaries reported that the main user of the phone was not themselves, but rather their husbands or sons.
Furthermore, it is important to assess how useful an intervention is to end users not only when delivered perfectly but in its actual, imperfect state. For NextDrop, this would have meant understanding the usefulness of providing information even when it was not completely accurate. Development engineers should not only continually refine their technology, but they also should invest in continual refinement of market research in order to evaluate the utility of their intervention as it evolves.
Incorporating the end users themselves, such as women residents, across the phases of a project may improve design and implementation overall. The Global Clean Cooking Alliance, an organization that delivers cleaner-burning cookstoves in low- and middle-income countries, aims to involve women from target countries in all phases of their projects, including design, production, distribution, promotion, investment, education, and product servicing. This practice increases the probability of creating a product that is actually useful and attractive to the intended end user.
Along with end users, it is also important to know what the innovation is worth to the people who help implement the innovation (e.g., the utility’s employees in our study). Our research showed that human intermediaries, like the valvemen, are often ignored in business models or their needs are oversimplified. Often technological interventions, particularly ICT interventions, aim to eliminate the need for intermediary workers altogether. As we show, however, the intermediaries and the “last-mile” delivery for which they are responsible can be key to the success of an innovation.
For example, the valvemen in Bangalore focused mainly on providing water to the public, and sending data to NextDrop seemed unimportant to them in comparison. NextDrop could have worked more on convincing valvemen of the connection between their data input and helping the public. It could have been helpful for valvemen to meet satisfied NextDrop customers, either in person or through video, and get their feedback. The core problem here, however, is that neither the utility nor NextDrop understood the valvemen’s jobs the way the valvemen understood them; what the company saw as non-compliance, the valvemen saw as compliance to their greater task—helping the public.
Relatedly, intermediaries are important sources of market information. Instead of attempting to simply align incentives, engineers and designers should understand that such intermediaries have insight into on-the-ground realities, which can hint at what is “worth” doing or not. In our study, we observed on the ground that the valvemen were “complying,” i.e., actually helping customers, but in ways that the utility possibly did not sanction or would not approve of. Development engineers should account for their perspectives and needs along with those of the end users, especially when planning to scale up.
Lastly, it is crucial to consider differences in context when scaling up innovations, especially in terms of people’s needs. When a pilot is deemed to be successful in one location, implementers and funders often decide to replicate it in another location. Yet, both the process for creating the innovation and its effects may vary from location to location. Indeed, even within areas of the same state and country (i.e., Bangalore and Hubli-Dharwad), differences in context were a key aspect of the difficulties NextDrop faced. In both cities, valvemen had similar job descriptions, but NextDrop found that incentives that seemed to work well for Hubli-Dharwad valvemen did not work well for those in Bangalore, reflecting differences in their needs. “Human behavior is really hard to change,” Kumpel reflected, underscoring the key challenge of NextDrop’s system in Bangalore.
The potential effects of the intervention also varied across the two locations. While customers in Hubli-Dharwad seemed to find the innovation useful, the relevant individuals in Bangalore were, for reasons of intra-household dynamics, prevented from even receiving the messages. Moreover, a 30-min advance water notification was actionable in Hubli-Dharwad, but in Bangalore—a larger city with traffic congestion—household members may have needed longer to reach home before their water was turned on. This longer interval between notification and water services was difficult to achieve with NextDrop’s valveman-based system. It seems, therefore, that in this new context, the end users’ needs could not be met.
All of these lessons ultimately point to the importance of not just understanding the technology that one designs but also the human ecosystem in which it is deployed and used. Kumpel believes that, in order to understand the human ecosystem, designers must know the context personally. She says, “You have to move there! There is no way NextDrop would have gotten off the ground without moving there.” Reflecting on the NextDrop experience, Sridharan added another important point: “A lot of tech startup manuals and guides are based on companies that are based in Silicon Valley [USA], but the constraints faced by the customers and employees of places like PayPal and Dropbox are very different from those we saw.” Understanding the diverse aspirations, constraints, and priorities of key stakeholders is fundamental to the success (or failure) of technology-based innovations designed to solve large-scale development challenges.
Discussion Questions
1.
What types of information might have been helpful for NextDrop to collect prior to rolling out their services in Hubli-Dharwad? In the megacity of Bangalore? How would you design a process for collecting information over time as the company grew and developed?
2.
How would you design a preliminary needs assessment to understand (a) whether women would have access to NextDrop notifications and (b) whether they really needed it? What information would you consider collecting (e.g., how women spend their time, their water use, how regularly they leave their home, and what you would want to know about these trips)? Consider why each type of information may or may not be useful.
3.
How would you design an information collection process to learn about how women used (or might use) the innovation? Why might simply surveying women not be effective here?
4.
What type of data collection and ongoing monitoring could NextDrop have performed to better understand the valvemen that were so critical to their system? What sort of information could they have collected on a continual basis about “compliance,” and how could they have analyzed these data? What sorts of meetings with and/or programs for the valvemen might have improved valvemen’s performance or led NextDrop to reconsider the way it worked with them?
5.
How can we guide development engineering toward thinking about human systems first, allowing the technological interventions and/or designs to follow? How does NextDrop’s experience illustrate the distinction between systems and interventions?
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Before this, Kumpel decided to conduct her dissertation research in Hubli-Dharwad. From 2007 onward, Kumpel and other UC Berkeley researchers established relationships with the municipality, so that when the NextDrop team put together its first business plan and won a grant in 2009, it could leverage this relationship to launch its system in Hubli-Dharwad.
From 2007 to 2008, the Board installed new pipes with continuous water supply through a World Bank-funded project. However, only 10% of consumers received this 24/7 provision, and even then, there have been reports of service interruptions (Ray et al., 2018).
Our research team was also advised by several others who had worked with NextDrop in Hubli-Dharwad, including Zachary Burt, Kara Nelson, and CS Sharada Prasad. This work was funded by a grant from UC Berkeley’s Development Impact Lab (USAID Cooperative Agreement AID-OAA-A-13-00002), which aims to help students and researchers scale up and evaluate science and technology innovations for development.
