Improving the Quality of Survey Data Using CAPI Systems in Developing Countries
Abstract and Keywords
Controlling field interview quality is a major challenge in survey research. Even in high-quality surveys, interviewers often make mistakes that ultimately result in added error in results, including visiting the wrong locations, skipping questions or entire pages, failing to read the complete wording of the questions, or even committing fraud while filling out responses. Survey research conducted in developing countries has to deal with these problems more frequently than research conducted in advanced industrial countries. Computer assisted personal interview (CAPI) systems provide an ideal opportunity for improving the quality of the data by eliminating many sources of error and allowing unprecedented control of the field process. The Latin American Public Opinion Project’s (LAPOP) experience using ADGYS, an Android-based CAPI system, provides useful information on how this technology reduces interviewer-related error, offers opportunities to control the field process, and ultimately significantly improves the reliability and validity of survey data.
If it can be said that advancement in science depends on improvement in the precision of measurement, then the development of modern survey research can easily be counted as one of the, if not the, greatest advances in social science in the twentieth century. Notwithstanding that claim, researchers also must admit that survey data are plagued by error, from a variety of sources. Since error can attenuate true relationships that are in the data, we constantly risk making Type II errors: reporting that there is no relationship, when in fact there is. In surveys there are so many different sources of error, and error is so common in each stage of survey research, the fact that researchers observe any statistically significant relationships between variables is truly an impressive demonstration of the robustness of this form of research. Yet just because researchers have made enormous progress in using surveys, that does not mean survey data are free of error.1
Because of its pervasiveness, error takes its toll on the quality of our research. Given that these errors are mostly unsystematic (not the product of a particular bias), they result in noise that weakens the statistical relationship among variables. Bivariate correlations are attenuated by error, affecting the precision of survey results. Yet some of the error is indeed systematic, the results of which can produce statistically significant findings that are misleading (a Type I error). The most important of these systematic (p. 208) errors in survey research are those that emerge from interviewing individuals and entire regions that were not intended to form part of the sample. When this happens, as we suspect it often does, researchers can no longer be certain that each element in the sample (in this case the respondent) has a known probability of selection, which is the sine qua non of any scientifically drawn probability sample.
For decades, face-to-face surveys were based on paper and pen interviews (which are sometimes called PAPI surveys).2 Indeed, even today interviewer-conducted surveys that are recorded on paper still represent the largest proportion of all face-to-face surveys conducted in developing countries. But in our experience, paper-based surveys are responsible for much survey error. Surveys conducted using paper and pencil technology are prone to a number of different forms of error, both systematic and unsystematic, with consequent negative effects on the precision and accuracy of results.
Questionnaire Application Error
Error can come from the interviewer improperly applying the questionnaire. As most professionals with experience in the field know, interviewers can sometimes skip questions, either intentionally (to save time or to avoid complicated or sensitive items) or unwittingly (because their eyes skipped a row on the page, or they mistakenly thought they had already filled in the answer). In our experience, both types of error are all too common, especially when field conditions are difficult (e.g., poor lighting, threatening surroundings). Interviewers can also incorrectly fill in the answers for filtered or conditioned questions, producing inconsistent response patterns. That is, it is not uncommon to find blocks of questions that are to be administered only to females, or only to respondents of specific age cohorts, being asked of all respondents. Sometimes, because pages of surveys can stick together, interviewers can skip entire pages unknowingly as they move from one page to the next in a paper questionnaire. Blank answers are usually coded as missing data by office coders, which results in a lower N for the skipped items and thus a reduced chance of finding statistically significant results. When groups of items that should have been skipped are asked, office coding has to be done to filter out those responses, but even then, inconsistency can emerge between those who were asked the correct batteries and those who were asked batteries that should have been skipped. For example, if a battery on domestic violence that is to be asked only to women is inadvertently asked to men, those respondents may condition their answers to subsequent questions in ways that differ from those men who were not asked those batteries.
But of all the errors in survey data, probably one of the most frequent and damaging occurs not in the field but back in the home office, when coders incorrectly record the results in the response columns of the paper surveys, and data entry clerks add error (p. 209) by entering the data incorrectly. While verification (i.e., double entry) of 100% of data entry is typically required in most survey contracts, systematic violation of that requirement is commonplace in a world in which survey firms attempt to maximize profit by minimizing costs (the work conducted by data entry clerks is costly and adds to the overall cost of the survey). Even in nonprofit settings, where presumably the quality of the data is more important than the “bottom line” of the firm, the drudgery of double entry of survey data quite likely causes all too many instances of data sets being partially or entirely unverified.
One vignette from our own experience drives home this point. Some years ago the senior author of this chapter contracted with a very well-known survey firm in Latin America to carry out the fieldwork for a survey. At the end of the project, he received the “data” from the survey, which turned out to be no more than a series of tabulations. When he explained to the firm that he would be doing an extensive multivariate analysis of the data, and that he needed the individual-level survey data, the head of the firm responded, “OK, but nobody has ever asked us for that before.” When the data were examined and compared against the tabulations, discrepancies of all kinds emerged. The most common was that the tabulations were all neatly coded, with no codes being out of range. But the data set was filled with out-of-range codes. When the author asked for an explanation of the inconsistency, he was told, “Oh, it is our standard practice to sweep all out-of-range codes into the missing category.” In other words, not only was no double entry performed, but the firm never went back to the original paper survey to find out what the true answers were.
Yet not all error is attributable to the coding/data entry phase. Interviewers can also easily mark an answer incorrectly, because they misheard or misunderstood the answer, or simply wrote it down wrong. They can also sometimes mark the answer into the coding box for a different question printed on the page in front of them. Some of this error is ultimately unavoidable, but paper questionnaires provide no range checks and therefore allow the entry of impossible responses for age, income, and education. Hence, interviewers can report a respondent of 239 years of age, when the correct answer should have been 39, or income of 3,000, when 300 was the actual response, or education of 16 years rather than 6 years.3 Some of these responses can be corrected in the office, but more often than not one is not certain what the correct answer should be. We cannot be certain if the correct response was 16 years of education or 6 years, although we can make a guess based on other items, such as occupation, income, or other variables.
Even when there is no problem of skipping, incorrect filtering, or incorrect recording of responses, there is often a subtler problem related to the style of delivery of the question itself. In order to move quickly through the interview and to save time, some interviewers systematically abbreviate the text of the questions they are required to ask. For example, the question might read, “How much would you say you trust the people of this town or village; would you say you trust them (1) a lot, (2) somewhat, or (3) not at all?” Interviewers who are trying to complete the survey quickly might just ask, “Do you trust people or not?” Such distortion of questions is common, yet it affects the (p. 210) comparability of the responses, as the questions asked of different interviewees are not exactly the same.
The most serious errors involve fraud, a problem that can be greatly attenuated by the new technology we describe later in this chapter. Interviewers perpetrate fraud by partially or completely filling out questionnaires on their own without reference to a genuine respondent, in effect self-interviewing, providing random answers to questions in an effort to shirk the often tedious and sometimes dangerous work of carrying out door-to-door surveys, while maximizing (though fraudulently) their earnings in a given period of time. Some of this fraud can be caught by attentive supervisors and partial recalls, but collusion between interviewers and supervisors is also possible, in which both parties benefit from the fraud (perhaps splitting the earnings from fraudulent interviews). Another type of fraud occurs when poorly supervised interviewers “outsource” the work to others (e.g., a younger brother or sister), thus allowing the interviews to be conducted by untrained personnel.
Other sources of error can produce biased survey estimates. An example of this is failing to interview the individual who was selected via the random procedures that guarantee lack of bias. Paper questionnaires place a heavy burden on interviewers to correctly implement the household selection process. Without proper fieldwork tools, interviewers can over- or undersample some segments of the population (e.g., gender or age groups), resulting in a data set that produces biased averages.
Interviewers can also visit the wrong geographic area, either knowingly or unknowingly, conducting the survey in a place other than where the sample was selected. Ceteris paribus, interviewers will usually visit easier to reach places, resulting in the population that lives in harder to reach or more dangerous areas having less opportunity to be included in the sample, and thus potentially biasing the results of the survey.
In survey research conducted in developing countries, many of these error sources are exacerbated by contextual conditions. One of the main issues is the quality of work that interviewers perform and the difficulties in supervision. For many individuals involved in the activity, interviewing is a part-time and occasional source of income. They rarely have a permanent contract with the polling company, and their earnings are based on a combination of daily and per-interview wages. Interviewers usually have low levels of education and, despite receiving training, are likely to make errors while administering the questionnaire. Under these conditions, interviewers’ work has to be closely supervised to minimize error and fraud. But field supervisors may also work part time and therefore suffer many of the same limitations as the interviewers.
(p. 211) Another factor that defines the conditions under which survey research is conducted in developing countries is the absence of complete and updated geographical information and maps. Census offices and other government sources of official information often do not have complete listings of residential and building areas, and mapping is seldom up to date and complete. In other instances, where census maps are available, government agencies may refuse to make them available to researchers. This makes it difficult for interviewers to locate a selected area or home to start the interview according to the sample design.
Finally, some relevant infrastructure limitations need to be considered. One is poor quality roadways, which make it hard for interviewers to visit some areas, particularly during rainy or winter seasons. Another is the lack of complete phone coverage; the fact that not every home has a phone makes personal, face-to-face interviewing in the respondent’s home the only option to produce a probability sample of the national population in many developing countries. Cell phone numbers, of course, are not geocoded, so a phone with an exchange for a rural area might actually be in the possession of someone from the capital city.
To a limited extent, many of the errors noted above can be prevented or attenuated using conventional methodologies. Foremost among them is increasing the intensity and quality of field supervision. Well-trained, responsible, and motivated field supervisors can make a world of difference in the quality of surveys, but this is a costly element that can significantly increase the overall budget of a project. In small sample projects, having the Principal Investigator (P.I.). in the field supervising a small team of interviewers is perhaps the best guarantee of quality. Yet in large-scale surveys such means are impractical, lest the fieldwork extend over many, many months, and only rarely would a P.I. have the time for such an effort. Further, the field supervisor cannot be in all households at the same time, leaving some interviewers to get it right only when under direct supervision. Finally, there is no ultimate guarantee that the field supervisors have not colluded with interviewers to cheat.
CAPI Surveys: Benefits and Costs
In light of these grim realities of the survey fieldwork process using paper questionnaires, the question is how to reduce or minimize each of these sources of error and deal with the contextual obstacles while conducting survey research so the results are as precise and reliable as possible. Academics, survey professionals, survey data users, and others interested in survey results care about the quality of the data, and they should understand the paramount importance of the survey collection process to guaranteeing that quality.
One strategy for dealing with these sources of error and limitations is to use computer assisted personal interview (CAPI) systems in handheld devices provided to the interviewers who conduct the fieldwork (this approach is sometimes referred to as (p. 212) MCAPI, mobile computer assisted personal interviews). The CAPI surveys can help by displaying the questionnaire in a way that is less prone to error than paper, showing one question at a time per screen and automatically including logical checks and skip patterns. These systems also produce paradata, information about the context and the conditions in which an interview was performed, allowing for better control of the fieldwork process and facilitating the supervision of the interviews (Couper 2005; Olson 2013).
Since advancements in computer technologies have made CAPI systems possible, social researchers and survey professionals have looked at their potential benefits for the quality and speed of data collection (Tourangeau 2005). Research has been conducted comparing computer assisted interviews with traditional paper-based surveys; some recount the differences in large government studies that started applying CAPI systems as soon as they became available, such as the British Household Panel Study (Banks and Laurie 2000) and the U.S. General Social Survey (Smith and Kim 2003). Some others recall the experience of innovating the use of CAPI data collection methods in developing countries (Caviglia-Harris et al. 2012; Shirima et al. 2007). Most of these studies conclude that CAPI surveys reduce error compared to paper and pen interviews, and that they reduce the length of the data collection process (De Leeuw, Hox, and Snijkers 1998; van Heerden, Norris, Tollman, and Richter 2014).
One of these systems is the Android Data Gathering System (ADGYS). It was developed by a team working in Cochabamba, Bolivia, in close partnership with LAPOP, the Latin American Public Opinion Project at Vanderbilt University, and Ciudadanía, Comunidad de Estudios Sociales y Acción Pública, LAPOP’s local academic partner in Bolivia. The beta version of the software was developed in 2011 and used in the AmericasBarometer survey of 2012 in Bolivia. Since then the software has been improved, and new versions have been developed and used, with a new version of the system becoming commercially available in 2015. The software programming company in charge of the development is GENSO Iniciativas Web, based in Cochabamba.4
ADGYS has a client-server architecture, with a mobile application and a Web server. On the server side, ADGYS was designed using open source technologies, including Scala programming language and a Liftweb framework. The databases are managed under MySQL and MongoDB. The client side was designed under W3C standards and uses Html5, Css3, jquery, and bootstrap. ADGYS mobile is a native Android SO application that uses Java technology and SQLite for database management. The synchronization with a Web server is via RestFul Web services, and all data are encrypted during transmission and while stored in the mobile devices.
The software was designed to deal with the needs and challenges arising from the kind of work that LAPOP carries out in Latin America and with some of the most common problems of field survey research enumerated earlier in this chapter. ADGYS was designed entirely from scratch, making use of available technological resources. This means that the system was specifically conceived to comply with specific requisites and demands, including (1) administering complex questionnaires with logical checks and conditional skips, (2) being able to manage complex samples and quota assignments, (3) using inexpensive smartphones and tablets, and (4) providing enough information to allow extensive control of the quality of the fieldwork.
(p. 213) ADGYS allows each survey to include multiple language versions, an important feature in countries that are language diverse. In the United States, for example, the system would allow the interviewer to change from English to Spanish when encountering respondents who feel more comfortable in, or can only speak, that language. In Guatemala, one of the countries in which LAPOP works, a wide variety of indigenous languages is spoken, and each of those can be programmed into ADGYS and be available for the same survey simultaneously.
The ADGYS mobile application works on devices using the Android operating system, versions 2.2 and newer, and was programmed using Android compatible Java technology. Since Android is currently on version 5, compatibility with the system back to 2.2 allows for the use of older, less expensive smartphones and tablets, rather than using only state-of-the-art, and hence more costly, systems. This feature is crucial for conducting work in low-income countries, where the cost of electronic devices is often quite high because of import duties.
Interviewers can use the application to conduct an interview with the device either online or offline; this feature partially deals with the limitation of not having complete cell phone coverage over a given territory (which is common not only in developing countries, but also in remote areas even in developed countries). Unlocking new sample areas for an interviewer can be done online or by entering a code generated by the system for each survey area (e.g., a sample segment). Uploading the data to the server can, of course, only be done while the mobile device is connected to an Internet provider (either via Wi-Fi or using a data connection plan from the cell phone service provider).
The mobile application requires a personalized login for interviewers and other levels of users, such as field supervisors, so that each user is properly noted and tracked. The sample assignment, defined for each interviewer, is also downloaded onto the phones or tablets using the application. This means that each member of a team of interviewers may log into the application and will only see and work on his or her unique personal assignment of interviews, including different studies (or survey projects). With this feature, all of the information generated using ADGYS is produced and reported to the server under the personalized settings for each user.
The second element in ADGYS is the Internet-based server, which is hosted at www.Adgys.com. The server is the most important part of the system, storing and managing the data uploaded from the mobile devices. Questionnaire construction and sample design programming are done from the server, as well as user creation and editing, including assigning new sample areas and quotas to specific users.
The server allows users personalized login with different levels of access. Higher level users can produce a number of reports on the advance of the fieldwork process, including reports on sample completion by interviewer or area. Authorized users can also generate the complete data set at any moment, even if the survey project is still in the field. This feature makes it possible to get virtually real-time information from the field, an important element when using ADGYS in disaster reporting and assessment surveys. A separate data set with the duration of each question for each case is also available for download from the system.
The server also produces an Excel spreadsheet or an Internet-based form, unique for each survey project, that allows the client to program a questionnaire according (p. 214) to the specific goals of that particular study. This feature enables different types of questions with various levels of measurement to be included in the electronic form the interviewer sees. Logical checks and conditional skips can be used here, as well as random assignment of questions and other tools that allow experimental research to be conducted using the system.
Besides the cost of purchasing Android phones or tablets, the use of ADGYS and other CAPI systems for fieldwork has some other costs, related to licensing of the software and server and data traffic and storage. These costs are absent in PAPI surveys, but researchers conducting paper and pen interviews need to budget the cost of printing and transporting the questionnaires to/from the field, and the data entry and data verification phase, which also adds considerable time to the process, not to mention the cost of errors in the final survey results. These costs can vary from one context to another; depending on the local availability and costs of labor and copies, paper interviews could be less expensive in some areas, while in other places they can cost more than CAPI studies. However, once the initial investment in equipment is made, CAPI surveys are almost certain to be less costly and more convenient for most polling companies.
There are two other common concerns related to the use of CAPI systems in handheld devices by interviewers. The first is usability of the system, considering interviewers’ potential lack of familiarity with personal computers, particularly among older and poorly educated interviewers (Couper 2000). The second is safety concerns for the interviewers carrying expensive equipment in the field. Both concerns are at least partially solved with the use of an Android-based CAPI system, such as ADGYS. Given the almost universal penetration of cell phones (and smartphones over the last few years), Android mobile devices such as phones and even small tablets are inconspicuous when they are carried and employed by interviewers. And almost all interviewers own and operate a cell phone on a daily basis, so they are already familiar with the operating system and how one of these devices works.
LAPOP’s experience with ADGYS shows that, as happens with most other consumer electronics, younger interviewers get used to the ADGYS interface more quickly than their older counterparts do, but in the end all interviewers are able to use the system without difficulty. Further, we have found that the number of interviewers mugged or robbed in the field has not increased with the use of Android devices when compared to previous rounds of the AmericasBarometer survey, in which paper and pencil interviews were used, so concerns about interviewer safety are unfounded.
Using ADGYS to Improve the Quality of Survey Data in LAPOP Studies
LAPOP used the ADGYS system extensively in its 2014 round of the AmericasBarometer. The system was employed by LAPOP and its local partners in nineteen of twenty-seven national surveys conducted as part of that AmericasBarometer.
(p. 215) LAPOP’s experience with ADGYS reveals five ways in which this CAPI system can help improve the quality of survey data. Two are defined ex ante, and conditions influence interviewers’ administration of the survey. The other three employ the paradata produced by the ADGYS system to develop mechanisms for quality control.
Conditioning Ex Ante How the Survey Is Administered
There are two ways in which the use of a CAPI system on a handheld device during the interview has improved the quality of the data from a survey. First, it displays in electronic format the questions and response choices in a way that is much less prone to error than paper and pen questionnaires. Second, it does so by assigning sample segments to specific interviewers.
ADGYS displays one question at a time and does not allow interviewers to move to the next one until a proper response has been entered for that particular item. A “proper response” means a substantive answer to the question, a “don’t know,” or “no reply.” Absent one of these choices, the next question cannot be asked and is not displayed on the screen of the mobile device. This format therefore substantially mitigates the error caused by the interviewer skipping questions or entire pages, or entering responses in the wrong location in the questionnaire. If properly programmed, this feature of CAPI systems can also eliminate the inconsistent response patterns that occur as a result of the incorrect use of skips in the questionnaire by the interviewer.
Assigning specific segments of the sample to each interviewer reduces the chances that two interviewers will cover the same area, or that one area will be left uncovered during fieldwork. ADGYS allows gender, age, or other socioeconomic quotas to be assigned to interviewers, which improves the chances of having an unbiased sample at the end of fieldwork. While this form of sample and quota assignment is also possible using paper questionnaires, it is greatly facilitated by the use of handheld devices that only display the areas assigned to the particular interviewer.
Employing Paradata for Controlling the Quality of the Fieldwork Process
Paradata, or the data that refer to the conditions in which a specific interview was conducted, can be automatically produced by CAPI systems and represent a valuable opportunity to reduce error and improve the quality of the data. Paradata can be used in at least three different forms to control data quality: accessing GPS information for each interview, reviewing the total time of the interview, and the time for each question.
Geographical coordinates can be produced by smartphones and other handheld devices in the field using the Global Positioning System radio (GPS) existing in most devices. The ADGYS application turns the GPS radio on automatically, without involvement of the interviewer, and records the coordinates using the satellite information (p. 216) as well as cell phone signal via the device’s Assisted-GPS or A-GPS functions. Under proper conditions (clear skies and a good cell phone signal), all interviews will have a proper GPS reading recorded. This information can be used by the supervisory team to make sure that the interviews were conducted in the place where they were supposed to have been carried out.5
There are some variations in average duration times between interviewers that can be attributed to their age cohort and familiarity with smartphone technology (Böhme and Stöhr 2014), but in general the total duration of the interview can be seen as a proxy for the quality of that interview. Most interviews should fall close to the average time of a particular study (every questionnaire has a minimum duration time, which should include the amount of time it takes to read the complete wording of each question, plus the appropriate response time for the interviewee). Interview time is usually recorded by CAPI systems using the device’s internal clock. ADGYS records interview time automatically as part of the paradata recorded for each interview. Interviews that fall under this minimum time, or that exceed it significantly, should be closely scrutinized and, more often than not, be excluded from the database and replaced.
Partial question time is the number of seconds that the screen for every item in the questionnaire was displayed. This information can be used to identify odd patterns in the flow of the questionnaire. In some cases, it can be used to identify interviewers who attempt to perpetrate fraud, but understand the importance of keeping their total interview time within the expected range.
Partial question time can also be used for improving the quality of the questionnaire and its design, by providing information that can be related to the time the respondent takes to understand and answer a particular question or a series of them within a questionnaire. Mean values across a relatively large number of cases in a survey can reliably show the flow of the interaction between interviewer and respondent during the interview and suggest corrections in the design of the data collection instrument.
Beyond these ways in which CAPI systems have been and are being used, uses are also emerging that could further expand their utility. First, the increasingly large screens on smartphones, as well as the declining costs of tablets, open many possibilities to the survey researcher for respondent–screen interaction. It is now possible to consider showing the respondent small video or voice clips and then ask questions about what he or she saw. These clips could be randomly altered for some experiments or be selected based on prior questions in the survey. For example, if a respondent were to identify herself as belonging to a certain ethnic group, the video or voice clip chosen could focus on that group. Male respondents might receive one clip, females another.
Virtually all Android devices contain cameras, of varying quality. With the permission of the respondent, photos could be taken of the home, which could then later be coded in terms of the appearance of its quality. However, taking photos in the home could sharply raise interviewer suspicions (fear that the survey was really a ruse to set up a future home break-in). Therefore, one would have to proceed very carefully, and with full respondent permission, before photos in the home could be taken. Further, (p. 217) Institutional Review Boards (IRB ) requirements would almost certainly mandate the removal of such photographs before the data set is publicly released.
This expansion in the possibilities of capturing different forms of paradata also increases the potential ethical implications related to the privacy of respondents. While informed consent from the respondent should be necessary for gathering these data, it does not seem to be sufficient to protect the identity of respondents. The authors of this chapter want to highlight the responsibility of the researchers for protecting the subjects who make their research possible by their willingness to answer a survey interview, and that protection depends on the anonymity of responses. All necessary efforts should be made by both the polling company and the research team to ensure that the anonymity of respondents is guaranteed and their identities fully protected, even if they have agreed to the recording of certain data that could put them at risk.
The experience of using a CAPI system in a large, hemisphere-wide public opinion study in the Americas offers substantial evidence of the advantages of this mode of research for the quality of the data produced by surveys in developing countries. LAPOP’s use of ADGYS offers a good example of the specific pros and cons of this mode of data collection in survey studies.
By constraining ex ante the way in which the interviewer sees the items and the questionnaire and by forcing the interviewer to enter one response for each question, CAPI systems reduce the chances that the interviewer might add error to the study. CAPI systems prevent the inclusion of some error that is caused by the interviewer at the moment of conducting the interview and entering the data.
By providing information related to the conditions in which the interview was conducted, particularly GPS coordinates and partial and total interview time, CAPI systems provide the team in charge of a survey study with the opportunity to exert wider control over the field process. Paradata analysis drastically reduces the opportunities for the interviewers to select and collect data from areas not included in the sample. Interview duration can also help control fieldwork by giving the team in charge a better understanding of how data are really collected in the field. As a result of these possibilities, paradata discourage fraud being committed by interviewers.
While CAPI surveys do not solve all problems related to fieldwork or prevent all sources of error in a survey study, they provide useful resources for improving the quality of the data in surveys conducted in developing countries. As computer technology and cell phone infrastructure and connectivity advance swiftly, researchers should take advantage of the increasing opportunities for improving the conditions under which data are collected.
Banks, R., and H. Laurie. 2000. “From PAPI to CAPI: The Case of the British Household Panel Survey.” Social Science Computer Review 18 (4): 397–406.Find this resource:
Biemer, P., R. Groves, L. Lyberg, N. Mathiowetz, and S. Sudman, eds. 1991. Measurement Errors in Surveys. New York: Wiley.Find this resource:
Böhme, M., and T. Stöhr. 2014. “Household Interview Duration Analysis in CAPI Survey Management.” Field Methods 26 (4): 390–405.Find this resource:
Caviglia-Harris, J., S. Hall, K. Mullan, C. Macintyre, S. C. Bauch, D. Harris, . . . H. Cha. 2012. “Improving Household Surveys Through Computer-Assisted Data Collection: Use of Touch-Screen Laptops in Challenging Environments.” Field Methods 24 (1): 74–94.Find this resource:
Couper, M. 2000. “Usability Evaluation of Computer-Assisted Survey Instruments.” Social Science Computer Review 18 (4): 384–396.Find this resource:
Couper, M. 2005. “Technology Trends in Survey Data Collection.” Social Science Computer Review 23 (4): 486–501.Find this resource:
Couper, M. 2011. “The Future of Modes of Data Collection.” Public Opinion Quarterly 75 (5): 889–908.Find this resource:
De Leeuw, E., J. Hox, and G. Snijkers. 1998. “The Effect of Computer-Assisted Interviewing on Data Quality: A Review. In Market Research and Information Technology: Application and Innovation, edited by B. Blyth, Amsterdam: ESOMAR.Find this resource:
Lyberg, L., P. Biemer, M. Collins, E. De Leeuw, C. Dippo, N. Schwarz, and D. Trewin, eds. 1997. Survey Measurement and Process Quality. New York: Wiley Interscience.Find this resource:
(p. 219) Olson, K. 2013. “Paradata for Nonresponse Adjustment.” Annals of the American Academy of Political and Social Sciences 645: 142–170.Find this resource:
Shirima, K., O. Mukasa, J. Armstrong-Schellenberg, F. Manzi, D. John, A. Mushi, . . . D. Schellenberg. 2007. “The Use of Personal Digital Assistants for Data Entry at the Point of Collection in a Large Household Survey in Southern Tanzania.” Emerging Themes in Epidemiology 4 (5).Find this resource:
Smith, T., and S. Kim. 2003. “A Review of CAPI-Effects on the 2002 General Social Survey.” GSS Methdological Report 98.Find this resource:
Tourangeau, R. 2005. “Survey Research and Societal Change.” Annual Review of Psychology 55: 775–801.Find this resource:
United Nations. 2005. Household Sample Surveys in Developing and Transition Countries. New York: United Nations.Find this resource:
van Heerden, A., S. Norris, S. Tollman, and L. Richter. 2014. “Collecting Health Research Data: Comparing Mobile Phone-Assisted Personal Interviewing to Paper-and-Pen Data Collection.” Field Methods 26 (4): 307–321.Find this resource:
(1.) For an ample discussion of error in survey studies see Biemer et al. (1991); for a more specific discussion of error in studies conducted in developing countries see the methodological report prepared by the United Nations (2005).
(3.) Some techniques that deal with this type of inconsistency have been developed and are available to survey researchers (Herzog, Scheuren, and Winkler 2007). While the different techniques available can improve the quality of a data set, they do so only partially and cannot be considered a replacement for good data coding and verified data entry.
(4.) LAPOP surveys can be accessed via the Internet at www.lapopsurveys.org. The research conducted by Ciudadanía is available at www.ciudadaniabolivia.org. Genso Iniciativas Web can be visited at www.genso.com. bo.
(5.) There are ethical implications regarding the collection of paradata, as it could potentially lead to the identification of respondents. Human subject protection standards recommended by professional associations such as WAPOR and enforced by most institutional IRB offices require that all information that could potentially lead to the identification of the respondent of an anonymous survey (as is the case in most public opinion studies) be removed from the public database. LAPOP and the ADGYS administration comply with this standard and do not include GPS data or any other information that, combined with the responses in the questionnaire, could lead to the identification of individual respondents, their homes, or their families.