MArketing research notes chapter8

UNIT II LESSON 8: TECHNIQUES OF DATA COLLECTION

Students so far we have studied about various research processes and the writing of report. The write up part we have done in detail. Now, we will be discussing in detail each and every step of research. After the research problem is framed along with the hypothesis the next step is the collection of required information and data. In this class we will be focusing specially on the collection of qualitative data for market research. The list of techniques and sources of data are

The audit process includes an opening meeting, factory tour, document review, interviews with employees and a closing meeting. The key parameters that we look at when carrying out retail audits are: In-store availability of product/brand;
• Types of outlets (by owner, location, specialty); • Sales volume cross-tabbed with type and location; • Pricing of product/brand cross-tabbed with type/location

RESEARCH METHODOLOGY

Sources of Market Data
Retail Audit Consumer Panel TV Meters Diary Method Internet as a source of Data Secondary Data Sources of Secondary Data RBI Economic Survey CSO Investment Data Foreign Trade Survey Data Types of Survey Techniques Now let us discuss these in detail Retail Audit Retail Audit is a common term in marketing research Audits During the 1990s, it became increasingly important to develop a strong brand image. It’s not just the product that needs to be sold, but also the brand, charged with values such as ethics, quality, feelings and identity that put over a positive message to consumers. Today, many companies are moving their production from their home countries to nations where manufacturing costs are considerably lower. However, the role of the company extends beyond just financial issues; every organisation has a social responsibility. Consumer and pressure groups are increasingly concerned about the social conditions in which workers from developed and developing countries are subjected. They expect companies to accept its responsibilities and to conduct its activities in accordance with the ethical and moral values accepted in the country in which their product is sold. Forced labor, child labor, low pay, poor conditions and dangerous working environments are all areas of serious concern to the reputable retailer or brand owner.

of outlet;
• Display value; • Customer demand; • Resulting market share and rank/position of product/

brand. It must be noted that there are no readily available retail universe data. The design of a retail audit is critical to the success of the project. The data obtained from the retail audit is useful for carrying out
• Identification of market opportunities • Trend analyses and forecasting • Studying market structure • Prioritisation of markets • Conducting analyses of competitors • Product portfolio analysis • Understanding changes in distribution • Pricing trend analyses • Product Categories Covered

This Audit covers more than 100 product categories including
• Baby products (oil, powder, diapers, milk food, weaning

food.)
• Beverages (coffee, soup mix, squash and juice, syrup, tea,

concentrated drinks.) • Contraceptives
• Cosmetics (colognes, deodorant, perfume, lipstick, nail

polish.)
• Environmental hygiene (air freshener, floor cleaner, floor

polish, etc.)
• Fabric care (fabric bleach, washing powder, liquid, whitener,

soap, detergent.)
• Food products (butter, margarine, salt, packaged food, etc.) • General toiletries (mouthwash, talcum powder, toilet soap,

toothpaste, toothbrush, sanitary napkins.)
• Hair care (conditioner, dye, oil, shampoo.)

48

© Copy Right: Rai University

11.556

• Health products and OTC (analgesic, digestive, medicated

dressing, etc.)
• Liquor (beer, brandy, gin, rum, vodka, whisky, wine, liquor.) • Milk products (milk, condensed milk, milk powder, Cheese.) • Semi-durable products (batteries, bulbs, lubricants, paint,

7. Usually the sources can be broken down into three basic groups: a. “White area”: from official stats sources to fully legal retail; b. “Grey area”: includes medium and small wholesale, and kiosks (partial reporting) original, but locally unauthorized product; c. “Black area”: private entrepreneurs operating without a license, ad-hoc open air markets, van sales, babushkas, etc. 8. Analysis and report writing After verification, the data are punched in (software and formats to be determined based on client needs), structured, analyzed, and presented as text, graphics, customized databases, or a combination of these.=Consumer Panel There’s nothing (consumer) panel data can tell us that we don’t already know from scanner data. Consumer panels are a unique tool that can enable a clever researcher to examine dynamic longitudinal changes in behaviors, attitudes, and perceptions. Consumer panels can also be an overly costly, excessive generator of unused data What are Consumer Panels? There are two basic kinds of consumer panels. In the first kind, respondents report essentially the same information repeatedly over some period of time. The chief examples of these kinds of panels are the syndicated purchase panels using store and home, termed as, continuous panels. The second kind of panel consists of samples of pre-screened respondents who report over time on a broad range of different topics, termed as discontinuous access panels. Both kinds of panels come in all different forms. Panel studies can involve data collection at widely different intervals varying anywhere from a day to several years between waves of interviews. Panel operators are continuously faced with the decision about how often panel members should be contacted and asked to report. Contacting the panel either too frequently or too infrequently may lead to reduced cooperation,
The Benefits of Continuous Consumer Panels

RESEARCH METHODOLOGY

tube lights, etc.)
• Shaving products (after-shaves, blades, razors, etc.) • Skin care (cream, cold cream, lotion, face-wash, etc.) • Snack foods and soft drinks (biscuits, chocolates,

confectionery, etc.) Measures
• Market size in terms of units sold, volume and value • Market share by volume and value • Numeric distribution • Weighted distribution • Share among handlers • Out-of-stock retailers • Per dealer off-take • Purchases by retailers • Stock levels with retailers • Stock turnover ratio • Trends for market, company, brand and SKU - for size and

shares Following Steps can be Followed We never assume that our clients will mean the same thing under “retail audit”. We always strive to define exactly the specific knowledge needs, and design the approach, methodology, and sample accordingly. Our experience has taught us that there can be no long-term representative samples. Each new project requires a revision of the existing sample size and structure in order to achieve credible results. 1. Draft the research plans and schedule, indicating.
• • • •

Scope and goals; Optimal sample size, methods of collecting quantitative & qualitative data, etc.; Deadlines; Structure and format of reports.

3. Fine-tuning and approval of research approach. 4. Design and production of customized research tools. 5. Launch and management of field research As a rule, we use the following field research methods:
• Observation • Face-to-face POS interviews • Mystery shopping

1. The effect of a special offer can be measured through a before-and-after design using a panel approach. Thus, a sample of families might be interviewed initially to gather information on their purchases of soft drinks, possibly over several weeks to obtain a good idea of their “steady state” purchasing patterns. A special deal for a particular brand is then introduced, and the purchases of the same sample are monitored for perhaps every week for three months. In this way, sampling variation is minimized and both short-term and long-term effects of the deal are obtained. 2. A static consumer panel of families with young children might be set up to monitor the acceptance of new line of toys. In this case no type of experimental treatment is involved. Rather, information is obtained, say, every month on the toy purchases of the families. In this way, data are compiled on the types of families that are buying any of the new toys, how soon the toys are purchased after they have

Note Do not expect data on opening stock/deliveries/closing stock, bar code data (scanning d-bases), audit code levels, etc. They are mostly non-existent. 6. Data collection

11.556

© Copy Right: Rai University

49

been placed on the market, and how many of the toys are purchased by each family. 3. A dynamic consumer panel might be used to keep track of the purchases of frozen foods of one brand in relation to other brands. By obtaining such data every week for several years, very detailed information can be obtained on what sorts of families are purchasing each major brand and on the change in market shares of the different brands over time among different groups of consumers. Also estimates can be derived of the extent to which purchasers remain loyal to different brands. 4. A continuous consumer panel may be used to obtain more detailed and reliable information on different types of behavior. It has been demonstrated that data on consumer financial holdings are obtained much more reliably if this information is sought over a period of time, thus allowing the respondent to build up confidence in the validity and trustworthiness of the study. Similarly, information on medical care events is obtained much more accurately from panels than from one-time surveys. 5. A continuous consumer panel is the only means of obtaining information on a series of events extended through time. For example, reactions to the weekly episodes of a television program are best obtained by monitoring the viewing of the same family and at the same time getting their reactions to the different programs. In this way it becomes possible to measure changes in program acceptance and to relate attitudes and behavior at one time to viewing and attitudes toward earlier episodes. 6. Only through continuous consumer panels is it possible to monitor changes in the behavior of particular cohorts. For example, the purchase habits of teenagers might be monitored over a number of years to ascertain how these purchase habits change as the subjects move into a different stage of life. By monitoring the behavior of peers at the same time, it becomes possible to distinguish effects due to history (i.e., changes in economic and social conditions) from effects due to the aging process. It is possible to use a series of demographically identical discontinuous access panels for the purposes of continuous tracking. Selecting demographically identical samples containing different panelists at predetermined intervals across time can do this. These different groups of panelists can then be used separately in the separate waves of the panel. Since the sample is not static, traditional static panel analytics, such as measures of trial and repeat and brand switching, are lost. What is gained, however, means of obtaining other insights in at a lower cost than it would be to maintain a continuous, full-time panel.
The Benefits of Discontinuous Consumer Access Panels

1. Screening for special populations (especially for rare special populations), 2. Evaluation of new product concepts and formulations, 3. Marketing and advertising experimentation The following examples illustrate some uses of discontinuous consumer access panels: 1. A manufacturer of tennis racquets is considering alternative shapes for a new racquet that would make it easier to handle. Initially, sheets with pictures and a description of the new racquets might be sent by mail or e-mail to pre-screened samples of respondents who play tennis. Any one respondent would receive only one of the alternatives, but the manufacturer could determine which racquet was preferred from the different samples. Alternatively, respondents might receive pictures of two racquets with the order of the pictures randomized, and asked for their preference between the two. At a later stage, respondents might receive the actual racquets for use testing. 2. Instead of a new product, a marketer might be considering a new advertising campaign for an existing product, and might wish to choose between several alternatives that had been proposed by the advertising agency. Again, samples of each of the alternatives would be sent to relevant panel members for their evaluation. As above, they might be asked to evaluate a single advertisement or to choose from among multiple advertisements. The testing could also be done by the advertising agency before the recommendation was made to the manufacturer. Similarly, panel members could be asked to evaluate different designs or layouts for a web page or a brochure. In all cases, the objective is to screen different ideas or executions inexpensively by having a panel evaluate them singly or side-by-side. It is obvious that similar information could be obtained from one-time surveys, but with greater difficulty and at greater expense. Two reasons for using discontinuous panels are because they can provide greater relevance and better quality. They are more relevant because respondents can be easily screened on the basis of prior questions (e.g., pet owners, users of denture cream, recent car purchasers) They are often better quality because respondents are experienced and can easily be pre-qualified as panel members on the basis of the quality of the previous survey responses. In the next section we discuss problems with discontinuous consumer panels that sometimes make one-time surveys the better alternative.

RESEARCH METHODOLOGY

Challenges of Panels?
Essentially, a continuous consumer panel operation poses four problems: 1.Gaining and maintaining cooperation, 2.Information validity and reliability, 3.Panel conditioning, and 4.Record maintenance. 1. Gaining and Maintaining Cooperation Mail, Internet, or the World Wide Web does even when panels are recruited by personal methods, the initial rate of cooperation can be as low as 50%, and may be much lower if recruiting. Those

The benefits of discontinuous consumer access panels are primarily related to reductions in the cost and time required obtaining market research information. Although these kinds of panels are used in a wide variety of ways, three uses are especially common

50

© Copy Right: Rai University

11.556

who cooperate are more likely to be more educated, in professional or clerical occupations, in middle or upper middle-income levels, and in the younger and middle age brackets. As a result, a panel at the beginning of the operation may not be representative of the population from which it was selected. This is only the beginning of the problem, however. Attrition can be substantial. About half the people that even consent to participate may drop out after the first two or three rounds, especially if they are asked to keep extensive written records. As a result, a panel operation can become increasingly unrepresentative of the population from which it originally came, something that could be a problem even for a static panel. Of course, panel operators take steps to reduce this non-representativeness through the use of a variety of methods such as selective recruiting and weighting the panel. 2. Information Validity and Reliability This problem of sample representativeness is also the key problem for discontinuous consumer panels. Almost all of these panels are recruited by mail with initial cooperation rates usually below 5%. As with continuous panels, there are also very high initial dropout rates. Operators of discontinuous panels make initial efforts to balance their samples for major demographic characteristics by selectively recruiting respondents from groups least likely to cooperate and by dropping from their panels respondents from groups that are over-represented. It is often claimed by operators of such panels that the response rate to an individual survey is 70 percent or higher, but this refers to respondents who had already previously agreed to participate. As with continuous panels, the data from individual surveys are weighted to further control for major demographic biases. Sample representativeness is also a concern when recruiting online panels. Critics argue that because on-line panels necessitate computer literacy and the means to access the Internet, it is biased against low-income groups and technological laggards. These concerns of representatives may be exacerbated depending on how the panel is recruited. Surfers who inadvertently stumble on to a site, or ones who are attracted by the lure of a lottery, may be even less representative than those recruited through more deliberate or personal means. Yet, just as efforts are made to make off-line panels more representative, so are many of these same efforts being used to make-to-make on-line research more representative. It is important to realize that “purpose defines precision.” The representatives and precision required to determine which of six package designs is most appealing is different (perhaps less important) than that needed to estimate the impact of a price change on market share. Independent of the population being sampled is the reliability of the information obtained from panel members. For discontinuous panels, these problems are identical to those conducting one-time surveys. Since many of the uses of the panel relate to attitudes and buying intentions, the only way of ultimately verifying the quality of responses is to observe marketplace results. The extensive and increased use of discontinuous panels suggests the responses obtained from these panels do provide information that is sufficiently accurate for making marketing decisions. For continuous panels that are more often measuring behavior, shipment data can validate results, particularly at an aggregate level. Certainly the introduction of portable household scanner equipment and electronic meters for

television viewing has increased validity, although even such equipment and meters do not prevent errors caused by respondents forgetting to use them. Many panels still rely on diaries. Although diaries significantly reduce reporting error as compared to recall, reporting errors do occur if panel members forget to make their entries, or attempt to recall and record earlier behavior at the end of the recording period instead of at the time that it occurred. This is especially a problem for behaviors that are infrequent and of low salience to respondents. 3. Panel Conditioning The third major problem that affects continuous panels, but probably not discontinuous ones is the danger of conditioning effects. That is, the possibility that behavior or attitudes of panel members will be influenced or contaminated by their participation in the panel. For example, respondents who keep diaries about visits to restaurants may become aware of the large amount of money they are spending on restaurant meals and either reduces the frequency of the visits to restaurants or switch to lower cost ones. In a similar fashion, a family asked month after month about ownership of savings accounts may decide to open a savings account, even though they originally had no such intention, Panel conditioning effects are both erratic and pervasive. They exist in some sorts of studies, but do not seem to exist in others. As with panel mortality, methods exist for detecting and correcting such effects. 4. Record Maintenance The fourth major problem of a continuous panel study is not so much methodological as of the researcher’s own making. This is the need for some systematic means of keeping easily accessible records on the activities of panel members and on changes in the characteristics of these panel members over time. Since members of a panel are usually households or families, rather than single individuals, there is the problem in a long-term panel study of keeping track of changes in the composition of these households and changes in their characteristics. A family member may leave the household, another may be born or move into it, a household may be dissolved, or a new household may be formed. In addition, the employment status and other characteristics of the individual members will change over time. All of these changes have to be recorded so that the data can be used for analytical purposes when required. The attitudes and behavior of the panel members also have to be recorded in such a way that the data are readily accessible, particularly so that analyses can be made either on a cross-sectional or longitudinal basis. Some idea of the magnitude of the problem can be obtained from the fact that in many panel studies a single round of data collection may provide information on 500 to 1,000 variables. If a panel has 10,000 families, which is not unusually high, and information is obtained every month for, say, five years, the number of pieces of information could be as high as 600,000,000. Fortunately, the capacity of computers has expanded so rapidly that the computers themselves are no longer the problem. The major problem was and remains the designing of computer systems that make storing and accessing the data straightforward. It is especially important for syndicated services to design systems that make client access to data fast and easy. Worldwide Major Panel Operators Country/ Firm/ Started/ Phone/ URL
51

RESEARCH METHODOLOGY

11.556

© Copy Right: Rai University

1. USA AC Nielsen 1933 203-961-3330 www.acnielsen.com 2. USA NFO Worldwide 1946 203-629-8888 www.nfow.com 3. USA Maritz Marketing Research, Inc. 1973 636-827-1610 www.maritz.com 4. USA Market Facts, Inc. 1946 847-590-7000 www.marketfacts.com 5. USA NPD Group, Inc. 1953 516-625-0700 www.npd.com 6. USA Opinion Research Corporation Intl. 1938 908-2815100www.opinionresearch.com 7. USA Taylor Nelson Sofres Intersearch 1960 215-442-9000 8. USA Roper Starch Worldwide, Inc. 1923 914-698-0800 www.roper.com 9. USA Burke, Inc. 1931 513-241-5663 www.burke.com 10.USA MORPACE International, Inc. 1940 248-737-5300 www.morpace.com 11.USA Creative and Response Rsch Services, Inc. 1960 312828-9200 www.crresearch.com 12.USA Harris Interactive, Inc. 1956 716-272-9020 www.harrisinteractive.com 13.USA Lieberman Research Worldwide 1973 310-553-0550 www.lrw.com 14.USA Ziment 1976 212-647-7200 15.UK BJM Research and Consultancy Ltd 1973 44-207891.1200 16.UK BMRB International 1933 44-20-8566.5000 www.bmrb.co.uk 17.UK The Gallup Organization 1937 44-208-939.7000 www.gallup.com 18.UK GfK Marketing Services Ltd 1992 44-870-603.8100 www.gfkms.co.uk 19.UK Harris Research 1965 44-20-8332.9898 20.UK Information Resources 1992 44-1344-746000 www.unfores.com 21.UK INFRATEST BURKE GROUP LTD. 1974 44-208782.3000 22.UK Ipsos-RSL Ltd. 1946 44-208861.8000www.ipsos.rslmedia.co.uk 23.UK Isis Research plc 1973 44-208788.8819www.isisresearch.com 24.UK Martin Hamblin 1969 44-20-7222.8181 www.martinhamblin.co.uk 25.UK Millward Brown UK Ltd 1973 44-1926-452233 www.millwardbrown.com 26.UK MORI (Market & Opinion Rsch Intl)) 1969 44-207222.0232 www.mori.com 27.UK MVA 1968 44-1483-728051 www.mva-research.com 28.UK ORC International 1938 44-20-7675.1000 www.opinionresearch.com 29.UK The Research Business International 1981 44-207923.6000 www.trbi.co.uk 30.UK Research International 1962 44-20-7656.5000

31.UK Research Resources Ltd 1986 44-20-7656.5555 32.Canada Angus Reid Group, Inc. 1979 1.613.241.5802 www.angusreid.com 33.Canada CF Group Inc. (ARC, Canadian Facts, 1932 1-416924.5751) Burke International Research 1. France B.V.A. 1970 33-1-30.84.88.00 www.bva.fr 2. France CSA (CSA TMO Group) 1983 33-1-41.86.22.00 www.csafr.com 3. France IFOP 1938 33-1-45.84.14.44 www.ifop.com 4. France Ipsos France 1975 33-1-53.68.28.28 5. France IRI-SECODIP 1993 33-1-30.06.22.00 6. France MEDIAMETRIE 1985 33-1-47.58.97.58 www.mediametrie.fr 7. France Research International 1952 33-1-44.06.65.65 www.researchint.com 8. France SECODIP / Groupe SOFRES 1969 33-1-30.74.80.80 www.secodip.com 9. Brazil IBOPE GROUP IBOPE Ad Hoc 1942 55-11-3066.1587 www.ibope.com.br 10. Brazil INDICATOR Pesquisa de Mercado Ltda. 1987 55-113365.3000 www.indicator.com.br 11.Brazil Instituto de Pesquisas Datafolha 1983 55-11-224.3933 12.Brazil MARPLAN BRASIL Pesquisas Ltda. 1958 55-113361.2033 www.marplan.com.br 13.India Indian Market Research Bureau (IMRB) 1970 91-22432.3636 www.imrbint.com 14.India Indica Research Pvt. Ltd. 1994 91-22-265.1741 www.indica.com 15.India MBL Rsch. & Consultancy Group Pvt. Ltd. 1987 91-40335.5433 www.mblindia.com 16.India ORG-MARG Research Ltd. 1961 91-22-218.6922 17.Indonesia PT AMI Indonesia 1996 62-21-521.3420 18.Japan Marketing Intelligence Corporation (MiC) 1960 81424.76.5164 www.micjapan.com
(K.K.Shakai-Chosa KenkyUsho)

RESEARCH METHODOLOGY

1. Japan Video Research Ltd. 1962 81-3-5541.6506 www.videor.jp 2. Korea Hyundai Research Institute 1986 82-2-737.2685 3. Turkey Procon 1. Japan NIKKEI RESEARCH INC. 1970 81-3-5281.2891 www.nikkei-r.co.jp 4 GfK Research Services 1997 91-212-216.21.91 www.procongfk.com TV Meters It’s easy to get lost in the details of audience research, so today; I’d like to begin with a very simple model for television research quality assessment. Specifically, I believe that an acceptable research service has to be thought of as having four quality components. If you’re trying to decide whether to use a new source of research, or if you’re trying to influence the priorities of an existing service, you’ve got to be able to address these four issues.

52

© Copy Right: Rai University

11.556

Good research always requires:
• A sample of the right people... • In sufficient quantity for your purpose... • Who provide accurate data about their behavior... • To a supplier with high-quality process controls.

Nelson AGB, which uses picture matching as an alternative means of identifying the channels tuned. One of the strengths of the BBM system was their open approach to testing this new meter. This is the way to pursue changes in technology-with an unusually rigorous test design, with an industry steering committee representing all parts of the industry, and conducted openly, with provision for two independent audits. It doesn’t get much better than that- and it’s often much less. Which leads nicely into the “how” issues. The methodology issues of
• Who, • How Many, and • What Data can only be considered?

RESEARCH METHODOLOGY

Take away any one of those, and you don’t have usable research. The “who” questions-getting a sample of the right people to participate in the research-often get the lion’s share of our attention. Sometimes I think we go overboard with meter panels, especially, sacrificing our knowledge of the other quality issues while putting samples under the microscope. But there’s no question that sampling issues are critical. We have to start off well, with a good knowledge of the population we’re measuring. And of course, there are response rates, the golden yardsticks of survey research. I only wish that other quality concerns had such appealing summary measures; maybe then we’d pay them more attention. Some people think that if your sample appears to mirror the population demographically, then your panel is “representative.” Panels and surveys are frequently judged more on composition than cooperation, and even getting appropriate data about cooperation can sometimes be a challenge. But that’s not enough. We have to insist on open disclosure of cooperation data for all major segments of the population. There are many shortcuts to a balanced panel, and we have to consider not just whether we have “enough,” but whether the ones we have are representative of their population segments. The importance of sample size doesn’t need much additional stress from me. But I’d like to remind you quickly that the overall reliability of surveys could have multiple components. So be wary of simplistic assumptions about sampling error. As for overall sample size: Frankly, there’s never enough sample, with meters, or diaries, or anything else. We’d always like to have more stable data. The real issue, though, is reducing the total error in our surveys. Having stable data isn’t very helpful if all you’ve done is make a bias more consistent. Here’s the one that I wish were easier to measure, because in my mind, it’s at least equal in importance to the other three quality factors. As important as it is to have a good sample of adequate size, it’s just as important to collect good data from that sample. I place a number of distinct issues under this heading. Obviously, we have the respondent’s ability and willingness to provide accurate data:
• Can they answer the question, and does the question make

in the applied context of how the work is actually being executed. That’s why it’s critical to have a transparent, verifiable system in place, with full disclosure of defined, objective policies and procedures. Obviously, research quality and usability can be very difficult to assess. The details can quickly overwhelm you. But in the end, you have to ask whether you’re really going to receive: • A sample of the right people...
• In sufficient quantity for your purpose... • Who provide accurate data about their behavior... • To a supplier with high-quality process controls.

I would argue that you need to have knowledge, and an opinion, on each of those four dimensions in order to make a good decision. Those are some of my core beliefs about media research. Let me segue from that to a few observations specific to meter measurement. Under the heading of “getting the right people,” television measurement, is challenged in at least two ways, one obvious and one not so obvious: Response Rate It would be easy to simply decry the fact that the television ratings have response rates that are painfully low; they are low, and we’re honestly not sure how much that affects us. It would also be easy to make excuses. There do appear to be major factors beyond the direct control of individual suppliers, causing an accelerating overall trend toward lower cooperation. But that’s too easy-on us. There are fixes to the declining response rate problem, but most of them cost money. Cash incentives, for example, are still the most potent way to affect response rates, and they work universally. But they also directly affect the unit costs of a research supplier. For a supplier to invest in response rates, they have to believe that it’s a priority with customers. Ability to Estimate the Universe Another sampling challenge concerns our ability to estimate the universe that we project our numbers to. Television measurement is dependent on many non-government measures of the population. Cable and satellite penetration and multi-set distribution are just two of them. To at least some users, those population estimates are of growing concern, too. One of the greatest concerns is the extent to which the population can change rapidly without us knowing about it.

sense?
• Will they push their buttons, and what do they mean when

they do so?
• And just how much of this task will they tolerate, anyway?

Beyond the capabilities of the respondent lie the technical capabilities of any equipment used in the survey or panel. There are tremendous variations in meter equipment around the world, each with its own strengths and weaknesses, and more variations are coming. For example, the BBM in Canada had rolled out an intriguing new people meter design for use nationally and in local markets. BBM has licensed a newly designed people meter from Taylor

11.556

© Copy Right: Rai University

53

Under the heading of “what”-of trying to collect better quality data from our samples-let me focus today just on meter measurement of set tuning. Internationally, there’s a lot of effort directed at better measurement of set tuning with meters. New media technologies pose significant near-term threats to the current generation of television set meters. The next challenge is directto-home satellite, and digital television in general. That’s not measured today. Statistical Sampling This is the same technique that pollsters use to predict the outcome of elections. A “sample audience” is created, and then count how many in that audience view each program. The researcher then extrapolates from the sample and estimates the number of viewers in the entire population watching the show. That’s a simple way of explaining what is a complicated, extensive process. The researcher relies mainly on information collected from TV set meters that it installs, and then combines this information with huge databases of the programs that appear on each TV station and cable channel. To find out who is watching TV and what they are watching, the research company gets around 5,000 households to agree to be a part of the representative sample for the national ratings estimates. Then TVs, homes, programs, and people are measured in a variety of ways. To find out what people are watching, meters installed in the selected sample of homes track when TV sets are on and what channels they are tuned to. A “black box,” which is just a computer and modem, gathers and sends all this information to the company’s central computer every night. Then by monitoring what is on TV at any given time, the company is able to keep track of how many people watch which program. Small boxes, placed near the TV sets of those in the national sample, measure who is watching by giving each member of the household a button to turn on and off to show when he or she begins and ends viewing. This information is also collected each night. The national TV ratings largely rely on these meters. To ensure reasonably accurate results, the company uses audits and quality checks and regularly compares the ratings it gets from different samples and measurement methods. This research is very costly. Advertisers pay to air their commercials on TV programs using rates that are based on these data. Programmers also use these data to decide which shows to keep and which to cancel. A show that has several million viewers may seem popular to us, but a network may need millions more watching that program to make it a financial success. That’s why some shows with a loyal following still get canceled. Using Diaries in Social Research Biographers, historians and literary scholars have long considered diary documents to be of major importance for telling history. More recently, sociologists have taken seriously the idea of using personal documents to construct pictures of social reality from the actors’ perspective (see Plummer’s 1983 book Documents of Life). In contrast to these ‘journal’ types of accounts, diaries are used as research instruments to collect detailed information about behaviour, events and other aspects of individuals’ daily lives.

Self-completion diaries have a number of advantages over other data collections methods. First, diaries can provide a reliable alternative to the traditional interview method for events that are difficult to recall accurately or that are easily forgotten. Second, like other self-completion methods, diaries can help to overcome the problems associated with collecting sensitive information by personal interview. Finally, they can be used to supplement interview data to provide a rich source of information on respondents’ behaviour and experiences on a daily basis. The ‘diary interview method’ where the diary-keeping period is followed by an interview asking detailed questions about the diary entries is considered to be one of the most reliable methods of obtaining information. The Subject Matter of Diary Surveys: A popular topic of investigation for economists, market researchers, and more recently sociologists, has been the way in which people spend their time. Accounts of time use can tell us much about quality of life, social and economic well being and patterns of leisure and work. The ‘time-budget’ involved respondents keeping a detailed log of how they allocated their time during the day. More qualitative studies have used a “standard day” diary, which focuses on a typical day in the life of an individual from a particular group or community. One of the most fruitful time-budget endeavors, initiated in the mid 60s, has been the Multinational Time Budget Time Use Project. Its aim was to provide a set of procedures and guidance on how to collect and analyse time-use data so that valid crossnational comparisons could be made Two other major areas where diaries are often used are:
• Consumer expenditure and • Transport planning research.

RESEARCH METHODOLOGY

Other topics covered using diary methods are social networks, health, illness and associated behaviour, diet and nutrition, social work and other areas of social policy, clinical psychology and family therapy, crime behaviour, alcohol consumption and drug usage, and sexual behaviour. Diaries are also increasingly being used in market research. Using Diaries in Surveys Diary surveys often use a personal interview to collect additional background information about the household and sometimes about behaviour or events of interest that the diary will not capture (such as large items of expenditure for consumer expenditure surveys). A placing interview is important for explaining the diary keeping procedures to the respondent and a concluding interview may be used to check on the completeness of the recorded entries. Often retrospective estimates of the behaviour occurring over the diary period are collected at the final interview. Diary Design and Format Diaries may be open format, allowing respondents to record activities and events in their own words, or they can be highly structured where all activities are pre-categorized. An obvious advantage of the free format is that it allows for greater opportunity to recode and analyse the data. However, the labour intensive work required to prepare and make sense of the data may render it unrealistic for projects lacking time and resources, or where the sample is large. Although the design of a diary will depend on the
11.556

54

© Copy Right: Rai University

detailed requirement of the topic under study, there are certain design aspects, which are common to most. Below are sets of guidelines recommended for anyone thinking about designing a diary. Furthermore, the amount of piloting required to perfect the diary format should not be under-estimated. 1. An A4 booklet of about 5 to 20 pages is desirable, depending on the nature of the diary. Disappointing, as it might seem, most respondents do not carry their diaries around with them. 2. The inside cover page should contain a clear set of instructions on how to complete the diary. This should stress the importance of recording events as soon as possible after they occur and how the respondent should try not to let the diary keeping influence their behaviour. 3. A model example of a correctly completed diary should feature on the second page. 4. Depending on how long a period the diary will cover, each page denoting either a week, a day of the week or a 24 hour period or less. Pages should be clearly ruled up as a calendar with prominent headings and enough space to enter all the desired information (such as what the respondent was doing, at what time, where, who with and how they felt at the time, and so on). 5. Checklists of the items, events or behaviour to help jog the diary keeper’s memory should be printed somewhere fairly prominent. Very long lists should be avoided since they may be off-putting and confusing to respondents. For a structured time budget diary, an exhaustive list of all possible relevant activities should be listed together with the appropriate codes. Where more than one type of activity is to be entered, that is, primary and secondary (or background) activities, guidance should be given on how to deal with “competing” or multiple activities. 6. There should be an explanation of what is meant by the unit of observation, such as a “session”, an “event” or a “fixed time block”. Where respondents are given more freedom in naming their activities and the activities are to be coded later, it is important to give strict guidelines on what type of behaviour to include, what definitely to exclude and the level of detail required. Time budget diaries without fixed time blocks should include columns for start and finish times for activities. 7. Appropriate terminology or lists of activities should be designed to meet the needs of the sample under study, and if necessary, different versions of the diary should be used for different groups. 8. Following the diary pages it is useful to include a simple set of questions for the respondent to complete, asking, among other things, whether the diary-keeping period was atypical in any way compared to usual daily life. It is also good practice to include a page at the end asking for the respondents’ own comments and clarifications of any peculiarities relating to their entries. Even if these remarks will not be systematically analysed, they may prove helpful at the editing or coding stage.

Data Quality and Response Rates In addition to the types of errors encountered in all survey methods, diaries are especially prone to errors arising from respondent conditioning, incomplete recording of information and under-reporting, inadequate recall, insufficient cooperation and sample selection bias. Diary Keeping Period The period, over which a diary is to be kept needs to be long enough to capture the behaviour or events of interest without jeopardizing successful completion by imposing an overly burdensome task for collecting time-use data, anything from one to three day diaries may be used. Household expenditure surveys usually place diaries on specific days to ensure an even coverage across the week and distribute their fieldwork over the year to ensure seasonal variation in earnings and spending is captured.

RESEARCH METHODOLOGY

Reporting Errors
In household expenditure surveys it is routinely found that the first day and first week of diary keeping shows higher reporting of expenditure than the following days. This is also observed for other types of behaviour and the effects are generally termed “first day effects”. They may be due to respondents changing their behaviour as a result of keeping the diary (conditioning), or becoming less conscientious than when they started the diary. Recall errors may also extend to ‘tomorrow’ diaries. Respondents often write down their entries at the end of a day and only a small minority are diligent (and perhaps obsessive!) diary keepers who carry their diary with them at all times. Expenditure surveys find that an intermediate visit from an interviewer during the diary keeping period helps preserve ‘good’ diary keeping to the end of the period. Literacy All methods that involve self-completion of information demand that the respondent has a reasonable standard of literacy. Thus the diary sample and the data may be biased towards the population of competent diary keepers. Participation The best response rates for diary surveys are achieved when diary keepers are recruited on a face-to-face basis, rather than by post. Personal collection of diaries also allows any problems in the completed diary to be sorted out on the spot. Success may also depend on the quality of interviewing staff that should be highly motivated, competent and well briefed. Appealing to respondent’s altruistic nature, reassuring them of confidentiality and offering incentives are thought to influence co-operation in diary surveys. One research company gives a 10-pound postal order for completion of their fourteen-day diary and other surveys offer lottery tickets or small promotional items. Coding, Editing and Processing The amount of work required to process a diary depends largely on how structured it is. For many large-scale diary surveys, the interviewer while still in the field does part has the editing and coding process. Following this is an intensive editing procedure, which includes checking entries against information collected in the personal interview. For unstructured diaries, involving coding of verbatim entries, the processing can be very labour intensive; in much the same way as it is for processing qualitative interview

11.556

© Copy Right: Rai University

55

transcripts. Using highly trained coders and a rigorous unambiguous coding scheme is very important particularly where there is no clear demarcation of events or behaviour in the diary entries. Clearly, a well-designed diary with a coherent pre-coding system should cut down on the degree of editing and coding. Relative Cost of Diary Surveys The diary method is generally more expensive than the personal interview, and personal placement and pick-up visits are more costly than postal administration. The interviewers usually make at least two visits and are often expected to spend time checking the diary with the respondent. If the diary is unstructured, intensive editing and coding will push up the costs. However, these costs must be balanced against the superiority of the diary method in obtaining more accurate data, particularly where the recall method gives poor results. The ratio of costs for diaries compared with recall time budgets are of the order of three or four to one Computer Software for Processing and Analysis Probably the least developed area relating to the diary method is the computer storage and analysis of diary data. One of the problems of developing software for processing and manipulating diary data is the complexity and bulk of the information collected. Although computer assisted methods may help to reduce the amount of manual preparatory work, there are few packages and most of them are custom built to suit the specifics of a particular project. Time-budget researchers are probably the most advanced group of users of machine-readable diary data and the structure of these data allows them to use traditional statistical packages for analysis. More recently, methods of analysis based on algorithms for searching for patterns of behaviour in diary data are being used. Software development is certainly an area, which merits future attention. For textual diaries, qualitative software packages such as The ETHNOGRAPH can be used to code them in the same way as interview transcripts. Archiving Diary Data In spite of the abundance of data derived from diary surveys across a wide range of disciplines, little is available to other researchers for secondary analysis (further analysis of data already collected). This is perhaps not surprising given that the budget for many diary surveys does not extend to systematic processing of the data. Many diary surveys are small-scale investigative studies that have been carried out with very specific aims in mind. For these less structured diaries, for which a common coding scheme is neither feasible, nor possibly desirable, an answer to public access is to deposit the original survey documents in an archive. This kind of data bank gives the researcher access to original diary documents allowing them to make use of the data in ways to suit their own research strategy. However, the ethics of making personal documents public (even if in the limited academic sense) have to be considered Internet as a Source of Data: The expansion of the Internet over the past decade has provided the researcher with a range of new opportunities for finding information, networking, conducting research, and disseminating research results.

Through the use of tools such as online focus groups, electronic mail, and online questionnaires, the Internet opens up new possibilities for conducting research. It offers, for example: 1. Shorter timeframes for collecting and recording data: e-mail messages can be saved and analyzed in qualitative data packages, for example, while online surveys can be captured directly into a database 2. The possibility of conducting interviews and focus groups by e-mail, with related savings in costs and time 3. New “communities” to serve as the object of social scientific enquiry. 4. Opportunities for including mixed multiple media in questionnaires.On the other hand, these opportunities also raise new challenges for the researcher, Such as
• Problems of sampling • The ethics of conducting research into online communities • Physical access and skills required to use the technologies

RESEARCH METHODOLOGY

involved
• Accuracy and reliability of information obtained from online

sources
• The changed chronology of interaction resulting from

asynchronous communication Internet is a useful media to get valuable information and results of various surveys. Access to computer-led data becomes handy in solving many complex mysteries, related to the market place. The 10 ‘C’s outlined here, provide criteria to be considered while evaluating Internet resources: 1.Content What is the intent of the content? Are the title and author identified? Is the content “juried?” Is the content “popular” or “scholarly”, satiric or serious? What is the date of the document or article? Is the “edition” current? Do you have the latest version? (Is this important?) How do you know? 2.Credibility Is the author identifiable and reliable? Is the content credible? Authoritative? Should it be? What is the purpose of the information, that is, is it serious, satiric, humorous? Is the URL extension .edu, .com, .gov or .org? What does this tell you about the “publisher”? 3.Critical Thinking How can you apply critical thinking skills, including previous knowledge and experience, to evaluate Internet resources? Can you identify the author, publisher, edition, etc. as you would with a “traditionally” published resource? What criteria do you use to evaluate Internet resources? 4.Copyright Even if the copyright notice does not appear prominently, someone wrote, or is responsible for, the creation of a document, graphic, sound or image, and the material falls under the copyright conventions. “Fair use” applies to short, cited excerpts, usually as an example for commentary or research. Materials are in the “public

56

© Copy Right: Rai University

11.556

domain” if this is explicitly stated. Internet users, as users of print media, must respect copyright. 5.Citation Internet resources should be cited to identify sources used, both to give credit to the author and to provide the reader with avenues for further research. Standard style manuals (print and online) provide some examples of how to cite Internet documents, although these standards are not uniform. 6.Continuity Will the Internet site be maintained and updated? Is it now and will it continue to be free? Can you rely on this source over time to provide up-to-date information? Some good .edu sites have moved to .com, with possible cost implications. Other sites offer partial use for free, and charge fees for continued or in-depth use. 7.Censorship Is your discussion list “moderated”? What does this mean? Does your search engine or index look for all words or are some words excluded? Is this censorship? Does your institution, based on its mission, parent organization or space limitations, apply some restrictions to Internet use? Consider censorship and privacy issues when using the Internet. 8.Connectivity If more than one user will need to access a site, consider each users’ access and “functionality.” How do users connect to the Internet and what kind of connection does the assigned resource require? Does access to the resource require a graphical user interface? If it is a popular (busy) resource, will it be accessible in the time frame needed? Is it accessible by more than one Internet tool? Do users have access to the same Internet tools and applications? Are users familiar with the tools and applications? Is the site “viewable” by all Web browsers? 9.Comparability Does the Internet resource have an identified comparable print or CD ROM data set or source? Does the Internet site contain comparable and complete information? (For example, some newspapers have partial but not full text information on the Internet.) Do you need to compare data or statistics over time? Can you identify sources for comparable earlier or later data? Comparability of data may or may not be important, depending on your project. 10. Context What is the context for your research? Can you find “anything” on your topic, that is, commentary, opinion, narrative, statistics and your quest will be satisfied? Are you looking for current or historical information? Definitions? Research studies or articles? How does Internet information fit in the overall information context of your subject? Before you start searching, define the research context and research needs and decide what sources might be best to use to successfully fill information needs without data overload.

other institutions or other parts of the world can work at the times that suit them. It is easy to share resources of all kinds amongst a group of researchers and to make results available to as many or as few people as you desire. Once you get used to map reading on the Internet you will find that you have access to a huge variety of resources including the expertise of the many other network users, the power of many computers and programs and the stored information from millions of documents. Getting Connected If you are working in a higher education institution you should have easy access to a network connection of some sort. Since the institution’s connection and use of it are paid for en bloc, you will not incur any charges for using the network, though to start with you may have to buy an add-on such as an Ethernet card for your machine. If you are a researcher connected with, or working for, a higher education institution but you do the majority of your work at home, then you should talk to your computer support staff about connecting to the institution via a modem, using your home telephone. Most institutions will allow this, and once you have made the initial connection you should be able to get “out” onto the Internet without incurring any charges other than the local phone call between your home and the institution. If you are an independent researcher with no links to an academic institution, then you must use a commercial service to access the Internet. A list of such commercial services appears below. There are various levels of service available from such providers. Make sure that the service you get is the service you need - if you want to be able to use telnet and FTP (described below) then don’t subscribe to an e-mail only service. Etiquette - a Sense of Timing Most services on the Internet are made available through volunteer effort. Many services, particularly in the USA, are very busy during their working day. Please try to use US- based services in the morning, before their day starts. You will find that access times are much improved and that you get less “system busy” messages. Interactive Access (telnet and PAD>) Sometimes known as Terminal Access, this is the process in which you make a connection between the machine on your desk and another computer (a “remote host”). The information stored on the remote computer appears on your screen. In this way you can read news items and bulletin boards, search through library catalogues and data archives, browse through articles and sometimes books and, if you find them useful, ask the remote computer to e-mail them to you. There are many thousands of computers, which freely allow public access in this way. If you have never used interactive access before try accessing the NISS Gateway, a sort of information supermarket from whose menus you can choose a variety of other services. From a PAD> prompt type: call uk.ac.niss or make a telnet call to: niss.ac.uk If you don’t know how to make either of these calls please ask your computer services.

RESEARCH METHODOLOGY

Why use the Networks?
International computer networks provide a very cheap and effective vehicle for collaboration and communication. Long distances and time zones do not disrupt the process and your colleagues in

11.556

© Copy Right: Rai University

57

The NISS Gateway is an example of a centrally funded, publicly accessible service. Other services only permit access once you or your institution has paid a subscription. Probably the best known such service in the UK is the BIDS ISI service run from the University of Bath. If your academic institution subscribes to BIDS, you can obtain a username and password from your library or computing service. This allows you to search the Social Science Citation Index looking for citations (references) quoted in articles from several thousand journals, published from 1981 onwards. BIDS is a service provided for the academic community, but there are also commercial services, which require subscription and/or make access charges. This article concentrates on using freely accessible resources on the Internet with no charges attached. The program, which allows you to interactively access other computers on the Internet, is called telnet. You may be able to run telnet directly from the machine on your desk, or alternatively you may be able to run it from an account you have on a larger local computer, perhaps the one that handles your e-mail. Many sites are not yet able to use telnet directly, if you are in this position you will probably access the UK JANET via a PAD> prompt. The NISS Gateway at Bath runs a Guest Telnet Service to allow you to use telnet to access the rest of the world via the Internet. If you have a telnet name of a service you wish to call, at your PAD> prompt type call uk.ac.niss then from the menus choose: General Services and then: NSF-Relay Guest Telnet Service You will then be prompted for the name or the number of the service you wish to call using telnet. All services on the Internet have a unique number, which is called the IP address (Internet Protocol). Most also have a name (the Domain Name) which is a lot easier to remember. For example, the NISS Gateway’s domain name is: niss.ac.uk And it’s IP address is: 196.63.76.1 When you are using telnet you can usually use either of these, they will get you through to the same place. But sometimes, if you are away from your home site, particularly abroad, or if the machine which works out (“resolves”) which number matches which name has gone wrong, then you will need to know the number. If you are relying on reading your e-mail when you are away from home it is always a good idea to have a note of the IP address (the number) as well as the name. E-mail and More Many academics have discovered the advantages of using e- mail. Your correspondent doesn’t have to be available to take your call - she may be in a meeting or, if she’s on the other side of the world, still asleep - yet your message will be waiting for her as soon as she switches on her machine. E-mail allows quick question and answer sessions, rapid revisions and corrections to documents and is an ideal medium for collaboration and supervision. Where e-mail really scores is in-group communication. A group of geographically disparate researchers can continually consult and
58

keep in contact using the discussion list facilities available over email. Your one message can be circulated to the whole group and any replies or comments also seen by the whole group in a matter of minutes. Messages are archived for future reference and some systems also allow important files to be stored and retrieved by list members. There is a general discussion list for UK sociologists. To join this list sends an e-mail message to: [email protected] Also there are many other academic discussion lists on the UK service Mail base, and many thousands of lists on other systems worldwide. A few lists that may be of interest to sociologists are mentioned at the end of this article. File Transfer Anything that can be stored as a file on a computer can be transferred over the networks from one computer to another. This includes word-processed documents, datasets, software and graphics. File transfer over the Internet is known as FTP (file transfer protocol). Many sites on the Internet have set up repositories of files, which are freely available for you to transfer. These repositories are known as FTP Archives. The process of transfer is known as Anonymous FTP because you don’t need to identify yourself with a password in order to use such systems. If you can use telnet from your machine then you can probably use ftp too. Example of a file transfer session: When the location of a file available for anonymous ftp is quoted it will probably look something like this: Filename: Readme Site: ftp.bris.ac.uk Directory: pub/info/networks/generic To transfer this file to your local machine you would follow this procedure: At the system prompt type: ftp ftp.bris.ac.uk This will connect you to the ftp archive at the University of Bristol. You will be prompted for your username, type: anonymous You will then be asked for your password, it is polite to enter your e-mail address e.g.: [email protected] You should then see on your screen the prompt: ftp> Now type: dir This will give you a directory listing of the contents of the root or base directory. Now type cd pub/info/networks/generic You have now changed into the appropriate directory, use dir again to look at the contents. You will see that one of the files is called README (note the capital letters - case is important in such systems. Now type: hash

RESEARCH METHODOLOGY

© Copy Right: Rai University

11.556

this asks the system to put hash symbols (#) on your screen while it is working. This is reassuring - sometimes the process takes several minutes. Now type: get README While the process is working the hash symbols will appear on your screen. When the transfer is complete you will get a message saying so. Leave the system by typing: bye and look for the file on your home system. It will explain to you the rest of the contents of the directory: pub/info/networks/generic. The directory contains a set of practical exercises designed for social scientists to explore the networks. You may want to print them out and investigate them at your leisure.

• Information on the ESRC research awards and the

RESEARCH METHODOLOGY

surrounding publications resulting from these awards (journals, audio-visual material, books, databases, software, etc)
Access
• PAD> call uk.ac.ed.ercvax Username: rapid Password: rapid

or telnet ercvax.ed.ac.uk Username: rapid Password: rapid Discussion Lists When you join an e-mail discussion list it is often called subscribing (but you don’t have to pay). You will then receive all the messages that are sent to that list. To join a list you send a specific message to the machine (or sometimes the person) that runs the list, NOT to the list itself. Messages sent to the list itself are distributed to ALL the list members. The lists below each appear with their subscription address and the correct text of the e-mail message to send. In each case replace first name last name with your names. Social-Theory
• This list aims to provide a forum for the discussion of social

Resources for Sociologists
CoombsQuest - Social Science Research Data Bank
• Collection of 21 databases on material specific to the Pacific

region and South and North East Asia. Includes papers, bibliographies, directories and abstracts of theses. Server points to other social science information sites around the world. Access
• telnet info.anu.edu.au login: info

theory with the social sciences. Particular emphasis will be placed upon the relationship between psychology and sociology with special reference to the individual and social processes. Subscription address: [email protected] Message text: subscribe social-theory first name lastname List Details POR - Public Opinion Research
• Unmoderated discussion list for academics and professionals

Many files from the Coombs archives are available for file transfer, using ftp: sitename: unix.hensa.ac.uk directory: pub/uunet/doc/papers/coombspapers BIRON - Bibliographic Information Retrieval Online
• Online bibliographic details to the ESRC catalogue and

interested in public opinion research, useful to researchers currently conducting survey research projects. Subscription address: [email protected] Message text: subscribe por first name lastname PSN - Progressive Sociologists Network
• International discussion list for sociologists concerned with

subject index. Contains over 3000 datasets including the General Household Survey and Census of Great Britain.
• Access • PAD> call uk.ac.essex.solb1 Login: biron Password: norib

or Telnet dasun.essex.ac.uk Login: biron Password: norib
Pen Pages
• An American information server concerning all aspects of

progressive issues and values such as civil rights struggles, women’s rights, community development, etc. Membership is mostly from the US, Canada and Western Europe. Subscription address: [email protected] Message text: subscribe psn firstname lastname SOCORG-K - Social Organisation of Knowledge Discussion
• Moderated discussion list for sociologists.

rural life, it contains several databases including MAPP - the national Co-operative Extension family database. This is provided by the Dept of Agricultural Economics and Rural Sociology at Pennsylvania University and contains research briefs, bibliographies, census data, reference materails, publications, etc. Some is of a local nature but there is also general interest material available. Server also hosts the Senior Series Database and the 4-H Youth Development Database.
Access
• Telnet: psupen.psu.edu

Subscription address: [email protected] Message text: subscribe socorg-k firstname lastname SOS-DATA - Social science data discussion list
• Provides a forum for discussion of any topic related to social

Login: world RAPID - Research Activities and Publications Information Database

science data. The list is used most frequently to ask for references to sources of data on some particular subject but it also includes announcements of conferences and new data resources. Subscription address: [email protected] Message text: subscribe sos-data firstname lastname Electronic Journal: Psycoloquy

11.556

© Copy Right: Rai University

59

• Refereed electronic journal intended to implement peer

review over the network. The journal is primarily for psychologists but it is interdisciplinary in the topics covered and includes articles, book reviews, queries and announcements, etc. Subscription address: [email protected] Message text: subscribe psych firstname lastname Many files from the Psycoloquy journal are available for file transfer, using ftp: sitename: ftp: princeton.edu (128.112.128.1) directory: /pub harnad

were not present. A more radical response is to argue that the design, conduct and analysis of both qualitative and quantitative research are always contingent upon the contextualisation and interpretation of subjects’ situation and responses. Thus, secondary analysis is no more problematic than other forms of empirical inquiry, all of which, at some stage, depend on the researcher’s ability to form critical insights based on inter-subjective understanding.
2. Origin

RESEARCH METHODOLOGY

Secondary Analysis
Secondary analysis involves the use of existing data, collected for the purposes of a prior study, in order to pursue a research interest, which is distinct from that of the original work. In this respect, secondary analysis differs from systematic reviews and metaanalyses of qualitative studies, which aim instead to compile and assess the evidence relating to a common concern or area of practice. The approach may either be employed by researchers to re-use their own data or by independent analysts using previously established qualitative data sets. Why do Secondary Analysis? It has been contended that the approach can be used to generate new knowledge, new hypotheses, or support for existing theories; that it reduces the burden placed on respondents by negating the need to recruit further subjects; and that it allows wider use of data from rare or inaccessible respondents. In addition, it has been suggested that secondary analysis is a more convenient approach for particular researchers, notably students. It should also be noted that use of the approach does not necessarily preclude the possibility of collecting primary data. This may, for example, be required to obtain additional data or to pursue in a more controlled way the findings emerging from the initial analysis. There may also be a need to consult the primary researchers in order to investigate the circumstances of the original data generation and processing. Despite the interest in and arguments for developing secondary analysis of qualitative data, the approach has not been widely adopted to date. This raises questions about the desirability and feasibility of particular strategies for secondary analysis of qualitative data discussed below: Methodological and Ethical Considerations: Before highlighting some of the key practical and ethical issues, which have been discussed in the literature, there are two fundamental methodological issues to be considered.
1. Tenable

The second issue concerns the problem of where primary analysis stops and secondary analysis starts. Qualitative research is an iterative process and grounded theory in particular requires that questions undergo a process of formulation and refinement over time. For primary researchers re-using their own data it may be difficult to determine whether the research is part of the original enquiry or sufficiently new and distinct from it to qualify as secondary analysis. For independent analysts re-using other researchers’ data there are also related professional issues about the degree of overlap between their respective works. There is no easy solution to these problems except to say that greater awareness of secondary analysis might enable researchers to more appropriately recognise and define their work as such.
Compatibility of the Data with Secondary Analysis

Are the data amenable to secondary analysis? This will depend on the ‘fit’ between the purpose of the analysis and the nature and quality of the original data. Scope for additional in-depth analysis will vary depending on the nature of the data; for example, while tightly structured interviews tend to limit the range of responses, designs using semi-structured schedules may produce more rich and varied data. A check for the extent of missing data relevant to the secondary analysis but irrelevant to the original study may also be required; for example, where semistructured interviews involved the discretionary use of probes. More generally, the quality of original data will also need to be assessed.
Position of the Secondary Analyst

Was the analyst part of the original research team? This will influence the decision over whether to undertake secondary analysis and, if so, the procedures to be followed. Secondary analysts require access to the original data, including tapes and field notes, in order to re-examine the data with the new focus in mind. This is likely to be easier if they were part of the original research team. If not, then ideally they should also be able to consult with the primary researchers in order to assess the quality of the original work and to contextualise the material (rather than rely on field notes alone). Further consultation may also be helpful in terms of crosschecking the results of the secondary analysis. Finally, whether conducting secondary analysis in an independent capacity or not, some form of contractual agreement between the secondary analyst and the primary researchers, data archive managers, and colleagues involved in the primary research but not in the secondary analysis may have to be negotiated. Reporting of original and secondary data analysis Such is the complexity of secondary analysis, that it is particularly important that the study design, methods and issues involved are reported in full. Ideally this should include an outline of the original study

The first is whether secondary analysis of qualitative studies is tenable, given that it is often thought to involve an inter-subjective relationship between the researcher and the researched. In response, it may be argued that even where primary data is gathered via interviews or observation in qualitative studies, there may be more than one researcher involved. Hence within the research team the data still has to be contextualised and interpreted by those who

60

© Copy Right: Rai University

11.556

and data collection procedures, together with a description of the processes involved in categorising and summarising the data for the secondary analysis, as well as an account of how methodological and ethical considerations were addressed. Ethical Issue How was consent obtained in the original study? Where sensitive data is involved, informed consent cannot be presumed. Given that it is usually not feasible to seek additional consent, a professional judgment may have to be made about whether re-use of the data violates the contract made between subjects and the primary researchers. Growing interest in re-using data make it imperative that researchers in general now consider obtaining consent which covers the possibility of secondary analysis as well as the research in hand; this is consistent with professional guidelines on ethical practice. Developing the Approach To see if the potential of secondary analysis can be realised in practice, developmental work still needs to be undertaken:
• First, there should be a more comprehensive review of the

• A sample of voters is questioned in advance of an election

RESEARCH METHODOLOGY

to determine how the public perceives the candidates and the issues.
• A manufacturer does a survey of the potential market before

introducing a new product ... A government entity commissions a survey to gather the factual information it needs to evaluate existing legislation or to draft proposed new legislation. Not only do surveys have a wide variety of purposes, they also can be conducted in many ways-including over the telephone, by mail, or in person. Nonetheless, all surveys do have certain characteristics in common. Unlike a census, where all members of the population are studied, surveys gather information from only a portion of a population of interest -the size of the sample depending on the purpose of the study. In a bona fide survey, the sample is not selected haphazardly or only from persons who volunteer to participate. It is scientifically chosen so that each person in the population will have a measurable chance of selection. This way, the results can be reliably projected from the sample to the larger population. Information is collected by means of standardized procedures so that every individual is asked the same questions in more or less the same way. The survey’s intent is not to describe the particular individuals who, by chance, are part of the sample but to obtain a composite profile of the population. The industry standard for all reputable survey organizations is that individual respondents should never be identified in reporting survey findings. All of the survey’s results should be presented in completely anonymous summaries, such as statistical tables and charts.
How Large must the Sample Size be?

literature on secondary analysis and studies, which have explicitly (and perhaps implicitly) used this approach. This could include examination of the methods used, as well as the quality, value and impact of this work.
• Secondly, further work on the protocols for conducting

secondary analysis of qualitative data, particularly with regard to the re-use of other researchers’ data should be carried out.
• Thirdly, there should be greater consideration of the issues

involved in the secondary analysis of single, multiple and mixed data sets.
• Finally, some more specific guidelines are needed for

researchers about the ethical issues to be considered when undertaking qualitative work that may be re-used in the future

Conclusion
Despite growing interest in the re-use of qualitative data, secondary analysis remains an under-developed and ill-defined approach. Various methodological and ethical considerations pose a challenge for the would-be secondary analyst, particularly those who were not part of the primary research team. Further work to develop this approach is required to see if the potential benefits can actually be realised in practice. Survey This is an “information society.” That is, our major problems and tasks no longer mainly center on the production of the goods and services necessary for survival and comfort. Our “society,” thus, requires a prompt and accurate flow of information on preferences, needs, and behavior. It is in response to this critical need for information on the part of the government, business, and social institutions that so much reliance is placed on surveys. Then,What is a Survey? Today the word “survey” is used most often to describe a method of gathering information from a sample of individuals. This “sample” is usually just a fraction of the population being studied. For example:

The sample size required for a survey partly depends on the statistical quality needed for survey findings; this, in turn, relates to how the results will be used. Even so, there is no simple rule for sample size that can be used for all surveys. Much depends on the professional and financial resources available. Analysts, though, often find that a moderate sample size is sufficient statistically and operationally. For example, the well-known national polls frequently use samples of about 1,000 persons to get reasonable information about national attitudes and opinions. When it is realized that a properly selected sample of only 1,000 individuals can reflect various characteristics of the total population, it is easy to appreciate the value of using surveys to make informed decisions in a complex society such as ours. Surveys provide a speedy and economical means of determining facts about our economy and about people’s knowledge, attitudes, beliefs, expectations, and behaviors. Who Conducts Surveys? We all know about the public opinion surveys or “polls” that are reported by the press and broadcast media. They conduct surveys on national public opinion on a wide range of current issues. State polls and metropolitan area polls, often supported by a local newspaper or TV station, are reported regularly in many localities.

11.556

© Copy Right: Rai University

61

The major broadcasting networks and national news magazines also conduct polls and report their findings. The great majority of surveys, though, are not public opinion polls. Most are directed to a specific administrative, commercial, or scientific purpose. The wide variety of issues with which surveys deal is illustrated by the following listing of actual uses
• Major TV networks rely on surveys to tell them how many

by devices attached to a sample of TV sets that automatically record the channels being watched. Mail surveys can be relatively low in cost. As with any other survey, problems exist in their use when insufficient attention is given to getting high levels of cooperation. Mail surveys can be most effective when directed at particular groups, such as subscribers to a specialized magazine or members of a professional association. Telephone interviews are an efficient method of collecting some types of data and are being increasingly used. They lend themselves particularly well to situations where timeliness is a factor and the length of the survey is limited. In-person interviews in a respondent’s home or office are much more expensive than mail or telephone surveys. They may be necessary, however, especially when complex information is to be collected. Some surveys combine various methods. For instance, a survey worker may use the telephone to “screen” or locate eligible respondents (e.g., to locate older individuals eligible for Medicare) and then make appointments for an in-person interview. What Survey Questions Do You Ask? You can further classify surveys by their content. Surveys are concerned with:
• Opinions and attitudes (such as a pre-election survey of

RESEARCH METHODOLOGY

and what types of people are watching their programs
• Statistics Canada conducts continuing panel surveys of

children (and their families) to study educational and other needs
• Auto manufacturers use surveys to find out how satisfied

people are with their cars • The U.S. Bureau of the Census conducts a survey each month to obtain information on employment and unemployment in the nation
• The U.S. Agency for Health Care Policy and Research

sponsors a periodic survey to determine how much money people are spending for different types of medical care
• Local transportation authorities conduct surveys to acquire

information on commuting and travel habits
• Magazine and trade journals use surveys to find out what

their subscribers are reading
• Surveys are conducted to ascertain who uses our national

parks and other recreation facilities. Surveys provide an important source of basic scientific knowledge. Economists, psychologists, health professionals, political scientists, and sociologists conduct surveys to study such matters as income and expenditure patterns among households, the roots of ethnic or racial prejudice, the implications of health problems on people’s lives, comparative voting behavior, and the effects on family life of women working outside the home. What are Some Common Survey Methods? Surveys can be classified in many ways. Size and type of sample. Surveys also can be used to study either human or non-human populations (e.g., animate or inanimate objects-animals, soils, housing, etc.). While many of the principles are the same for all surveys, the focus here will be on methods for surveying individuals. Many surveys study all persons living in a defined area, but others might focus on special population groups - children, physicians, community leaders, the unemployed, or users of a particular product or service. Surveys may also be conducted with national, state, or local samples. Method of Data Collection Surveys can be classified by their method of data collection. Mail, telephone interview, and in-person interview surveys are the most common. Extracting data from samples of medical and other records is also frequently done. In newer methods of data collection, information is entered directly into computers either by a trained interviewer or, increasingly, by the respondent. One wellknown example is the measurement of TV audiences carried out

voters), • Factual characteristics or behaviors (such as people’s health, housing, consumer spending, or transportation habits). Many surveys combine questions of both types. Respondents may be asked if they have heard or read about an issue ... what they know about it ... their opinion ... how strongly they feel and why... their interest in the issue ... past experience with it ... and certain factual information that will help the survey analyst classify their responses (such as age, gender, marital status, occupation, and place of residence). Questions may be open-ended (“Why do you feel that way?”) or closed (“Do you approve or disapprove?”). Survey takers may ask respondents to rate a political candidate or a product on some type of scale, or they may ask for a ranking of various alternatives. The manner in which a question is asked can greatly affect the results of a survey. For example, a recent NBC/Wall Street Journal poll asked two very similar questions with very different results: 1. Do you favor cutting programs such as social security, medicare, medicaid, and farm subsidies to reduce the budget deficit? The results: 23% favor; 66% oppose; 11% no opinion. 2. Do you favor cutting government entitlements to reduce the budget deficit? The results: 61% favor; 25% oppose; 14% no opinion. The questionnaire may be very brief - a few questions, taking five minutes or less - or it can be quite long- requiring an hour or more of the respondent’s time. Since it is inefficient to identify and approach a large national sample for only a few items of information, there are “omnibus” surveys that combine the interests of several clients into a single interview. In these surveys, respondents will be asked a dozen questions on one subject, a half dozen more on another subject, and so on.
11.556

62

© Copy Right: Rai University

Because changes in attitudes or behavior cannot be reliably ascertained from a single interview, some surveys employ a “panel design,” in which the same respondents are interviewed on two or more occasions. Such surveys are often used during an election campaign or to chart a family’s health or purchasing pattern over a period of time. Who Works on Surveys? The survey worker best known to the public is the interviewer who calls on the telephone, appears at the door, or stops people at a shopping mall. Traditionally, survey interviewing, although occasionally requiring long days in the field, was mainly part-time work and, thus, well suited for individuals not wanting full-time employment or just wishing to supplement their regular income. Changes in the labor market and in the level of survey automation have begun to alter this pattern - with more and more survey takers seeking to work full time. Experience is not usually required for an interviewing job, although basic computer skills have become increasingly important for applicants. Most research organizations provide their own training for the interview task. The main requirements for interviewing are an ability to approach strangers (in person or on the phone), to persuade them to participate in the survey, and to collect the data needed in exact accordance with instructions. Less visible, but equally important are the in-house research staffs, who among other things- plan the survey, choose the sample, develop the questionnaire, supervise the interviews, process the data collected, analyze the data, and report the survey’s findings. In most survey research organizations, the senior staff will have taken courses in survey methods at the graduate level and will hold advanced degrees in sociology, statistics, marketing, or psychology, or they will have the equivalent in experience. Middle-level supervisors and research associates frequently have similar academic backgrounds to the senior staff or they have advanced out of the ranks of clerks, interviewers, or coders on the basis of their competence and experience. What about Confidentiality and Integrity? The confidentiality of the data supplied by respondents is of prime concern to all reputable survey organizations. Several professional organizations dealing with survey methods have codes of ethics that prescribe rules for keeping survey responses confidential. The recommended policy for survey organizations to safeguard such confidentiality includes:
• Using only number codes to link the respondent to a

• Presenting statistical tabulations by broad enough categories

RESEARCH METHODOLOGY

so that individual respondents cannot be singled out. What are Other Potential Concerns? The quality of a survey is largely determined by its purpose and the way it is conducted. Most call-in TV inquiries (e.g., 900 “polls”) or magazine write-in “polls,” for example, are highly suspect. These and other “selfselected opinion polls (SLOPS)” may be misleading since participants have not been scientifically selected. Typically, in SLOPS, persons with strong opinions (often negative) are more likely to respond. Surveys should be carried out solely to develop statistical information about a subject. They should not be designed to produce predetermined results or as a ruse for marketing and similar activities. Anyone asked to respond to a public opinion poll or concerned about the results should first decide whether the questions are fair. Another important violation of integrity occurs when what appears to be a survey is actually a vehicle for stimulating donations to a cause or for creating a mailing list to do direct marketing.

References
Donald R. Cooper – Business Research Methods, Tata McGraw –Hill Publication Easterby-Smith M et al- Management Research-an introduction (Sage Publications, 1991) Gallagher, J. William, “Report Writing for Management”, Addison-Wesley Golen, P. Stevan, “Report Writing for Business and Industry”, Business Communication Service Kothari C R – Quantitative Techniques (Vikas Publishing House 3rd ed.) Levin R I & Rubin DS - Statistics for Management (Prentice Hall of India, 2002) Nargundkar R – Marketing Research Text and Cases (Tata McGraw- Hill 2002) Notes

questionnaire and storing the name-to-code linkage information separately from the questionnaires
• Refusing to give the names and addresses of survey

respondents to anyone outside the survey organization, including clients
• Destroying questionnaires and identifying information

about respondents after the responses have been entered into the computer
• Omitting the names and addresses of survey respondents

from computer files used for analysis
© Copy Right: Rai University

11.556

63