Survey Research

Develop knowledge that is otherwise unavailable by developing an effective survey.

Surveys are a series of questions, which are usually presented in questionnaire format. Surveys can be distributed face-to-face, over the phone, or over the Internet.

Developing, Designing, and Distributing Surveys

Surveys are usually developed to obtain information that is otherwise unavailable. For example, suppose you feel that nationwide statistics on college-age drinking or cheating fail to adequately represent life on your campus. You could then develop a survey to demonstrate (or not demonstrate) your point of view.

Note: This section presents detailed instructions for developing, designing, and distributing surveys. Because English teachers are primarily concerned with how your survey is written, they may encourage you not to worry about some of the design issues addressed below, such as census, random, and stratified census sampling procedures

Determine your survey design by creating questions with your purpose and audience in mind.

To develop a credible survey, you must first organize a systematic approach to your study. This involves insuring your study sample actually represents the population of interest. For example, if you were conducting a survey of whether or not college students ever plagiarized or cheated on a test, you would not want to submit your survey to teachers or parents; you would want to hear from college students. A well-designed survey will reduce the chances of this type of misuse of the data.

Ask the Right Questions

Surveys most commonly take the form of a questionnaire. However, a questionnaire is not just a collection of questions, it is a set of questions focused on a particular objective or set of objectives. Defining objectives simply demonstrates you know what you are doing before you do it. If you have selected a topic like “The education of the 20-year-old,” you would most likely need to narrow the topic before designing your survey. One way to narrow the topic is to consider the when question. When does the education of the 20-year-old interest us? It could be cave people, residents of Europe during the plagues, or some time in the 20th century. Take your choice, but the application of a single survey to cover all the periods of time is highly unlikely.

Limit the Survey

Limit your focus by restricting a geographical region. You may want to survey 20-year-olds who live within 1,000 miles of the North Pole, or only those who live in Palaski County, Missouri. By establishing a geographical target population you can expect reasonable closure. If you wanted to interview all the subjects in your original category (the education of 20-year-olds), you may never get around to writing your report. Another factor that influences your survey is the extent to which you want to examine your population. Do you want to look at all the 20-year-old-students or can you focus on a specific subgroup (such as 20-year-old-students in trade schools or 2-year vs. 4-year colleges or, perhaps, the 20-year-old-students who are self-directed learners attending no formal institution)? What do you want to know about these people anyway? Are you interested in how they learn, why they learn, what they learn or perhaps, how much they spend?

Define Objectives

Ask yourself: What am I going to do with the information once it is revealed? You may plan to simply report your findings, make comparisons with other information or take the results of your survey to support or reject a whole new social theory. All these considerations should be met before beginning your survey research. A well-defined set of objectives eases the burden on the researcher.

Create a Survey Instrument

Learn to write effective survey questions.

After defining the population, surveyors need to create an instrument if a proven survey instrument does not exist. As discussed in more detail below, survey creation parallels the computer axiom “GIGO”–Garbage In—Garbage Out. A well-designed, accurate survey is an excellent way for researchers to gather information or data about a particular subject of interest. When prepared in an unprofessional manner, surveys can also become an inaccurate, misused, misunderstood conveyer of misinformation. If a researcher has developed an idea or hypothesis and seeks information only to prove that the idea is correct, the process, the ethics, the motivation, and the very essence of the researcher are worthy of rigorous challenge and should, most likely, be rejected.

Writing Survey Questions

A survey is more than a list of questions. Not only should each question relate to a specific part of your objectives, but each question should be written in a concise and precise manner, and the survey population must be able to understand what is being asked.

There are four common question formats, as discussed in detail below.

Demographic Questions

Demographic questions attempt to identify the characteristics of the population or sample completing the survey.

Your English instructors do not expect you to generalize from a small sample to a larger population, so your sample does not need to reflect characteristics in the overall population. However, you will need to talk meaningfully about your sample, so you will want to describe the characteristics of your respondents as specifically as possible. The kinds of characteristics you should identify are determined largely by the purpose of your research. Gender, income, age, and level of education are examples of the sort of demographic characteristics that you may want to account for in your survey.

Demographic Example:

Circle One: Male or Female?

Closed Questions

With the closed question, the survey provides responses and the subject selects the one that is most appropriate. When constructing closed questions, you need to provide choices that will allow you to make meaningful interpretations.

Closed questions allow the researcher to deal with the responses in a more efficient manner than open ended questions. Both forms of questions can produce similar information.

Closed Question Example:

How often do you study?

a. less than 6 hours a week

b. 6-10 hours a week

b. 11-15 hours a week

c. more than 15 hours a week

The closed question allows for greater uniformity of response, and the categories can be easily matched to the areas of interest by the researcher. Whether you select one type or a combination of questions for your survey, each question must be clear, unambiguous, and relate to a specific item in your objectives.

A common error is to create a double-barreled question such as, “Do you walk to school or carry your lunch?” Clearly, this question is attempting to ask two different things and a simple yes or no response won’t really answer the question. The original question should be rewritten as two questions: “Do you walk to school?” “Do you carry your lunch?” Now the respondent can provide you useful data.

Rank-Ordered Questions

When your library research has informed you that your audience is likely to be concerned about several issues, you can determine which concern is most troubling by asking a rank-ordered question. Essentially, as illustrated by the following example, the rank-order question presents respondents with several alternatives and requests that they rank them according to priority:

Rank-Ordered Question:

Please rank your five most important reasons for not contributing to our company’s blood drives. Put a #1 by your most important concern, a #2 by your second most important concern, and so forth.

_____ Lack of time
_____ Lack of awareness of former blood drives
_____ Perception that the blood bank has plenty of blood
_____ Concern about possible pain
_____ Concern about infection
_____ Concern about fainting
_____ Concern about vomiting
_____ Concern about bruising
_____ Concern about being infected with AIDS
_____ Belief company should provide some compensation

Open Questions

An open question requires respondents to answer in their own words, like a short-answer or essay type response. Open questions allow for free-flowing responses and are less restrictive than the closed questions. If you are going to use an open-ended question format for mailed surveys, don’t expect a high response rate. People seem more inclined to circle or mark a response than to write a lengthy response.

Open Question Example:

How often do you study?

(Ans)_________________________________

Because open-ended commentaries can enrich your interpretation of the statistics, you would be wise to include a few such questions in all surveys that you conduct. One generic question that you might find useful is “What important item(s) has/have been left out of this survey?” However, because they are difficult to tabulate and because they require time on the part of your respondents, be sure to limit the number of open-ended questions you ask.

How Many Questions?

Of course, when you are freewriting possible questions, you should attempt to generate as many questions as possible. You might also ask friends and colleagues what questions they would ask if pursuing your topic. Consider using questions that other researchers have asked, and base additional questions on the sources you have consulted to develop an informed survey. Ultimately, however, you will need to limit the questions so that you have a better chance of getting the surveys back. Before submitting your survey to your targeted sample, ask a few people to take it and time how long it takes them to complete it. Many people are willing to give 15 minutes to a survey, yet find a survey that takes more than 30 minutes intolerable.

Formatting Guidelines

Your questionnaire should be prepared so that someone will read it and respond. Try to make this process as easy as possible for the respondent. Make the questionnaire look like a questionnaire. Have it neatly organized, with questions in a logical sequence, numbering each question and page. If you mix open-and closed-ended questions, put the open-ended questions at the end of the questionnaire. An open-ended response is more threatening than a “select a, b, c, or d” response. Keep your questionnaire focused and as short as possible. Each page added reduces the number of potential respondents who will complete the survey.

Sample the Population

Understand the basics of census and random sampling techniques.

A critical step in survey research involves sampling the population. Now that you have narrowed your objectives to something achievable, who are you going to sample? For many, this is a rather simple step: Ask the people who are able to answer your questions! If you want to learn about the educational perceptions of 20-year-olds, ask them. If you want to learn about the parental perceptions of the education of 20-year-olds, ask the parents.

Sampling

Since data collection involving humans can raise legal, ethical, and moral issues, if you chose to survey the 20-year-olds on a particular college campus, find out what procedures are required before distributing your questionnaire or conducting interviews (see Informed Consent).

  1. Surveys and questionnaires may be presented:
  2. In paper format.
  3. Over the phone.
  4. Face to face.
  5. Online, perhaps with results viewable in real time.

Census Sample

A researcher may elect to include all the subjects within the population of interest, otherwise known as a census sample. If one is investigating a particular group of people, such as students enrolled in Composition 1 at 8:00 on Monday morning at a particular university, one would survey all the members of the class. The investigation of all the class members is referred to as a census sample.

Random Sample

If you are seeking information about a statewide issue, you would not attempt to interview everyone in the state. Information is still available if you use a technique that allows a sample from the population of interest to represent the population as a whole. A common technique is to employ the random sample. Random sampling implies that each member of the population has an equal chance of being selected in the sample from the population. As a general rule, the higher the sample size the higher the representativeness. The more representativeness you have in the sample, the more it tends to “look like” the population you are studying. Further study in sampling techniques will show how sample size can be determined to afford the researcher a particular level of confidence in the research outcomes.

Random Sample Techniques

Simple random samples may be made using a variety of techniques. One of the most common techniques is the use of a random number table (see Table 1). If you wanted to select 15 people from a classroom of 60 students, begin by obtaining a class roster of all the students. Assign each student a number from 1 – 60. Now we can use the table. Select a row and column. Any row/ column can serve as a starting point. For instance, if you select row 5, column 7 as your starting point, look at the first two digits (34). This number falls within your population range (1 – 60) and therefore can be used as the first of your required 15 people to be selected. Continue down the column. The next number is 68, but that is outside your range, so you should simply skip it and go to the next number, which is 47. When column 7 is exhausted, go to columns 8, 9, and 10 and if you haven’t accomplished your sample of 15, go to column 1 and continue. Note that column 9, rows 2 and 4 repeat the number 60. Since you have selected 60 the first time, simply skip the second and subsequent times. Using this technique, the 15 people from your class size of 60 are represented by numbers 34, 47, 32, 29, 33, 08, 25, 05, 41, 60, 03, 38, 16, 15, and 28. You have just accomplished a simple random sample. As with many things, once you become familiar with the technique, it is often easier to accomplish than to explain.

Table 1—TABLE OF RANDOM NUMBERS

12345
28564664564545654775745
32852143564627883454645
447832456457368264564
5249566557434534546456

Stratified-Random Sample

Another method of random sampling is called a stratified-random sample. For instance, one may look at the national census for a particular city or state. By applying percentages of age, ethnicity, or gender within the survey design, the researcher is able to gain representativeness within the survey. If the population of a city under study has a population that is 10% Afro-American, 10% Hispanic, 70% Caucasian, and 10% of other ethnic origins, then the researcher would insure the sample of the population contained 10% Afro-American, 10% Hispanic, and so on. The key to random-stratified sampling is to place the population into various categories, called strata, such as age, gender, ethnicity, income, etc. Then select individuals at random from each category. This technique can be used to identify a highly representative sample using a smaller sample size than a purely random sample.

Transmit the Survey

For the mailed survey there are several tactics you can employ to increase the response rate. Write a letter of transmittal stating the purpose and importance of the study, the reason why the individual was selected to participate, insurance of confidentiality, and an offer of thanks for the individual’s participation. If a separate letter is not appropriate, then consider a survey coversheet that contains similar information. Another technique is to send out a mailing informing intended participants that a questionnaire is forthcoming. Advance notification informing individuals they have been selected for your survey may also spark supportive interest.

Sending the Survey and Following Up

Always pick a date for respondents to reply, and be sure that date is at least a few weeks after the survey is mailed. Otherwise you’ll spend most of your time waiting for a response that will never come. About two weeks after your initial mailing, send a follow-up letter. In the letter thank those who have responded and indicate the necessity of obtaining outstanding surveys. Sometimes a statement like “If you have lost the original survey, please contact (researcher’s name, address, phone/fax number) and a new survey will be sent immediately” may also be helpful.

How many follow-ups are required?

That number is up to the researcher. If you can complete your research with a 20% response rate and your sample represents the population from which it is drawn, then don’t follow up. Do be prepared to explain in your report why you stopped. Some researchers will attempt as many as three times to gain subject participation. Given that you have had a particular number of responses now, ask yourself, “Would my results be different if everyone responded?” If you had a 90% initial response, it is unlikely that the remaining 10% would disrupt your findings. If the initial response was only 10%, then you have a considerable amount of work ahead of you.

Read More: