Chapter 7. Administer the Survey
Chapter 7 in PDF format
At this point, the agency has completed all the steps necessary to administer the survey:
- Identified which program or services to assess;
- Selected the appropriate survey instrument;
- Determined the best method to collect data;
- Selected a service recipient sample; and
- Developed a work plan and budget.
The following sections provide a step-by-step guide to administering the survey instrument by telephone or mail.
The Telephone Survey
Pilot the Data Collection Procedures
The POMP Survey Instruments and procedures have undergone reliability and validity testing and have been successfully administered over the past ten years. An Agency, however, may want to conduct a pilot (i.e. administer the survey in the proposed manner to a small number of recipients) to refine its data collection procedures.
Notify the Involved Agencies
The agency sponsoring the survey sends correspondence to organizations such as AAAs or providers that have direct contact with service recipients who are sampled for the survey. The correspondence introduces the survey and its purpose, explains the survey process, and includes the study timeline. The correspondence ensures that agencies are aware of the survey and are able to answer questions from sampled service recipients. After the correspondence is distributed, the agency sponsoring the survey may want to schedule a conference call with agencies to answer any questions.
Notify the Service Recipients
The agency sends advance correspondence to inform service recipients of the survey to reduce the likelihood that they will think that it’s a telemarketing call. The correspondence includes a description of the purpose of the survey, the requirements of participation, notification that participation is voluntary, and contact information for obtaining additional information about the survey. Include a statement that participation in the survey will not affect services in any way. The correspondence should be delivered 3-5 days prior to data collection.
Appendix B for the
Catawba Council on Aging Sample Letter of Invitation.
The agency selects interviewers who are not agency employees. Agency staff members may bias the results of the survey by inhibiting the service recipients’ responses. If it is impossible to use outside interviewers, the agency should select staff members who do not work directly with service recipients. The characteristics of good interviewers are listed below:
Good interpersonal skills — An interviewer needs to have a clear and reasonably loud speaking voice with good projection. This is very important when interviewing older persons who may have some hearing loss. Interviewers may need to speak slowly to older people. However, it is important to show sensitivity to the service recipients and tailor the approach to the individual.
A positive attitude toward aging — Interviewers should have a positive attitude toward aging. Interviewers should show respect and sensitivity to the service recipients. They need to refer to the service recipients by their last names unless invited to use their first names.
Dependability — Interviewers often work without supervision, so it is important to hire people who are dependable. Interviewers will often conduct interviews in evenings and on weekends, especially when interviewing caregivers who may work during regular office hours. This often means that interviewers work after regular office hours and have irregular schedules.
The topics to include in interviewer training include:
- the background and purpose of the survey,
- survey ethics,
- survey materials and procedures, and
- general office procedures.
An important part of interviewer training is providing opportunities to practice administering the survey and the responses to frequently asked questions.
Background and Purpose of the Survey
At the beginning of the training, the trainer explains the background and purpose of the study; including the sponsoring agency, the reason for the study, the potential uses of the results, and the timeline.
Confidentiality. Collecting data from service recipients to assess programs or services involves human subjects. Therefore, data collection is subject to a number of legal and ethical requirements. The agency should instruct interviewers to protect the names, telephone numbers, and other personal information of the people they are calling. This is especially true if interviewers call from locations other than an agency’s office building because identifying information may be disseminated via email attachments (most likely with encryption and password protections) or transported—on paper forms or by electronic devices such as laptops or data storage devices—in cars or on public transit. Agencies typically require interviewers to sign a confidentiality agreement.
Appendix B for a
The agency must adopt special procedures to protect the confidentiality of data in transport or sent via email. If the agency has a confidentiality form that employees must sign, include it in the discussion of agency policies. Protect information from unauthorized disclosure and use. This includes information that is stored outside of locked file cabinets and locked rooms (i.e., data that are transported by laptop, CD, DVD, or memory stick). End the discussion of confidentiality by asking each interviewer to sign the agency’s confidentiality form. The exception to confidentiality is the disclosure of elder abuse/neglect or the intent to harm self or others. Interviewers should inform respondents beforehand that they are required to report abuse or neglect to the appropriate parties. The agency needs to have a plan in place for interviewers to follow in the event that this happens.
Appendix B for a
Train interviewers to assure service recipients that their responses are confidential and will in no way affect their services. This sets the stage for service recipients to feel free to give honest responses. If interviewers are employed by the agency that provides services, service recipients may worry that their responses are not confidential and may feel less inclined to honestly respond to items about consumer assessment of services. Service recipients may also hesitate to give an honest response for fear of “hurting the feelings” of service providers or fear that they will get the service provider in trouble.
Respondents’ right to refusal. It is important for interviewers to understand that respondents may skip any question for any reason. Interviewers are provided with a script to read to respondents before administering the survey that informs them of this right. If the agency is under the authority of a human subjects institutional review board (IRB), this disclaimer likely will be mandatory. Interviewers should encourage people to participate, but never put any type of pressure on them to do so.
Respondents requiring services. Sometimes respondents may reveal the need for additional services. When this happens, instruct interviewers on how to handle this information.
Interviewer bias. Train interviewers to avoid introducing bias. Interviewers may introduce bias by showing empathy or a change in tone of voice. Interviewers are advised not to interview persons they already know. It is important that the interviewers understand that they cannot interpret a question for the service recipient because doing so will introduce bias.
The majority of the interviewer training covers the content of the survey, procedures for conducting the survey, and record keeping.
The survey instrument. The training is a good opportunity to carefully review the survey instrument. This ensures that interviewers are familiar with the survey items and that their questions are answered.
Introductory script. The introductory script establishes contact with the service recipient and ensures that he/she understands the informed consent information in the cover letter. As with the survey items, interviewers are required to read the introductory script verbatim.
Appendix B for a
Script on Establishing Contact.
Answering machine/voice mail protocol. When attempting to reach the respondent, the interviewer may encounter voice mail. Since it is not known whether the respondent is the only person with access to the voice mail, instruct interviewers to avoid leaving any personal service recipient information, including the fact that he/she receives services. Using an agreed-upon voice mail protocol ensures that the interviewers do not reveal any identifying information.
Appendix B for a
Script for Interviewers to Use for Answering Machines or Voice Mail.
Recording and survey tracking materials. Instruct interviewers on how to track the number of respondents contacted and the outcomes of the contacts. There are many options to track this information. One option is to maintain a master logbook with all of the respondent contact information to record all the statuses. Each interviewer is provided with “call sheets” copied from the master logbook that contain the respondents’ names and contact information.
Appendix B for a
Caregiver Sample List and Results/Telephone Interviewer Caller Log.
Another option is to issue individual respondent information sheets paired with a notebook for keeping track of which interviewer has the information sheet for each respondent, time of check out and return of the information sheet, and disposition of the attempts to contact the respondents. The advantage of this option is that the information sheets are easily transferred from one interviewer to another.
Appendix B for a
POMP Survey Contact Sheet.
Regardless of which option is selected, interviewers must maintain a log to record each attempt to reach a respondent, including the date and the disposition of the contact. A general guideline is for interviewers to attempt to contact respondents at least five times. When respondents are difficult to reach, instruct interviewers to vary the times of the call attempts. The service recipient may have a standing appointment or activity that takes him/her from home. Schedule calls from 9:00 a.m. to 8:00 p.m.
Survey instructions. Interviewer trainees have different learning styles. While the most important lessons are taught and practiced in training, provide written instructions. When a question arises, it is helpful to refer interviewers to the written instructions on interviewing techniques.
Appendix B for
the Sample Instructions for
POMP Survey Interviewers.
Practice administering the survey. At the end of the training, interviewers are provided with an opportunity to practice the survey administration procedures (introductory script and interview). The trainer should pair interviewers with a senior member of the survey team who is familiar with the survey instrument and procedures. Practicing with a senior member of the survey team provides an opportunity to reinforce good skills and correct any mistakes in a helpful and supportive manner. Another option is to let interviewers practice interviewing each other and listen for strengths and weaknesses. Give feedback individually and discuss any common mistakes with the entire group. A standard practice is to monitor the first few calls made by each interviewer to identify additional training needs.
Back to top
The Mail Survey
The initial mailing includes the following components: an envelope addressed to the recipient, a cover letter, the survey with a cover sheet and instructions, and an agency’s self-addressed, stamped return envelope. After the initial mailing, send nonrespondents a reminder card and a second mailing of the survey instrument. If the agency’s response rate is still inadequate after sending the reminder postcard and the second mailing, the agency might consider calling nonrespondents to ask them if they are willing to complete the survey over the telephone. The following paragraphs discuss this approach more thoroughly.
Mailing envelope. The size of the mailing envelope should accommodate all materials and include the agency’s return address preprinted on the envelope or on a label in the upper left-hand corner. Include any special instructions to the post office regarding special services required such as a change of address or return requests. The recipient address label must be properly placed for postal scanning and sufficient postage must be printed, stamped, or affixed to the envelope.
Cover letter. Similar to the advance notification letter, the cover letter explains the purpose and importance of the survey and encourages participation. In addition, the cover letter explains how the agency plans to use the results. Print the cover letter on agency letterhead and include the agency director’s signature. Include a “please return by” date. A due date 10-14 days from the day of the mailing provides service recipients sufficient time to respond. Include a telephone number to call if there are any questions and thank the service recipient in advance for his/her participation.
Appendix B for
the New York State Office on Aging cover letter.
Processing survey returns. Develop a protocol for processing completed survey instruments. Adhering to the protocol ensures that survey instruments are not misplaced and that each service recipient returns only one completed survey instrument. Some of the completed survey instruments may arrive later than the deadline stated on the cover letter. Establish a set of rules regarding what constitutes the closing date for accepting returned surveys. For example, will the closing date be the actual calendar date, the postmarked date, or some specified number of days after the actual calendar date?
Survey instrument with instructions. Include a cover page with instructions on how to complete the survey. Proxy respondent instructions for people authorized to respond on behalf of service recipients are also included on the cover page. The survey instrument itself has instructions for responding to each item or set of items.
Stamped self-addressed envelope. Include a stamped, pre-addressed envelope with sufficient postage for the service recipient to return the completed survey instrument.
Reminder postcard. The reminder postcard should be mailed to nonrespondents approximately 1 week after the initial mailing. The reminder postcard may help increase the response rate. It reminds service recipients of the survey and provides them with a contact number if the original survey instrument did not arrive.
Appendix B for a
Reminder/Thank You Postcard.
Follow-up letter with survey instrument. Mail a follow-up letter with the survey instrument to nonrespondents after the completion deadline. It is important to include a second copy of the survey instrument for those who may have thrown out or misplaced the original survey. Depending on the response rate, an agency may consider sending additional rounds of the follow-up letter with the survey instrument to encourage participation.
Appendix B for a
New York State Office on Aging reminder letter.
Follow-up telephone call. A follow-up telephone call can be made to nonrespondents to give them the option of completing the survey over the telephone. Providing service recipients with another way to participate in the agency’s survey may increase the response rate.
Back to top
Back to top
- Prepare materials: cover letter, copies of survey instruments and envelopes
- Mail survey
- Enter data into an electronic data entry utility as survey instruments are returned
- Contact nonrespondents at designated time periods
- Enter remaining data
- Check for accuracy of data entry
- Review database for outliers and anomalies in the data
- Code responses to open-ended questions