This chapter covers the project activities and deliverables necessary to collect data by mail at SRO.

SRO Standard Project Procedures

Following are mandatory and conditional pre-award deliverables for SRO mail data collection projects:

 Deliverables

• Project charter;
• Financial Services project initiation meeting with Budget Analyst;
• Project review meetings (project initiation, interim, closing);
• Scope statement;
• Project management plan;
• Project kickoff meeting;
• Requests for change to the project management plan or work scope;
• Cost reports;
• SRO monthly project reports;
• Client reports;
• Production goal spreadsheets and charts;
• Ongoing production reports (e.g., cost, outcome rates assessment, productivity, data and process quality);
• Study manual with study protocols;
• Mailing schedule;
• Bulk materials checklist;
• Respondent materials (e.g., persuasion, debriefing, and data collection materials);
• Project 800# and/or email support services;
• Agendas and meeting notes for team meetings;
• De-identified password-protected data set containing the survey responses, data dictionary, and variable formats;
• Deliverables sign-off;
• Lessons-learned report;
• Project master file(s); and
• Project close-out checklist.

 Deliverables

• Survey instrument(s);
• Ancillary mailing materials sent to respondents;
• Materials used to locate and/or track respondents;
• Materials and instructions for household listing and respondent selection;
• Respondent incentive payments;
• Work scope change documentation; and
• Logging documentation.

Many mail studies are inter-departmental efforts. In the Project Design and Management Group (PDMG), the Project Leader will be a Project Manager; in the Survey Services Lab (SSL), the Project Leader will be a Production Manager. Also in the SSL, telephone interviewers are referred to as “Survey Technicians.” As the majority of mail studies are carried out through the SSL, the terms “interviewer” and “survey technician “will be used interchangeably.
While the specific steps for survey implementation vary from one project to the next, Figure 10C.1 provides a diagram of typical mail data collection activities. SRC best practices for mail studies are described in sections 10C.2 through 10C.17:

10C.2 Project Management;
10C.3 Project Staff;
10C.4 Project Protocols;
10C.5 Project Team Materials;
10C.6 Training;
10C.7 Ethical Considerations;
10C.8 Respondent Materials;
10C.9 Shipping;
10C.10 Respondent Support;
10C.11 Managing Bulk Materials;
10C.12 Receipt Control;
10C.13 Data Security;
10C.14 Monitoring and Reporting Progress;
10C.15 Changing Study Protocol;
10C.16 Monitoring and Reporting Cost; and
10C.17 Project Closeout and Documentation.

Figure 10C.1 Mail Data Collection

See Chapter 3 for SRO best practices for project management, which covers the following activities:

3.2 Project Initiation

Receive funding notification
Assign key project staff
Gather existing project information
Create project names
Obtain IRB approval
Obtain a Certificate of Confidentiality
Complete the Financial Services project initiation process
Set up the Cost Reporting System
Set up the Monthly Project Reports
Develop and obtain approval of the project charter
Meet with the Senior Project Advisor
Hold an initial project management review meeting
Set up the network location for project files
Monitor and control project initiation

3.3 Project Planning

Set up the Costing Reporting System
Set up the Monthly Project Reports
Create cost projections
Assemble a project team
Develop the project management plan
Monitoring and controlling project planning

3.4 Project Implementation

3.4.1 Project Team Coordination

Schedule a kickoff meeting
Obtain approval of deliverables and to proceed
Implement the project management plan and manage the project
Obtain approval of major changes to the project management plan

3.4.2 Financial Monitoring and Management

Monitor expenses
Estimate costs
Account for slack
Create and review updated cost reports
Produce cost reports at least monthly
Review and obtain approval of cost reports

3.4.3 Reporting

Schedule Project Management Reviews
Submit Monthly Project Reports
Generate Progress Reports

3.4.4 Requesting Approval of Changes in Work Scope

3.4.5 Obtaining IRB Approval of Changes in Procedures and Materials

3.4.6 Monitoring and Controlling Project Implementation

3.5 Project Closeout

Complete the project deliverables sign-off checklist
Complete the project closeout checklist
Write a final summary report
Hold a project management review meeting
Assemble project materials for archiving
Monitoring and controlling project closeout

After SRO receives funding notification, the SRO Administrative Group will assign a Senior Project Advisor (SPA) and a Project Leader to the project. The Project Leader develops and implements production staffing plans and defines them in the project’s management plan. Project staff for a mail study typically includes:

• The Principal Investigator(s) (PI’s);
• A SPA;
• A Project Leader; and
• Survey Technicians.

A larger study may also include:

• A Survey Director;
• One or more Team Leaders (each to supervise the activities of 6-8 Survey Technicians); and
• One or more Production Coordinators (to supervise the activities of the Team Leaders).

All SSL and field management team members are contingent staff and receive an hourly differential during the time they are serving in a supervisory role on a project. The Project Leader is generally a Survey Specialist, a regular salaried University employee.

Project protocols are worked out in meetings between the Project Leader and the PI(s), with ongoing oversight by the SPA who represents SRO Administration. Typically this is done through an iterative series of working meetings and individual work sessions in which the project team drafts and refines project materials. At this time, the finer details of the project protocols are worked out and due dates and tasks are assigned to project teams or team members.

These include materials used by the project team in training and throughout production.

10C.5.1 Survey overview materials

The study team receives an overview of the survey via a “kick-off” meeting and a study management plan and/or a study project manual.

Study management plan

The management plan summarizes the entire life cycle of the project, from its background and objectives to its final documentation and deliverables. It includes the history, management team, a communication plan, sample design and methodology, a detailed description of the materials to be mailed and how they are to be packaged, information about the technical systems that will be used to support the project, the data collection process, project monitoring and reporting to the PI’s, any workscope changes with their dates and financial impact, and a timeline, with all updates noted. It is most efficient to use a management plan as the base document when survey technicians will only prepare and check mailings and log returns.

Study project manual

The project manual is a comprehensive reference guide written for the use of interviewers and/or survey technicians working on the project. This document provides a study overview, history of previous waves of research conducted on the study, and the goals for the study. It summarizes the PI’s research objectives and how the data will be used. In addition, the project manual describes the sample selection method and the methodological procedures for production data collection (e.g., persuasion letters, incentives, and the sample management system).

The manual also describes study-specific protocols for handling respondent inquiries on the SSL 800-line or the dedicated study 800-line, along with study information that can be shared with respondents. It summarizes information that should be included in call notes or logging system entries. The project manual is organized to follow the flow of the Survey Technicians’ tasks from the initial steps of organizing mailing materials to final-coding lines in the study’s logging system and packaging study materials for shipping to the client. It is most effective to use a project manual as the base document when the mailing is a portion of a larger study and when survey technicians will be involved in other aspects of the work in addition to preparing mailings and logging returned materials.

Example of Project Manual Outline
Formatting Guidelines for Project Manuals

10C.5.2 Mailing schedule

A mail study typically involves many departments within ISR — Duplicating prints materials, the ISR Business Office provides Imprest Cash for incentives, SSL Survey Technicians assemble mailings, and the Mailroom sends and receives questionnaires. The mailing schedule is used to coordinate work across departments and to help Survey Technicians organize their time and plan their availability for assembling mailings and logging in returned materials.

In studies with multiple mailings, it is very helpful to maintain a file that summarizes the number of different mailings sent, the type of mailing (e.g., initial questionnaire mailing, follow-up questionnaire mailing, follow-up postcard mailing, etc.), the dates of the initial and final sample pulls, the date on which the mailing was sent, and the total number of respondents to whom it was sent.

Example of Schedule of Mailings and Sample

10C.5.3 Materials checklist

Materials checklists help Survey Technicians organize outgoing and incoming mailings. They also provide information about contact persons for supplies, problem resolution, and respondent requests.

Example of Survey Technician Checklist

10C.5.4 Communication plan

Email

SRO staff rely heavily on email for daily communication. Contingent staff, whose schedules do not necessarily conform to a standard 9-5, in-office schedule, are expected to:

• Check email at least once daily;
• Respond to requests made via email within 24 hours;
• Use email only for University business;
• Never include identifying respondent information (e.g., name, address, phone numbers) in email; and
• Not use email to communicate with respondents unless this is part of the study protocol and has been approved by the IRB.

Telephone

When attempting to contact respondents or informants, staff are requested to leave only the respondent 800-line number or a dedicated line number that has been set up for the study. Answering machine and voicemail messages on dedicated lines must be professional and identify the University of Michigan’s Survey Research Center. When contacting respondents, staff members typically log the time, nature, and outcome of the conversation in the study’s logging program, so that there is a historical record of the interaction.

800-Line and Email Response Protocol
Contact Attempt Logging Protocol

Team meetings

On most mail surveys, the Project Leader meets regularly with the Survey Director and/or the Project Team Leaders. Typically, these management team meetings are used to:

• Update team members on production progress;
• Share news and reminders regarding protocol, study procedures, materials, etc.
• Engage in decision-making on questions concerning protocols;
• Provide continuing education and retraining; and
• Receive feedback from the Survey Technicians on production experiences.

In addition to management team meetings, Team Leaders or the Project Leader may conduct group meetings with the Survey Technicians to provide them with updates on study progress and with information about upcoming tasks and work requirements.

Contact Attempt Logging Protocol

Generally, training takes place at SRO during regular office hours and takes 1-2 hours.  Interviewers are certified to work on the study after successfully completing training and practice.

If the mailing is part of a larger field study (i.e., materials are collected in the field and then mailed back to SRO), training on mailing protocols is usually included within overall field training.   Alternately, the mailing training can take place independently, via phone or in-person, for a subset of the total contingent field staff.

Project Leaders are responsible for establishing and enforcing ethical standards for the project. They work with the PI(s) to submit an application to the appropriate Institutional Review Board (IRB), describing the project’s consent procedures, any clinical contact protocol, and safeguards to protect respondent anonymity. The IRB must approve the proposed project protocol before data collection can begin; any subsequent changes to the project design or materials must also be approved by the IRB. The IRB requires that lead members of the project staff take the Program for Education and Evaluation in Responsible Research and Scholarship (PEERRS) training, available on the internet without cost to University of Michigan employees, and be PEERRS certified.

PEERRS Training and Certification Web Site

Respondent materials are developed with input and approval from the PI and must be reviewed and approved by the IRB before they can be shared with sample members. They typically include brochures or flyers to introduce the study, letters, letterhead, postcards, envelopes, and incentives.

Each study must consider issues of study branding (i.e., Is it more appropriate for the study to be associated with the University of Michigan, the Institute for Social Research, or the sponsoring body?) and personalization (i.e., whether to invest the time and effort needed to put stamps on each mailing piece or whether to run the mailing through a postage meter). Both of these may affect response rates and a cost/effectiveness assessment is typically made. For further information, see Dr. Donald A. Dillman’s research on mail surveys at http://www.sesrc.wsu.edu/dillman.

10C.8.1 Screening questions

Mail studies usually use list samples with known names and addresses but occasionally respondent selection is based on criteria that are assessed after the mailing. This is done by placing screening questions at the beginning of the survey. If the mail recipient does not meet screening question criteria, he/she is instructed to give the survey and accompanying materials to the first available individual in the household who does meet the study’s selection criteria. If more than one individual meets the criteria for selection, instructions can be provided for randomizing the selection process (e.g., by using elements of the Kish table selection process). However, without an interviewer present, elements of convenience sampling may be introduced – that is, recipients may give the packet to the household member of their choosing.

Kish Tables Documentation

10C.8.2 Questionnaire

The Project Leader typically works closely with the Survey Director to develop a concise, contextually clear, and attractively-formatted mail questionnaire. The presentation should be clear and easy-to-read, with instructions readily distinguishable from the question text.

Typically, instructions describe and provide examples to the respondent of how he/she should fill in the form. The instructions are usually printed in boldface font before the first question. They provide examples of the various question formats that can be found on the questionnaire (e.g., checkbox, write-in, checkbox with associated skips) and how they should be filled in. Instructions about how to answer specific questions immediately follow the question text, and are usually displayed in italic font, to set them off visually from the questions, themselves.

Typical instructions include:

• “Please select one box in each row.”
• “Please select all that apply.”

Self-administered questionnaire conventions include:

• When yes/no categories are given, the affirmative response is placed to the left;
• When rating scales (e.g., Excellent/Very good/Good/Fair/Poor) are employed, they are ordered from high-to-low, best-to-worst, for an intuitive progression;
• Key words such as “Any,” “All,” and “Ever” are underlined to bring them to the respondents’ attention;
• A study contact telephone and/or email address is printed at the end of the questionnaire; and
• Materials are written at the 8th-9th-grade reading level (the reported U.S. adult average).

Example of Survey Completion Instructions

10C.8.3 Letters and postcards

Cover letter

A cover letter, introducing the study to respondents and signed by the Project Leader or a Principal Investigator, is enclosed in the initial mailing packet. The cover letter typically includes information about:

• The survey—its purpose, its coverage (i.e., national or local), its subject matter, whether any questions are sensitive or include the need for physical measures or specimen collection, and its general length.
• The sample–how the sample was selected, why the study is important, and which group(s) it will likely benefit.
• The data–how the data will be used and disseminated. The cover letter will typically name the sponsoring bodies and their interest in the research. It may also refer to steps that respondents can take in the future to obtain a copy of the study’s results, although this is more likely to be sent at the end of the study in the form of a debriefing, or thank-you, letter.
• The importance of the respondent’s participation—how the respondent’s participation is vital to the success of the study, the scientific selection process, and how the respondent cannot be replaced by anyone else. The letter will also describe any incentives that are offered for participating in the research.
• The voluntary and confidential nature of the respondent’s participation–how confidentiality is protected, that the respondent can refuse to answer any question that he/she does not wish to answer, how collected data are separated from identifying information about the respondent, and how the respondents’ answers are reported only in aggregate form, together with other people’s. The letter will also provide a telephone number to call with any questions about the study and/or an email address to which to send queries and will include the study’s IRB contact information.
• The letter may also indicate any risks associated with participating in the study.

Example of Cover Letter with IRB Merge Fields in Footer

Authority or endorsement letter

An authority, or endorsement, letter may be sent to public officials and/or difficult-to-reach professional groups. This letter explains the reputation of the Survey Research Center and describes the credentials of the professional endorser(s) and funding organization(s).

Persuasion letter or postcard

Once the study is underway, persuasion letters or postcards can be sent to respondents who have been difficult to enlist, who have shown some resistance, or who have specific, initially unforeseen, concerns about participating in the study. The letters or postcards are tailored to specifically address these issues. SRO has also developed a “Refusal Postcard” to be sent to respondents who fail to participate throughout the duration of a study. The “Refusal Postcard” attempts to determine the primary reason for their nonparticipation.

Example of Refusal Postcard with IRB Merge Fields in Footer

Thank you letter

Thank you letters are often sent out in response to completed interviews. They may include respondent incentives if incentives were not mailed with the initial cover letter.

10C.8.4 Brochure

Projects teams often design a brochure to introduce their study to respondents, contact people, and officials. The brochure includes study-specific information, confidentiality assurances, and the toll-free number for inquiries, and describes the qualifications of the Survey Technicians. The project brochure is a public relations tool and should be very professional in appearance (e.g., glossy, including pictures where appropriate). Brochures must be reviewed and approved by the IRB.

Example of a Project Brochure

10C.8.5 Envelopes

Mail studies may use standard University of Michigan envelopes or envelopes customized to reflect a different “branding.” Sometimes the study’s funding organization (e.g., the American Bar Foundation, the International Reading Association) will use its own envelopes to emphasize the fact that the study is being underwritten and endorsed by that particular organization.

It is important to liaise with the ISR Mailroom if you develop customized envelopes or if you set up a campus address for the study’s returned mail. The Mailroom will need a sample of the outgoing and return envelopes, and the Project ShortCode. They can give advice on the placement and formatting of the bar code(s) and confirm the permit number on the return business reply envelopes.

Example of a Customized Business Reply Envelope

10C.8.6 Respondent incentives

Most projects offer an incentive to encourage sample members to participate. Incentives can be monetary, non-monetary, or some combination of the two. The decisions whether or not to offer an incentive and what any incentive should be are made in collaboration with the project staff, usually at the budgeting stage. Monetary incentives can be paid in cash or by check. Non-monetary incentives should be lightweight and compact for ease of mailing; past non-monetary incentives include gift cards, music download cards, and leather bookmarks.

Incentives may be given before or after the interview. Including the incentive with the pre-notification letter or with the mail questionnaire may motivate the respondent to participate. The inherent risk is that the respondent may decide to keep the incentive and still refuse to participate.

Any respondent compensation plan must be approved by the IRB and is usually described in the Project Charter or the Project Management Plan,

Example of a Mail Study Respondent Compensation Decision

Paying by cash

Cash for the payment of respondent incentives is obtained through the ISR Business Office via the Imprest Cash Fund. Imprest Cash forms and a checklist for the process of requesting an account can be found in “Accounting Forms” in the Forms Gallery on the ISR Intranet. Instructions for developing an Imprest Cash Request Memo are found in the “Guidelines to Follow for Acquiring Imprest Cash Fund,” part of the “Imprest Cash Fund Request Authorization” packet. The account Custodian (the person setting up the account and withdrawing the funds) is governed by the University’s tenets related to Imprest Cash accounts in the Standard Practice Guide (SPG).

Standard Practice Guide Imprest Cash Funds
Example of an Imprest Cash Fund Request Memo
Imprest Cash Fund Request & Authorization
Imprest Cash Checklist
Imprest Cash Withdrawal Request
Imprest Cash Fund Change Request Form
Safe and Imprest Cash Access

The account Custodian is responsible for accurately accounting for all sample members to whom the fund was disbursed. Since the mail survey mode does not facilitate the use of receipts, a file containing the SID, each Respondent’s name and complete mailing address, and the dates on which the funds were disbursed is provided to the ISR Business Office, along with a cover memo. The total amount disbursed will tally with the total value of the fund, by the end of the project.

Imprest Cash Fund Receipt Summary Cover Memo
Example of imprest Cash Fund Receipt Summary Format

The ISR Business Office will send monthly statements against which it is the Custodian’s responsibility to reconcile the fund, within a few days of receipt of the statement, and to respond to the representative managing the fund in the Business Office concerning the reconciliation status. Final reconciliation occurs after all funds have been disbursed and the data collection period has ended, thereby allowing returned monies – either sent directly from respondents, or by means of returned, undeliverable mail — to be collected and then credited back to the project ShortCode.

Example of Imprest Cash Fund Monthly Statement Information

Paying by check

Large mail studies more commonly issue checks. Data Operations processes requests for respondent payment in the Check Processing System (CPS) and requests printed checks via the ISR Business Office. Typically, the checks are generated from the respondent address information in the project sample file, provided to Data Operations in an Excel spreadsheet. This file is then run through the CPS, the files containing the data for generating the checks are produced, and they are provided via secure transmittal to the ISR Business Office. Checks are typically generated within 3-5 working days, although more time may be needed for a very large number of checks. They are reviewed for accuracy upon receipt and are kept in a locked file cabinet until they are mailed by the project team.

Incentive Specifications Form for CPS

Non-monetary incentives

These can be just about anything, but they are usually limited to small items that are easy to mail or carry and that are customized to be relevant to the study. Some examples from past projects are refrigerator magnets, coffee cups, pens, calendars, music download cards, leather bookmarks, and notepads, with the study logo or the UM logo, and sometimes with a study telephone number or email address imprinted on them. Inexpensive non-monetary incentives can be purchased using a P-Card (or the vendor may accept a project account number). Totals over a certain amount must go out for bids. DCO staff can help with the process.

Modifications to incentives

Projects may choose to increase the incentive amount at the end of the study to encourage reluctant respondents to participate. This end-game strategy may have been part of the respondent compensation plan from the beginning and included in the initial IRB submission. If the decision to increase incentives is made later, it will need to be submitted for IRB approval as an amendment.

The frequency of mailings and carrier used vary by study and can affect the survey’s visibility and impact on respondents.  Most mail studies at SRO use the U.S. Postal Service (USPS) because of its lower cost and because address labels for USPS-mailed envelopes can be easily produced on the in-house printers.  UPS, DHL, or FedEX are sometimes used when sending to business professionals at work, when sending documents overseas, or when the study requires making a distinctive impact,

Study materials may be returned to a dedicated ISR post office box or to the existing ISR box.  In either case, the ISR Mailroom should be directly involved in the design of outgoing and return mail to ensure accuracy of all permit numbers, bar codes, ZIP Codes, and post office box numbers.

Respondent telephone support may be provided via a dedicated study 800-line or a dedicated Field Interviewer 800-line. Communication with respondents can also occur via email.

10C.10.1 Dedicated 800-line

A dedicated 800-line is most appropriate if the study anticipates a large volume of respondent calls or if the study design requires the administration of a complex decision tree regarding how the incoming calls are handled. The 800-line is set up through IT Com at UM Information and Technology Services (ITS) ([email protected]). The service can be forwarded from one desk to another to allow coverage during vacation periods.

800-Line Responses to Respondent Queries
University of Michigan ITCom 800-Line Service

10C.10.2 Field toll-free coverage

While the primary service provided by the field toll-free staff is to connect respondents to appropriate resources, it also provides respondents with a voice that legitimizes the study. However, the field toll-free staff should not be used as a data collection end-point unless previous arrangements have been made. It is not the primary mission of the field toll-free staff to provide study specific instructions on navigating a questionnaire or to update respondents’ address information; these matters should be addressed by the Project Staff. If using field toll-free staff for respondent telephone support,

• Include the toll-free coordinator in the project kick-off meeting;
• Include the toll-free staff in project trainings, particularly if there are study specific protocols that they should be aware of (e.g., eligibility requirements, empanelment, physical measures, survey length, survey subject matter); and
• Provide the toll-free staff with copies of all respondent materials and timelines.

10C.10.3 Email

Often studies set up a project email account to reply to respondents’ queries. To set up an email account for a study, place a Footprints Helpdesk request by going to the ISR Intranet website and clicking on the link to “Request for Computing Help.”

Once the account has been set up, staff members should check the mailbox regularly and respond to enquiries, typically within 24 hours. Protocols for email communication with respondents include:

• Responses should include the staff member’s name, the organization name (ISR or UM) and the 800-line number;
• Responses should be stored in the study’s Sent Items folder; and
• The respondent request and response should be entered into a logging program.

Replying to a Respondent Message in the Study Email Box

Mail studies rely more heavily on bulk materials to communicate with respondents and to collect data than do computer-assisted surveys. And, unlike computer-assisted surveys, in which changes can be made to the instrument up to or even during production, mail studies’ strict timelines and high cost-per-unit make this unfeasible. All of the study’s main mailing materials should be thoroughly reviewed, finalized, and approved prior to the initial mailing. Once the final documents are drafted, PI sign-off is essential before printing.

10C.11.1 Materials spreadsheet

A materials spreadsheet helps to coordinate the steps involved in materials design, ordering, and delivery. The spreadsheet typically includes:

• A detailed description of each mailing piece (e.g., customized letterhead, cover letter, postcard, information sheet, brochure, questionnaire, outgoing envelope, return envelope, etc.), including:
  • Paper size (dimensions), weight, colors, type (parchment, printer, coverstock, etc.)
  • Type/colors of ink to be used
  • Who will design (in-house, out-of-house)
  • Type of binding, if any
  • Whether watermarked
  • Whether imprinted with a study logo
  • Whether an electronic signature will be imprinted
  • Whether edge trimming will be needed
  • Whether collating of documents will be needed
  • Cost per thousand, per unit for materials
  • Cost per thousand, per unit for preparation
  • Total volume required;
• Dates by which orders must be placed;
• Additional materials needed with order placement (e.g., sample listing);
• From whom we are ordering (e.g., Duplicating, an outside vendor, etc.);
• Agreed date of delivery of materials to SRO; and
• Inventory check, including qualitative observations (date materials were delivered and any observations about them, such as whether all materials arrived in the number and condition requested and, if not, what the issue was and whether redress was requested from the supplier).

This document will be referenced frequently during the pre-production phase to ascertain the status of materials and to confirm that required dates and deadlines are being met.

Sample Printing Quantities for After the JD2

10C.11.2 Selecting a supplier

Unless the cost is very nominal, Project Leaders typically review past mail studies to short-list the best supplier candidates in terms of quality and price. Typically, ISR Duplicating prints mail-merged cover letters, information sheets, and some questionnaires. Questionnaires that will be scanned require an outside vendor, due to in-house equipment limitations.

Postcards may be printed by ISR Duplicating or an outside vendor. ISR Duplicating subcontracts the printing of envelopes to an outside supplier but they will coordinate this task if provided with electronic files of the base documents. Because of the color artwork that is typically involved, brochures and customized letterhead are usually sent to an outside vendor for printing.

Any items or services that are quoted in excess of $5,000 must be put out for outside vendor bid in order to ensure that a fair and reasonable pricing is obtained in an open, competitive market.

University of Michigan Procurement Services Website

10C.11.3 Customized mailing materials

Projects sometimes design a specific study logo to be placed on letterhead, envelopes, brochures, and the cover page of the questionnaire. The client may provide some of these customized materials. While customized materials create an impact, they can also be expensive and labor-intensive to set up.

Customized Letterhead Disclosure Mail Survey Cover Letter

10C.11.4 Materials storage

Materials that are generic in nature and which have not yet been used (e.g., envelopes, letterhead, non-personalized surveys) may be stored in offices or unlocked filing cabinets. Materials that are personalized or that contain respondents’ answers (e.g., mail-merged letters, mailing labels with names/addresses, sample listings, completed surveys, CDs of data files, removable disk drives containing data, returned letters with respondent-identifying information) must be kept in locking file cabinets or locked desks, within locked areas. The latter should be available only to project staff on an as-needed basis. At the end of the study, confidential materials are typically returned to the Principal Investigators’ office area for long-term storage or are shredded.

10C.11.5 Mail assembly

Before mail preparation takes place, a mailing assembly protocol is defined to summarize:

• How the work area will be set up (e.g., where mailing assembly and quality control (QC) will take place);
• How the sample control sheets with respondent information will be used;
• What items will be included in each mailing packet;
• How the packets will be assembled;
• Whether mailing labels will be used or whether names and addresses will be printed directly on the envelopes;
• Whether and what denominations of postage will be affixed to outgoing envelopes;
• How many packets will be assembled before quality control (QC) starts;
• What the QC steps will be; and
• Which interviewers will be assigned to assembly and QC.

A printed copy of the mailing preparation protocol is provided to all assemblers at training and another is left at each assembly station.

If incentives are involved, this document will also describe where the incentives are stored, who will be responsible for getting them out during mailing assembly, and how they will be managed, monitored, accounted for, and secured at the end of the shift.

Typically, quality control is performed on all packets in an incentive mailing and on 15 to 50% of the pieces in a non-incentive mailing. The first several “sets” of a new assembler’s work are always checked in full with feedback provided immediately. After the assembler has gained some experience, a random but representative distribution of that person’s work is checked for accuracy. Anyone whose work is consistently found lacking is dismissed from working on assembly.

Due to space constraints, mailings are frequently assembled in different locations around the building, so a schedule indicating exactly where the work is to take place on different dates is necessary. Minimizing the movement of materials and workers is ideal, although not always possible. The work is usually performed during regular business hours in shifts of 4-hour time periods.

Mailing assembly can continue into the evening and/or over the weekend but a permanent staff member must always be present.

Sample Mailing Assembly and QC Instructions
Imprest Cash Insertion Step
Sample Assembly Venues
Sample Mailing Assembly Schedule

Logging procedures and timelines must be defined for all hard copy materials (e.g., consent forms, self-administered questionnaires, event history calendars, time diaries, etc.).

10C.12.1 Logging system

Logging applications are a place to record, for each sample member, any outgoing contact attempts and any incoming (respondent-generated) contacts. Smaller projects often use Access or Excel; mid-to-large projects generally use the SRO Web Logging application. For the latter, a programmer and the Project Leader build the program, test it, and publish it for the use by the project team. If the program is a particularly complicated one based on the study’s design, a pre-publication demo may be arranged for key project stakeholders.

The logging program can be used to:

• Display outgoing contact summaries;
• Log incoming contact summaries;
• Log the status of incoming material (complete, partial, blank, refused, etc.);
• Manage sample member addresses, including original and updated addresses;
• Record new respondent information;
• Record additional contacts (friends, family, etc.);
• Record respondent requests for additional materials;
• Log interim disposition codes/final result codes;
• Manually modify result codes (supervisor use only);
• Record undeliverable mail returned/bad addresses; and
• Record returned incentives.

10C.12.2 Protocol for returned paper materials

Check returned materials for completeness and accuracy. Depending on the study methodology, respondent follow-up may be required for incomplete materials; more typically, missing data codes are used.

Log and file materials daily. Include:

• Sample ID number
• Date received
• Status of the piece (complete/partial/refused/blank)
• Whether the piece was returned as undeliverable and whether tracking is required
• Any specific notes recorded by the Respondent (e.g., that might indicate that the R is non-sample, that the piece is a duplicate, etc.)
• Create a reconciliation report to flag cases that have been mailed but not returned. Typically a follow-up mailing is sent to these cases.

10C.12.3 Consent and incentive receipts

Consent is usually not elicited with a Self-Administered Questionnaire (SAQ). The cover letter includes the purpose of the survey, any potential harm arising from participation, and an explicit statement that the respondent may skip any question. Survey completion is accepted as implicit consent. Likewise, receipts are not required for cash incentive payments included with the mailing. Instead, the Project Leader submits a list of all those to whom the funds were disbursed.

Survey Research Operations (SRO) places a premium on the processes and technologies that provide security during data collection, as well as on the storage of electronic and paper media containing respondents’ data. There are several controls in place to prevent the unauthorized access, modification, and/or disclosure of sensitive data, to protect capital hardware assets, and to provide the ability to recover information.

Safeguarding Respondent Confidentiality

10C.13.1 Electronic files

The Project Leader works with members of SRO’s Technical Services Group (TSG) to ensure all survey data and sample management databases are stored on secure servers. All hardware used for storing and/or accessing respondent information is located in secure areas with restricted access. For example, all SRO areas have 24-hour key-card controlled access, individual computers are password protected, and computers are locked when the employee is away from the machine. Furthermore, user access to respondent data records is restricted to authorized personnel only.

There are a variety of media that can be used for sharing electronic files containing confidential or identifying information with other project staff members and/or the client (when permissible). Examples include:

• E-Mail: A single email message cannot contain both the sample identification number and personal identifying information of a respondent (e.g., name, phone number, address). Any attachments with respondent information should be password protected.
• Shared Directories: Files containing respondent information should not be kept on shared directories with public access.
  Secure SRO Shared Directories: Within SRO, files should be placed in a study folder on a secured directory. Since only SRO members have access to this server, it is secure for any type of file delivery or sharing. Projects may decide to establish their own secure SRO drive that only members of that study can access.
• ISR/SRC Secure Drives: ISR/SRC project staff generally have secure directories on other ISR servers. They can give members of their SRO project team access to these secure directories for transfer of data files and any other files with information about or from respondents.
• eRoom: eRoom is a secure web-based server. All data transmissions are encrypted. To prevent the interception of private information across the network, eRoom.net utilizes secure sockets layer to encrypt data while in transit. Users need a license to use this software. Licenses are limited and must be obtained via TSG. Access to the project’s eRoom is limited and controlled by the room administrator. Users need authorization and a password to gain access. Files with identifying information can be stored in eRoom, but password protection is advised.
• Secure FTP Site: The project staff can provide the SRO project team with a secure FTP site for the delivery of project data (this may require assistance from CMT staff).
• Removable Media (e.g., CD-ROMs, DVDs, or jump drives): Large files may be burned to password-protected removable media and shipped via secure carrier between organizations. Not only should confidential information copied to removable media be password protected (or otherwise encrypted), these media must be kept in locked cabinets when not in use. Usage of removable media for files with confidential information should be extremely limited.

After the electronic data is no longer needed and/or permission is no longer granted, the SRO Project Leader works with CMT to ensure these data are either appropriately archived or properly disposed.

10C.13.2 Paper materials

It is the responsibility of the SRO Project Leader to identify all study-specific paper materials containing identifying information and to make sure there are locked file cabinets available to store these materials. Access to the locked file cabinets is restricted to authorized personnel. The locked file cabinets are located within the areas of SRO that have 24-hour key-card controlled access. Documents containing identifying information about respondents are stored in a separate file cabinet from documents containing their collected survey data. As soon as paper materials are received at SRO, they must be stored in the appropriate file cabinet(s). When in use, materials with information identifying respondents are never left unattended.

If the client has an interest in receiving the paper materials, the Project Leader reviews all text data and removes confidential information, unless permission has been given by the IRB and/or SRC to deliver identifying information to the client.

To protect paper materials during long-term storage, all forms, printouts, and reports are archived in climate- and access-controlled off-site facilities. These materials are maintained for the time period designated by the contract. Once released for destruction, they are shredded or otherwise appropriately destroyed. If long-term storage is not needed, paper materials are shredded as soon as there is no further need of them.

Before data collection begins, the Project Leader works with the Supporting Statistician to determine which production indicators will be monitored during data collection and what the target goals are for each of those indicators. They then share these goals with others on the project.

Once the project is underway, the Project Leader monitors actual progress against the production goals, typically sending a weekly summary report to the client. On most studies, the reports that chart production progress are posted daily in WebLog.

See Section 3.4.3 (Project Management, Reporting) for more information on reporting progress.

10C.14.1 Result codes and outcome rates

All SRO projects use the same standard result codes to record the outcome of attempts to contact sample members. The result codes are then used to calculate outcome rates (e.g., response rate, cooperation rate, contact rate). The outcome rates help predict how production will compare to the goals set for the study. SRO’s result codes and outcome rates are based on the American Association for Public Opinion Research (AAPOR) Standard Definitions.

AAPOR Standard Definitions: Final Case Dispositions and Outcome Rates

10C.14.2 Client progress reports

The client and the Project Leader determine the frequency of client progress reports, although SRO recommends that each project provide client reports at least once per month. The content and format of the report is determined by the Project Leader, with input from the Senior Project Advisor (SPA) and client. Client reports may need to be approved by the SPA before being released to the client.

10C.14.3 Monthly project reports

“Monthly Project Reports” is accessible via the SRO Intranet and gives the reader basic information about the project and a monthly snapshot of project activities. All projects, both funded and developmental, are entered into this system.

The link “Monthly Project Report (MPR)” allows view-only access, while the “MPR Administration” link allows read-write access to Managers.

SRO Monthly Project Report System

10C.14.4 Project reviews

Each Project Leader schedules a minimum of three project review meetings with SRO Admin: 1) At the inception of the project; 2) When data collection begins; 3) After data collection or at the close-out phase of a project. Additional meetings may be scheduled as necessary. The Project Leader may choose to invite other members of the project team who are relevant to the current phase of the project. For the initial meeting, the Project Leader provides a copy of the project charter. For all other meetings, the Project Leader brings the current version of the project management plan and reports on data collection progress and project costs. There is a Project Management Review Template on the SRO Intranet, which is accessible through the “Project Management” link.

You will find the Project Management (or Administrative) Review Template in the right-hand column, under the Documents header. There are two versions, one for “TIG” or “Technical Infrastructure Group”-funded projects, and all others. Non-TSG or non-technical development projects use the standard Project Management Review Template (i.e., the upper link).

Project Management Review Summary Template

10C.15.1 Responsive design

Responsive design is the ability “to alter the design during the course of data collection in order to improve survey cost efficiency and achieve more precise, less biased estimates.” In order to implement responsive design, production goals must be set before data collection begins and then monitored throughout. Key measures are identified and, if the goal(s) on these key measures are not being achieved by a specified date during production, a predetermined responsive design is enacted.

10C.15.2. End game strategies

End game activities are implemented at the end of the study to achieve production goals. They vary among projects, depending on the resources (budget, study-specific materials, etc.) and the time available. Common end-game strategies include increasing respondent incentives and using UPS, Priority mail, or other envelopes that are likely to be noticed (and opened) by the respondent.

10C.15.3.Documenting change

Any changes made to materials or procedures once production data collection has begun are documented and added to the Management Plan. This includes memos to interviewers outlining changes to procedures and/or materials and copies of the revised materials.

In addition to end-game activities, the scope of work that SRO and the funding agency/investigator agreed to will most likely change over time. To make certain that the necessary resources are available to handle scope changes, the Project Leader sends an email, with an attachment defining the workscope change, to the SRO Operations Team.

See Sections 3.4.4 (Project Management, Requesting Approval of Changes in Scope of Work) and 3.4.5 (Project Management, Obtaining IRB Approval of Changes in Procedures and Materials) for more information on requesting a change in workscope.

See Section 3.4.2 (Project Management, Financial Monitoring and Management) for information on monitoring and reporting cost.

See Chapter 12 (Archiving and Documentation) and Section 3.5 (Project Management, Project Closeout) for information on project closeout and documentation.