At the preliminary/technical planning stages of a project, a technical team is put together based on resources available in each department.  Each team is composed of a Technical Leader, the Project Leader, Production Manager and Production Assistants, a Database Administrator (DBA), a Programmer for each technical system, a Data Manager, and an Interviewer Help Desk representative.

During the system and instrument design phase of development, the Project Leader and Technical Leader discuss issues related to quality assurance with their Data Manager.  These discussions are reflected in the following documents that the Technical Leader provides the Data Manager:

• Instrument technical specifications;
• System technical specifications;
• Description of the dataset’s content and format; and
• Testing checklist.

8.4.1 Create training projects

In preparation for interviewer training, the Data Manager may create a number of projects. All newly hired interviewers go through a separate General Interviewer Techniques (GIT) training to learn standard interviewing procedures and to become familiar with the sample management system, SurveyTrak. The Data Manager adds the new interviewers to the most current GIT project and creates the project-specific projects for both training and certification. Because many studies involve more than one SurveyTrak project, some of the tasks in the following sections are duplicated for each project in the SurveyTrak database related to the study.

The Data Manager is notified that the staff list is available once it is finalized and posted on eRoom. Then the Data Manager adds the interviewers to the GIT, training, and certification projects and generates lines in each using LineGenerator.  The Data Manager must work with the Interviewer Help Desk staff responsible for loading the interviewer laptops and ensure that the projects and sample lines are on the laptops and that quality checks are performed before training begins.

8.4.2 Prepare sample lines

Project Leaders provide scenarios for training. This is usually done in the testing phase, so they can thoroughly test the scenarios they will be using at training.   There are two types of preload to prepare:  (1) the sample management preload (respondent names, addresses, behind the scenes flags (i.e., data to load into SurveyTrak)), and (2) the Blaise preload. 

Often, the Blaise preload for the training lines will come in a long Excel file.  This file must be manipulated to create the Blaise preload string, usually done in SAS and then exported to a caret (^) delimited text file to load into the SurveyTrak tCapi table. 

It is necessary to run quality checks on the Blaise preload data. This could involve importing the preload into the datamodel and looking for import errors, running through testing lines to make sure the values line up correctly, and looking at the variable lengths and distributions in SAS to confirm that the preload fits the data model. Data Managers should understand that the preload is a critical piece of the data collection process, and ideally the Survey Director and Blaise programmer will work with the Data Manager to establish appropriate QC checks and follow-up.

8.4.3 Attach RCLS calls to sample lines

The Respondent Contact Logging System (RCLS) is an application designed to track incoming contacts made or attempted. These are separate from interviewer attempts to contact respondents. Contacts logged in this system are incoming calls to the respondent toll-free line and voicemails, returned or undeliverable mail, and other forms of communication coming into the Ann Arbor office. Emails containing pertinent contact information are sent to the interviewer from this system, and the sample lines are flagged for the interviewer to see that there is information for them to review.

There is a section of the GIT training that covers RCLS and it is standard practice for the Data Manager to attach RCLS calls to two of the GIT lines. Because attaching RCLS calls has not been added to the LineGenerator application, this process is done through a standard set of MS Access queries. The Access database, GIT12_RCLS Prep ST11.mdb, is located in the Training folder in the Template project folder
(\\SRC-Douvan\SRO\groups\TSG\dataops\TEMPLATE\NEWPROJECT\Training).  For further information and steps to add RCLS calls to training lines, see the Data Management Wiki. Contact a Data Manager supervisor for access. 

There are four lines in the tSampleLine_Supplemental table for each call. A view in the SurveyTrak Reporting database, DBA.vRCLS_view, organizes the RCLS data in single rows for each call. This is the best place to look at the RCLS data.

The Data Manager sometimes needs to close RCLS calls, set RCLS to send messages to the right people, or back up records when resetting lines. In addition, throughout the life of a SurveyTrak project, RCLS action requests come up from time to time for the Data Manager. These tasks are discussed in Section 8.6, Data Collection Support.

8.4.3 Create profiles 

Data from previous waves of data collection, such as address or household information, can be displayed in SurveyTrak, Webtrak, and RCLS.  These are called respondent profiles, which are available for the interviewer to reference during data collection. Either the project staff or the Data Manager creates the profiles as text or html files. After the production sample is loaded into SurveyTrak, the Data Manager uses the application RProfile to load the profiles. If the Data Manager is to create the profiles, the Project Leader provides the specifications and signs-off on the profiles before they are loaded into SurveyTrak.  If project staff creates the profiles, the Data Manager should review and test them.

Data for the profiles generally comes from a wide range of sources, including: SurveyTrak data from past waves (call records, RCLS notes, and any BCP pulls from the instruments); address updates from other sources; Census data, etc. There is example SAS code to generate production profiles as well as profiles on training lines in the project production template folder (see Section 8.3.1).  It is also standard practice to run the HTML code through an HTML validator such as that in UltraEdit.

HRS 2012 introduced using Javascript to create profile sections that open and close, as well as imposing a maximum length on lines of text to increase readability.  Due to their size, the profiles were generated into four files and run on the Linux server.

When creating large profiles, the Data Manager needs to work closely with the DBA team to coordinate the loading of profiles. Data Managers give the DBA team at least 24 hours’ notice that profiles need to be loaded. This advance notice is necessary even if only a subset of the profiles is being re-loaded. 

8.5.2 Create assignment file 

The purpose of the sample assignment procedure is to allocate production sample to the interviewing staff assigned to a production project. Some projects assign sample at the sample line level, while other projects assign sample at the Primary Sampling Unit (PSU) or PSU segment level. After cleaning and reformatting the sample file, the Data Manager creates an assignment file based on the project needs.

The Data Manager sends the file(s) to the Production Manager or designated assigner, who determines the assigned interviewer for each sample line. They add the Interviewer’s 8-digit University ID to the assignment file and return it to the Data Manager. The Data Manager then performs certain quality checks, and may need to work with the assigner to clarify or correct information. The assigned interviewer field is used to load sample into SurveyTrak. The Data Manager also may need this information to process respondent prepayments if they are to be sent to the interviewers.  For further information and steps to create sample assignment files, see the Data Management Wiki. Contact a Data Manager supervisor for access.

Sample Assignment Examples

The Data Manager works with the assigner, and if necessary the Production Manager, to minimize any delay in receiving and preparing the assignment file on schedule.  At the Production Manager’s request, Data Managers may assign sample directly to a Production Coordinator or Team Leader, who will later transfer the lines to interviewers. This may happen if there is uncertainty about the availability of interviewers at the time sample assignments need to be made.

It is good practice to send the Project Managers and Production Manager (and Team Leaders in some cases) a summary of sample assignments after the lines have been loaded into SurveyTrak, but before they are activated. This gives the Managers another chance to easily make changes before the sample “goes live.”

Sometimes sample assignments may change after the initial processing, for example, if an interviewer is terminated during training. If the sample is inactive (tSampleLine.sActiveStatus = “02” or “03”), the Data Manager reassigns the sample in tSample_Line to another interviewer or the Team Leader as requested. If the sample is active, it can be transferred from the interviewer’s laptop via SurveyTrak, or forced by the Data Manager if necessary. If there were prepayments, they may also need to be redirected.

8.5.3 Release sample

Once the assignment file has been approved, production sample activation can differ from project to project. Data Managers coordinate with the Project Leaders and the Interviewer Help Desk on when and how to release the sample. Sample cannot be released for interviewers who have not yet passed certification. Typically when sample is released, the Data Manager sends an email to Project Managers and the Interviewer Help Desk. To release (or activate) sample, tsample_line.sactivestatus is changed from “03” to “01.”

Sometimes Data Managers are asked to perform sample management tasks, such as release sample in different batches or pull sample back. These tasks should be discussed with the Project Manager and documented.

8.5.4 Set up data extraction

During overnight processing, the Interview Data Merge (IDM) takes all the completed interviews from the interviewers’ laptops and merges them into one master file with the SurveyTrak Reporting System Manager (SRSM).  It is the Data Manager’s responsibility to set the merge criteria and monitor IDM’s progress.  Once the IDM is set up for the project’s instruments, additional criteria changes will need to be made when datamodels are updated, which is best done through STAdmin.  For further information and steps for setting up a merge, review error codes and merge criterion, see the Data Management Wiki.

The merge examines Blaise audit trail files (file extensions .adk and .adt) to determine the interview length and, for some projects, block lengths. Occasionally audit trail files are corrupt. If a file is corrupt, it cannot be relied on for determining interview length. The merge performs a number of checks to determine whether the file may be corrupt. So that interview data can successfully merge even if an audit trail file is corrupt, audit trail files are processed in batch after interview data is merged.

Depending on interviewers’ send and receive times, some interviews (mostly contact observations) get missed. Before producing any data set for delivery to the clients, it is necessary to set the date in the merge criteria back to either the beginning of the study, or another reasonable date, in order to identify any missed data.

For example, the National Survey of Family Growth (NSFG) sets the date back to the beginning of every quarter on Mondays, Wednesdays and Fridays.  This is because there is ongoing data analysis throughout the quarter and this keeps the daily files up-to-date.  In the NSFG example, the date in tmerge_criteria is updated in an Access macro, and then the IDM runs with error handling through a scheduled task on the processing computer.

8.5.5 Set up datamodel migration

When a datamodel is updated during production, it is necessary to migrate the master data file to the new datamodel so that all interviews have the same data structure.  The Blaise programmer should include the migration script (blaTobla.asc) in the datamodel.  Sometimes the migration scripts are straightforward, and in other cases there is a complicated algorithm. Occasionally it is necessary to manually migrate to a newer or older datamodel.    For further information and steps for datamodel migration, see the Data Management Wiki.

8.5.6 Set up daily reports archiving

Daily Reports, or Field Progress Reports (FPRs), are standard .pdf files that programmers set up for each project. They display data from tables in the SurveyTrak reporting database. These reports allow Project Managers and Supervisors to see what production (hours per interview, response and completion rates, etc.) looks like on a given day. They are archived daily, and are used to compare production across waves of data collection or during budget preparation for a similar study.

The Daily Report Archive System (DRAS) runs a scheduled task twice a day at 8:30am and 3:00pm.  At each run, the system looks for new reports, archives marked previous reports, and checks disk space.

DRAS sends emails at each runtime including:

1. Number of reports archived;
2. New reports in the WebTrak folder;
3. New reports not approved within DRAS; and
4. When the archive folder has reached 90% of available space.

The archive folder is located at \\SRC-Hess\data\CO_rpts.  Each project has a separate subdirectory. There is also a subdirectory called “Archive.”  When a project ends, the final reports are kept in the project folder under the Archive subdirectory. If there is a request for final daily reports, a Data Manager can pull reports from the Archive folder.

When the out of disk space message comes from DRAS, a Data Manager moves the archived reports from the network drive to the SRO archive server.. This practice ensures that we have reports for whenever they are needed for review.

For further information and steps for archiving daily reports, see the Data Management Wiki. Contact a Data Manager supervisor for access.

8.6.1 Perform quality assurance checks

It is impossible to anticipate all problems that may occur during data collection, but it is important to identify and respond to problems quickly. The Data Manager is responsible for identifying errors in survey data and problems related to the procedures, processes, systems, and instruments used to create the data. This includes associated metadata (data about data) and paradata (data about processes).  To be able to uncover problems that affect data quality as soon as possible and to take corrective action, the Data Manager creates a project quality assurance program. The goal of the program is to ensure that all data will fully meet the analysis requirements of the study and conform to SRO Standards.

The program has two components. The first outlines SRO standard checks that must be done for every project. This involves intensive monitoring of specific data items and study procedures and processes throughout the data collection period.

The second component of the program consists of any additional data quality checks the Principal Investigator(s) and project staff feel are necessary to ensure that that data will meet the research and analysis requirements of the study. Projects vary widely in requirements, and each project may enhance standard data quality control procedures, as well as define unique measures. The checks performed in this portion of the program are designed in close collaboration with the Technical Leader, Project Leader, and the Principal Investigator and project staff.

There are nine standard QC SAS programs called by a tenth program (_RunAll.sas) in the directory \\SRC-Douvan\SRO\groups\TSG\dataops\DOCUMENT\SAS\QC.  The nine programs are:

1. QC Checks Addresses.sas
2. QC Checks Call Records.sas
3. QC Checks Contact Obs.sas
4. QC Checks Interviewer.sas
5. QC Checks Merge Track.sas
6. QC Checks Misc Standard.sas
7. QC Checks Preload.sas
8. QC Checks Rpay.sas
9. QC Checks Timings.sas

The _RunAll.sas program in the template folder points to the above standard SAS programs, so they do not need to be copied.  If project specific changes need to be made, they can be made in a separate SAS program called in the the _RunAll.sas program.

For further information and steps of the QC process, see the Data Management Wiki. Contact a Data Manager supervisor for access.

8.6.2 Respond to help desk calls

Requests come to the Data Managers through the Footprints help desk system. The Data Manager is notified by email about Interviewer Help Desk calls that require their attention. There are a number of different types of Help Desk calls, ranging from adding a user to a project to an interviewer losing interview data. For further information and steps to address Help Desk calls, see the Data Management Wiki.

Remove interviewer from project (“closeout”) 

The Data Manager’s role in closing out an interviewer’s project assignments is to identify what production data is left on the interviewer’s laptop when they leave a project. This includes sample lines, recording files, RCLS calls, and any other project-specific data.

The closeout request is initiated when an interviewer quits, is terminated, or is otherwise finished working on part or all of a project. A closeout request may come in for one project, while the interviewer continues to work on another project and can happen before a project begins production, in the middle of data collection, or as the project wraps up production.

Closeout requests are handled similarly for each of these scenarios, but can differ based on project needs. Data Managers approve an interviewer for closeout when there are no production data left on the laptop and the interviewer has been deactivated on the project. It is best practice to flag and remove all lines and deactivate the interviewer from all GIT, training, certification, and production projects associated with the study. If the interviewer will remain on another project, then the GIT project and sample lines can remain active on their laptops. The Data Manager’s approval then goes through the Footprints system to either (1) Inventory when the laptop has been returned to Ann Arbor, or to (2) the Interviewer Help Desk, if the interviewer is being removed from one project but is still active on other projects.

Once the Data Manager sends approval for closeout through the Footprints system to Inventory, the closeout process continues through the different channels—DBAs revoke replication and Computing and Multimedia Technologies (CMT) removes them from the active directory for those interviewers who returned their laptop.

Remove sample from interviewer databases (“force transfer”)

Circumstances around removal of sample from interviewers databases (“force transfers”) are rare, but can be requested by project staff when a laptop is lost or stolen, an interviewer is terminated quickly, or other instances where a regular line transfer is not possible.

The Data Manager FIRST confirms with the DBAs that replication has been revoked for the interviewer who currently holds the lines.

Reset sample line

Interviewers sometimes screen at the wrong household or persons are added or missed in the household rostering, affecting the sampling process.  There are other reasons for line resets, however, these are uncommon and project-specific.  Lines need to be thoroughly reviewed and approved, generally by Team Leaders, Project Coordinators, or Project Leaders before resetting because this process can impact the sample selection. There have been times when an interviewer has asked for a line reset simply because the person selected through the household rostering was not home.  Resetting this kind of case would be considered data manipulation and is grounds for termination.

Before production starts, discuss the protocol for handling line resets with the project manager.  For example, NSFG Cycle 8 requires the interviewer to get approval from their FOC and project staff before submitting a help desk call.  Before resetting the line, the Data Manager confirms that there has been approval from one of three project staff.

Recover lost interview

Sometimes interviews get lost, or appear missing. Notification when this occurs is when the Interview Data Merge sends an error saying “interview data is null.”

Often the interview can be recovered from the SurveyTrak Incremental Backup logs.  The SurveyTrak database is backed up in 15 minute increments. The Data Manager uses a .sql file to translate the log file. If the interview is found in this file, then it can be loaded back into tCAPI using iSQL.

8.6.3 Remove sample from interviewer laptop (“scoop”)

Data Managers are expected to keep the “Removing lines from laptops – Current procedures” document for on-going projects. This document is available to all on\\SRC-Douvan\SRO\share\Data Ops.

After the interview is completed, sample lines will be pulled from the Interviewer’s laptop at a pre-defined, regular interval. This process aids in securing the data from unexpected incidents (i.e., a stolen laptop). Standard protocol is to run this process at least once a week, although some projects, such as NSFG, run this process once a quarter.

Because each project is unique, the Project Leader defines what constitutes a completed sample line. Some projects want to keep sample lines active in the field until Self-Administered Questionnaires (SAQs) are returned and logged in Ann Arbor, while other projects wait for incentive checks to be processed.

To be removed from the laptop, the sample line must meet the following conditions:

• The sample line must have a final result code;
• There must be consistency between the final result code and interview status according to project specifications;
• The interview must have been merged into the main master file(s) at least 7 calendar days prior;
• All contact observations and post-interview observations for the sample line must be completed according to project specifications;
• Housing unit observations must be completed, if housing unit observations are required for the project;
• If payments are being recorded in SurveyTrak, a payment record must exist according to project specifications; and
• If the project is using digital recording, the digitally recorded interview (DRI) files must be logged as received by the Ann Arbor office, and files removed from the interviewer’s laptop.

Sample is also removed from laptops is when the interviewer leaves the project.  This process is triggered when the Data Manager is assigned a help desk call notifying them that an interviewer is leaving a project, regardless of whether they are keeping the laptop or not.  In this case, all lines associated with the study need to be cleared from the laptop.  This process is described in the Interviewer Close Out section under Help Desk Calls.

For further information and steps for sample removal, see the Data Management Wiki.

8.6.4 Document lost or stolen laptop

In the rare circumstance of a lost or stolen laptop, actions are taken to document what was on the laptop.

The Data Manager sends a report to the Director of Data Collection Services (DCS) and copies the Interviewer Help Desk Supervisor and the Project Leader(s). The Data Manager then stays available in case any further questions come up and to help facilitate communication with the DBAs if the interviewer’s database replication needs to be revoked and/or sample needs to be forced to another interviewer. Data Managers should not force sample without first receiving confirmation that replication has been revoked.

The Help Desk Supervisor, the DBAs, DCS, and Inventory usually step in to handle the rest of the process of trying to track down the laptop. Eventually, the Data Manager may need to force transfer that interviewer’s sample to someone else. (See force transfer section under Section 8.6.2, Respond to Help Desk Calls.)

For further information and steps to address lost laptops, see the Data Management Wiki. Contact a Data Manager supervisor for access.

8.6.5 Maintain documentation

Throughout the development, testing, and production stages of a project, it is standard practice to maintain documentation for each task that is performed on the project. This also includes retaining emails relevant to decision making and process flow. The documentation should be easy to follow for another Data Manager who may serve as backup if the primary Data Manager is ill, on vacation, or otherwise indisposed. The SAS code written for processes should also be well commented. If there are ever any questions on how or why a task is performed or where the data is, it is necessary for the documentation to be able to answer these questions.

The Data Manager also keeps a problem database to track helpdesk calls or issues from other sources.The project data dictionary must also be kept up-to-date, as this is used by many within and outside of SRO.

Much of our communication is through email.  It is standard practice to maintain a project email folder within your project email and to archive it to the project folder.

8.6.6 Maintain automated tasks

There are many tasks that Data Managers perform regularly. Once the task has been set up and has been thoroughly tested, it is efficient to put the task on the task scheduler. There are processing computers set aside to perform these tasks that are monitored by the DBA on duty. It is our best practice to use these processing computers rather than a personal computer to run the scheduled tasks.

Some common tasks that Data Managers perform are running Access queries or macros, SAS programs, and Blaise Manipula scripts.  These tasks are set up via batch files on the processing computer and then run from the Task Scheduler program.  The processing computers may have different versions of the programs that are on your personal computer and therefore, Data Managers should always check that a scheduled task runs on the scheduler.

Passwords are not to be written into batch files. If a password is needed to pass into an application, it can be included within the Scheduled Task.  It is the Data Manager’s group responsibility to maintain passwords in the ODBC DSNs and the SAS autoexec used on the processing computers.

For further information and steps to set up scheduled tasks, see the Data Management Wiki.

8.6.7 Set up telephone study reporting fields

Most studies conducted in the Survey Services Lab (SSL) use a Computer-Assisted Telephone Interviewing (CATI) Sample Management System (SMS).  SSL Project Leaders use SMS for their real-time reports and production monitoring. Generally, the project Data Manager  is less actively involved in SSL projects, with major tasks including setting up the SMS-RT (SMS-Real Time) process, report archiving, and processing data for delivery.

SMS-RT transfers via an overnight process interviewing paradata from SMS to the SurveyTrak Reporting Database. This allows SSL Project Leaders to review reports on data collection effort that are similar to those generated for decentralized (field) studies. This paradata includes information in the SurveyTrak tSample_Line, and TCall_Information tables.

The Data Manager is responsible for setting up telephone real-time reporting standard and project-specific fields in SurveyTrak tables.

For further information and steps for setting up SMS-RT, see the Data Management Wiki. Contact a Data Manager supervisor for access.