12. Archiving & Documentation
12.1 Introduction
This chapter covers the project activities and deliverables necessary for documenting and archiving a SRO project, and adheres to the Survey Research Operations Standards (2020).
SRO Standard Project Procedures
Archiving processes result in the following mandatory deliverables:
Deliverables
- Final technical report;
- Final cost report;
- Final production and quality control reports; and
- Archived project files and materials.
Figure 12.1 provides a flow diagram of archiving and documentation processes. SRO best practices for archiving and documentation are described in Sections 12.2 through 12.7, and cover the following:
- 12.2 Project reports;
- 12.3 Archiving and storage guidelines;
- 12.4 Archiving survey metadata;
- 12.5 Archiving process data;
- 12.6 Archiving survey data; and
- 12.7 Data and materials delivery.
Project Records Retention Schedule
File Naming Guidelines
Archives Materials Checklist
Appraisal Workflow for Archives
Figure 12.1 Archiving and Documentation Processes
12.2 Project Reports
At the end of each project, the SRO Project Leader will prepare final summary reports of survey procedures, methods, and basic summary results. Activities in this process include, but are not limited to:
12.2.1 Write final technical report
The project lead will prepare a final report which describes activities, deliverables, and results of the project. The final report must be approved by the SPA before delivery to the client.
12.2.2 Prepare final cost report
After all costs have been posted, the budget analyst will run a final cost report from the SRO Cost Reporting System (CRS). Final cost report(s) will be converted to a generic format (e.g., .PDF) and submitted to the SRO archivist for inclusion with the survey metadata files (See Section 12.4).
12.2.3 Prepare final production and quality control reports
After data collection is complete, final production and quality control reports will be produced by the appropriate team member. Final reports will be converted to a generic format (e.g., .PDF format) and submitted to the SRO archivist for inclusion in the metadata files (See Section 12.4).
12.3 Archiving & Storage Guidelines
The project lead will coordinate archiving and storage with project team members, in accordance with contract requirements, SRO standards, and procedures and deliverables negotiated with the PI. Survey Research Operations (SRO) archiving protocols are designed to serve the needs of Survey Research Center (SRC) staff and its principal investigators. It is not intended to be a “library” of material to be used by the general public. General guidelines in this regard include, but are not limited to:
Meet project legal and contractual obligations
SRO projects will meet legal and contractual obligations. Some contracts specify the number of years materials should be kept. In addition, IRB committees currently require that consent forms be kept for three years. Archiving practices will follow these expressed requirements.
Code of Federal Regulations CFR 46.115(7)(b)
Collect information in electronic format
SRO’s goal is to limit the amount of hard copy material (e.g., coversheets, consent forms, audio tapes, completed questionnaires, etc.). Where feasible, the project lead will consult with Technical Services Group (TSG) and the Principal Investigator (PI) to design procedures to collect information in electronic format.
12.3.1 Hard copy materials storage
Following are the principal project requirements for the management and storage of hard copy materials.
Secure storage
The SRO project lead will arrange for secure storage at SRO for the first year after the end of data collection, and in a secure remote location (e.g., airport storage facility) for subsequent years.
Minimum period for storage of consent forms
In compliance with the Code of Federal Regulations CFR 46.115(7)(b), consent forms will be stored for a minimum of three years.
Storage costs
Storage costs will be included in the project budget and charged to project accounts.
Destroy-by-date
Hard copy materials in long-term storage will be marked with a “destroy by” date approved by the Principal Investigator. Destruction will be done in a manner that maintains the security of the information on the forms. The cost of materials destruction will be included in the project budget and charged to project accounts.
Periodic review of storage inventory
The SRO inventory of hard copy materials in storage will be reviewed annually by the SRO archivist. Material being held past the “destroy by” date will be destroyed. The project lead (or PDMG director if the project lead is no longer at SRO) will notify the PI before materials are destroyed, and the PI will have the option of modifying the “destroy by” date and paying the additional storage fees, or requesting to have the materials transferred to him/her in accordance with SRO policies for delivery of materials.
12.3.2 Electronic files
Short-term storage
SRO will maintain key electronic files for a period of 1-3 years after the end of data collection on the SRO short-term archive server. The files will be maintained with full system functionality. The length of time the files will be kept on the short term archive server will be negotiated with the PI, and costs associated with this storage will be charged to the project. Files stored will vary by project, but common electronic files are:
- Survey instruments;
- Survey instrument documentation;
- Instrument and system technical documentation;
- Process documentation;
- Codebooks;
- Survey data (if Blaise, include both .asc and .adt files);
- Reports;
- Training materials;
- Survey project metadata; and
- Process data.
Long-term storage
Long-term storage for key electronic files will be available for up to 10 years after the end of data collection. The Principal Investigator can negotiate a longer storage period if desired. Data Managers will set up a Survey Trak merge process for incomplete suspended interviews, even though they may not be initially needed by the project or by SRO analysts; this will ensure that the responses and keystroke files for these suspended interviews remain available. SRO long-term storage will be in generic format, so that the files are readable but do not have full system functionality. TSG will maintain, for at least 10 years, all versions of Blaise, Illume, and any other pertinent interview data collection software.
Storage costs
Storage costs will be included in the project budget and charged to project accounts.
12.4 Archiving Metadata
Metadata is high-level information that describes the purpose of the project, how it was implemented, and presents summaries of results. Typical metadata files include the project charter, management plan, final technical report, and final cost and production reports. Metadata files are maintained by SRO indefinitely, to provide historical information that retains the institutional memory of SRC.
Activities for this task could include, but are not limited to:
12.4.1 Submit files to archivist
The project lead will submit metadata files to the SRO archivist at the conclusion of each project. Although these files can be hard copy format, electronic format is preferred.
12.4.2 Place metadata in long-term storage
SRO will preserve materials to allow current and future SRC investigators to replicate studies, and will maintain administrative information to be used by SRO in budgeting and planning future studies. This includes sample information, study materials, study procedure manuals, project technical reports, cost data, staffing lists, project management plans, programs for special reports, and project manuals. SRO will export data and documents to a generic format that is likely to be usable for the next 10 years and will save documents in both their original and generic forms.
12.4.3 Add project to archive database
The SRO archivist will maintain a database with metadata information. As projects are completed, they will be added to the SRO archive database. This database is available for review on the SRO intranet.
12.5 Archiving Process Data
Paradata files (also called process files) include global process measures such as response rates and coverage rates, call history data, and also micro data at the individual case level such as item-level time stamps or keystroke files. Process data files are stored for 1 to 3 years after the close of data collection (the exact time will be negotiated with the PI and documented in the project plan) in fully editable files, and then moved to a static storage format and kept for a total of 10 years after the close of data collection.
Activities for this task could include, but are not limited to:
12.5.1 Prepare files for archiving
The TSG data management project team member will prepare the file(s) for archiving. The following information will be archived in electronic format and stored in a secure SRO server for a minimum of 10 years: key sample line information including sample ID, final disposition, and listing, housing unit, and interview observations, call history records, key interviewer demographics and performance data, and audit trail data that have been extracted from the interview audit trails (item-level times, function keys, keystrokes, and backup data).
12.5.2 Submit files to archivist
The project lead will work with TSG staff to archive these materials as part of the study close-out process.
12.5.3 Place paradata in long-term storage
The SRO archivist will store dated paradata files electronically in long-term storage with project metadata electronic files and paper materials. The files will be stored in generic format.
12.5.4 Update archive database
The SRO archivist will update the appropriate fields when adding dated paradata files to the archives.
12.6 Archiving Survey Data
Survey data (data from questionnaires or interviewer observation files) will be stored for 1-3 years after the close of data collection (the exact time will be negotiated with the PI and documented in the project plan) in fully editable files, and then moved to a static storage format and kept for a total of 10 years after the close of data collection.
Activities for this task could include, but are not limited to:
12.6.1 Identify survey data files
The TSG data management team member, in consultation with the project lead, will select the files containing survey data that are appropriate for archiving. This will usually include only production data, but for longitudinal studies where pretest samples will be contacted in future waves, the pretest data may also be archived.
12.6.2 Prepare and submit files to archivist
The TSG data management project team member will prepare the file(s) for archiving, and submit them to the SRO archivist.
12.6.3 Update archive database
The SRO archivist will update the appropriate fields when adding dated survey data files to the archives.
12.7 Data & Materials Delivery
Delivery or release of all hard copy and electronic materials will comply with ISR’s Policy on Safeguarding Respondent Privacy.
Activities for this task could include, but are not limited to:
12.7.1 Remove household and respondent identifiers
The TSG data management project team member will work with the project lead to flag variables in the survey dataset that contain identifying information. Those variables will be removed from the dataset before it is delivered to the PI. Common identifiers include:
- Sample line preload data;
- Household roster information; and
- Open ends and string data that may contain confidential information.
12.7.2 Obtain receipt from ISR PI’s for data with identifying information
The project lead will obtain a receipt for datasets that are delivered to ISR PI’s. The receipt will include a confidentiality memorandum assuring that the data or materials will be stored in a secure location. With this receipt and assurance, materials or files containing identifying information as well as survey data may be delivered to the PI.
12.7.3 Obtain receipt from external PI’s for data without identifying information
PI’s external to ISR will be given materials and files that have been stripped of identifying information after a receipt has been signed that assures proper use of the data and materials. SRO is responsible for stripping obvious identifiers from datasets but does not conduct a disclosure analysis prior to releasing data, unless it is explicitly included in the contract.
12.7.4 Obtain IRB permission to release identifying information to external PI’s
PI’s external to ISR must obtain explicit IRB permission before data or materials containing identifying information will be released to them.
12.7.5 Obtain Memorandum of Understanding
PI’s external to ISR must negotiate a Memorandum of Understanding that is signed by the SRC Director and approved by the Senior Staff Advisory Committee (SSAC) before data files or materials containing identifying information will be released to them.