Technical Topic Papers
Rocky Flats Public Exposure Study

Quality Assurance for the Public Exposures Studies on Rocky Flats

Quality assurance is a system of procedures to increase the reliability of the results of a process such as a research study. The procedures require that specific standards be met throughout the process of planning, control, evaluation, correction and documentation, and help ensure that the study's results are valid. Generally, quality assurance in producing products or services means that the results meet the requirements of the end users. Each quality assurance program must be tailored to the specific study.

Quality assurance on the Historical Public Exposures Studies on Rocky Flats was performed by an oversight panel known as the Health Advisory Panel (HAP). Panel members included experts in toxicology, epidemiology, risk assessment, biochemistry, meteorology, environmental modeling, medicine and radiation health physics, plus a citizen and a local government representative.

The panel held technical work sessions several times a year to review and discuss the work performed by the study contractors and to provide direction. At the outset of each phase of the study, the contractors were required to develop work plans for each of the research tasks. The panel members individually reviewed and commented on the work plans and all technical documents. When required, a panel member with security clearance reviewed and analyzed classified information to confirm the contractor's research findings. Panel members with applicable technical expertise often recommended alternative approaches to the research and analyses and gave direct feedback, both written and oral, to the contractors on changes and additions to be made.

Quality control is only one component of an overall quality assurance program. It refers to inspection activities focused on specific aspects of the production, study process or final product. For instance, a manufactured product is inspected to make sure that it meets all design specifications and functional requirements. In a scientific study, one example of a quality control step is having calculations or data entry checked and validated by a second qualified person.

The Colorado Department of Public Health and Environment staff in epidemiology and radiation invited experts to help design the two-phased study approach and develop the Request for Proposals for Phase I of the research. Input was given by experts from the University of Colorado Health Sciences Center, federal Centers for Disease Control, Illinois Department of Health, Texas A&M University, the University of California at Berkeley and the American Federation of Scientists. The Phase II Request for Proposals was designed by scientists on the Health Advisory Panel, with input from many of the same sources as Phase I and additional expertise from Clemson University in South Carolina and Clark University in Massachusetts. During both phases of the research, other experts were called upon for consultation on specific study aspects that were outside the panel members' areas of expertise. Several were invited to make presentations to the panel. At least two scientific peer reviewers were asked to comment on all major technical reports.

• Incident Investigation - performed additional research on past incidents such as the 1957 fire and other significant release events.

• Lessons Learned - compiled a summary of lessons learned for others conducting similar studies.

• Environmental Sampling - evaluated the need for additional soil or other sampling data to identify contamination from past releases.

• Technical Oversight - resolved various scientific differences and issues related to the study.

• Health Effects Information - identified information needed by health practitioners and data that would be necessary if subsequent epidemiologic studies were to be conducted.

• Public Involvement planned and recommended activities to keep the public informed and involved in the study.

The panel formed subcommittees to focus on study issues requiring more intensive discussion or research. Panel members and others with special expertise and/or interest in a particular topic area devoted additional time to work on important issues raised by the panel and interested citizens in the course of the study. The subcommittees performed additional research, explored alternative analyses and developed recommendations for the entire panel to consider.

New insights and information were often provided by the general public. Many citizens had special knowledge of past plant operations and technical expertise that was valuable to the research effort. Public input helped scientists see a different perspective and identified special concerns or interests of community members. External scrutiny by diverse groups and individuals improved the study results and was an important component of overall quality assurance.

Several methods were used to document suggested changes or comments from the Health Advisory Panel members and the general public. At its technical work sessions, the panel used feedback forms to provide individual written comments or direction to the contractors. To facilitate review and follow-up, the panel encouraged comments on any aspect of the study to be submitted in writing. Professional stenographers prepared a verbatim transcript of each panel work session to document oral comments. The panel adopted policy guidelines for eliciting and responding to input. All comments on the study, whether from internal or external sources, were reviewed for applicability and acted upon as appropriate. Responses were provided for written public comments on draft reports. Briefing books with key study materials plus written comments and responses were submitted to the panel and placed in Rocky Flats public reading rooms which include the Colorado Department of Public Health and Environment.

1. Experts in the fields of health risk assessment, epidemiology, health physics and dose reconstruction assisted the Colorado Department of Public Health and Environment in designing the study approach.
2. The Health Advisory Panel appointed to provide oversight included experts in toxicology, epidemiology, risk assessment, meteorology, biochemistry, environmental modeling, medicine and radiation health physics, plus a citizen and a local government representative.
3. Quality control and assurance were built in by designing two study phases with different contractors to allow for independent review.
4. Additional scientific consultation and review of technical reports were provided on specific topics by experts not directly involved in the study.
5. Special subcommittees were established by the panel to focus on project issues.
6. An open study process was established to invite public participation and review.

Every project task report went through several rounds of review and revision. Internal quality control measures were followed by the contractors in performing their research and preparing preliminary draft reports. They cross-checked data and methods used in the analyses, comparing relevant sources of information for consistency. If there were data gaps, panel members and contractors searched for other sources of information and applied accepted scientific techniques to make appropriate assumptions. Individual panel members reviewed all draft study documents and frequently performed additional calculations or analyses to check the contractors' work

Everyone involved in the study shared responsibility for quality assurance. The acceptability of the results ultimately depended on an all-inclusive search and consideration of all available data, under stringent quality assurance guidance. Continued scrutiny and critical review from individual panel members, project staff, reviewers and interested members of the public provided an added measure of quality.

Technical Issues Focused on During the Exposures Studies

• Assessing Health Risks from Nuclear Facilities
• Exposure Pathways
• Quality Assurance for the Health Studies on Rocky Flats
• Risk to Human Health and the Environment
• Uncertainty in Analyzing Health Risks
• Contaminants Released to Surface Water from Rocky Flats
• Estimating Plutonium Releases from Unmonitored Sources at Rocky Flats
• Source Term Estimates
• Research on Adverse Health Effects Related to Rocky Flats
• Movement of Contaminated Groundwater from Rocky Flats
• Verifying Historical Data
• Plutonium Health Risks

This series of Technical Topic papers explains the research design, methods and terminology used in the State of Colorado's Historical Public Exposures Studies related to the Rocky Flats Environmental Technology Site. The purpose of the project was to identify past contaminant releases from the plant and assess potential health risks to the community.