Three Evaluative Tools to Empower Local Communities in the Environmental Clean Up of Sediment Contaminated Sites: Social and Environmental Research Institute (SERI)

Project funding from the Environmental Protection Agency, EPA 831219-01-3.

October 1, 2003 – March 31, 2008.

Project Team

PI:
Seth Tuler (Social and Environmental Research Institute)
Tom Webler (Social and Environmental Research Institute)

Subgrantee:
Susan Santos (University of Medicine and Dentistry of New Jersey, School of Public Health, Heath Education and Behavioral Sciences Division).

Consultant:
Caron Chess (Department of Human Ecology, Rutgers University).

Project Overview

In spite of long commitment to community involvement and recent scholarship in the field, comprehensive efforts to evaluate "what works" and translate this information into improved practices is wanting. This project addresses this need at the level of theory and practice. The theoretical question this proposal addresses is: What are the comparative strengths and limitations of three practical and proven evaluative techniques to evaluate community relations processes and community satisfaction with progress toward policy objectives? We propose a comparative evaluation of three practical and promising evaluative tools. They are: the questionnaire, focus groups, and Q methodology. We propose to implement all three approaches in two real-world case applications (cases to be selected in collaboration with EPA and other stakeholders) and to perform three evaluations of each case. We then propose to have an body of local stakeholders assist in comparing the strengths and limitations of the three techniques. We also propose to give stakeholders and the scientific community practical guidance about how to evaluate community involvement programs in contaminated sediment sites using these three evaluative techniques. This project will improve community involvement and empower local communities to participate more effectively by providing feedback about the quality of the community involvement process and community satisfaction with the clean-up outcomes.

The two case study sites are:

Toms River, New Jersey, Ciba Geigy Site
Waukegan, Illinois, Waukegan Harbor

The expected Results of the project will be comparisons of the strengths and limitations of the three methods for getting feedback on (1) effectiveness of community involvement processes in contaminated site clean-up activities and (2) community preferences for outcomes related to clean-up. This study is not intended to be an evaluation of agency or responsible party community involvement activities. While we will be collecting information about how participants view the involvement process and clean-up decisions at the site, we will focus on the effectiveness of methods of obtaining feedback. We will not be judging the involvement process or the clean up remedies proposed.

At the end of the project we will complete

A Guidance document for how to use focus groups to gather feedback about how well the community involvement process is working and how satisfied are people with the clean-up decisions.
A Guidance document for how to use Q Method to gather feedback about how well the community involvement process is working and how satisfied are people with the clean-up decisions.
Information for local citizens and activists about participatory evaluation and the roles that citizens can play in ensuring a high-quality community involvement process and in understanding community opinion about clean-up decisions.
Recommendations for using focus groups, Q method, and mail surveys, including comparison of their strengths and shortcomings for different evaluation objectives.

Publications, Invited Presentations, and Reports

Guidance documents:

Guidance document describing how to use Q method for evaluation.

Guidance document describing how to use focus groups for evaluation.

An article entitled Selecting the Right Tool for Evaluations: Guidance for Community Involvement Practitioners was published by EPA in its EPA Public Involvement Network News issue from winter 2008. Download the article or read the entire newsletter.

Danielson, S., S.L. Santos, T. Webler, and S.P. Tuler. 2008. Building and Breaking a Bridge of Trust in a Superfund Site Remediation, International Journal of Global Environmental Issues 8(1-2):45Ñ60. (PDF Available)

Doing it Right: Measuring the Success of Public Participation in Watershed Management presented at a workshop on the 9th of November sponsored by the Statewide Public Advisory Council for Michigan's Areas of Concern Program and the EPAs Great Lakes National Program Office on Getting the Message Out on Great Lakes Restoration: Communicating with Local Communities on Restoring Michigan's Great Lakes Areas of Concern.

A series of presentations were given to staff from the Community Involvement Section (Office of Public Affairs) and Great Lakes National Program Office at EPA Region 5 in Chicago on September 19th:

overview of the project

primer on using focus groups for evaluation of community involvement

short primer on using Q method for evaluation of community involvement

comparison of the two methods

Presentation to EPA on results of first case study, May 6 2005 (PowerPoint Available)

			Three Evaluative Tools to Empower Local Communities in the Environmental Clean Up of Sediment Contaminated Sites 278 Main Street, Suite 404, Greenfield, MA 01301 phone: 413-773-9955, fax: 530-348-7325, e-mail:


All material © SERI unless otherwise noted.			Project funding from the Environmental Protection Agency, EPA 831219-01-3. October 1, 2003 – March 31, 2008. Project Team PI: Seth Tuler (Social and Environmental Research Institute) Tom Webler (Social and Environmental Research Institute) Subgrantee: Susan Santos (University of Medicine and Dentistry of New Jersey, School of Public Health, Heath Education and Behavioral Sciences Division). Consultant: Caron Chess (Department of Human Ecology, Rutgers University). Project Overview In spite of long commitment to community involvement and recent scholarship in the field, comprehensive efforts to evaluate "what works" and translate this information into improved practices is wanting. This project addresses this need at the level of theory and practice. The theoretical question this proposal addresses is: What are the comparative strengths and limitations of three practical and proven evaluative techniques to evaluate community relations processes and community satisfaction with progress toward policy objectives? We propose a comparative evaluation of three practical and promising evaluative tools. They are: the questionnaire, focus groups, and Q methodology. We propose to implement all three approaches in two real-world case applications (cases to be selected in collaboration with EPA and other stakeholders) and to perform three evaluations of each case. We then propose to have an body of local stakeholders assist in comparing the strengths and limitations of the three techniques. We also propose to give stakeholders and the scientific community practical guidance about how to evaluate community involvement programs in contaminated sediment sites using these three evaluative techniques. This project will improve community involvement and empower local communities to participate more effectively by providing feedback about the quality of the community involvement process and community satisfaction with the clean-up outcomes. The two case study sites are: Toms River, New Jersey, Ciba Geigy Site Waukegan, Illinois, Waukegan Harbor The expected Results of the project will be comparisons of the strengths and limitations of the three methods for getting feedback on (1) effectiveness of community involvement processes in contaminated site clean-up activities and (2) community preferences for outcomes related to clean-up. This study is not intended to be an evaluation of agency or responsible party community involvement activities. While we will be collecting information about how participants view the involvement process and clean-up decisions at the site, we will focus on the effectiveness of methods of obtaining feedback. We will not be judging the involvement process or the clean up remedies proposed. At the end of the project we will complete A Guidance document for how to use focus groups to gather feedback about how well the community involvement process is working and how satisfied are people with the clean-up decisions. A Guidance document for how to use Q Method to gather feedback about how well the community involvement process is working and how satisfied are people with the clean-up decisions. Information for local citizens and activists about participatory evaluation and the roles that citizens can play in ensuring a high-quality community involvement process and in understanding community opinion about clean-up decisions. Recommendations for using focus groups, Q method, and mail surveys, including comparison of their strengths and shortcomings for different evaluation objectives. Publications, Invited Presentations, and Reports Guidance documents: Guidance document describing how to use Q method for evaluation. Guidance document describing how to use focus groups for evaluation. An article entitled Selecting the Right Tool for Evaluations: Guidance for Community Involvement Practitioners was published by EPA in its EPA Public Involvement Network News issue from winter 2008. Download the article or read the entire newsletter. Danielson, S., S.L. Santos, T. Webler, and S.P. Tuler. 2008. Building and Breaking a Bridge of Trust in a Superfund Site Remediation, International Journal of Global Environmental Issues 8(1-2):45Ñ60. (PDF Available) Doing it Right: Measuring the Success of Public Participation in Watershed Management presented at a workshop on the 9th of November sponsored by the Statewide Public Advisory Council for Michigan's Areas of Concern Program and the EPAs Great Lakes National Program Office on Getting the Message Out on Great Lakes Restoration: Communicating with Local Communities on Restoring Michigan's Great Lakes Areas of Concern. A series of presentations were given to staff from the Community Involvement Section (Office of Public Affairs) and Great Lakes National Program Office at EPA Region 5 in Chicago on September 19th: overview of the project primer on using focus groups for evaluation of community involvement short primer on using Q method for evaluation of community involvement comparison of the two methods Presentation to EPA on results of first case study, May 6 2005 (PowerPoint Available)