Lawmakers and the public have pushed for increased accountability in health and social programs. As a result, evaluators and evaluation skills have become very important to both large and small organizations providing health and social service programs. However, much of the public health workforce is not well equipped to handle these increased demands for outcomes data and research. Many federal agencies (e.g., Health and Human Services, Department Labor, Department of Justice) and private foundations are beginning to fund only organizations that deploy evidence-based interventions demonstrating impact on behaviors (e.g., individual, organizational, community). Our firm, through its national evaluation work has witnessed, first hand, the challenges faced by the staff and leadership of many organizations engaged in planning and responding to health issues, including (1) lack of expertise in evaluation and the ability to capture data related to program outcomes; (2) adverse experiences with external evaluators and researchers; (3) lack of resources, limited time and inadequate funds for independent evaluations; and, (4) fear of negative evaluation results. These inhibitors are also cited in literature and reports on evaluation capacity building. Evaluation can require considerable time and resources and many organizations cannot afford to create separate evaluation departments or hire external evaluators. EVALUATION CAPACITY BUILDING - VIRTUALLY (ECB-V) is a product concept that has emerged from our firmâs 23 years of work in program planning and evaluation with government and nongovernment organizations. We envision ECB-V as customized, multimedia Elearning course (ECourse) with a supplemental EAssistant to transfer learning from the individual to the program and organizational levels. The goal of this multi-level Elearning and engagement tool is to improve the capacity of the program staff of community-based organizations and state and local health departments to plan, implement and evaluate programs that advance population health in the U.S. The REESSI staff will focus on two key aims in carrying out this project â (1) Our team will develop the ECB-V prototype through iterative activities with experts and end-users and (2) evaluate the ECB-V prototype using quantitative and qualitative methods. The transdisciplinary team of population and public health experts, evaluators and software developers will engage in this proposed project to develop the prototype and conduct a feasibility study to determine the degree to which program staff and independent evaluators find the ECourse and EAssistant appealing, usable and valuable (feasibility) and how the ECourse and EAssistant affect program staff knowledge of program planning and evaluation (merit). The development of the user process for the product will be guided by the Evaluation Capacity Building Theory of Change Model to design and deliver stimulating and interactive content with electronic surveys in a digital environment. ECB-V will consist of a comprehensive ECourse with eight learning modules and an EAssistant for on-the-job tasks. The proposed topics (content) for the ECourse are based on (1) the Centers for Disease Control and Prevention Evaluation Guide and (2) the Office of Minority Health Evaluation Protocol. The Phase I activities will provide the foundation for the construction of the full and final components of the ECourse and EAssistant. The Phase II activities will consist of a multi-site randomized controlled evaluation that focuses on both individual and organization outcomes (e.g., attitudes, evaluation knowledge, public health core competency skills, reported changes in organizational systems and decision making).
Public Health Relevance Statement: Medical advances and new technologies have provided people in America with the potential for longer, healthier lives more than ever before. However, persistent and well-documented health disparities exist between different racial and ethnic populations and health equity remains elusive. Addressing disparities at the population level must involve both new and well-established efforts. A major goal of the U.S. Department of Health and Human Services (HHS) Health Disparities Action Plan is to increase the efficiency, transparency, and accountability of HHS programs. This includes internal strategies and externally funded activities. A primary strategy proposed in Healthy People 2020 and the action plan is to reduce disparities in population health by increasing the availability and effectiveness of community-based programs and policies. This involves developing the capacity of frontline program staff to plan, implement and evaluate interventions and programs. The Centers for Disease Control and Prevention (CDC), which funds a significant portion of the projects that focus on health equity/health disparities views evaluation as a major tool for program improvement and accountability.
Project Terms: Accountability; Address; Adverse event; Affect; Americas; Attitude; base; Behavior; Centers for Disease Control and Prevention (U.S.); Cognitive; Communities; Competence; Computer software; Custom; Data; Data Analyses; Data Collection; Decision Making; design; Development; digital; disparity reduction; E-learning; Economics; Educational Curriculum; effective intervention; Effectiveness; Environment; Evaluation; Evidence based intervention; Feasibility Studies; Focus Groups; Foundations; Fright; Funding; Goals; Government; Health; health disparity; health equity; Health Personnel; Health Services; Healthcare; Healthcare Systems; Healthy People 2020; Human; improved; Individual; inhibitor/antagonist; intervention program; Interview; Justice; Knowledge; Leadership; Learning; Learning Module; Link; Literature; Measures; Medical; Methods; minority health; Modeling; Monitor; Multimedia; new technology; Occupations; Outcome; Participant; Phase; phase 2 testing; Policies; Population; Population Distributions; population health; Population Programs; Positioning Attribute; prevention evaluation; Privatization; Process; Program Evaluation; programs; Progress Reports; Protocols documentation; prototype; Psychological Transfer; Public Health; Qualitative Methods; racial and ethnic; Randomized; recruit; Reporting; Research; Research Infrastructure; Research Personnel; Resources; Sampling; service programs; Services; Site; skills; social; Social Work; Surveys; System; Testing; Theory of Change; Time; tool; United States Dept. of Health and Human Services; usability; virtual; W
