Presenting Author:

Emma Reidy, M.P.H.

Principal Investigator:

George Greene, Ph.D.

Department:

Medical Social Sciences

Keywords:

HIV, HIV prevention, evaluation, PrEP, capacity building, technical assistance

Location:

Third Floor, Feinberg Pavilion, Northwestern Memorial Hospital

PH9 - Public Health & Social Sciences

Assessing Organizational Capacity to Successfully Evaluate HIV Programs in Chicago

Background: Men who have sex with men (MSM) and transgender persons are heavily impacted by HIV infection in the US. In response to these HIV health disparities, the CDC’s Project PrIDE is taking a program evaluation approach to assess two novel strategies to address the HIV epidemic. As a PrIDE grantee, the Chicago Department of Public Health is collaborating with Northwestern University’s Evaluation Center (EC) to evaluate local PrEP implementation and Data to Care (use of surveillance re-engage people living with HIV in medical care) demonstration projects. The EC is providing capacity-building and technical assistance (TA) as the first step in evaluating these projects locally. Capacity-building assistance is a core public health function and consists of “technical assistance, training, information sharing… that enables an organization… to operate in a more comprehensive, responsive, and effective manner”. The aims of this research are to: a) determine the evaluation capacity of Chicago-based Project PrIDE sites; b) assess differences across sites by type of organization and staff role; and c) assess TA needs among sites. Methods: The EC administered a one-time online survey to assess organizational and individual evaluation capacity and TA needs. The “Evaluation Capacity Survey” was administered to 33 individuals across 15 sites who participated in implementation and evaluation activities with the EC. A total of 33 individuals completed the survey, including administrators (n=7), healthcare providers (n=10), and field staff (n=16). The survey consists of closed- and open-ended questions about agency resources and capacity, individual knowledge/experience with evaluation, and TA needs. Results: All respondents indicated that program evaluation was a priority at their agency. In terms of evaluation skills, about 27% of participants reported that they could not create an evaluation plan before working with the EC, 24% did not understand what a logic model was, 60% could not develop a fidelity assessment plan, and 27% could not develop evaluation data collection tools. Preliminary analysis suggested evaluation TA needs varied across program role (e.g., conducting needs assessments). Discussion: Given the varying levels of resources, including staff, at each organization, it is understandable that the needs differ across organizations. It is interesting to note differences in needs among staff members at each organization. This is likely due to levels of education and training among different roles. Clinical providers, for example, are likely to have had research training and to be more familiar with evaluation methods, whereas field staff may not have encountered such concepts previously. Conclusions: Once the analysis is completed, we will be able to determine evaluation capacity among grantee agencies. Additionally, the EC will suggest next steps for additional training and TA, both through Project PrIDE grant and for future EC projects.