Assessing the effectiveness of CMs approaches in achieving goals and objectives.

As part of the DRG Learning, Evaluation, and Research II Activity (DRG-LER II), NORC was contracted to do a mid-term performance evaluation of the United States Agency for International Development (USAID) Comunitatea Mea (CM) Activity. CM was a $20.5 million five-year (2018-2023) flagship local governance program funded by USAID/Moldova and implemented by IREX and its partners Tetra Tech ARD, National Democratic Institute (NDI), and Urban Institute. CM’s goal was to strengthen local government to become more effective, transparent, and accountable to citizens through four primary components.  

NORC partnered with IMAS, a Moldovan research firm to complete a mixed methods evaluation to assess the progress of CM in achieving its goals and provide recommendations to inform the strategic design of future USAID local governance programs in Moldova. The evaluation was guided by the following four main questions: 

(1)  How is Comunitatea Mea making progress toward the achievement of its goals and objectives?   

(2)  To what extent were the approaches employed by IREX/Comunitatea Mea effective in reaching the stated results?  

(3)  How effective is CM’s phased approach? How feasible is the total number of LPAs over the course of the project? 

(4)  Looking forward and building on the successes and failures of Comunitatea Mea, which areas should USAID support further to build stronger local governments and better local democracy where USAID has a competitive advantage? 

NORC's mixed-methods approach obtained information from varied stakeholders. 

NORC worked with local research partner IMAS, based in Chisinau, as well as two local experts who led data collection (quantitative survey, observational data, focus group discussions, and key informant interviews) for this evaluation.

NORC programmed, piloted and conducted the survey in Qualtrics, and trained IMAS supervisors and enumerators on the different modules with complex skip patters and loops based on 7 types of respondents. The Qualtrics program allowed the team to predetermine which modules to present based on information included in the respondent database or on real-time screening questions based on the respondents knowledge of the assistance provided by CM to the local public administration staff. The survey was made available in both Romanian and Russian and administered by IMAS enumerators over the phone and inputted directly into the computer after obtaining informed consent from respondents.

CM was appreciated by LPA staff, but sustainability of various approaches used was a challenge.

Overall CM helped LPAs improve the quality of and access to several services at least moderately through a variety of approaches. Several CM approaches were effective and could be further supported by USAID. CM contributed to the improvement in LPAs’ transparency, and some initial improvements in meaningful citizen participation, engagement and trust. It provided a flexible and tailored approach to specific needs and priorities of beneficiary LPAs for financial management.

There was high interest from LPAs to continue using CM approaches, but they faced challenges in ensuring their institutionalization and sustainability. There was some sharing of lessons learned among LPAs, but it was organic and not sufficiently stressed by CM. Stakeholders suggested potential new approaches to expand effectiveness, and improved donor coordination in future USAID activities would be fruitful.