The Institutionalization of Monitoring and Evaluation Systems within International Organizations: a mixed-method study Open Access
Downloadable ContentDownload PDF
Abstract of Dissertation The Institutionalization of Monitoring and Evaluation Systems within International Organizations: a mixed-method studySince the late 1990s, Results-Based Monitoring and Evaluation (RBME) systems have seized the development discourse. They are institutionalized, and integrated as a legitimate managerial and governance function in most International Organizations. However, the extent to which RBME systems actually perform as intended , make a difference in organizations' performance, and their roles in shaping actors' behaviors within organizations, are empirical questions that have seldom been investigated. This research takes some steps towards addressing this topic. Drawing on an eclectic set of theoretical strands stemming from Public Administration theory, Evaluation theory and International Organizations theory, this study examines the role and performance of RBME systems in a complex international organization, such as the World Bank. The research design is scaffolded around three empirical layers along the principles of Realist Evaluation: mapping the organizational context in which the RBME is embedded; studying patterns of regularity in the association between the quality of project-level monitoring and evaluation and project outcome, and eliciting the underlying behavioral mechanisms that explain why such patterns of regularity take place, and why they can be contradictory.. The study starts with a thorough description of the World Bank's RBME system's organizational elements, and its evolution over time . I identify the main agent-based driven changes, and the configurations of factors that influenced these changes. Overall, the RBME institutionalization process exhibited key traits of what Institutionalist scholars call "path dependence." The RBME system's development responded to a dual logic of further legitimation and rationalization, all the while maintaining its initial espoused theory of conjointly promoting accountability and learning, despite some evidence of trade-offs. The second part of the study uses data from 1,300 World Bank projects evaluated between 2008 and 2014 to investigate the patterns of regularity in the association between the quality of monitoring and evaluation (M&E;) and project performance ratings as institutionally measured within the organization and its central evaluation office. The propensity score matching results indicate that the quality of M&E; is systematically positively associated with project outcome. Depending on whether the outcome is measured by the central evaluation office or the operational team, the study finds that projects with good quality M&E; score between 0.13 and 0.40 points higher—on a six-point outcome scale— than similar projects with poor quality M&E.; The study also concludes that the close association between M&E; quality and project performance reflects the institutionalization of RBME within the organization and the socialization of actors with the rating procedures. The third part of the inquiry uses a qualitative approach, based on interviews and a few focus groups with operational staff, managers and evaluation specialists to understand the behavioral factors that explain how the system actually works in practice. The study found that, like in other International Organizations, the project-level RBME system was set up to resolve gaps between goals and implementations. Yet, actors within large and complex IOs are facing ambivalent signals from the external stakeholders, that may also conflict with the internal culture of the organization; and organizational processes do not necessarily incentivize RBME. Consequently, the RBME system may elicit patterns of behaviors that can contribute to further decoupling goals and implementations, discourse and actions.