Abstract
Rationale. Development partnerships are essential for achieving the 2030 Agenda for Sustainable Development owing to the necessary level of investment (USD 4.3 trillion a year) and the intrinsic limitations of individual organizations. However, one in five strategic partnerships are unsuccessful, incurring higher costs for a lesser impact on beneficiaries. We estimate that unsuccessful partnerships cost a multilateral development bank USD 10.9 billion in 15 years and that forming a partnership requires 150 clerical and management work days over 4 to 8 months per organization. Therefore, identifying the most promising partners or partnerships is a pressing concern.
Focus. To help form partnerships more selectively, we narrow down to multilateral development organizations and assume that partnership performance is proportionate to the ratings of co-financed projects at the national or regional level.
Toolkit. Our proposed engine processes project ratings, a corpus of 1,135 project evaluation reports of four multilateral development banks, international financial institutions, and UN agencies, and a list of 753 factors of influence into prioritized factors for decision-making. The engine integrates text mining, semantic and sentiment analysis–including the Stanza sentiment analysis model re-trained on project evaluations–, a novel double-weighing system for factors, supervised machine learning with four alternative explainable artificial intelligence (XAI) and opaque models, and SHAP values to rank factors by criticality, and bootstrap sampling to compute confidence intervals.
Results. We ranked factors for partnership success and failure by criticality. Explainable Artificial Intelligence (XAI) models achieved a Matthews Correlation Coefficient (MCC) of 0.749 in rating prediction, and opaque algorithms achieved an MCC of 0.780 in a matter of hours. These results are comparable to current success rates of 81% for project co-financing partnerships and 75% for knowledge partnerships. Higher predictive performances are achieved with a mix of XAI and opaque algorithms.
Utilization. Our prototype engine and top prioritized factors can be used for strategic guidance (utilizing the prioritized factors to guide multilevel stakeholder discussions on future partnerships), operational focus (enhancing and shortening appraisal and due diligence processes by focusing on critical factors), project co-financing studies (comparing the critical factors influencing success or failure in co-financed versus non-co-financed projects to identify key determinants of co-financing outcomes), and–with additional development–predictive studies on partnerships (predicting partnership performance ratings based on ex-ante documents).
Outlook. Future work includes refining the factor formulations and the contextualization feature, deploying models with incremental learning, and introducing causality analysis.
Biography
Nicola Drago is a private sector development consultant, who has collaborated with multiple UN agencies, International Financial Institutions, Bilateral Cooperation Agencies, National Development Agencies, and Academia (the University of Padova, Italy, and the Technical University of Denmark) for 20 years in 25 countries, preceded by a five-year experience in industrial corporations.
His current interest is to provide actionable solutions for multilateral organizations to step up to more impactful policymaking, harnessing the wealth of their data repositories and elements of Artificial Intelligence technology.
Holding an M.S. in mechanical engineering (1994), since 2021, he is a PhD student at the University of Padova, Italy, researching the classification and the automation of predictive factors for interagency partnership success. He is a visiting researcher at UNU EGOV.
