Evidence Based Social Impact Design and Monitoring

Lack of social impact is by design and no longer a surprise in Non-profit interventions 

Dr. Muteshi Hilda 

Most non-profits and governments struggle to demonstrate the long-term benefit of their efforts. Convincing donors and partners has become an extreme sport. Donors and partners want to see clearly documented results with impact linked to their funding. Some organizations could have delivered impact but failed to document it and link it to their action and the funds that they received. Because of this problem, non-profit organizations have long been criticized of having no impact. To mitigate intangible results, most non-profits would prefer to build infrastructure, construct or procure material goods to distribute, and photograph their work to demonstrate what they have achieved.

Most programs focused on tokenism and consumables are not sustainable, and this leads to over-dependency on donor funding, a vicious cycle of the problem without clear solutions. In order to achieve sustainable development, there is a need for systemic change. This change is progressive; the ecosystem, the enabling environment, and all interventions have to be monitored to make sure programs are on the right path towards sustainability. So, what are some of the reasons why most non-profits have not been able to demonstrate results?

1.     Fragmented program design.

The program design stage is critical; it’s at this stage that the problem is clearly defined, the root causes of the problem, the intervention logic, and the means of measuring results are established. Non-profit organizations (NPOs) have made program design aevent;-time event most of the programs are designed in a hurry, as a response to calls for proposals. They are designed based on the call for proposal indicators and not based on a well-thought-out process. Ideally, product development takes time; NPOs should have tested models that can be funded to scale and not just respond to calls for proposals.

A project/program is a social impact product that should be clearly thought through; this means it has to undergo a thorough research process of product development, product piloting, and testing before implementation. Because of the nature of operations of NPOs, there is always a call for proposals and a rush to respond within 2-3 weeks. With the rush in proposal development, different teams are assigned duties: one section drafts the problem, a different team does budgeting, and the monitoring team develops log frames. This team is likely to come up with an uncoordinated intervention, whose results cannot be measured. The moment the proposal is approved, the business development team is disconnected from the implementing team. The program team implements, and the monitoring and evaluation team comes in at baseline, output indicators, and end-of-project evaluation.

The fragmented teams have let the key results drop through the cracks. The program-implementing team does not understand the indicators, the monitoring team does not understand the process of implementation, while the finance team is busy tracking the rate at which the organization is ‘burning’ the budget without considering the quality of implementation. At the end of the project, the donor receives a clean finance report and a photo of the activity. It remains unclear whether the intended results were achieved. Sometimes the organization can have a very good program design, but they suffer poor monitoring and evaluation systems. So, what is the role of a proper M&E system? And what does it mean to have a poor program monitoring system?

2.     Poor Monitoring and Evaluation (M&E) systems

Monitoring and evaluation are key aspects in program design, implementation, and impact. Non-profit leaders should provide clarity in their business case that demonstrates a clear return on investment for donor funding. However, due to the fragmented design process, the monitoring team sets out indicators that are not clearly related to the designed activities. In other situations, the program starts without a baseline, hence there is no reference point to measure progress or impact. Some non-profits fear failure; they know it takes a lot of effort to bring the change but are sometimes constrained by donor demands and community expectations. For this reason, the monitoring team sets low-hanging fruit indicators at the output level so that throughout the project, they mention photos and participant lists as means of monitoring. This type of monitoring does not point out situations under which the program is likely to fail to achieve the desired impact and what needs to be done as corrective measures.

At the end of the project, the non-profit organization advertises for a consultant to evaluate the project based on the OECD evaluation criteria—relevance, coherence, effectiveness, efficiency, impact, and sustainability. What organizations don’t understand is that evaluation has different stages; it should not be done at the end of the project. If you did not have a baseline based on OECD evaluation criteria, you cannot evaluate a project based on the same. If you did not have process indicators to monitor the OECD model, you cannot evaluate the project. Any consultant producing such an evaluation report is just fixing things. This article is not about evaluation models; hence we will have another time to discuss models, but if organizations don’t monitor the process, they have no business monitoring the impact. Let’s look at the effect of poor process monitoring of programs.

3.     Fragmented ineffective, inefficient processes monitoring.

In order to achieve impact, there has to be a clear result and the process that will be followed in order to achieve the results. There has to be a commitment to monitor the process and corrective measures taken to ensure the desired impact. Throughout implementation, there has to be a commitment to learning that seeks to improve the program and service delivery. In this case there has to be a clear set of process indicators that organizations can track. Most of the time, the proposal does not accommodate process indicators. For example, a project for youth or women's economic empowerment monitors the number of participants trained, fails to monitor the delivery of the training contents, and the journey towards economic recovery is not monitored. Organizations have indicators like the number of people trained but have no training manual or have a training manual with content that is not impactful, or the mode of training does not guarantee impact.

The failure to track the process leads to training that has no impact in the community.  With clear process indicators, it's possible to tell when a project is going to fail to achieve its results. Monitoring and Evaluation (M&E) has to focus on program improvement, constantly checking on the process, evaluating the process effectiveness, and communicating the changes that need to be actioned in the process in order to guarantee impact at the end. With the fragmented departments, where business development, program design, implementation, and monitoring and evaluation remain disconnected, how then will organizations close the loop to demonstrate impact? Do organizations have the capacity to close the loop?

4.     Lack of capacity of staff to monitor and evaluate and document project results.

The only way to show evidence in non-profit work is through documentation, learn and sharing. Without proper planning, without proper design on what should be documented and at what point, the non-profit misses out on the key aspects that should have been documented. It becomes impossible to demonstrate impact where there is no data to show before and after intervention. The ability of the program teams to understand monitoring and evaluation determines whether the organization will have documented results or not. Most of the time, the teams do not have the skills required to run through design, implementation, monitoring, and evaluation. There is a growing demand for M&E capacity to meet the demand for evidence to support policy decisions, guide resource allocation, demonstrate development results, and enhance overall performance. The challenge is that most monitoring and evaluation training do not focus on the entire program development; they focus on the smaller part of indicators.

Most programs focused on tokenism and consumables are not sustainable, leading to over-reliance on donor funding, creating a vicious cycle with no visible solutions. Sustainable development requires systemic transformation. This is a progressive transformation; the ecosystem, the enabling environment, and all interventions must be monitored to ensure that programs are on track for sustainability. So, what are some of the reasons why so many non-profits have been struggling to demonstrate results?

In conclusion, support should focus on systems strengthening through processes that produce evidence. Structures must be established to implement, monitor, document, and report effectively. Organizations should commit to allocating resources by budgeting for adequate staffing, training, systems, and infrastructure. Capacity strengthening is crucial for public sector officials, development practitioners, non-profits, NGOs, donors, and humanitarian actors to enhance the design and execution of their projects. Without proper design, monitoring, evaluation, and documentation, achieving social impact and sustainable development remains an illusion. Additionally, without building staff capacity, sustainable development programs risk becoming endless funding sinks.