Image: If you don't count your fruit and veg intake you don't know if you're hitting your 5-a-day
I cut my professional teeth in the world of monitoring, evaluation and learning (fondly referred to as MEL these days), but readily admit that there is something completely terrifying about the language of MEL, even to native English speakers. It sounds so technical, academic and obtuse. In this blog I hope to break it all down a bit and highlight how a MEL lens can help companies think about human rights in their supply chains.
First and foremost, MEL should be about asking “what works?”
Monitoring and evaluation (M&E) sprung up in projectized settings including international development where I spent my early career. In that space in the early 2000s, donors needed to understand how money was being spent and to what result. Reporting was about being accountable – ensuring money wasn’t going missing, and that good financial due diligence was being practiced so that the British public could sleep easy at night that their hard earned tax was being spent efficiently. Recipients of donor aid felt squeezed to do what was demanded of them by stakeholders far removed from their reality, without room for innovation or freedom to determine what aid should be spent on.
In time, everyone realised that M&E focused only on accountability to tax payers wasn’t exactly leading to any real insight into whether change was being achieved. We were missing investigation into effectiveness. The OECD Development Assistance Committee helpfully came up with some principles about how we understand change and later the M&E community added the word “Learning” to our process acronym, so MEL was born. The UK Department for International Development (DFID) and Harvard University were producing ground-breaking new methods in evidence-based development programming, and projects were even encouraged to fail! (so long as they learned and adapted quickly). But these were outliers in practice - not all institutional donors had the same approach, and many remained in an accountability mindset, some to this day.
The reason for going off on such a tangent is because I see the same challenges today for businesses when it comes to their supply chain management. Driven by a need to report to consumers and investors, many are stuck in a limited accountability trap without really using their data to understand what’s going on and then learn and implement solutions. And I’m not sure just yet that consumers or investors would be too forgiving of failure, if that were genuinely ever reported on. Maybe we can change that.
Good MEL starts with a strategy
Any MEL professional can talk to you about the pros and cons of using “Theory of Change” to determine your approach to impact. In fact, it’s our favourite topic of conversation. I am aware this language can make people switch off, however. I was once challenged on using the phrase theory of change this with the line "But why am I wasting time on a theory? It's irrelevant!"Back to basics we go:
In short – you can’t control your impact unless you know what you’re trying to achieve.
[Aside: Maybe we should just rename it as "strategy to achieve your goal" (If you can think of anything more catchy please let me know).]
The business community is actually getting better at this, even if they're not calling it theory of change. We’ve got ways to diagnose a company’s sphere of influence and have tools to understand highest risk and priority areas by applying a materiality lens. But there is a gap around creating purposeful social impact, and this is where a theory of change is so much more than a set of targets. It’s all about understanding what your organisation does, what you hope to achieve, then identifying the assumptions that will prevent you from achieving more, or the risks involved in each step along the way. This then allows for later investigation into how you’re faring so you can make any necessary adjustments. It’s the logic that enables MEL to take place. It’s completely insightful and empowering and creates more “ah-ha!” moments than any 80s pop band.
Without such a strategy, we're stuck in an endless cycle of risk identification, closure, repeat. And if the human rights argument doesn't convince you of that insanity, perhaps a reflection on the cost might.
Yes, monitoring sounds boring but it is massively underestimated as a data source
Monitoring is my least favourite aspect of MEL. It's detail-oriented, systematic, detail-oriented… did I say that already? All of which I am not. However, a good monitoring system will tell you a great deal about conditions in supply chains.
Most global businesses undertake social auditing, and while we can reel off the weaknesses of audit as a tool to understanding actual outcomes, it makes a pretty good starting point for a monitoring system.
Social audit data can help you understand:
- What issues are getting the highest and most frequent non compliances in your supply chain?
- How do these issues stack up against other risk data, or expectations on where the issues should be in your sourcing contexts?
And if you want to get really clever about it:
- Does audit frequency drive any long term improvement in supplier performance or labour standards?
- Is there any connection between your commercial practices with your social audit data on wages or working hours? (the interesting one!)
Regular collection of social audit alongside procurement data will yield a huge insight into what’s going on with your supply chain. Take a look at Cornell University Professor Sarosh Kuruvilla’s analysis on drivers of compliance if you want a deep dive insight into what audit data can help you to understand.
The elephant in the room is of course the weakness in social audit as a methodology to identify the difficult stuff, or then to instigate effective action. For issues like modern slavery, child labour and discrimination, we need different data sources and methods of collection. We need direct worker input into what’s really going on, we need strengthened freedom of association and worker representation. But we’ll get into that more another time. For now, just know that social audit data can be useful.
Evaluations needn’t be scary
In MEL-speak, an evaluation is a periodic assessment digging deeper into what is working, why and why not. You look at intended and unintended consequences. It’s the moment when monitoring data is handed over to an anonymous third party, focus groups and interviews are conducted, and everyone holds their breath hoping they don’t get told off.
But it isn’t an exam. Going back to the very first point – a periodic review of what is working is a good thing, because it can help identify those activities that are starting to create seeds of change, as well as those activities that are a big old waste of time and money.
For example, if you have a responsible sourcing policy that requires you to undertake social audits of all suppliers annually, but then (through evaluation) realise that audits themselves don’t improve anything and are in fact a massive money pit for you or your suppliers, then you might consider reducing their frequency, or focusing them only on key suppliers, or paying for them directly. Evaluations will test the assumptions behind your work and give you evidence (and backbone) to do things differently.
So how does a business go about conducting evaluations? Getting someone independent in is always the best way to ensure a level of objectivity, deploying both quantitative and qualitative methods to get under the skin of your data. You need to establish a timeframe that’s under review and the key questions you want answered, focused on outcomes rather than outputs (ie behaviours, not things you can count). You need to think carefully how you're going to consult rightsholders, in a meaningful way - and commit to feeding back to them over the long term. And – critically – you need to get your internal stakeholders on board with engaging with the process and acting on the results. Human Rights Impact Assessments (HRIAs) are increasingly being used as an evaluative method for global businesses which is really heartening to see, especially if they can spur the action required from the insight they provide.
Companies have the opportunity to build evaluation into business systems through annual reporting cycles. And while I’m not convinced this is what happens just yet, the potential is there (and I stand to be corrected, always!).
I’d be remiss not to briefly cover “Learning”. This is about what happens in between the structured moments of reporting and evaluation. Is the organisational culture conducive to critical thinking and exploring effectiveness? Are questions welcome, especially by those without decision-making authority? How do teams communicate and understand each others’ context, to build a coherent approach to responsible sourcing? Smart organisations ask these questions and support staff-led innovation that helps to "make things work". Conversely, a robust M&E system without any focus on the L won't get very far in the real-world.
In conclusion
I started this meander into the world of MEL by talking about projectized spaces and the international development sector. This is clearly not the same context as supply chain management by a global business, but there are some similarities to draw across. Budgets are finite in both circumstances and companies increasingly work to projectized timelines when it comes to ESG targets. The need for accountability to the stakeholders with the money (the donors, or investors) is rightly expanding towards the intended beneficiaries of that action (the project’s targets, or in a business’s case, their global workforce). Spurred on by legislation, companies are starting to talk about their intent to create impact. And when there is intent, we need ways to measure if we’re being effective. A MEL lens can only help to get you started.