EVEN 2909: Introduction to Sustainability Engineering — Week 13
University of Colorado Boulder
“Without data, you’re just another person with an opinion.” — W. Edwards Deming
Monitoring and Evaluation (M&E) is the systematic process of collecting and analyzing data to determine whether an intervention is working, for whom, and why. It answers three fundamental questions:
For sustainability engineers: Good intentions are not enough. Your designs must be validated with data. M&E is how you prove (or disprove) that your technology actually creates the impact you intend.
A Theory of Change (ToC) is a roadmap that explains how and why you expect your intervention to produce a desired outcome. It makes your assumptions explicit and testable.
IF we install chlorine dosing systems at rural water points, AND communities are trained to maintain them, THEN water quality will improve at the point of collection, WHICH LEADS TO reduced waterborne disease, BECAUSE pathogens are eliminated before consumption.
Sources: Weiss, Evaluation 1997; USAID Theory of Change guidance; Taplin et al., 2013
The Logical Framework (logframe) translates a Theory of Change into a structured table that connects activities to impact through a chain of results.
| Level | Description | Indicator | Data Source |
|---|---|---|---|
| Impact | Reduced child mortality from diarrheal disease | Under-5 diarrhea mortality rate | National health statistics |
| Outcome | Households consume safe drinking water | % of households with <1 CFU/100mL E. coli | Household water quality testing |
| Output | Water treatment systems installed and functional | # of systems installed; % functional at 6 months | Field monitoring reports; sensor data |
| Activity | Train technicians; procure materials; install systems | # technicians trained; # systems procured | Training records; procurement logs |
| Input | Funding, staff, equipment, partnerships | Budget spent; staff FTEs | Financial reports |
The logframe gap: Most projects are good at tracking inputs and activities (we spent the money, we built the thing). Far fewer track outcomes and impact (did it actually improve lives?). That gap is where M&E adds the most value.
Sources: USAID Logframe guidance; World Bank Operations Manual
Indicators are the specific, measurable signals that tell you whether your intervention is on track. Good indicators follow the SMART criteria:
Sources: UNDP M&E Handbook; Bamberger et al., RealWorld Evaluation 2012
Mixed methods: The strongest M&E combines quantitative data (what and how much) with qualitative data (why and how). Neither alone tells the full story.
Sources: Patton, Qualitative Research & Evaluation Methods 2014; J-PAL Research Resources
The fundamental question of impact evaluation: What would have happened without the intervention? This is the counterfactual — and it’s the hardest thing in evaluation to establish.
Sources: Gertler et al., Impact Evaluation in Practice (World Bank) 2016; J-PAL Handbook
Science is built on reproducibility — but a growing body of evidence shows that many published findings cannot be replicated. This has profound implications for evidence-based sustainability.
Sources: Ioannidis, PLoS Medicine 2005; Open Science Collaboration, Science 2015
Measurement, Reporting, and Verification (MRV) is the backbone of carbon markets, climate finance, and environmental compliance. Digital tools are transforming MRV from periodic, manual auditing to continuous, automated monitoring.
Connection to carbon markets: The credibility crisis in carbon markets (recall Week 12) stems partly from weak MRV. If you can’t accurately measure and verify emissions reductions, credits have no integrity. Digital MRV is the technological solution to this trust deficit.
Sources: World Bank Digital MRV report; Gold Standard Digital MRV framework; Virridy
Geographic information systems (GIS) and remote sensing have democratized access to spatial data, enabling sustainability analysis at scales from local to global.
Sources: Google Earth Engine; Our World in Data; NASA SEDAC; Global Forest Watch
Visualization is how data becomes knowledge. Good charts inform; bad charts mislead. As engineers, your ability to communicate data visually is as important as your ability to collect it.
Sources: Tufte, The Visual Display of Quantitative Information 2001; Schwabish, Better Data Visualizations 2021
Collecting data about people creates power dynamics and responsibilities. Ethical M&E requires centering the rights and dignity of the people whose data you collect.
The extractive pattern: Too often, researchers collect data from communities, publish papers, advance their careers, and never return results to the people who provided them. Ethical M&E requires reciprocity.
Sources: CARE Principles for Indigenous Data Governance; Belmont Report; GDPR
A real-world example of how rigorous M&E transformed a water technology intervention from an assumption into peer-reviewed evidence.
IF sensors enable rapid detection of water system failures, AND repair teams respond quickly, THEN communities will have more consistent access to safe water, WHICH LEADS TO reduced waterborne disease.
Sources: Thomas et al., The Lancet Global Health; Virridy (virridy.com)
For your course project, develop a Theory of Change that maps how your proposed intervention leads to the sustainability impact you claim.
Assignment: Draft a 1-page Theory of Change for your course project. Include: problem statement, intervention description, causal chain with assumptions, and at least 3 SMART indicators at different levels of the logframe.