Monitoring & Evaluation
McGovern-Dole Food for Education: Mauritania
I served as the education assessment specialist on the team at baseline (2020) and the quantitative analyst and assessment specialist at midline (2022). This required designing two parallel Early Grade Reading Assessments in both French and Arabic. This required understanding reading skills acquisition in both languages, conceptualizing how to properly assess reading sub-skills and designing culturally and linguistically appropriate tools. Analysis required great care, as the assessment spanned students whose primary language in some cases was the language of assessment, a related language, or an unrelated language. Evaluation was conducted during the COVID-19 pandemic, requiring creative solutions for training and monitoring that included a mixture of live and recorded video, as well as ongoing collaboration throughout data collection.
Photo "Manuscripts" by ammar01uk is licensed under CC BY 2.0
Every Adolescent Girl Empowered and Resilient
UKAID’s Girls’ Education Challenge has a special fund for out-of-school girls name Leave No Girl Behind. In Sierra Leone, International Rescue Committee leads a consortium to empower out-of-school marginalised young women through a mixture of literacy, numeracy, business skills, and mentoring support. The baseline evaluation required design of literacy, numeracy, business and life skills tools appropriate for girls with no schooling experience. Interviews with mentors, community members, caregivers, and beneficiaries The evaluation provided the client with statistically rigorous information on literacy, numeracy, and life skills for a myriad of subgroups of interest, as well as suggestions on how to further create more supportive community norms through a nuanced mixture of mass media and community discussion.
Saving Lives in Sierra Leone Baseline
UKAID funded an International Rescue Committee- led consortium to save women and children’s lives by improving the quality, availability and accessibility of (reproductive, maternal, new born and child health (RMNCH) services. The baseline evaluation required visiting over 60 hospitals and rural health clinics. As the quantitative analyst, I helped design the tools, conduct analysis, and work with health enumerator teams. The evaluation required a mixture of client and staff interviews, a thorough recording of data from past hospital records, and matching definitions of success with nationally-defined indicators. The evaluation required working closely with implementing NGOs to identify novel ways to monitor success while accounting for the challengies of missing records in hospitals and health clinics.
McGovern-Dole Food for Education: Tanzania Mara Region
Trembley Consulting led the entire scope of the evaluation’s anlaysis. Project Concern International supported the USDA-funded initiative to improve food security, reduce incidence of hunger, improve literacy and primary education, and contribute to more self-reliant and productive societies in the Bunda, Musoma, and Butiama districts in the Mara region. The mixed-methods evaluation included implementing an Early Grade Reading Assessment, and thorough interviews with staff, families, and community stakeholders. External challenges had fundamentally altered both what beneficiaries needed and what the implementer was able to provide since baseline. The evaluation provided insights and re-established a significantly clearer and improved methodology, while giving insights into how to adjust the project in ways that would allow it to better support its intended beneficiaries.
Synthesis of Findings and Lessons Learned from USAID-Funded Evaluations
USAID commissioned a study to assess the quality of USAID-funded evaluations in the education sector from 2013 to 2016 and, based on a subset of these evaluations that met minimum quality standards, to synthesize findings and lessons learned for topics related to the Agency’s 2011-2015 Education Strategy. I was on the team that prepared the report.
Girls' Education Challenge
2017 Evaluation Baseline Sierra Leone
The Girls' Education Challenge (GEC) aims to help the world’s poorest girls improve their lives through education and supporting better ways of getting girls in school and ensuring they receive a quality of education to transform their future. A consortium of organizations led by Plan International implemented the project in Sierra Leone, focusing on marginalized girls and children with disabilities. I lead the quantitative analysis on a team coordinated by International Solutions Group to conduct a mixed-methods baseline. I designed a joint sampling strategy to measure minimum detectable effects in transition and learning outcomes using difference-in-difference analysis. I designed the data collection methodology, trained a team of 25 enumerators, and supported them in the field. I conducted the entire quantitative analysis using school and classroom data from over 200 schools, 1,100 student learning assessments, and 1,000 household surveys to extremely rigorous funder specification. Our team provided a thorough descriptive analysis of the beneficiaries, highlighting potential course corrections for implementers, and possibilities and risks to consider for the upcoming midline and endline evaluations.
Evaluation: Guatemala Education Quality & Secondary Education Project
A team of two consultants and a World Bank staffer evaluated an $80 million IBRD loan to the Guatemalan government. This complex project was revised five times, spanned five changes in World Bank supervision and six presidential administrations. We needed to estimate enrollment rates to understand project effects, but there hadn't been a census in 13 years and government estimates varied by million people. I created a new population prediction model by merging national census, school enrollment, and birth and death record data to estimate populations by municipality -- something never done before there. Triangulating using student achievement data, focus groups, and field visits helped us understand how the project succeeded at improving school access, but at the cost of education quality. I conducted an economic analysis that demonstrated that some of the least popular interventions were actually the most cost-effective. The report received commendation by the World Bank's independent evaluation group.
Indicator Design: Tanzania Big Results Now in Education
In one of the World Bank's first results-based education financing, our team created a series of indicators that not only measured success, would determine how much funding the government received from the World Bank, and linked to how much funding schools would receive from the government.
We developed the indicators with the government through a series of innovative workshops. The indicators needed to be nuanced enough to work in a highly diverse country, but simple enough that teachers and school leaders understood them, balancing measuring growth vs. proficiency. Ultimately we created a series of six disbursement-linked indicators, which included portions relevant to school leaders, and separate portions relevant to the government. The project served as a blueprint for numerous other education projects.
Gender and Economic Development in Millennium Challenge Corporation Indicators
The Millennium Challenge Corporation uses a series of indicators to decide how it apportions funding to developing and middle-income countries. The team reviewed the gender sensitivity of current indicators and proposed possible alternatives for evaluating applicant countries. It required approaching the series of indicators from a unique perspective to understand how indicator definitions and global context influenced whether the indicators were supportive, indifferent, or detrimemental to gender equity, and identifying creative alternative measures.