Microfinance Impact Assessments: The Perils of Using New Members as a Control Group

Are the problems of dropout biases inherent in cross-sectional impact evaluation solvable?

The paper states that dropout biases inherent in a cross-sectional impact evaluation are problematic but solvable. However, the selection and institutional dynamics problems are more difficult. Using appropriate examples, the paper examines:

  • Importance of cross sectional studies in microfinance impact evaluation;
  • Informs practitioners about the perils of using such a strategy;
  • Problems created by dropout, the selection process into an MFI, and the dynamic nature of credit policy;
  • Suggests solutions to some of the larger problems with the approach, which makes many assumptions that are untested, and others that are tested and false.

It argues that:

  • If the studies are conducted, then MFI operators can learn more about whether MFIs can alleviate poverty, who to help the most, and how best to help them;
  • Impact assessment studies should have proper control groups, which follow all members, including dropouts.

The paper concludes that:

  • Solid understanding of the selection process, economic environment and institutional dynamics is important in deciding whether or not to employ mid-level cross-sectional approach;
  • Alternative to mid- level impact assessments should be a two-prong approach, with many "targeting" evaluations, and a few methodologically rigorous longitudinal evaluations;
  • "Targeting" evaluations should be small, frequent tools, which monitor client targeting (but do not claim to measure impact), combined with institutional analysis, which examines, from a management perspective, the efficiency and flexibility with which a program delivers its services.

About this Publication

By Karlan, D.