AdobeStock_177744399.jpeg
VoxEU Column Education Financial Regulation and Banking

Financial education is effective and efficient

Financial education programmes are sometimes dismissed by critics claiming ‘mixed evidence’ on their effectiveness. This column analyses a full set of available randomised controlled trials evaluating the causal impact of financial education around the world to show that this claim is misleading. Financial education is effective in improving both knowledge and behaviour, even adjusting for publication selection bias. Moreover, available estimates indicate these improvements come at relatively low costs.

Research on financial literacy has been gaining momentum, and the number of articles that contain the term ‘financial literacy’ in their title or abstract has grown rapidly in the past ten years (see Figure 1). The popularity of financial literacy is further evident in policy and policymaking: the large majority of OECD member countries now have national strategies for financial education (OECD 2015). 

Figure 1 Number of articles on ‘financial literacy’ per year in the Web of Science, January 2022

Still, there is resistance to introducing financial education in schools or the workplace. Two major arguments against financial education are that it would be ineffective and inefficient. We explore the relevant evidence and show that these worries are misguided.

Evidence from randomised controlled trials

The evidence on financial education consists of more than 1,000 published studies. Here, we focus on a relatively small set that is considered the gold standard of rigorous evaluation – randomised controlled trials. We retrieved 76 randomised controlled trials, covering 33 countries and over 160,000 individuals (Kaiser et al. 2020). The reported interventions typically target several treatment effects within one study, generating 673 treatment effects in total: 50 randomised controlled trials report 215 effects regarding financial knowledge and 64 report 458 effects regarding various domains of financial behaviour.

Meta-analyses with various methods

We use a meta-analysis to determine the general effect of financial education, using the 670 treatment effects as observations. The treatment effect on each outcome is measured in standardised mean differences. These effects are then aggregated. 

There are two main approaches in the literature to measure effectiveness. Meta-analysis in fields such as medicine typically uses the common-effect approach, which assumes that there is a common true effect and that observed differences in treatment-effect estimates are due to random sampling error. 

In financial education, however, most meta-analyses rely on the random-effects approach, which assumes that the studies under consideration are quite heterogeneous, in particular regarding target groups, intervention objectives, or intensity (e.g. Miller et al. 2015). Thus, the general effect is assumed not to be a fixed parameter but, rather, a distribution of possible true effects. We use this latter approach, though we are careful to show that our findings are not driven by the choice of the estimation method (Kaiser et al. 2020, Appendix B).

Financial education improves knowledge and behaviour

We find that financial education works and works effectively. Effectiveness is assessed by two benchmarks: first, by considering the estimated general effect and, second, by comparing the magnitude of treatment effects to other fields of education interventions. The mean effect of financial education on knowledge is about 0.2 standard deviations, and the mean effect on behaviour is 0.10 standard deviations. The fitted distribution of these results is shown in Figure 2.

Figure 2 Raw distribution of treatment effect estimates (Kaiser et al. 2020)

Notes: Dotted lines show meta-analysis weighted averages at 0.1 and 0.204 standard deviations for the outcomes of financial behaviours and financial knowledge, respectively.

The confidence intervals associated with these estimates rule out zero-effects of financial education. Still, the full distribution of observed treatment-effect estimates suggests that not all interventions succeed (determinants of success include, for example, teacher quality or motivating parents; see Borowiecki 2022, List et al. 2021). This fact, however, does not justify the claim of ‘mixed evidence’ for financial education because such failures are common and observed in other fields of education too. Moreover, some failures are often expected in fields with little experience, and financial education is a young field, as also indicated by the increasing effect sizes among more recent randomised controlled trials relative to the first interventions.

Personal finance education is as effective as education in other domains

We compare the effectiveness of financial education to other fields to assess what can be expected from financial education. The order of magnitude of 0.20 standard deviations (i.e. the average effect on financial knowledge) is comparable to average effects of other educational interventions. Relying on the classification of Kraft (2019), our result of 0.203 can be considered as a medium to large effect.

Regarding the impact on financial behaviour, behaviour may be more difficult to affect than knowledge. In this sense, the relatively smaller size of the statistical effect of financial education on behaviour makes sense. An average effect of 0.1 standard deviations is comparable to other domains of educational interventions, such as health or energy conservation.

Result is robust to adjusting for publication bias

Even if the estimated effectiveness is robust to the estimation method, there is concern that published results might be distorted by the selective publication of studies. Reasons for such a publication bias are that academics may be more likely to publish papers with statistically significant results than with null results, and that funding institutions may prefer positive findings. There is indeed strong evidence for publication bias and selective reporting of results in economics (e.g. Brodeur et al. 2020). Thus, it seems important to account for potential biases before making a judgement on the effectiveness of financial education.

We apply the approach by Andrews and Kasy (2019) to control for a possible publication bias. Results show that this bias is present and considerable, with conditional publication probabilities of insignificant results being between 25% to 40% compared to results that pass tests for conventional levels of statistical significance. Consequently, the reported effectiveness of financial education may be inflated due to publication bias, but the adjusted effect sizes remain sizeable (about 0.15 standard deviations on knowledge and about 0.06 standard deviations on behaviour) and the associated confidence intervals rule out zero-effects.

Financial education is a low-cost intervention

While financial education programmes are effective on average, little is known about their costs and cost-effectiveness. In our sample of randomised controlled trials, 20 papers also report costs. 

The mean cost per outcome and participant is about $60, which is a ‘low cost’ educational intervention according to Kraft (2019). Thus, the medium-sized treatment effects appear to be at low costs, resulting in a generally favourable cost-effectiveness ratio.

Conclusion

The debate about financial education programmes is sometimes hindered by the argument that the evidence about their effectiveness is ‘mixed’. This stems from focusing on randomised controlled trials which generally appear to report smaller estimates of statistical effect than impact-evaluation designs with lower degrees of internal validity (Fernandes et al. 2014, Kaiser and Menkhoff 2017). 

However, when we analyse recently available randomised controlled trials that include a large number of studies across 33 countries, we find a sizeable general effect of financial education on knowledge and behaviour at a relatively low cost. This result holds true for various empirical models and adjusting for publication bias. These results provide a solid basis to extend research into better understanding which types of programmes are most impactful, cost-effective, and scalable and for whom.

References

Andrews, I, and M Kasy (2019), “Identification of and correction for publication bias”, American Economic Review 109(8): 2766–94.

Borowiecki, K J (2022), “How teachers influence creativity: Evidence from music composition since 1450”, VoxEU.org, 29 January.

Brodeur, A, N Cook and A Heyes (2020), “Methods matter: p-hacking and publication bias in causal analysis in economics”, American Economic Review 100(11): 3634–60.

Fernandes, D, J G Lynch Jr and R G Netemeyer (2014), “Financial literacy, financial education, and downstream financial behaviors”, Management Science 60(8): 1861–83.

Kaiser, T, A Lusardi, L Menkhoff and C Urban (2020), “Financial education affects financial knowledge and downstream behaviors”, CEPR Discussion Paper 14741 and Journal of Financial Economics, forthcoming.

Kaiser, T, and L Menkhoff (2017), “Does financial education impact financial behavior, and if so, when?”, World Bank Economic Review 31(3): 611–30.

Kraft, M A (2019), “Interpreting effect sizes of education interventions”, Educational Researcher, forthcoming.

List, J, J Pernaudet and D Suskind (2021), “Addressing the roots of educational inequities by shifting parental beliefs”, VoxEU.org, 12 December.

Miller, M, J Reichelstein, C Salas and B Zia (2015), “Can you help someone become financially capable? A meta-analysis of the literature”, World Bank Research Observer 30(2): 220–46.

OECD (2015), National strategies for financial education, OECD/INFE Policy Handbook, Paris: OECD.

1,050 Reads