When behavioural economics meets randomised control trials: Examples from Canadian public policy

Robert French, Philip Oreopoulos 05 December 2016

a

A

Over the past decade there has been a tremendous rise in the application of behavioural economics to public policy the world over. Combatting known cognitive and perceptual biases in our decision-making processes, the application of behavioural economics to public policy is occurring in diverse settings, from encouraging the timely and honest payment of taxes, to promoting energy conservation at home (BEworks 2014).1 Such public policy has typically emanated from centralised labs or innovation hubs specifically set up within government departments to rigorously trial and disseminate findings from the literature.

The most well-known example of these centralised hubs is the UK’s Behavioural Insights Team (BIT). Established in 2010 after Richard Thaler and Cass Sunstein’s book, Nudge, made it onto the UK Conservative Party’s summer reading list, the BIT has been advising the UK government on incorporating findings from behavioural economics (and the behavioural sciences more generally) into its policies (Thaler 2015). The BIT has been remarkably successful in enhancing the effectiveness of public policy in the UK, perhaps best evinced by the demand for its consulting services from governments internationally.2 Many other countries have also followed similar paths to the UK; in 2014, the Economic and Social Research Council estimated that at least 51 countries are home to some type of government-led initiative directly influenced by the behavioural sciences (Whitehead et al. 2014).

Although initially slow to warm to the trend, Canada is witnessing behavioural economics play an increasingly important role in its policy formulation (French and Oreopoulos, forthcoming). Agencies promoting the application of behavioural economics to public policy have now been established at both the federal and provincial level, covering a broad array of policy issues. A particularly notable feature of these agencies is the emphasis they place on rigorously testing their prospective behavioural interventions through randomised control trials. While not unique to the Canadian agencies, we believe this feature is worth highlighting. Many of the country’s behavioural interventions have occurred within an existing policy framework in which small tweaks to the status quo – influenced by behavioural economics – have had a marked impact on policy outcomes. Using randomised control trials to determine how best to tweak existing policy has been critical to the success of these interventions to date.3

One of the first explicit Canadian behavioural interventions to this effect involved promoting organ donation registration rates in the Province of Ontario. While much has been written about the power of default options affecting organ donation registration rates (e.g. Li et al. 2013, Johnson and Goldstein 2003), changing Ontario’s current opt-out default policy to one of presumed consent remains publicly unappealing.4,5 Instead, the province sought to promote organ donation registration through a behavioural intervention that operated within the province’s opt-out default policy framework. The intervention was designed with assistance from the province’s recently established Behavioural Insights Unit (BIU) – a government agency established to design and test behavioural interventions in the province.

Currently, when conducting a health card, driver’s licence, or photo card transaction at a provincial service-centre, Ontarians are asked by customer service representatives whether they would like to register as a donor and are provided with a registration form. The BIU’s intervention sought to raise registration rates by increasing the salience and simplicity of the registration process through altering the content of the registration form as well as the time at which potential donors received the form during their visit to a provincial service-centre. The intervention also included variations in nudge statements present on the registrations forms.6 This simple trial was considered a remarkable success by the Ontario government, as organ donation registration rates increased by up to 143% during the trial (Treasury Board Secretariat 2016).

Importantly, the BIU’s intervention tried to mimic a randomised control trial by comparing the effects of the different treatment periods to both a pre- and post-intervention control period. This approach to testing the effectiveness of the intervention was pursued not only to assess the impact of the intervention against the status-quo, but also to determine which treatment – each grounded in findings from behavioural economics – was most effective in increasing registration rates. That there were distinguishable differences between the registration rates during the different treatment periods perhaps validates this intention, and may have important consequences given that the most successful treatment is currently official practice in Ontario.

Figure 1 The intervention associated with the greatest increase in organ donation rates asked potential donors to consider themselves or a loved one needing an organ donation, highlighting a sense of personal relevance

The BIU’s attempt to mimic a randomised control trial while testing a behaviourally based intervention within an existing policy framework is emblematic of many other recent initiatives in Canadian public policy. One particularly interesting example involves Canada’s Revenue Agency (CRA) testing the impact of moving the signature box on personal income tax returns – where filers agree in writing that information provided is complete and accurate – to the beginning of the form. This intervention – based upon previous research suggesting that people are more truthful after they have been prompted to think about honesty (e.g. Shu et al. 2012) – was conducted in two Canadian cities wherein paper tax returns containing the altered signature block were distributed among some of its citizens. The CRA is currently comparing the amount of income reported (especially in areas more susceptible to dishonest disclosure) between the treatment and control groups, and pending differences between the two groups may roll out this intervention more extensively in the future. This example is especially interesting since it is a quintessential example of how combining insights from behavioural economics and rigorously testing such insights can affect dramatic change through public policy. Even a marginal difference in reported income across the two groups could garner substantial savings for the government if this costless intervention were implemented nation-wide.

Other interventions trialed in Canada include testing the effectiveness of different weblinks incorporating behavioural insights in promoting the use of a government funded job-search programme (Parent & Audet 2016); trialing college/university application assistance programmes among high-school students (Oreopoulos and Ford 2016); and testing the effectiveness of motivational interviewing in assisting unemployed individuals into the workforce (Ford et al. 2014). Notably, almost all of these interventions are occurring and being tested within existing policy frameworks. In complementing traditional public policy, we believe this approach holds much promise.

Incorporating behavioural economics into public policy initiatives is no panacea to solving all of today's most important public policy concerns; but details in how policies are prescribed matter, and testing different approaches to program take-up and communication can reveal significant impacts from simplification and friendlier messaging.  Canada is finally making strides forward in this regard.  

Authors’ note: The views expressed herein are the views and opinions of the authors and do not reflect or represent the views of Charles River Associates or any of the organizations with which they are affiliated.

References

BEworks (2014), Analyzing and Nudging Energy Conservation and Demand Shifting Through Time of Use Compliance, prepared for the Ontario Energy Board.

Ford, R, J Dixon, S-W Hui and I Kwakye (2014), Motivational Interviewing Pilot Project: Project Final Report, Social Research and Demonstration Corporation.

French, R  and P Oreopoulos (forthcoming), “Applying Behavioural Economics to Public Policy in Canada”, Canadian Journal of Economics.

Johnson, E J and D Goldstein (2003), “Do Defaults Save Lives?”, Science 302(5649), 1338-1339.

Li, D., Z Hawley and K Schnier (2013), “Increasing organ donation via changes in the default choice or allocation rule”, Journal of Health Economics 32(6), 1117-1129.

Oreopoulos, P and R Ford (2016), “Keeping College Applications Open: A Field Experiment to Help All High School Seniors Through the College Application Process” NBER Working Paper No. 22320.

Parent, A and M Audet (2016), “Job Match Nudge Trials: Job Postings”, Behavioural Economics and Service Innovation Research Unit, Strategic Policy Branch in partnership with Employment and Social Development Canada.

Shu, L L, N Mazar, F Gino, D Ariely and M H Bazerman (2012), “Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end”, Proceedings of the National Academy of Sciences 109(38), 15197-15200.

Thaler, R (2015), Misbehaving: The Making of Behavioral Economics, New York: W. W. Norton & Company Ltd.

Treasury Board Secretariat (2016), "Behavioural Insights Pilot Project - Organ Donor Registration".  

Whitehead, M, R Jones, R Howell, R Lilley and J Pykett (2014), Nudging all Over the World: Assessing the Global Impact of the Behavioural Science on Public Policy, Economic & Social Research Council.

Endnotes

[1] See also http://www.behaviouralinsights.co.uk/category/tax/.

[2] The UK’s BIT has international offices in New York and Sydney, and has collaborated with and consulted for various federal and provincial/state governments; see, for example, http://www.behaviouralinsights.co.uk/who-we-work-with/ and http://www.behaviouralinsights.co.uk/behavioural-insights-team-australia/.

[3] One important feature of these RCTs is that they typically operate within the exact context of the policy which they seek to refine, reducing concerns surrounding external validity.  

[4] Opt-out default policies require individuals make an active decision to enroll as an organ donor, whereas policies of presumed consent assume consent is granted and require individuals to actively decide against being an organ donor.  

[5] Within the province of Ontario, despite their apparent willingness to register, citizens report viewing organ donation as a personal choice and favour the current opt-in default (see https://www.giftoflife.on.ca/img/Opt-In-vs-Opt-Out-Infographic-Dec2015.png).

[6] Research in consumer behaviour and marketing suggest that small changes in design and messaging can have large impacts on individuals’ decision making processes. Treasury Board Secretariat (2016) notes the success of similar behaviourally based interventions that tested changes in the messaging content of letters/text messages in the UK to reduce fraud and error among its citizens.

a

A

Topics:  Frontiers of economic research Health economics Welfare state and social Europe

Tags:  Behavioural economics, randomised control trials, public policy

Associate , Competition and Antitrust Practice, Charles River Associates

Professor of Economics and Public Policy, University of Toronto

Events