Aid for trade: Can it be evaluated?

Olivier Cadot, Jaime de Melo 14 April 2013

a

A

The demand for accountability in aid for trade is on the rise…

At their annual conference in Hong Kong in 2005, WTO trade ministers called for expansion of aid for trade to help “developing countries, particularly LDCs, to build the supply side capacity and trade-related infrastructure that they need to implement and benefit from the WTO Agreements and more broadly to expand trade”. A WTO task force was set up in 2006 to implement this ‘positive agenda’ to enhance competitiveness. Multiple goals were adopted,1  but clear guidelines on how to conduct evaluations were largely absent even though pressure for greater accountability was mounting. So far, the quest to improve accountability has produced case studies and a digest of a large collection of projects and case stories – many voluntarily supplied and thus heavily selected – feeding into meta-analyses built around word counting (OECD 2011).

… but aggregate data shows no discernible aggregate effects in spite of rising aid-for-trade volumes

Start with aid-for-trade volumes, the yardstick keenly followed by negotiators, in particular from developing countries. By the commitment measure, the 2005 initiative has indeed been highly successful, reversing the long-term decline in the share of trade-related assistance in foreign aid. The trade-related share rises from 30% in 2005 to 35% in 2010, boosting annual commitments (from $25 billion in 2005 to over $45 billion in 2010). Tracking disbursements (Figure 1) shows that the 'big push' to the agenda came mostly from multilateral agencies rather than from bilateral aid from countries with no noticeable change in the allocation categories, as hard infrastructure is still taking the lion’s share (63%, of which 35% was for roads and 16% for rail), against 29% for behind-the-border policies (a hodgepodge of projects including sector-targeted ones), 6% for technical assistance on trade policy – arguably the most 'T'-related in aid for trade – and 2% for trade facilitation. Of course, these numbers reflect not just donor priorities but also intrinsic cost differences, as building a bridge is costlier than computerising a border post.

Figure 1. Aid for trade disbursements, 2002-2010 (share of total foreign aid)

Aid for trade’s potential to increase exports by reducing trade costs has been the centrepiece of this agenda. Figure 2 checks whether a simple correlation between lagged (to allow for delays) disbursements and export growth is visible to the naked eye. It splits the set of recipients within quintiles of the (baseline) export per capita distribution by the median into two cohorts, 'low recipients' and 'high recipients', based on average 2000-2005 receipts. Thus, Q1 is the worst-performing quintile in the baseline period, Q2 is the second-worst, and so on. Results are striking. Only in the top two quintiles can one see a positive export-growth differential between high- and low-recipients (Panel a). Aid for trade could, however, have an indirect effect on export performance by working primarily through improved logistics markets. Panel b checks for this by carrying out the same exercise for the time to export, with similarly disappointing results. Inconclusiveness also resulted from inspection of the write-ups of voluntarily supplied case studies (Cadot et al 2012).

Figure 2. Export growth and time to export vs lagged aid for trade, by quintile of the export per capita distribution

Source: Cadot et al. (2012).

… but the evidence does support the reduction of trade costs as a means to increase trade

In spite of the ambiguous prima-facie evidence that aid for trade has raised export growth, trade costs calculated from the ‘gravity equation’ have fallen over the last fifteen years for a large group of countries (Arvis et al. 2013). However, this decline has been less for low-income countries where aid for trade expenditures are concentrated which still suggests marginalisation for this group of countries (Carrère and de Melo 2009). 

Where should aid for trade be directed: Hard or soft?

If aggregate trade costs have been falling, it is difficult to disentangle their many determinants even though recent estimates (Arvis et al. 2013) suggest that endogenous costs (logistics performance and trade barriers) are as important as bilateral exogenous factors (geographical distance, language). So should aid for trade target ‘hard’ infrastructure (roads, ports, railways) as suggested by Limaõ and Venables (2000)? Or should it target ‘soft’ infrastructure (the provision of backbone services, regulation, competition) as suggested by Teravaninthorn and Raballand (2008) in their study of trucking corridors across Africa? Answering this allocation question requires going beyond cross-country aggregate estimates of trade costs.

Impact evaluation: No panacea, but a credible alternative

Impact evaluation identifies the impact of projects by comparing the performance evolution of treated entities with that of a control group. The key data requirement is a baseline survey administered on a sufficiently large sample including both beneficiaries and non-beneficiaries prior to the intervention as well as a follow-up survey.

Here is where difficulties start. First, by construction, treatment effects capture only effects that are internalised by the beneficiaries. But then, why shouldn’t they pay for them? Subsidised interventions (most aid for trade takes the form of grants or concessional loans) should be justified by some sort of market failure such as non-appropriability of the gains, as funds have an opportunity cost. But if gains are not appropriable, they won’t show up in a treatment-effects test. Thus, the absence of estimated treatment effects suffers from a basic ambiguity; it could be that the programme was ineffective, in which case it should be discontinued, but it could also be that its effects spread to the control group, in which case it should be continued (it could also be that the test does not have sufficient power to reject the null, a sample-size problem). In plain English, impact evaluation can be a key piece in the monitoring-evaluation nexus, but it should be interpreted cautiously.

Second, situations of 'clinical' policy interventions in trade are rather rare. Targeted programs such as technical assistance for export promotion could be amenable to randomised control trials or other forms of impact evaluation, but the more numerous non-targeted reforms like customs reforms, port improvements or other institutional improvements are less easily amenable to the usual methods (although sometimes it is still possible to go down from the intervention level, say a border post, to the firm or transaction level, as in Volpe and Graziano 2012).

Third, implementation faces two types of constraints, i.e. incentives and costs. As for incentives, project manager buy-in would be facilitated if impact evaluation could be fully decoupled from their evaluation, but no organisation could commit to that without facing a time-consistency problem. As to costs, bottom estimates for an evaluation are around $300,000 . For large-scale social or health projects, typically this will be only a few percentage points of programme cost. But trade-related projects are much smaller, so containing evaluation costs to 5% of project costs (requiring project cost above $6 million) will put the majority of aid-for-trade projects outside the range of feasibility. Cadot et al. (2012) estimate a median commitment size of $700’000 (aggregated over all donors) for trade policy and regulations. In conclusion, randomised control trials face an uphill road in trade-related assistance but quasi-experimental methods relying on existing data from customs and industrial surveys provides a second-best alternative.

The way forward: Using benchmarking to identify programme effects

For both hard and soft infrastructure, causal links from policy intervention to export performance are strongly suggested by theory but non-trivial and often elusive to estimate empirically. Cross-country evaluations will continue to be needed because they are the safest route in terms of 'external validity', in spite of their limitations in terms of 'internal validity' (ability to establish causality from intervention to effects). As to impact evaluation methods, given the typically small size of trade-related projects, In order to generalise the use of impact evaluation in trade-related interventions, what is needed is to make it practically feasible in terms of design (project and evaluation using quasi-experimental methods), incentives (impact evaluation results should be decoupled from individual performance evaluation), and resources (get government buy-in to release confidential data). Governments will be more willing to relinquish semi-confidential data to researchers if they understand the value of the results generated.

References

Arvis, J F, Y Duval, B Sheperd and C Utokham “Trade Costs in the Developing World: 1995-2012” (2013), VoxEU.org, 17 March.

Cadot O, A Fernandes, J Gourdon and A Mattoo (2011), "Impact Evaluation in Aid-for-Trade: Time for a Cultural Revolution?”, VoxEU.org, 21 January.

Cadot O, A Fernandes, J Gourdon, A Mattoo and J de Melo (2012) "Evaluation in AFT: From Case-study Counting to Measuring", paper presented at the FERDI-ITC-WB workshop, December.

Carrère, C and J de Melo (2009) “The Distance Puzzle Resides in Poor Countries”, VoxEU.org, 10 November.

Limao, N and A Venables (2000), “Infrastructure, Geographical Disadvantage, Transport Costs, and Trade”, World Bank Economic Review, 15(3) : 451–479.

OECD (2011) “Strengthening Accountability in Aid for Trade” OECD, Paris

Teravaninthorn, S and G Raballand (2008), Transport Prices and Costs in Africa: A Review of the Main International Corridors, Washington, DC: World Bank.

Volpe, C and A Graziano (2012), “Customs as Doorkeepers: what are their effects on international trade?”; mimeo, paper presented at the FERDI-ITC-WB workshop.


1 The WTO task force listed in order: increasing trade, diversifying exports, maximising linkages with the rest of the economy, increasing adjustment capacities, regional integration, and contributing to inclusive growth and poverty reduction.

a

A

Topics:  Development International trade

Tags:  Impact evaluation, Aid for Trade, aid effectiveness

Professor, International Economics; and Director, Institute of Applied Economics, University of Lausanne

Senior Fellow, FERDI; Professor, University of Geneva; CEPR Research Fellow

Events