VoxEU Column Frontiers of economic research

The benefits of forced experimentation: Striking evidence from the London underground network

Suboptimal behaviour has been discussed but has not been studied extensively empirically. This column argues that the February 2014 London Underground strike enabled a sizeable fraction of commuters to find better routes. Due to the strike, many commuters were forced to experiment and around 5% stuck with these new routes after the strike was over. The strike produced an overall net benefit. Commuters seem to ‘satisfice’ and under-experiment during normal times.

Do people make first-best decisions? And to what extent do they get stuck with suboptimal habits (and at what cost)? Suboptimal behaviour has long been discussed (see, among many others, Simon 1955), yet it hasn’t been studied empirically using a large consumer dataset.

Utilising public transport travel card data

In Larcom et al. (2015), we aim to fill this gap. In the paper, we analyse a unique dataset that contains all individual travel movements on the London public transport system from 19 January to 15 February 2014. Our data include (recoded) travel card IDs, thanks to which we can track individual behaviour. From 4 to 6 February 2014, London Underground (or ‘Tube’) workers went on strike as a result of which some (but not all) Tube stations were closed down – forcing many commuters to experiment. In our paper, we use this event to study how repeat behaviour of commuters changes after a disruption-induced episode of experimentation.1

Descriptive statistics of our dataset can be found in Figure 1 (the strike days appear within vertical lines). The top-left panel shows the fraction of commuters who enter at their modal station (i.e. the station they used most frequently pre-strike), while the top-right panel shows the same for the modal exit station. It is clearly visible from the two panels that far fewer commuters were able to use their modal station during the strike, which implies that a substantial number of individuals were forced to explore alternative routes. The data also suggest that the strike brought about some lasting changes in behaviour, as the fraction of commuters that made use of their modal station seemingly drops after the strike (in the paper we substantiate this claim econometrically). The lower two panels provide information on journey times. The bottom-left panel shows that the duration of the average journey on London’s public transport system increased during the strike, while the bottom-right panel shows that dispersion went up as well.

As the network was only partially closed, some commuters continued to take their normal route to work – thereby enabling us to use a difference-in-differences approach (comparing the behaviour of ‘treated’ and ‘non-treated’ commuters). To ensure robustness, we define the treatment group in three different ways: those who deviated from their pre-strike modal journey during the strike; those whose pre-strike modal station (at entry, exit, or both) was closed during the strike; and those whose average travel times during strike days were sufficiently different from their average travel times during the pre-strike period. 

Figure 1. Descriptive statistics of the strike

Results

We find that those who were forced to explore alternative routes during the strike (‘the treated’) were significantly less likely to return to their pre-strike modal commute in the post-strike period, relative to the non-treated control group.

This result holds no matter how we define the treatment group and is robust to using different estimation strategies. By a revealed preference-type argument, this suggests that a fraction of commuters had failed to find their optimal journey before the strike. After all, post-strike all routes were available again (including the pre-strike modal one) so a failure to pick the latter option suggests that the commuter had found a better alternative during the disruption. In terms of magnitude, the fraction of post-strike switchers is about five percentage points higher among the treated.

As far as the mechanism is concerned, our results suggest that informational imperfections play a role in why treated commuters are more likely to switch post-strike. After digitising the London Tube map and comparing it to actual distances between stations, we find that the degree of distortion varies across London.  Exploiting this variation (which is unobserved to most commuters), our results suggest that those who live in (or travel to) more distorted areas were less likely to return to their pre-strike modal journey in the post-strike period – suggesting that those living in more distorted areas learned more from the strike. We also find that treated individuals were more likely to change their journey in the post-strike period if they were commuting on a relatively slow line before the strike (with train-speed being another characteristic where imperfect information plays a role, since it remains unobserved by commuters until a particular line is tried). 

Interpretation

  • Our results suggest that a significant fraction of commuters became aware of a better route to work thanks to the strike.

This is puzzling, since the alternative journey could have also been discovered beforehand through voluntary (as opposed to forced) experimentation.

This finding can be interpreted two ways. The first interpretation is that consumers were acting rationally and followed the optimal search rule, but due to search costs they (rationally) gave up on finding the best alternative before they had found their global maximum. In the terminology of Baumol and Quandt (1964), commuters under this hypothesis were not maximising, but optimising (which Baumol and Quandt define as acting rationally given the presence of search costs). The alternative interpretation is that agents were not adhering to the optimal search rule and experimented less than prescribed by the standard rational model. That is, they were neither maximising nor optimising. Under this second interpretation commuters were ‘satisficing’ (i.e. they continued to search until ‘some satisfactory outcome’ was found) in a way that is harder to rationalise (as hypothesised by Simon 1955). 

To see which interpretation fits our data best, we use Weitzman’s (1979) solution to the problem of optimal costly search (which is consistent with the Baumol-Quandt notion of optimisation). Using conservative numbers for the estimated time saving and its monetary equivalent, we calculate that if commuters were adhering to the optimal search strategy, the cost of trying the most attractive untried alternative would have to be greater than £380.2 Given this implausibly large number, it seems that commuters in our dataset were experimenting less than what is described by the standard rational model. Instead, agents seem to satisfice in a way that is not straightforward to rationalise.3

While a subset of commuters found better ways to get to work thanks to the strike, the vast majority (95%) only suffered from travel disruptions. However, somewhat surprisingly, when we compare the costs imposed on all treated commuters during the strike with the gains to the subset of beneficiaries, we find that the strike produced net benefits (the main reason being that the gains are much longer-lasting than the costs). Importantly, the net benefit from the strike came from the disruption itself, providing empirical support to Porter’s (1991) controversial hypothesis that imposing a constraint on an economic system can enhance efficiency over time (as constraints force agents to experiment, innovate, and re-optimise). In the specific context of the London Underground, this implies that commuters could be made better off if given an occasional external encouragement to experiment. Since partial closure of the network is a rather radical way to achieve this, it is worth investigating whether clever use of journey planner apps can ‘nudge’ travellers to experiment more.

Concluding remarks

More generally, our findings are relevant to government policies, to business practices, as well as to our personal lives. Given that a significant fraction of commuters on the London underground failed to find their optimal route until they were forced to experiment, perhaps we should not be too frustrated that we can’t always get what we want, or that others sometimes take decisions for us. If we behave anything like the satisficing commuters on the London Underground network and experiment too little, hitting such constraints may very well be to our long-run advantage. Encouraging ourselves to implement occasional routine-breaks could be beneficial as well.

Therefore, we ask, when was the last time that you did something for the first time?

References

Baumol, W J and R E Quandt (1964), "Rules of Thumb and Optimally Imperfect Decisions", The American Economic Review, 54 (2): 23-46.

Caplin, A, M Dean, and D Martin (2011), "Search and Satisficing", The American Economic Review, 101 (7): 2899-2922.

Larcom, S, F Rauch and T Willems (2015), "The Benefits of Forced Experimentation: Striking Evidence from the London Underground Network", University of Oxford Working Paper.

Porter, M E (1991), "America’s Green Strategy", Scientific American, 264 (4): 168.

Simon, H A (1955), "A Behavioral Model of Rational Choice", Quarterly Journal of Economics, 69 (1): 99-118.

Weitzman, M L (1979), "Optimal Search for the Best Alternative", Econometrica, 47 (3): 641-654.

Footnotes

1 We define commuters as those travellers who use the underground network on a daily basis during non-strike weekdays between 7am and 10am.

2 In computing this number, we use estimates by Stutzer and Frey (2008) on the monetary cost of commuting.

3 Recently, Caplin et al. (2011) provided evidence for satisficing behaviour in a laboratory experiment. However, they did not analyse whether their participants were optimising in the broader sense of Baumol-Quandt. Our results suggest that commuters do not.

3,465 Reads