When individuals make choices from lists, does the list ordering matter? There may be a ‘primacy effect’, where individuals are biased towards selecting items earlier in the list. Conversely, there may be a ‘recency effect’, i.e. a tendency to select items towards the end of the list. Studies have documented order effects in a number of settings, including artistic competitions (Wilson 1977, Ginsburgh & Van Ours 2003), elections (Miller & Krosnick 1998) and mutual pension fund choice (Karlsson, Massa & Simonov 2006). These effects can arise for example due to cognitive fatigue, short-term memory congestion, ‘satisficing’ behaviour (when search is costly), or any combination of the above.
With the advent of the internet and the digitisation of journals and working papers, consumers now typically select which papers to read from electronic lists rather than browsing among journals in libraries. Novarese and Wilson (2013) find that papers randomly listed first in lists of new papers from Research Papers in Economics (RePEc) receive more downloads, and that the effect is stronger when the lists are longer. Others have found evidence of order effects in email announcements of new astrophysics papers in ArXiv (see e.g. Haque and Ginsparg 2009). If presentation order in lists can lead to such bias in choice-making, those designing choice-mechanisms would want to account for these effects, particularly if the bias leads to downstream effects.
Email announcements
In a recent working paper (Feenberg et al. 2015), we investigate to what extent order effects matter in the reading and citing of economics research papers in the National Bureau of Economic Research’s Working Paper series. The National Bureau of Economic Research Working Paper series remains one of the most visible forms of distribution of cutting edge economics research in the US (average paper downloads are 195 compared to 5 downloads in the Novarese and Wilson (2013) RePEc sample.)
We measure the response by consumers to the ordering of working papers in the National Bureau of Economic Research’s ‘New This Week’ email announcement. Each Monday the National Bureau of Economic Research send the New This Week email that lists all of the working papers that have been issued in the past week. This email goes to more than 23,000 subscribers, both inside and outside academia.
National Bureau of Economic Research working papers are listed in the email announcement in ascending order of working paper number, which is allocated sequentially during the working paper production process at the National Bureau of Economic Research administrative offices. Working papers are assigned solely based on the order in which they were received and processed through the various filters that are required of authors (e.g. disclosure of conflicts). There is no evidence to suggest that authors are timing submission strategically to improve their list placement. Eventual list placement in the New This Week email announcement hence appears to be effectively random.
Order effects in new working papers announcements
We build a database of National Bureau of Economic Research working papers announced in 2013 and 2014 where we match each week’s New This Week ranking to abstract views and downloads. We find that papers listed first receive about 30% more abstract views and downloads. There are further declines as papers slide down the list. However, the very last position is associated with a boost in views and downloads.
The wide readership of the National Bureau of Economic Research New This Week announcement also enables us to ask not only if list ordering matters for readership (downloads), but also whether that extra readership converts into citations. We use citations from Google Scholar for National Bureau of Economic Research working papers announced in 2012 and 2013 (to allow citations to accrue). We find that papers listed first in the National Bureau of Economic Research New This Week announcements receive about 25% more cites than other papers.
Possible explanations
What is driving the results? We test several hypotheses regarding the mechanisms at work. First, we investigate whether the results are due to lack of attention among non-experts. We are able to distinguish between views and downloads for ‘.edu’ subscribers, a group presumably more expert than other subscribers. We find that the effects are present for both for experts and non-experts.
Second, we consider whether the effect is stronger when a list is longer than the average list. We find that the effect of being ranked first is stronger with longer lists for downloads, but not for cites. Finally, we consider whether the attractiveness of the first paper on the list (in terms of having a highly cited author) impacts search behaviour, and find some evidence supporting this mechanism.
Conclusions
In digital media, many academic papers (not to mention other media) are competing for the scant attention of readers. Our findings indicate that in this context, even details like presentation order can matter. They also confirm, predictably, that digital distribution matters for citations. More generally they suggest that far from being only driven by research quality and topic, visibility and citations are powerfully shaped by dissemination activities.
More generally, our findings confirm that presentation order can be a powerful determinant of choice in a list-based environment – and that this can have strong downstream effects, such as through paper citations in our sample. Accounting for the bias that can arise from presentation order is clearly important when designing choice mechanisms in list-based environments, and simple solutions include randomising the order of presentation, or by building this bias into list-ordering algorithms to achieve policymaker objectives.
References
Feenberg, D, I Ganguli, P Gaule, and J Gruber (2015) “It's Good to be First: Order Bias in Reading and Citing National Bureau of Economic Research Working Papers”, National Bureau of Economic Research Working Paper No. 21141
Ginsburgh, V A, and J C Van Ours (2003), “Expert opinion and compensation: Evidence from a musical competition”, American Economic Review: 289-296.
Haque, A U, and P Ginsparg (2009), “Positional effects on citation and readership in arXiv”, Journal of the American Society for Information Science and Technology 60(11): 2203-2218.
Karlsson, A, M Massa, and A Simonov (2006), “Portfolio Choice and Menu Exposure”, working paper.
Miller, J M, and Krosnick, J A (1998), “The impact of candidate name order on election outcomes”, Public Opinion Quarterly: 291-330.
Novarese, M, and C M Wilson (2013), “Being in the Right Place: A Natural Field Experiment on List Position and Consumer Choice”, MPRA Paper No. 48074.
Wilson, Vietta E (1977), “Objectivity and effect of order of appearance in judging of synchronized swimming meets”, Perceptual and Motor Skills 44: 295–298.
