andre7septfig1.png
VoxEU Column Frontiers of economic research

What’s worth knowing in economics? A global survey among economists

Research shapes policy. But what we choose to study is subjective. This column uses a global survey of almost 10,000 academic economists to find their opinions on what economic research should look like. Many economists think that economic research should become more policy-relevant, multidisciplinary, and disruptive, and should pursue more diverse research topics.

Science and research matter. They shape how we think about ourselves, how we live together, and how we design policies. What researchers work on, which topics they choose, and how they resolve trade-offs between different research objectives is therefore of central societal importance. However, as famously argued by Max Weber (1919), the question about what is ‘interesting’ and ‘worth knowing’ cannot be answered scientifically. Instead, researchers must retreat to their intuition and personal value judgments.

We therefore conducted a global survey of almost 10,000 academic economists and asked them about their views about what is worth knowing in economics (Andre and Falk 2021), First, we asked which fundamental research objectives economists should pursue. The questions were posed in the form of trade-offs between research objectives such as policy relevance versus establishing causality, pure versus applied theory, quantity versus quality, and the level of specialisation. Second, we investigated which topics economists think economics should study. For instance, how much weight should topics such as environmental economics, industrial organisation, or finance receive?

To ensure that the sample of economists is representative of the discipline, the study identified and invited all researchers who have actively contributed to the international economics literature in recent years (more than 50,000 scholars), Crucially, this approach allows us to document and correct for the possibility of selection bias: Some groups of scholars might have been more likely to participate in the survey in which case their views and opinions would be over-represented in the sample. Fortunately, we can compare our sample with the full community of economists along dimensions such as gender, year of first publication (a proxy for academic age), continent of residence, publication success, research field, and position in the discipline-wide co-author network. To address the (mostly small) observed imbalances, we report results derived with post-stratification weights. Indeed, our data show that the weighted sample broadly represents the population of academic economists.

Here, we first ask whether economists think economics should embrace different fundamental research objectives than it does today. Respondents indicated whether they believe that the current state of research in economics is “about right” or whether they would prefer more or less of a specific research objective (Figure 1), Of course, these trade-offs are sometimes more and sometimes less severe, but in many cases economics can have more of one research goal only at the expense of the other.

The first block of trade-offs revolves around policy relevance and the public importance of research. Research projects that are relevant for society or policymakers are not always the ones who satisfy the researchers’ own intellectual curiosity. Likewise, basic research rarely has an immediate benefit for practitioners. We find that – even at the costs of less intrinsic interest or basic research – the majority of respondents (54% and 52%) believe that more policy relevance is needed. Likewise, 56% of economists would favour working on more important research questions even if this comes at the cost of reduced emphasis on establishing causality. And for theoretical work, 61% would prefer more applied, evidence-related theory instead of pure theory.

Second, a majority of respondents (60%) thinks that individual researchers should be less specialised and their research more multidisciplinary (79%),

Third, economists endorse a shift towards riskier (74%) and more disruptive (67%) research. They further say that economic research should put a larger focus on quality versus quantity (66%), The responses to the final research objective question on the goal of theory (prediction versus explanation) are more balanced.

Figure 1 Distribution of survey responses to the research objective questions

 

Notes: Distribution of survey responses to the ten research objective questions (weighted sample), The overarching question is: “In comparison with how research in economics is currently conducted, how should economists conduct research?” The labels at the top left and top right of each distribution summarise which two research objectives a question contrasts. The legend displays the available response categories.

Next, we ask which share of papers should be written on which topic. To express their view, scholars could allocate 100 points between 19 primary research topics (1st layer of the JEL subject descriptions), Each point corresponds to 1% of the total research output in economics. This allows us to compare the average preferred distribution of research topics with the actual distribution of topics found in the top 400 EconLit-indexed journals in the last ten years.

While we find a similar relative ranking of JEL topics, we observe marked quantitative discrepancies between reality and economists’ preferences (see Figure 2 & 3), For instance, the respondents assign on average 9.8% less to the JEL topic Finance, the field with the highest actual share of publications. Also, they put less weight on the second (Microeconomics) and third (Industrial Organisation) most prominent fields. In contrast, economists put more weight on minor JEL topics such as Public Economics or Economic History. Thus, overall, they favour a more diverse and pluralistic distribution of topics in economic research.

Figure 2 Comparison of JEL topic distributions in economics journals with survey responses

 

Notes: Blue bars: Shares of JEL topics in our publication sample (EconLit publication data, top 400 journals, January 2009 - December 2019), Red bars: Weighted average survey responses with 95% confidence intervals. JEL topics: A General Economics and Teaching, B Econ. Thought, Methodology, Heterodox, C Mathematical and Quantitative, D Microeconomics, E Macroeconomics and Monetary, F International, G Financial, H Public, I Health, Education, and Welfare, J Labour and Demographic, K Law and Economics, L Industrial Organisation, M Business, Marketing, Personnel, N Economic History, O Development, Innovation, P Economic Systems, Q Agricultural and Environmental, R Urban, Z Cultural, Sports, Tourism.

Figure 3 Differences between the average preferred and the actual JEL topic distribution

 

Notes: Differences between the red and blue bars from Figure 2 with 95% confidence intervals.

Economists’ views are heterogenous

Weber's insight that any answer to the question of what is worth knowing is subjective and value-driven is empirically reflected in the substantial heterogeneity of views among economists. Importantly, this dissent cannot simply be attributed to a generic inability of economic experts to agree on certain issues. Past research shows that economists agree on many policy issues (Dahl and Gordon 2013), Hence, consensus among economic experts is possible, yet the question of which research objectives economics should pursue remains fundamentally disputed.

Moreover, by far the strongest predictor of the importance a scholar assigns to a topic is the extent to which their own work is within that field. Thus, economists tend to value their own fields most. We believe that this is an important insight to keep in mind when evaluating other researchers' work, whether as seminar participants, referees, or editors. Our own views about ‘what is interesting’ are valuable and irreplaceable, but also subjective.

Dissatisfaction with the current state of economics

Despite the large disagreement, economists are unified in their dissatisfaction. Across the ten research objective questions, only 13% to 31% of respondents reply that the current practice in economics is “about right”. Moreover, a majority of economists agree on the direction of change. Economists want more policy-relevant and risky research with a broader scope and stronger multidisciplinary orientation. Moreover, they put less weight on the most popular JEL topics and would prefer a more diverse set of research topics. The findings reveal a systematic dissatisfaction with the current state of economics. As a field, we do not appreciate and work on what we collectively prefer.

(Side note: The observed dissatisfaction is not simply reflecting the views of ‘unsuccessful’ or less experienced scholars. We find similar opinions among the discipline’s most successful scholars, namely editors and referees at top journals and authors with an excellent publication record.)

Moreover, data on researchers’ job satisfaction and stress experience reveal: Economists who disagree with the current objectives and topics of their discipline tend to have lower job satisfaction and report more job-related stress. Empirically, disagreeing with the current state of economic research is associated with a psychological and mental burden. The data also show that female scholars disagree more with the objectives and topics of economic research. This hints at another possible explanation for the underrepresentation of minorities in economics (Lundberg 2020): Under-represented groups such as women have comparatively less influence on the fields’ research agendas so that their research preferences remain under-represented. In turn, disagreement with economics’ practices could adversely affect who is willing to pursue an academic career.

An inclusive, open-minded debate is needed

There is a long tradition in economics to monitor the status quo of research in their own discipline, the topics chosen, and methods employed (e.g. Card and DellaVigna 2013, Oswald and Stern 2019), We know that economics has become more multidisciplinary and increasingly data-oriented in recent years (Angrist et al. 2017, Backhouse and Cherrier 2017), These trends have brought the field closer to economists’ research preferences. Thus, signs of progress are visible, but sustained change is needed to reduce the mismatch noticeably. For example, multidisciplinarity is still the research objective for which we document the highest degree of dissatisfaction today, with almost 80% supporting a continued shift towards more multidisciplinary research.

Since the choice of research questions and research objectives is arguably among the most important choices that a researcher makes, we hope that our results will contribute to an inclusive and open-minded debate about ‘what's worth knowing’.

References

Andre, P and A Falk (2021), “What’s Worth Knowing? Economists’ Opinions about Economics”, Working Paper.

Angrist, J, P Azoulay, G Ellison, R Hill and S Lu (2017), “Economics gets out more often. Using extramural citations to assess economic scholarship,” VoxEU.org, 17 November. 

Backhouse, R E and B Cherrier (2017), “The Age of the Applied Economist: The Transformation of Economics since the 1970s”, History of Political Economy 49: 1–33.

Card, D & S DellaVigna (2013), “Nine facts about top journals in economics”, VoxEU.org,  21 January. 

Dahl, G & R Gordon (2013), “Views among economists: Are economists really so divided?” VoxEU.org,  16 May.

Lundberg, S (Ed.), (2020), Women in Economics, CEPR Press. 

Oswald, A and N Stern (2019), “Why are economists letting down the world on climate change?”, VoxEU.org, 17 September. 

Weber, M (1919), “Wissenschaft als Beruf” in Geistige Arbeit als Beruf, Duncker & Humblot.

11,443 Reads