In the summer of 2020, the Society for Economics Dynamics Meeting had been postponed due to Covid-19 and many junior researchers from around the world would miss out on the opportunity to present their work in the largest macro conference. When the Virtual Macro Seminars (VMACS) team had the idea to organise an online conference for junior researchers in macroeconomics to fill in the gap, we enthusiastically volunteered to be part of the programme committee.
A two-step double-blind review
Ideally, an online conference would allow participation from a diverse and varied set of presenters from around the world. Yet, the field of macroeconomics and, as a result, its associated conferences have a noted lack of female and minority representation. This lack of representation is self-perpetuating and harmful to the discipline. To combat this, our design of the conference and the review process had two primary objectives. First, we wanted to deliver a high-quality programme on a par with top macroeconomics conferences. Second, we sought equal and equitable access for and representation of a diverse set of researchers. To achieve this outcome, we experimented with a two-step process involving a double-blind review of paper submissions.
In the first step of our process, we anonymised all submissions and had two programme committee members (the assigners) assign papers to the other six (the reviewers) to review. This assignment was made on the basis of the field of expertise of the committee members and to avoid conflicts of interest. Each reviewer blindly ranked a first list of “accepted papers after blind review” that would amount to 32 papers in total (i.e. two-thirds of the final programme).
In the second step, each reviewer blindly ranked a second list of “papers for consideration after deliberation” to complete the 16 remaining papers (i.e. one-third of the programme. The assigners then accepted the first list and uncovered the identity of the authors of the first and second list. The programme committee then met to deliberate on the second list with identities uncovered. Based on the outcome of the first step, the programme committee decided to prioritise papers in the second list by female presenters and advanced graduate students. The second list did not feature any presenters from an underrepresented racial minority to include in the programme.
The conference resulted in an acceptance rate of 14%. In terms of gender composition, 19.5% of the submissions were from women while 16.67% of the accepted were from women. In addition, 37.5% of the accepted were advanced grad students or junior economists presenting their job market paper.
Lessons for conference quality and institutional diversity
The word cloud below uses the titles of the papers presented to summarise the breadth of topics covered in the conference. The conference featured a variety of topics in macroeconomics. Together with old macro friends – “monetary”, “tax”, “cycles”, “dynamics”, “aggregate” – we see an increasing presence of the macroeconomics of the household –“income”, “consumption”, “credit”, “health” – and “policy” is paired with “insurance” and “inequality”. In addition, we received much positive feedback from participants on the quality of the presentations.
The word cloud below shows the representation of different institutions among presenters, excluding students. The conference also featured a more diverse list of institutional affiliations than similar conferences.
Lessons for the representation of women and underrepresented minorities
In terms of gender composition, the result from our two-step process is encouraging in that the acceptance rate of female presenters (16.67%) is close to the female submission rate of (19.5%). However, this number is still far from achieving a programme with equal representation of men and women.
One possible area for improvement is the number of papers that were put forward for consideration after blind review. We set this number to one-third of accepted papers in our experiment. To achieve a programme with better representation of women and underrepresented minorities, conference organisers may have to adjust this policy parameter.
Lessons for active participation in online conferences
The composition of the conference programme is only the first element that defines participation in a conference. The interaction in sessions determines what presenters and attendees get out of the conference. Online conferences, in general, struggle to emulate the in-person experience, and ours was no different. We found the interaction during sessions to be somewhat limited. Encouraging attendees to keep their video on did seem to promote participation on the margin.
To promote interaction in a more conducive setting, we organised a meeting with virtual ‘breakout rooms’ after each presentation session. An organiser set up a breakout room for each presenter from the session, then assigned attendees to the breakout room of the speaker with whom they wanted to discuss further. Interactions in the breakout rooms were livelier and a truer reflection of the in-person conference experience, allowing authors to expand on explaining their work beyond the short half hour presentations.
In conclusion
An online conference is a great opportunity for people from all over the world to participate and present their work as long as it is accepted as part of the programme. Reviewing papers for a conference is costly. Therefore, there is a trade-off between accepting papers based on the reputation of the known author and equal opportunity for a lesser known economist to present their great work. Recognising this challenge, we decided to experiment with a two-step double-blind review of papers. We found that the double-blind review was not much more costly than the status quo in economics conferences. The final programme turned out to be of high quality on broad macro questions presented by economists from a diverse set of institutions.
For representation of women and underrepresented minorities in particular, there is still room for improvement, and a double-blind review process by itself doesn’t solve the broader issue of unequal representation in the economics profession. Going forward, we would recommend future organisers employ one of two strategies in order to deliver a more equitable program. First, if using the review format discussed above, organizsers may experiment with a larger number of papers in the second group (“papers for consideration”) relative to the first group of papers (“papers accepted after blind review”). Second, it may be desirable to sort papers into two groups – one of which includes women and under-represented minorities – before blinding. Then, the share of acceptances from each group allotted to each reviewer can be specified ahead of time to achieve an equitable programme.
Large-scale conferences like this one serve two primary purposes: allowing people to discover presenters’ work, and providing those presenters with feedback on that work. We found the online format well-suited to the former, but less so to the latter. While breakout rooms helped overcome this shortcoming, more ‘feedback-centric’ conferences might consider smaller, more intimate settings, which would likely require different review structures to correct for different biases.
We hope these lessons are helpful, and we look forward to seeing how conference organizers adapt them going forward!
The VMACS Junior Conference Programme Committee: Kyle Dempsey, Elisa Giannone, Heejeong Kim, Rocio Madera, Simon Mongey, Abdoulaye Ndiaye, Omar Rachedi and Christian Wolf.
Message from the VMACS Organisers
The open-access Virtual Macro Seminar Series (VMACS) started in March 2020 as a response to the lack of standard academic seminars. The interest was enormous. Live audiences were often the size of a large conference and recordings of the talks have been watched more than 60,000 times on our YouTube channel.
However, we also quickly realised that there was a particular need to cater to junior scholars, due to cancellations of many conferences and a bias towards senior academics in this new virtual format. The VMACS organisers thus decided to dedicate July to juniors only and we had an open call for volunteer organisers. The response was amazing so we decided to not only have VMACS Junior seminars during July 2020, but also to organise a VMACS Junior Conference (held on 31 August 31 to 2 September 2020). Both events were extremely successful due to the great work by both organisers and participants.
In setting up these organising committees, we decided to aim for both excellence and diversity in terms of field, geography, gender, and minority groups. We also decided to give the committees all the power to freely innovate on the format. We think this has been extremely productive and the conference committee has taken the time to reflect on their efforts here, providing interesting feedback on their experiences which hopefully will be of value to the profession going forward.
VMACS Organisers: Stefania Albanesi, Ralph Luetticke, Kurt Mitman, Laura Pilossoph, Morten O. Ravn and Claudia Sahm.
