New technology in schools: is there a payoff?

Stephen Machin, Sandra McNally, Olmo Silva 14 December 2007

a

A

The view that information and communication technologies (ICT) are a useful tool for raising educational standards dates back to the 1950s and the findings of Harvard psychologist BF Skinner.1 More recently, support for the effectiveness of ICT as a teaching and learning device has come from the educational and psychological literature (recently reviewed by Heather Kirkpatrick and Larry Cuban),2 which tends to make enthusiastic claims for the value of new technology in schools.

Economists have generally been much more cautious, in particular raising concerns about the methodological validity of much of this research. Most of the positive findings are inferred from simple correlations between ICT and test scores. These cannot take account of unobserved school characteristics – such as more motivated teachers – that may lead to both ICT adoption and better attainment.

Indeed, starting with Joshua Angrist and Victor Lavy,3 a small number of economic studies, applying more rigorous methods of analysis, have found it hard to uncover any evidence of a positive causal relationship between computers (and/or computer software) and pupil performance.

This has not stopped governments around the world supporting widespread adoption of computers as teaching devices. A few remarkable examples include:

  • the United States, where major subsidies for internet and communications technologies were distributed to schools from 1998;
  • Israel, which equipped most of its schools with computers starting from 1994;
  • Italy, where since the early 2000s all main political parties (and in particular Berlusconi’s Forza Italia) have made ‘computers in schools’ part of their educational programmes;
  • and the Netherlands, whose government introduced subsidies earmarked for computers and education software in 2000 for schools with predominantly disadvantaged pupils.

In the UK, which is the focus of our research,4 the government has similarly emphasised the role of ICT investment in schools as ‘crucial to our drive to raise standards’ (former secretary of state for education Ruth Kelly), and envisaged ICT being widely used across the whole curriculum in all state schools.

The positive rhetoric has been backed up by considerable public investment. Between 1998 and 2002, ICT expenditure almost doubled in English secondary schools (those for children aged 11-18) – from an average of £40,100 to just under £75,300 per school, or 3% of overall expenditure – and more than trebled in primary schools (those for children aged 5-11) – from £3,600 in 1998 to £12,900 in 2002, or 2% of overall expenditure. Most of this dramatic acceleration took place from 2000 and the upward trend continued in 2003 and 2004.

Table 1 shows how primary and secondary schools used these resources in 2000 and 2002. Between these years, the share of ICT expenditure devoted to hardware and software actually decreased – by 16% in primary schools and 3.5% in secondary schools – but as the total amount of ICT funding was increasing, overall expenditure in these two categories grew steadily.

Table 1: How was ICT money spent in schools? (Percentage devoted to different items)
  1990/2000 2001/02 % change
Primary schools
Hardware 63 53 -16
Software 10 10 0
Internet 8 7 -12
Training 7 11 +57
Technical support 9 13 +44
Administration + other 3 6 +100
Total expenditure per school £10,000 £14,000 +41
Secondary schools
Hardware 57 55 -3.5
Software 9 9 0
Internet 4 3 -25
Training 4 6 +50
Technical support 14 17 +21
Administration + other 12 10 -17
Total expenditure per school £56,000 £76,000 +34.5

Source: Authors’ calculations from ICT Survey of Schools in England (UK Department for Education and Skills)

Interestingly, the share of resources devoted to teacher training rose by 57% for primary schools and 50% for secondary schools between 2000 and 2002. This suggests that the extra funding may have improved the quality of ICT use in schools rather than simply increasing the quantity of ICT equipment.

Table 2 provides a more complete picture of schools’ ICT use, showing how new technology and training grew in response to the extra resources. Although schools were already well-equipped, there were quite sizeable changes between 2000 and 2003. For example, the pupil-to-computer ratio in 2000 was 10 to 1 in primary schools and 7 to 1 in secondary schools; by 2003, the respective ratios were 6 to 1 and 4 to 1.

Table 2: Trends in ICT expenditure and use of ICT resources
  Primary schools Secondary schools
  99/00 02/03 % change 99/00 02/03 % change
Computers per pupil 0.10 0.16 +60 0.15 0.23 +53
% of teachers using ICT regularly 75 92 +23 38 55 +45
% of teachers trained to use ICT 81 95 +17 75 83 +11
% of teachers with recently updated training 57 85 +49 48 69 +44
% of schools connected to the internet 86 100 +16 99 100 +1

Source: Authors’ calculations from ICT Survey of Schools in England (UK Department for Education and Skills)

The indicators of ICT use in the classroom also show fairly high percentage increases over this short time period. What’s more, the fraction of teachers trained to use ICT grew substantially. This confirms our intuition that rather than just increasing the quantity of ICT equipment, schools also invested in the quality of ICT use.

Although secondary schools were better equipped with ICT in 2000, the greatest relative increase over time was experienced in primary schools. It is also notable that ICT is used regularly for teaching purposes in a much higher percentage of primary schools than secondary schools. Any effect of ICT on educational performance is therefore likely to be more evident in primary schools than secondary schools.

We also have evidence that ICT is widely used in primary schools to teach English: 65% of primary schools report that they ‘substantially use’ ICT for teaching this subject. The next most important ‘ICT user’ is Mathematics, where ICT is ‘substantially used’ in about 56% of primary schools, followed by Science (35%).

So has the big increase in ICT investment made a difference to educational standards?

Our research evaluates whether changes in ICT investment had any causal impact on changes in educational outcomes in English schools between 1999 and 2003. To do this, we rely mainly on administrative data at the level of local education authorities (LEAs), focusing on average achievements at the end of primary education in English, Mathematics and Science.5 Following government targets, we look at the proportion of pupils achieving level 4 or above in the three subjects at age 11 (national tests taken at the end of primary school).

Inferring a causal relationship between ICT investment and pupil achievement from simple correlations can be misleading. For example, we could imagine that schools or LEAs with more motivated teachers and head teachers are more likely both to adopt ICT and to produce higher attainment. In this case, if we did not control for motivation, a positive relationship between ICT and pupil test scores may emerge just because of this unobserved common factor (motivation) driving both observed outcomes.

To overcome this problem and identify the causal impact of ICT use on pupil achievement, we exploit a 2001 policy change that modified the rules for ICT investment in different regions of England. Before 2001, funding was allocated from central government to LEAs through a bidding process, aiming to direct money towards LEAs with innovative and interesting proposals for the use of ICT funds.

From 2001 onwards, allocations were instead made according to a formula based on school and pupil numbers in LEAs with an adjustment for population density. The change in the allocation mechanism created ‘winners’ and ‘losers’ among LEAs: areas that had benefited a lot under the old system stood to lose from the transition to a formula-based system, and vice versa.

In our analysis, we argue (and provide evidence) that money was reshuffled across LEAs in a ‘random’ way, that is, in a way unrelated to unobservable LEA characteristics that may give rise to a spurious relationship between ICT funding and test scores. We then use the changes in the ICT funding accruing to LEAs as a result of the policy change to estimate the effects of ICT expenditure on educational standards. Our approach identifies the effect of being a ‘winner’ or a ‘loser’ in the new system of ICT allocation, rather than the average causal effect for all schools in England.

We find a positive relationship between ICT funding per pupil and performance in English: a doubling of ICT funding per pupil in schools leads to a 2 percentage point increase in the proportion of pupils achieving level 4 or above at age 11. Changes in ICT funding of this magnitude really did happen for primary schools over this period, and the impact on performance in English is notable given that the average growth rate of pupils' scores in this subject was around 7% between 1999 and 2003.

For Mathematics, the impact of ICT on test scores is very close to zero. But there is a positive relationship between ICT and achievement in Science: in this case, a doubling of ICT funding per pupil leads to an increase of 1.6 percentage points in the proportion of pupils achieving level 4 or above.

So, unlike previous economic studies, we find evidence for a positive causal impact of ICT investment on educational performance in primary schools. This is most evident in the teaching of English, where we also find high use of ICT for teaching purposes. We also observe a positive impact for Science, though not for Mathematics.

How can we reconcile our evidence with previous research that finds no effect?

Our estimates identify the impact of being a winner or a loser under the new system. After the policy change, the average growth rate of ICT funds among LEAs mostly benefiting from the reform was roughly 60%. This contrasts with a much smaller change of 20% for LEAs that lost more from the introduction of the formula-based system. Intuitively, it is the comparison between these two groups that drives our identification of the impact of ICT on educational outcomes: our strategy mainly captures the impact of large changes in ICT investment on primary school performance.

Additionally, we have documented that new technology was already in place in English schools since the mid-1990s, and money redirected after the policy change was mainly spent in updating resources and teachers’ skills.

So it appears to be the joint effect of large increases in ICT funding – and a fertile background for making efficient use of it – that led to the positive effects of ICT expenditure on educational performance.

 


 

Footnotes

1 BF Skinner (1954), ‘The Science of Learning and the Art of Teaching’, Harvard Educational Review 24, 86-97; and BF Skinner (1958), ‘Teaching Machines’, Science 128, 969-77.
2 Heather Kirkpatrick and Larry Cuban (1998), ‘Computers Make Kids Smarter – Right?’, TECHNOS Quarterly for Education and Technology 7(2), 1-11.
3 Joshua Angrist and Victor Lavy (2002), ‘New Evidence on Classroom Computers and Pupil Learning’, Economic Journal 112, 735-65.
4 Stephen Machin, Sandra McNally and Olmo Silva (2007), ‘New Technology in Schools: Is There a Payoff?’, Economic Journal 117, 1145-67.
5 There are 150 LEAs in England. They are responsible for the strategic management of education services, including planning the supply of school places, intervening where a school is failing its pupils and allocating funding to schools.

 

 

a

A

Topics:  Education

Tags:  education, ICT, schools

Professor of Economics, University College London; Research Director at the Centre for Economic Performance, LSE; CEPR Research Fellow.

University of Surrey and the Centre for Economic Performance, London School of Economics

Lecturer at the Department of Geography and Environment at the London School of Economics

Events