How bad is deflation in Japan?

David Weinstein, Christian Broda

22 October 2007



On the face of it, Japanese deflation does not seem that severe. The latest monthly numbers suggest that over the past twelve months, the non-fresh food component of the CPI is falling at an annual rate of 0.1%. However, we believe that this number seriously understates deflation in Japan – maybe by an order of magnitude.

In a recent paper, “Defining Price Stability in Japan: A View from America,” we argue that the Japanese CPI is computed using methods that tend to overstate true inflation and that it differs markedly from a true cost-of-living index. Using a more sophisticated methodology – the one used in the US – inflation in Japan would be running at around minus 1%. This implies that between 1999 and 2006, deflation was probably double that suggested by the official statistics and that Japan is still immersed in a general deflationary environment. Moreover, if we compare the Japanese CPI to a true cost-of-living index, the gap widens to about 1.8 percentage points per year. The implications for the Japanese economy are hard to understate. These gaps are likely to significantly worsen Japan’s fiscal position in the future and have the potential to severely misguide the current debate about monetary policy in Japan.

Officially, none of these problems exist. For example, on March 9, 2006, the Bank of Japan issued a statement clarifying its thinking on price stability: “Currently, there seems to be no significant bias in the Japanese consumer price index.” This confidence in the lack of bias in the Japanese CPI is particularly surprising since the Japanese national statistics office does not correct for many of the most basic problems in standard index theory. By contrast, understanding biases in the US CPI has been a major undertaking that has produced many changes in the index in recent years.

Substitution bias in CPI: If its price falls, you buy more of it

A major problem in the Japanese CPI stems from what economists call substitution bias. This bias arises because consumers adjust their consumption when the prices of goods change, while “fixed-weight” indexes like the Japanese CPI do not allow such an adjustment. In particular, consumers tend to shift their purchases towards goods whose prices fall and away from goods whose prices rise. This implies that fixed-weight indexes like the Japanese CPI overweights goods whose prices are rising relative to the actual expenditure of consumers. This problem of “fixed-weight” indexes has the well-known implication of overstating the true inflation.

A simple example can illustrate the extent of the bias. Assume that the typical Japanese consumer buys one bottle of Kirin beer and one bottle of Asahi beer a month at a cost of 300 yen each. If the price of Kirin beer today doubles to 600 yen and the price of Asahi falls by half to 150, most consumers would switch their purchases towards Asahi, but a fixed quantity index like the Japanese CPI would record this set of price changes as inflation. If the Japanese CPI recorded the price of beer as 1 in the first year, the price index in the second year would be a 25% increase in the price of beer (i.e., 0.25 = ((600+150)/(300 + 300)-1). That is, the Japanese CPI methodology assumes that consumers continue to buy the same amount of Kirin and Asahi even after their prices change. As long as one believes that demand curves are downward-sloping – and last time we checked most economists do – this implies an upward bias in the Japanese CPI relative to a true cost-of-living index.

The US corrects for this problem in two ways. Like the Japanese CPI, the CPI in the US is comprised of two levels of aggregation. A lower level combines the prices of different goods within a product category and an upper level aggregates together the prices of the different product categories. This suggests that the substitution bias can be present at both levels of aggregation. Since 1999, the US has dealt with the problem of substitution bias at the lower level by randomly sampling prices and using geometric averaging at the lower level. The advantage of geometric averaging is that it allows for substitutability among the various goods that make up an item index. In the example discussed above, the US statistical office in charge – the Bureau of Labor Statistics (BLS) – would take the geometric average of the prices and produce an estimate of no inflation in the second year (0 = (2*0.5)^0.5-1). Various researchers have tried to assess the importance of geometric averaging of lower-level prices for the US CPI by computing the index with and without this correction. Reasonable estimates put this number at about 0.2% per year.

The US corrects for the upper-level bias, i.e. substitution across product categories like car and train transport, by using chained weights. One element of the Japanese method of computing prices that is quite confusing is that while both Japan and the US now report a “chained” CPI, the methodology for chaining is completely different in the two countries. The US chained CPI (the C-CPI) uses what is known in index number theory as a “superlative” formula at the upper level. Essentially this means that the US is using state-of-the-art indexes to avoid upper-level substitution biases. Japanese “chaining,” by contrast, is actually what the BLS calls “annual weight updating”. The BLS now updates the base weights in the CPI every two years. While more frequent weight updating may reduce the upper level substitution bias, it is not sufficient to eliminate it. In the aforementioned paper, we compute the magnitude of this bias in the Japanese case to be around 0.4% per year.

Quality biases: Same price for better laptop every year

The remaining difference between US and Japanese methodologies is related to the adjustment for quality growth in products. Since 1999, the US has expanded the use of hedonic pricing models to prevent quality increases in products to show up as price inflation.1 For instance, since 2002, the BLS has been using hedonics in a wide array of products including cell phones, computers, refrigerators, cable television, lodging away from home, and college tuition and fees. Japan only uses hedonics for computers since 2000 and for digital cameras since 2003. These quality corrections lower US inflation rates by around 0.2 percentage points per year. We believe that the Japanese methodology ignores this type of quality upgrades and incorrectly attributes the quality growth in these products to price inflation.

The upper and lower substitution biases, in combination with the hedonic price adjustment, imply that the US CPI moves upwards at a rate of approximately 0.8% per year slower than the Japanese CPI. This suggests that one should exercise extreme caution in assuming that the inflation differential between the two countries is captured by the difference in the two CPIs. If Japan used the same methodology as the US for computing inflation, it is likely that the official rate of inflation would be -0.9% per year not -0.1%.

In addition to these differences, it is also well-known that even the US CPI does not make corrections for the fact that the quality of most goods is continuously rising as new and improved products are introduced. Moreover, neither CPI makes adjustments for the fact that mass merchandisers like Wal-Mart have given consumers new and cheaper locations from which to purchase their goods. A series of recent papers (Broda and Weinstein (2007b), Hausman and Leibtag (2004) and Bils and Klenow (2001) have, for the first time, estimated the aggregate effects of these biases using micro data on consumer behavior. Combining the results in these papers, the overall bias is of the order of 1 percentage point a year, taking the total bias in the Japanese case to 1.8 percentage points per year. This suggests that instead of Japanese prices falling by 0.1 percent per year, the cost of living in Japan is probably falling at a rate closer to 1.9%.

Implications for Japan

The implications of this bias are enormous for Japan. Many Japanese government transfers, such as public pensions, are indexed to the CPI. The conventional justification for this indexing is that it is required to maintain the “standard of living” of the elderly constant over time. However, if the CPI is biased upward, then this means that the government is spending vastly more than it should to keep the standard of living of the elderly constant over time. If this overstatement in the Japanese CPI is not corrected it will imply higher government expenses for over 69 trillion yen – or 14 percent of GDP – over the next 10 years in increased social security transfers and debt service. For monetary policy, the overstatement of inflation suggests that if the BOJ adopts a formal inflation target without changing the current CPI methodology, a lower band of less than 1.8% would not achieve a goal of price stability.

In spite of the Bank of Japan’s contention that there is nothing wrong with Japanese statistics, the methodology used in Japan to compute prices is clearly out of date. When we spoke informally with Japanese bureaucrats about this, the main reason they give for why the government does not improve the statistics is cost. Japanese statistical agencies are terribly under-funded and lack the staff to improve the numbers. This suggests that providing adequate funds to these agencies so that they can generate accurate price measures may not be the most exciting thing on Prime Minister Fukuda’s agenda, but it may be the most important.


Bils, Mark, and Peter J. Klenow, 2001. “Quantifying Quality Growth.” American Economic Review, 91, 1006-1030.

Broda, Christian, David E. Weinstein (2007a), “Defining Price Stability in Japan: A View from America”, NBER Working Paper No. 13255, July 2007.

Broda, Christian and David E. Weinstein (2007b) “Product Creation and Destruction: Evidence and Price Implications,” NBER Working Paper 13041, April 2007.

Hausman, Jerry and Ephraim Leibtag, "CPI Bias from Supercenters: Does the BLS Know that Wal-Mart Exists? NBER Working Paper No. 10712 August 2004


1 These view prices as a function of the good’s characteristics, like speed of the processor, or size of the hard drive. Once the link between price and characteristics is established, quality adjusted prices can be simulated. For details, see here.




Topics:  Monetary policy

Tags:  CPI, Japan, deflation

Carl S. Shoup Professor of the Japanese Economy and Associate Director of Research at the Center for Japanese Economy and Business

Managing Director, Duquesne Capital Management