The impact of the corona crisis on compiling the CPI
About this publication
The corona crisis is having an impact on how the Consumer Price Index (CPI) is compiled. This is primarily due to the fact that certain services are currently unavailable (e.g. package holidays, hairdressing, a cup of coffee at a café), which means that no one can charge a price for providing these services. In addition, the purchasing behaviour of consumers has changed while, at the same time, it has become more difficult – and in some cases even impossible – to observe prices accurately.
Depending on the situation, Statistics Netherlands (CBS) employs different methods to arrive at a notional price for these goods and services. The methods CBS uses are in compliance with the guidelines drawn up by Eurostat in collaboration with all of the member states of the European Union and the European Central Bank. The solutions adopted will enable CBS to continue to publish the CPI in full and to ensure that in due course the price development calculated between the post-corona and pre-corona period will reflect reality.
1. Introduction
The corona crisis is having an inevitable impact on the way in which the Consumer Price Index (CPI) is compiled. Due to the fact that our daily lives have changed, we are also spending our money differently: the contents of our average shopping basket have changed dramatically. The hospitality industry has been almost completely shut down, we are unable to go to the hairdresser’s and our holidays have been cancelled. Instead, we are buying more products online, in supermarkets and in DIY stores. At the same time, it can be difficult to obtain an overview of all the prices we need in order to compile the CPI each month.
If we want to measure prices and price development for the CPI, how can we assess the price of a package holiday to Thailand if no one can actually book or make that trip? Yet this is a time when, perhaps more than ever, our users need the CPI information we compile. For services that are no longer being purchased, we have to make choices that reflect how people use the CPI. In this article, we will explain how we are approaching this task during the corona crisis. In Chapter 2 we will delve into the background of the CPI and set out the important aspects of how it is used. In Chapter 3, we will go on to describe the problems that arise in compiling the CPI during the corona crisis and the ways in which we have decided to solve them. Finally, Chapter 4 explains how we are approaching the publication of data in the CPI.
2. Why do we compile the CPI?
The main measure of consumer price development is the change in the CPI over a one-year period. We compare the price level in April 2020 with the price level in April 2019. Monthly price changes are more noticeable to consumers, but are also strongly influenced by seasonal patterns. After all, retailers sell clothing and shoes at a discount in their twice-yearly sales and there is clearly a strong seasonal element to the prices of hotels and package holidays. Comparing the price level now with the same period one year earlier cancels out the effect of these seasonal patterns.
This year-on-year change expressed in the CPI is an important measure of consumer price inflation. The CPI shows the extent to which household income has to increase in order to buy the same products and services, and is therefore widely used for indexation. If income rises in line with the CPI, the purchasing power of Dutch households remains stable. Indexation is used not only to adjust salaries or pensions, but also to help set tax rates, the price of renting a home and the financial component of all kinds of other contracts.
In addition to the national CPI, CBS also produces a European harmonised index of consumer prices (HICP). The main difference between these two indices is the scope: the CPI measures price development for the population of the Netherlands, while the HICP measures this for the Netherlands as a whole. The CPI therefore includes spending by Dutch citizens abroad, while the HICP includes spending by foreign visitors in the Netherlands. Because the HICP takes the country as its starting point, the HICPs of the various European countries can be merged into a single European HICP. The HICP also serves as an important reference point for the European Central Bank (ECB). In its efforts to maintain price stability, the ECB bases its monetary policy on the HICP to a considerable extent. Here, too, the primary focus is on change in prices relative to one year previous.
In addition to the national CPI, CBS also produces a European harmonised index of consumer prices (HICP). The main difference between these two indices is the scope: the CPI measures price development for the population of the Netherlands, while the HICP measures this for the Netherlands as a whole. The CPI therefore includes spending by Dutch citizens abroad, while the HICP includes spending by foreign visitors in the Netherlands. Because the HICP takes the country as its starting point, the HICPs of the various European countries can be merged into a single European HICP. The HICP also serves as an important reference point for the European Central Bank (ECB). In its efforts to maintain price stability, the ECB bases its monetary policy on the HICP to a considerable extent. Here, too, the primary focus is on change in prices relative to one year previous.
3. Problems affecting the CPI due to the corona crisis
During the corona crisis, three factors are having a significant impact on how we compile the CPI:
- Consumers’ purchasing behaviour has changed (e.g. hoarding, buying more DIY materials);
- There are obstacles to price observation (it can be more difficult or sometimes even impossible to observe prices);
- Certain services are currently unavailable (e.g. package holidays, hairdressing, a cup of coffee in a café).
For now, the first factor is having no effect on how the CPI is compiled, because we follow the price development of a fixed ‘shopping basket’ of goods and services. In the long term, however, it may have an effect if patterns of consumption alter permanently. As for the second factor, we have not had any major issues with price observation in the Netherlands, although this has been the case in many other countries. The almost complete absence of certain services at present is a unique situation for the CPI and poses a difficult problem for price statistics in general.
Eurostat has taken the lead in this regard by collaborating with all EU member states and the ECB to draw up guidelines on how statistical agencies should deal with these problems when compiling the CPI and the European HICP. These guidelines were published by Eurostat on 8 April and CBS has decided to follow them.
In the following sections, we will look at the three factors outlined above in greater detail and examine the solutions that CBS has chosen.
3.1. Changes in consumer purchasing
When compiling the CPI, we record the prices of a large number of goods and services. This package, referred to as the ‘average shopping basket’, covers the full spectrum of goods and services and can be broken down into various commodity groups (the COICOP1) breakdown). Each commodity group is given its own weighting, in line with the average amount that consumers in the Netherlands spend on goods in that particular group. The price development of a commodity group on which consumers spend a lot of money therefore counts for more than the development of a group on which less money is spent. The CPI’s focus on measuring price development means that this weighting2) is not adjusted in the course of a calendar year. If we were to do so, it would no longer be clear whether the results were due to changes in the weighting or changes in price.
Due to the stable nature of the weighting system, the consumption (purchase) of more or less of a certain product does not directly affect the composition of the CPI, not even if such a change in behaviour represents a shift from one commodity group to another (e.g. spending less on holidays and more on DIY materials or furniture). The various weightings remain the same throughout the year, with only month-to-month price changes being taken into account.
In the longer term, however, changes in consumption may well have an effect. The weightings we use in any given calendar year is based on the average spending pattern in previous years. As a result, the weightings gradually change in line with the changing consumption of Dutch households. The influence that the corona crisis is having on our spending patterns could therefore affect the weightings in years to come. For now, the extent to which we will actually take this effect into account in terms of weighting remains unclear. This will partly depend on whether the crisis lasts for a long time and brings changes that are more structural, or whether it lasts for a relatively short time and only leads to a temporary change in spending patterns.
3.2. Price observation becomes more difficult or even impossible
At CBS, we have encountered hardly any problems with price observation. To calculate the CPI, we compile price data for a very large number of products and services. Traditionally, price collectors visit retail outlets for this purpose: they go into shops with a list of representative products and record the prices of these products. This is a labour-intensive process and the availability of new resources for price observation has enabled us to steadily scale back this approach in recent years. On 1 January 2020, we became the first statistical agency in the Eurozone to move beyond observation by visiting shops. In the current circumstances, this milestone puts us at a considerable advantage. In many European countries it is currently impossible to carry out price observation in shops due to the corona restrictions that are in force, but CBS receives weekly transaction data (lists of items sold with price and turnover) from a large number of retailers: this means that we are continuing to receive an up-to-date overview of all prices. In addition, we use web scrapers to automatically observe prices on the internet. Lastly, we employ staff who use the internet, email and telephone to manually observe prices. Our approach enables us to accurately observe the prices of products and services that are offered and sold, even during the corona crisis.
Despite this almost entirely digital approach to observation, price data for certain products or services may still be unavailable even though they are being sold. Occasionally this happens under normal circumstances too. To ensure that the CPI we publish is accurate and complete, we use standard methods to estimate any missing prices (a process known as imputing). By imputing a price, we temporarily give it a notional value until its actual value can be ascertained. To determine this notional price, we use two different methods:
- Adjustment in line with the price development of a similar product;
- Sticking to the last recorded price (carry forward).
The first method assumes that the product whose price we do not know is likely to have undergone the same price development as other similar products. If we have been unable to record the actual price of peaches, for example, it is reasonable to assume that their price probably develops in much the same way as the price of other types of fruit.
The second method can be useful when estimating the price of a product or service with a very stable price development. If the price of museum admission is missing from the observation, for example, it is reasonable to assume that it will be the same as it was last month.
The second method can be useful when estimating the price of a product or service with a very stable price development. If the price of museum admission is missing from the observation, for example, it is reasonable to assume that it will be the same as it was last month.
If a price is observed again the following month, the estimated price is then replaced by the newly observed price. This applies in the case of both methods. As a result, the price level and the calculated price development always return to the right level after imputation, even if the imputation was not entirely correct.
In the current situation, we will continue to use these standard methods as we did under normal circumstances.
3.3. Discontinuation of services
As a result of the corona crisis, and more specifically the measures taken by the government, certain services are currently unavailable or subject to tight restrictions. Virtually all European countries have taken measures that are having a far-reaching impact on public life. The Netherlands has been in what the government refers to as an ‘intelligent lockdown’, with much of the retail sector permitted to remain open, while a number of sectors faced compulsory closure. This included events and festivals, cinemas, museums, the hospitality industry and professions involving close customer contact (e.g. hairdressing, massage). In addition, travel has been strongly discouraged, resulting in the cancellation of holidays and severely limiting flights on all routes.
This has led to a particular problem in terms of price statistics. Never before has a significant proportion of household spending simply disappeared, albeit temporarily. There are no methods for setting a price for goods or services in cases where transactions have ceased altogether. And even if a limited number of transactions continue to take place, it is questionable whether the prices involved are realistic. Statisticians across the world are looking into this problem and countries are implementing their own solutions. In Europe, Eurostat wasted no time in drawing up guidelines for this process, in consultation with all EU member states and the ECB. CBS is following these guidelines and has calculated the consequences of different methods, both on a purely mathematical basis and on the basis of historical figures.
In the Eurostat guidelines and the approach adopted by CBS, a distinction is made between services with a strong seasonal pattern (e.g. package holidays, air travel) and services with no marked seasonal pattern (e.g. museum admission, hairdressing). For services with no seasonal pattern, we are continuing to use the standard imputation methods (see Section 2.2) to estimate any missing prices. As there is no prominent seasonal pattern in these services, price changes will occur gradually. The standard imputation methods are therefore sufficient to provide an approximation of any current changes in price.
For services with a seasonal pattern, there are substantial price fluctuations that depend on whether a service is in or out of season. As there are no similar services, with a similar seasonal pattern, to facilitate the imputation of price development, we cannot apply this imputation method. Nor does it make sense to stick to the last observed price, given the strong fluctuations at play. As a result, there is no point in basing the price development on current information; this leaves us no choice but to turn to a different method. In doing so, our aim is to minimise the impact of this missing service on the recorded price change compared to one year earlier.
As stated in Chapter 2, the year-on-year change in a price is the CPI’s primary measure of consumer price inflation. It is therefore logical to ensure that services which consumers are no longer able to purchase do not affect price changes compared to the year before. In theory, this can easily be achieved by viewing these services as developing in line with the annual development of all other goods and services that are still available. This would mean that only goods and services that are actually available determine the annual development. In practice, however, this method is difficult to apply, not least because the systems we use to calculate the CPI are ill-equipped to deal with such an approach. However, one method that comes reasonably close7) is adopting the monthly change of the same service the year before: in other words taking the price change from April 2019 compared to March 2019 and using it to estimate the price change in April 2020 compared to March 2020. This is the method we have decided to use for unavailable services with a seasonal pattern, because it minimises the effect on the annual development of the CPI. In cases where limited supply and price observation exist but the prices observed are not representative of the missing price data, we also use this method to impute the missing prices. Of course, any prices that are observed will be used.
As soon as these services are offered once again and consumers are free to use them, we will revert to using the current prices. As with the methods described in Chapter 2, the current price level and the calculated price development between the post-corona period and the pre-corona period reflect reality. This is an important condition when deciding which method to use.
1) COICOP stands for Classification of Individual Consumption by Purpose.
2) CPI – Achtergrond jaarlijkse basisverlegging 2020 [Background to Annual Base Shift 2020] (only available in Dutch)
4. Publication of the CPI and the HICP
Every month, CBS publishes the price changes of hundreds of separate commodity groups and their aggregates (compilations) for the CPI and the HICP. As a result of the corona crisis, we currently have to establish a notional price development for dozens of commodity groups. For this purpose, we make imputations that are not based on actual price observations. We do not believe that it is right to simply publish these imputed price developments without providing transparency as to the assumptions that underlie them. Due to the different systems that apply, we approach this differently for the CPI and the HICP.
4.1. Publication for the CPI
We publish the CPI in Statline, the CBS online database. If a figure is unknown, insufficiently reliable or confidential, we do not publish it but instead replace it with a dot. We use this same approach for goods and services that are largely (over 50 percent) based on imputations as a result of the corona crisis.
To give users an insight into the assumptions we have made and to identify the values we have imputed in order to compile the CPI as a whole, we will be publishing a separate monthly customised table from April 2020 . This table shows the method we employ for each missing service and the value we impute for the index. This gives our users an understanding of the choices we are making and the impact they have on the results.
To give users an insight into the assumptions we have made and to identify the values we have imputed in order to compile the CPI as a whole, we will be publishing a separate monthly customised table from April 2020 . This table shows the method we employ for each missing service and the value we impute for the index. This gives our users an understanding of the choices we are making and the impact they have on the results.
Lastly, we measure the CPI for the average Dutch consumer, based on an average spending pattern. This spending pattern can be different for everyone. That’s why we have developed the personal inflation calculator. This online service enables everyone to calculate their own personal rate of consumer price inflation, based on their average monthly expenditure. The calculator also enables people to see how inflation is affected when certain services (e.g. air travel, package holidays, going out for a meal or a drink) are no longer available.
4.2. Publication for the HICP
Eurostat publishes the HICP of each individual country and for groups of countries. In its database, Eurostat uses flags to indicate the characteristics of a particular figure. The letter ‘u’ stands for ‘low reliability’ and flags goods and services that are largely (over 50 percent) based on imputations as a result of the corona crisis. All figures are therefore published in the HICP, but are marked where necessary.