Author: Rob van de Laar, Jacco Daalmans
Processing energy prices for the Consumer Price Index

3. Method

3.1 Introduction

The new method consists of a number of steps, which are explained in this chapter. Figures with outcomes have been included in the annex. Section 3.2 explains how the calculation prices – the annual energy bill amounts on which the price index is based – are determined. Section 3.3.1 describes how transaction data is aggregated into homogeneous products, delineated by price-determining characteristics. Section 3.3.2 sets out how the calculation prices are determined for each product based on the calculation prices of the individual contracts. Section 3.4 provides insight into how products are aggregated into one index for each respondent for electricity and one index for each respondent for natural gas. How these are incorporated into the CPI is described in Section 3.5. A number of other aspects are discussed in Section 3.6.

3.2 Determining the calculation prices

This section explains what prices are measured in the price index.

3.2.1 Definition

A household’s energy bill consists of several components: rates for the consumption of electricity and natural gas, transport costs, charges and taxes. All these components are included in the CPI. The purpose of this section is to explain how the different energy bill components are combined.

All components are annualised to arrive at a total amount per year for each connection. Each month, the amount per year is calculated based on the applicable rates. The annual price can be broken down into variable charges, which are calculated per kilowatt-hour (kWh) or cubic metre (m3), and fixed charges, which are calculated per connection. Several components of the fixed costs are expressed per year, which is why the CPI is produced based on annual rather than monthly prices.

Below are the various components, which are calculated both inclusive and exclusive of VAT:

  • per kWh or m3:
    • The variable rate per kWh or m3;
    • Renewable energy tax (ODE);
    • Energy tax.
  • per year for each connection:
    • Standing charge;
    • Transport rate;
    • Tax credit (‘energy tax refund’).

Rates and components can change monthly due to government measures (ODE, energy tax and tax credit) or decisions made by the energy companies (variable rates and the standing charge). CBS uses sample surveys to determine the average transport rate, which is set by the grid operator. The tax credit is shown within the expression for the calculation price as a negative amount, constituting a tax rebate. All other amounts are positive.

The annual energy bill amount is also referred to as the calculation price, which is defined as follows:

\begin{align*} Calculation\:price & = \left(\frac{100 + VAT\:rate}{100}\right) *\\ \biggl\{ &\bigg[Average\:annual\:consumption\:(kWh\: or\: m^{3})\bigg]\: *\\ *\: &\bigg[ Variable\:Rate+ODE+Energy\:tax \bigg]\: +\\ +\: &Standing\:charge + Transport\:rate + Tax\:credit \biggl\} \end{align*}

Average consumption quantities for electricity and natural gas are used to weight consumption-dependent rates and non-consumption-dependent rates/components of energy. These quantities, which are fixed for one year, are forecasts of energy consumption during the reporting year, prepared by the Netherlands Environmental Assessment Agency (2023). Annual consumption does not differentiate by contract type or by contract duration. For 2023, the average annual consumption per household is 1,064 m3 for natural gas and 2,151 kWh for electricity.

Some of the households with an electricity rate have double meters. These either pay an off-peak or a standard rate, depending on the time of day. For households with double meters, a fixed ratio between off-peak and normal consumption is assumed annually, which is assumed to be the same across all contract types. This ratio is based on consumption and is fixed for one year. 

The above leads to the following relationship:

\begin{align*} Variable\:&rate\:(Double\:rate) = \\ & Off-peak\:rate\: *\: \big[ Share\:off-peak\:consumption\:in\:total\:consumption \big]\: + \\ &Normal\:rate * \big[ Share\:normal\:consumption\:in\:total\:consumption \big] \end{align*}

From 2021, off-peak and normal consumption both account for 50% of total consumption.

In the next two sections, two specific aspects of determining calculation prices are discussed. Section 3.2.2 explains how the price cap, which was introduced in 2023, is incorporated into the variable rates in the calculation above. Section 3.2.3 discusses the tax credit in more detail.

3.2.2 Incorporation of the price cap

The price cap for energy, which was introduced on 1 January 2023, is a discount on the variable rates for electricity and natural gas up to a certain consumption cap per connection. Under this new policy, households whose consumption stays below a certain limit never pay more than the cap rate. Households that use more energy pay the variable rate specified in their contract for their energy consumption above the limit. The price cap is not incorporated into the index calculation at the individual level. This is because the discount a household receives due to the price cap is linked to consumption, and the provided data does not include information on individual consumption per connection. Instead, a consumption profile is used to estimate a new variable rate including the price cap using the originally submitted variable rates. This adjusted rate is based on estimates of expected consumption above and below the limit.

The consumption profiles correspond to estimated deciles of the annual energy consumption of Dutch households. For example, the first decile shows the consumption of households in the first 10% on the scale from low to high consumption. The consumption profiles are used for one year and – in the absence of more detailed data – are the same for all contract types.

The use of decile groups ensures more accurate estimates of the effects of the price cap. If the average energy consumption per household were used instead, this could lead to an underestimation of price developments in the CPI. The average consumption is below the limit set by the price cap, but there are also households whose consumption exceeds that limit. For example, if all contract prices are above the price cap rate, using average consumption would reduce the rate to that of the price cap, while some households would have to pay more due to their higher consumption.

Using the original rates, the new rates are calculated as follows:

  • For each decile group, an annual amount equal to the decile turnover is calculated as the product of a variable rate and decile group-dependent consumption. This includes the price cap discount based on the decile group’s consumption;
  • This annual amount is summed across all decile groups and then divided by the total consumption of all decile groups;
  • The result is a new variable rate that includes the price cap.

The above method implies the following relationship:
\begin{align*}
Variable\:&Rate\:inclusive\:price\:cap = \\
    &\alpha\:*\:Original\:rate +  (1-\alpha) * Price\:cap\:rate
\end{align*}
The factor α is the fraction of total consumption above the consumption limit. It is calculated here by adding the surplus of expected consumption above the price cap limit for each of the ten percentile groups and then dividing the result by the total consumption of the ten decile groups. The a factors in 2023 are 0.18 and 0.15 for electricity and natural gas, respectively.

The price cap is thus directly applied to the variable rates in the transaction data. For electricity, the price cap is applied separately to the off-peak and standard rates for households with dual-meter contracts. This corresponds with how the price cap is applied in practice.

3.2.3 Incorporation of the tax credit

As the government considers a certain amount of energy consumption to be a basic need, households receive a refund of part of their energy tax, also known as the tax credit. The energy tax refund is paid per electricity connection.

CBS has chosen to deduct the energy tax refund partly from the price of electricity and partly from the price of natural gas from 2022 onwards. On the one hand, this is more in line with the aim of the scheme, which is to offer compensation for all taxes on electricity and natural gas. On the other hand, this avoids including negative electricity prices in the CPI.

From 2022, the energy tax refund will be divided between electricity and natural gas in proportion to the average tax paid on electricity and natural gas consumption.

This ratio is calculated as follows:

  • The consumption-based tax on electricity and natural gas is calculated as the sum of the renewable energy tax (ODE) and the energy tax;
  • This consumption-based tax is multiplied by the average annual household consumption of electricity and natural gas;
  • The ratio between electricity and natural gas in the above sum is then determined. This ratio is applied to the tax credit in the following year.

For 2022, this leads to more than one-third of the refund being offset against the electricity bill and almost two-thirds against the natural gas bill. The incorporation of the tax credit is explained in more detail in CBS (2022).

3.3 Aggregation

This section explains the aggregation level of the index calculation.

3.3.1 Product definition

A price index is an indicator of the price development of a set of products, which can be defined at different levels of aggregation. At lower levels of aggregation, detailed distinctions between products are made, while at higher levels broader categories are used. A disadvantage of using very narrowly defined products is that some products are not sold in every period, leading to discontinuities that can cause the price index to become unstable. A disadvantage of using very broadly defined products is that, over time, there may be discrepancies between the products that are compared, leading to an undesirable loss of homogeneity. In defining products, there is thus a trade-off between continuity and homogeneity.

The MARS method provides a statistical approach to determining a product definition (Chessa, 2021a). This method selects price-determining characteristics and ignores non-price-determining characteristics in order to obtain as much data as possible per homogeneous product.

As electricity and natural gas are purchased by households through contracts with energy companies, the contracts are viewed as the purchased products. These products have several characteristics (see Section 2.3). For fixed-rate contracts, the delivery rate remains the same for the duration of the contract. The duration of a fixed-rate contract has a clear effect on the rate and is therefore also a quality determinant. Other characteristics can also determine price. MARS was used to investigate which contract features have the most significant impact in this context. Brand and contract duration were found to be the most important product characteristics when it comes to defining products. Thus, other characteristics, such as for electricity single versus double rates, and green versus grey, are not included in the product definition.

Using MARS and the data available in the datasets, the following classification was made for contract duration:

  • 0: duration of less than one year or indefinite (i.e. variable);
  • 1-2: duration between 1 and 2 years;
  • 3: duration between 2 and 3 years;
  • 4-5: duration of more than 3 years (usually 4 or 5).

This classification can be applied for all respondents.

3.3.2 Unit value aggregation of prices

As described above, a product definition delineated by brand and contract duration was chosen. Multiple contract types with the same brand and contract duration occur in the standardised transaction data. Different contract types for the same brand and with the same contract duration must first be aggregated before price indices can be calculated. This section explains how calculation prices and numbers of connections are aggregated.

Calculation prices are aggregated using the unit-value method, which means that the calculation price is calculated as an average of the calculation prices of the underlying contract types. This average is weighted by the number of connections: contract types with many connections are given a greater weight than contract types with fewer connections. For each record corresponding to one brand and one contract duration, the number of connections is derived as the sum of the numbers of connections of the underlying contract types.

3.4 The Geary-Khamis index 

The previous sections explained how individual contracts are aggregated into homogeneous products and how the corresponding prices are defined. Next is the aggregation of price changes across products, which is done in two steps. In line with standard CBS practice, price indices are first calculated per energy company. This is explained below. Subsequently, the price indices by respondent are aggregated into one index for electricity and one index for natural gas. Section 3.5 sets out how this is done.

3.4.1 Motivation

Several methods are available for calculating price indices per respondent (see for example Balk, 2008). Various index methods were considered for electricity and natural gas. These were compared based on their theoretical properties, simplicity, flexibility and compliance with international standards, on how well they fit the data, and on other criteria. In the end, the Geary-Khamis index was chosen, mostly because this is CBS’s standard method for transaction data. A more detailed motivation can be found in CBS (2023a)

3.4.2 Method

Below, the calculation of the Geary-Khamis index is discussed in more detail. This index is calculated according to the following formulas:
\[
P_t=\frac{\left(\sum_{i}p_{it}q_{it}\right)/\left(\sum_{i}\nu_{i}q_{it}\right)}{\left(\sum_{i}p_{i0}q_{i0}\right)/\left(\sum_{i}\nu_{i}q_{i0}\right)}
\]
and
\[
\nu_{i}=\sum_{t}\left(\frac{q_{it}}{\sum_{Z}q_{iZ}}*\:\frac{p_{it}}{P_{t}}\right)
\]
The index calculation requires prices pit and quantities qit, where i represents a product and t represents the period. Price is defined here as the calculation price of a contract for a specific brand with a specific contract duration for a specific month. The quantities qit are the corresponding numbers of connections.

In addition to the index itself, the index calculation Pt produces the implicit prices, vi. which are meant to correct for quality differences. As these are derived from the index calculation, they do not need to be specified. This is called an implicit quality correction. For electricity and natural gas, it is important to consider differences in quality. Contracts from different suppliers and with different durations are clearly subject to different price trends and are therefore not directly comparable.

If all vi's were equal, the method would be identical to a unit-value index, which is a simple method without quality correction. As the Geary-Khamis index can be viewed as an extension of the unit-value method (one that includes a quality correction), the method belongs to the class of quality-adjusted unit-value (QAUV) methods (Von Auer, 2014).

The price index Pt is calculated as the ratio of a turnover change \((\sum_{i}p_{it}q_{it})/(\sum_{i}p_{i0}q_{i0})\) and a volume change \((\sum_{i}\nu_{i}q_{it})/(\sum_{i}\nu_{i}q_{i0})\). Implicit prices vi are calculated as the weighted average of a product’s deflated prices.

The above two formulas for Pt and vi are dependent. The calculation of Pt depends on vi, and conversely vi depends on Pt. Usually, an iterative method is used to calculate Pt and vi. For a more detailed explanation of the Geary-Khamis index and how it is calculated, see Chessa (2016).

3.4.3 Application

This section discusses some of the choices made in applying the Geary-Khamis method.

In general, a distinction can be made between price index methods that calculate an index for two periods (months) and methods that calculate an index for a longer period. The former are called bilateral methods, the latter multilateral methods. Multilateral methods, which are increasingly gaining traction, have the advantage that the outcomes do not depend on the choice of the starting period. The Geary-Khamis method described above is a multilateral method.

To apply a multilateral method, several questions must be answered first. These include:

  • How long should the index series that are derived using Geary-Khamis be?
  • How should consecutive index series be linked together?
  • How should current year index series be created?
  • How is the index defined? (This has an impact on calculation prices).

These questions are answered below.

Index series length
In choosing an index series length, there is a trade-off between including enough prices on the one hand and correctly measuring short-term price changes on the other hand.

A disadvantage of short index series is that relatively few prices are used to create the indices. The implicit quality correction that is applied when using the Geary-Khamis method improves if more prices are available per product. To make an adequate estimate, prices from several periods are needed, and more prices means a longer index. A disadvantage of using long series is that short-term price changes may not be measured as accurately. In a multilateral index, price changes between any two periods are affected by all the periods included in the index calculation. In other words, the change from January to February is also influenced by August. To effectively calculate short-term price movements, index series should not be too long.

For the energy price index, 13-month index series were chosen. These index series start in December of the previous year and continue until the end of the current year. This is in line with standard practice, but there are also practical reasons for choosing this period.

Many elements that are used in the index calculation are set per calendar year, such as tax rates, average consumption and the weights used to combine the index series of electricity and natural gas with those of other products. It therefore makes sense to use annual index series, as this simplifies the calculations.

Linking index series over time
As explained above, it was decided to use 13-month annual index series. To arrive at one long, continuous index series, these annual index series need to be linked. How this is done is explained below.

Short index series start in December of the preceding year and continue until December of the current year at the latest.  Two indices are produced for the base month, December: one for the current calendar year’s series and one for next year’s.

Linking the annual series to create a long multi-year index series is done by applying a multiplicative level correction in December. Suppose, for example, that the current calendar year’s index series ends in December at 1.25 (or 125), and the new index series for the next year starts with 1.00 (or 100). By multiplying the new index series in its entirety by 1.25, it is aligned with the level of the old series. This is how the two indices for December are made equal.

Calculation of the index during the current year
In a given year, the index is calculated starting in December of the preceding year and continues to the current reporting month. The fact that the figures for the preceding months have already been published at the time of calculation, and that these may not be adjusted, presents a complication. For example, if the current reporting period is March, the index is calculated from December to March. In this index, the outcomes for January and February will usually differ from those published earlier. However, as figures that have already been published may not be adjusted, only the new figure for March is published.

The method described above is called the Fixed-Base Expanding Window (FBEW) method, where ‘Fixed-Base’ refers to the fact that the index always starts in a pre-determined base month (December), and ‘Expanding Window’ pertains to the index series getting longer with each passing month. FBEW is an extension method for linking index series over time. FBEW is not the only extension method that can be used, but it is highly useful for processing annual series with annually adjustable parameters, such as average energy consumption. A detailed description of extension methods can be found in Chessa (2021b).

Determining the calculation prices for the base period
As explained above, annual index series comprise 13 months. The first month is December of the preceding year, which is used to link the current, new annual series to the previous index series. Along with the annual base shift, we also update annual consumption according to the latest estimates. This has the following implications for the calculation of the base month:

  • For all 13 months, including December of the preceding year, the current year’s ‘standard’ consumption is applied;
  • For households with dual electricity meters, the ratio between the off-peak and standard rates remains the same over the 13 months. The current year’s ratio is also used for December of the preceding year.

The calculation of the tax credit deserves special attention. For December of the preceding year, the total discount for that year is used, but for subsequent months the total discount is that of the current year. The distribution of the discount between electricity and natural gas is that of the current year for all 13 months. This means that the distribution of the tax credit in December of the preceding year is corrected to align with that of the current year.

This way, the change in the total tax credit is included in the price index, as this is a real change in households’ energy bills. However, as the distribution is kept the same for December- to- January, potential shifts in the distribution of the tax credit between electricity and natural gas will not affect the index.

3.5 Aggregation of energy companies and of electricity and natural gas in the CPI

As explained in the previous section, index series for electricity and natural gas are first created for each individual respondent. These series are then aggregated to arrive at indices for electricity and natural gas for all respondents, which can in turn be aggregated with the index series of the other COICOPs, for example to arrive at the total CPI for all household expenditures. This section explains how this is done.

Index series can be combined by applying weighting. This means that different indices are multiplied by weights to produce an aggregate index. The weights are set at the beginning of the year and remain fixed for twelve months, in accordance with the Laspeyres index.

The aggregation weights for the respondents’ indices are derived from the electricity and natural gas suppliers’ market shares in the base month. These turnover fractions are based on the calculation prices and the numbers of contracts listed in the transaction data. The aggregation weights for product groups are derived from the consumption shares (final expenditure) of the relevant product groups in the national accounts. Within the CPI, a weighting scheme is used for this, with weights adding up to 100,000. In this scheme, electricity and natural gas have weightings of 2,627 and 3,013, respectively (2022). For more information on this, see CBS (2023b) and Eurostat (2018).

As the weighting factors change annually, the weighting is performed on the short 13-month index series. The aggregation across products (using Geary-Khamis) and respondents (using Laspeyres) precedes linking the index series over time, as explained in Section 3.4.3 under ‘Linking index series over time’.

3.6 Other aspects

This section describes relevant aspects of the new method’s application that have not yet been discussed.

3.6.1 Index series by subpopulation: by respondent or by contract duration

Using Geary-Khamis, index series for specific subpopulations can be calculated – for instance by respondent or contract duration – based on the selected microdata.

In Section 5.1, the index series are specified by contract duration (see annex, Chapter 5). It shows that variable-rate contracts increased the most in price, while long-term fixed-rate contracts increased the least. The latter are mostly contracts signed before the major price increases from mid-2021. The index for long-term contracts actually fell slightly due to the government’s tax measures.

3.6.2 Derived series

In addition to the standard publications of consumer price indices, derived series are also published. These are price index series that exclude the effect of changes in the rates of product-related taxes (such as VAT and excise duties on alcohol and tobacco) and subsidies, and of consumption-related taxes (such as the motor vehicle tax). In the derived series for electricity and natural gas, taxes and charges remain constant throughout the year, equal to the values in December of the preceding year (as with consumption).

The following factors remain constant:

  • Renewable energy tax (ODE);
  • Energy tax;
  • Tax credit (‘energy tax refund’);
  • VAT.

The following factors are variable:

  • Rate per kWh or m3;
  • Transport rate;
  • Standing charge.

The derived series are created by making the above adjustments to the microdata when determining the calculation price, as explained in Section 3.2. As with the standard series, the price cap is also included in the derived series.

For the derived series, see Section 5.2. These show that the derived series for electricity and natural gas are higher than the standard series. This is mainly due to the increase in the tax credit in January 2022 compared to December 2021, which is reflected in the standard series but not in the derived series. In January 2023, the tax credit decreased, but the effect of this was less significant.

From July 2022 to December 2022, VAT on energy was reduced from 21% to 9%. This tax reduction has been included in the standard series but not in the derived series. However, the effect of omitting the tax reduction from the derived series turned out to be limited compared to the effect of the variable components that have been included.

3.6.3 Switching to the new method

In June 2023, there was a switch from the old method (using ACM data) to the new method (using transaction data).

The figures up to May 2023 have not been adjusted as part of the transition; they are the outcomes produced by the old method. Switching methods can be done in several ways. In this case, arriving at an accurate long-term level had top priority, which is why an annual-overlap method was chosen. This method ensures that the long-term price index will be at about the same level as if the new method had already been introduced in 2020, before energy prices started to rise.

As a result, price increases are not counted twice. As price developments for new contracts have a delayed impact on existing contracts, the old method is ahead of the new one. Price increases that have already been accounted for by the old method should not be included again after the switch to the new method.

The annual-overlap method is explained in more detail below. The indices that were published until May 2023 were created using the old method; the ones published from June 2023 are based on the new method. These new indices are multiplied by a correction factor to align them with the old index. This factor is calculated by dividing the average level of the old index over the 12 months of 2020 by the average level of the new index.

Figures in the annex Section 5.3 illustrate how the switching method would have been used if it had been applied from April 2023 (a hypothetical scenario). Until March 2023, the old method is used. After that, the index series mostly follows the new method.

3.6.4 Alternative index methods

Besides the Geary-Khamis index method, several alternative index methods for energy transaction data have been explored, including the unit-value index. This is a simple method that measures the development of the average calculation price weighted using the number of contracts. Unit-value indices were calculated for electricity and natural gas, without differentiating by contract type and respondent.

Figures in Section 5.4 show that these unit-value indices are higher than the Geary-Khamis indices for electricity, natural gas and all expenditures. This is reflected not only in the indices, but also in the year-on-year changes for those indices (see Figures in Section 5.5). There is a logical explanation for this. In 2022, many households saw their fixed-term contracts expire. As virtually no new fixed-term contracts were offered in 2022, these households were then forced to switch to new variable-rate contracts. While the associated price increases are included in the unit-value index, they are not directly reflected in the Geary-Khamis index. The unit-value index calculates the average price across all contracts, which increases when cheap fixed-term contracts are replaced by expensive variable-rate contracts. The effects of this are less significant in the Geary-Khamis indices because of the quality correction that is applied.

Geary-Khamis is applied in production because it is more sophisticated than the unit-value method. As the analysis above shows, comparing methods can provide new insights. The intention is therefore to calculate a unit-value index in addition to the Geary-Khamis index, for internal evaluation.