Valuing the quantity and quality of product variety to consumers

Hedonic regression methods involve estimating shadow prices for the characteristics which constitute a product. These estimates can be used to value and adjust for changes in a product’s characteristics over time.

The BLS is a standout here with around one-third of the index adjusted using hedonic methods (BLS 2017). This high proportion primarily reflects the use of hedonics for owner-occupier rents which constitutes a significant share of the index. Globally, hedonic adjustments are used in a wide range of areas including used cars, clothing, footwear, household appliances and electronics. Amongst the countries they surveyed, Wells and Restieaux (2014) found that the following used hedonic adjustments: Australia, Canada, New Zealand, the USA, Germany, Sweden, the UK and Switzerland.

A notable example is the so-called Apple-Cinnamon Cheerios Wars involving Timothy Bresnahan and Jerry Hausman (see Bresnahan and Gordon 1996; Bresnahan 1997).

Note that the expenditure shares of new and disappearing goods will be influenced by the particular prices that retailers choose to charge in the respective periods. There is some evidence that new and disappearing products are priced differently from existing products (see Melser and Syed 2016, 2017). For example, disappearing products are subject to runout sales. The extent to which these expenditure shares are influenced by such life cycle pricing factors is an important question, but the one that we leave aside in this paper.

The infinite reservation price is a feature of the CES functional form which has raised questions about its ability to accurately value the gains from variety. In the closely related logit discrete choice approach to demand estimation it has been argued that the logit functional form—which has similar properties to the CES—tends to exaggerate the gains from variety (Ackerberg and Rysman 2005).

This index number formula is attractive, while it is not superlative, as defined in Diewert (1976) because the CES preferences structure for which it is exact is not sufficiently flexible, Diewert (1978) argued that the index was pseudo-superlative in that it approximated superlative indexes very closely.

More specifically, price indexes can be distorted by the differential weight given to price rises and falls as a result of sales. The price decline associated with a sale is given a large weight as purchases spike at this time. However, the price rise at the end of a sale is given less weight than the fall. This is because purchases in the period after the sale are diminished due to the stockpiling that occurred during the sale. This process can lead to significant chain index ‘drift’. Additionally, Melser and Webster (2017) document chain drift arising as a result of the large price falls observed in the last period in which a product is available for sale.

There is somewhat less concerned with the correlation of demand and supply shocks in our ‘goods’ demand framework than in alternative frameworks such as the logit discrete choice ‘characteristics’ approach. This is because we are not modelling demand as a function of the characteristics of a product, but are effectively using product fixed effects. This means that, unlike in the discrete choice approach, we do not have to worry about unobserved product characteristics possibly inducing correlation between supply and demand errors. All product characteristics are encompassed in the product-specific effect. Moreover, these characteristics do not change over time as we are considering products at the barcode level—any meaningful change in characteristics necessitates a change in barcode. However, as we note later, concerns about endogeneity remain, particularly with regard to advertising and in-store promotions which are not included in our model, but may change demand independent of price.

We are thankful to a referee for their insightful comments on this issue.

Our data are comprehensively summarized, for each city and product category, in Table A.1 of Online Appendix.

Complete results, by product and city, are reported in Table A.2 of the Online Appendix.

Table A.2 in Online Appendix provides data for all city–product combinations for the statistics shown in Table 2.

The full results, for each city and product combination, are shown in Table A.3 of Online Appendix.

In the bisquare weighting function we used the weights that had the form \(w_{it} = I(|r|<1) ( 1 - r_{it}^2 )^2\), where \(r_{it} = u_{it} / ( 4.685 * \hbox {MAD}/0.6745 )\) and MAD is the mean absolute deviation. The numbers represent various widely used tuning parameters, while \(I(|r|<1)\) is an indicator function which is 1 if the statement is true and 0 otherwise.

Full results, for each product category and city, are given in Table A.4 of Online Appendix.

While using a nested CES approach will likely better fit the data the nature of the nesting structure is somewhat contentious. Broda and Weinstein (2010) used brands as their subgroups. However, it is not clear that brand—as opposed to other characteristics of the product, such as size and flavour—is the most important feature across which consumers substitute. Our data did not include product grouping characteristics. Hence, we did not pursue the nested approach. The nature of consumer substitution amongst product groupings, however, is an important area of future research as it will allow more realistic applications of the CES functional form to expenditure data.

The effects on the variety-quality component are just the mirror image of this because the variety-quality and variety-quantity effects add up to the overall variety effect which is fixed.

The results for a span of 24 months are shown in Table A.6 of Online Appendix and are broadly similar to those using a 13-month span.

This is given in Table A.3 of Online Appendix.