Top

Empirical Economics

Published in:

11-02-2017

Combination of “combinations of p values”

Authors: Lan Cheng, Xuguang Simon Sheng

Published in: Empirical Economics | Issue 1/2017

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

We investigate the impact of an uncertain number of false individual null hypotheses on commonly used p value combination methods. Under such uncertainty, these methods perform quite differently and often yield conflicting results. Consequently, we develop a combination of “combinations of p values” (CCP) test aimed at maintaining good power properties across such uncertainty. The CCP test is based on a simple union–intersection principle that exploits the weak correspondence between two underlying p value combination methods. Monte Carlo simulations show that the CCP test controls size and closely tracks the power of the best individual methods. We empirically apply the CCP test to explore the stationarity in real exchange rates and the information rigidity in inflation and output growth forecasts.

previous article Demand for household sanitation in India using NFHS-3 data

next article Determining the number of factors after stationary univariate transformations

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Harvey et al. (2009) used the union–intersection method to combine two unit root tests, and their combination approach comes close to exploiting the available information efficiently as shown by Müller (2009). Dumitru and Urga (2012) used the similar strategy across nine procedures and across sampling frequencies to minimize spurious jump detection in financial assets. Bayer and Hanck (2013) applied a similar method to combine non-cointegration tests.

These two methods were introduced to panel unit root literature independently by Maddala and Wu (1999) and Choi (2001). See Costantini and Lupi (2013) and Sheng and Yang (2013a) for recent applications.

Hanck (2013) advocated Simes’ method in testing panel unit roots.

See Lehmann and Romano (2005, p.350) for further discussions on the power of Bonferroni procedure.

In cases where \(g(\gamma ;n)\) has a closed-form formula, e.g., combining Tippett’s and Simes’ methods, we find that a second-order polynomial is sufficient in approximating the function \(g(\gamma ;n)\). We also conducted the simulation study by combining Tippett’s and Simes’ methods. The simulated values of \(\gamma \) are virtually the same as the ones obtained analytically by solving the equation \(g(\gamma ;n)\), implying that the Monte Carlo simulation methods are reliable.

We experimented with different values of K and step size h. Our results show that the values of \(\gamma \) remain almost the same across different values of \(K=10\), 20, and 40. For the detailed results, see the online appendix. In our simulation, we used \(K=20\), \(h=0.01\), and \(M=10,000\).

In separate simulation studies not shown here, we combined the two methods from the same group. We find that Tippett’s and Simes’ methods are almost perfectly corresponded, and Fisher’s and Stouffer’s methods are highly corresponded. To save space, these results are not reported here. They are available upon request.

In separate simulation studies, we also combined the two methods within the same group. We find that the power of the individual methods is virtually indistinguishable from that of the CCP test. Thus, the gains from combining two highly correlated methods are very small. For this reason, we do not recommend combining the two methods coming from the same group.

The power of the CCP test when combining other methods, that is, Fisher’s and Simes’, Fishers’ and Tippett’s, and Stouffer’s and Tippett’s, is qualitatively similar to that of combining Stouffer’s and Simes’ and is thus not reported here for the sake of brevity.

The 82% is calculated as the ratio of the number of rejections of the null by the CCP test (18) over the number of cases where two constituent methods give conflicting results (22).

Andrade P, Le Bihan H (2013) Inattentive professional forecasters. J Monet Econ 60:967–982CrossRef

Bai J, Ng S (2004) A PANIC attack on unit roots and cointegration. Econometrica 72:1127–1177CrossRef

Baltagi B (2008) Econometric analysis of panel data, 4th edn. Wiley, Hoboken

Bayer C, Hanck C (2013) Combining non-cointegration tests. J Time Ser Anal 34:83–95CrossRef

Birnbaum A (1954) Combining independent tests of significance. J Am Stat Assoc 49:559–574

Choi I (2001) Unit root tests for panel data. J Int Money Financ 20:249–272CrossRef

Choi I, Chue TK (2007) Subsampling hypothesis tests for nonstationary panels with applications to exchange rates and stock prices. J Appl Econom 22:233–264CrossRef

Coibion O, Gorodnichenko Y (2012) What can survey forecasts tell us about informational rigidities? J Polit Econ 120:116–159CrossRef

Coibion O, Gorodnichenko Y (2015) Information rigidity and the expectations formation process: a simple framework and new facts. Am Econ Rev 105:2644–2678CrossRef

Costantini M, Lupi C (2013) A simple panel-CADF test for unit roots. Oxford Bull Econ Stat 75:276–296CrossRef

Dufour J-M, Khalaf L, Voia M (2015) Finite-sample resampling-based combined hypothesis tests, with applications to serial correlation and predictability. Commun Stat—Simul Comput 44:2329–2347CrossRef

Dufour J-M, Torres O (1998) Union-intersection and sample-split methods in econometrics with applications to MA and SURE Models. In: Giles D, Ullah A (eds) Handbook of applied economic statistics, vol 14. Marcel Dekker Inc., New York, pp 465–505

Dumitru A-M, Urga G (2012) Identifying jumps in financial assets: a comparison between nonparametric jump tests. J Bus Econ Stat 30:242–255CrossRef

Fisher RA (1932) Statistical methods for research workers, 4th edn. Oliver and Boyd, London

Hanck C (2013) An intersection test for panel unit roots. Econom Rev 32:183–203CrossRef

Harvey DI, Leybourne SJ, Robert Taylor AM (2009) Unit root testing in practice: dealing with uncertainty over the trend and initial condition. Econom Theory 25:587–636CrossRef

Hedges LV, Olkin I (1985) Statistical methods for meta-analysis. Academic Press, San Diego

Lahiri K, Sheng X (2008) Evolution of forecast disagreement in a Bayesian learning model. J Econom 144:325–340CrossRef

Lehmann EL, Romano JP (2005) Testing statistical hypotheses, 3rd edn. Springer, New York

Lopez C (2008) Evidence of purchasing power parity for the floating regime period. J Int Money Financ 27:156–164CrossRef

Loughin TM (2004) A systematic comparison of methods for combining \(p\) values from independent tests. Comput Stat Data Anal 47:467–485CrossRef

Maddala GS, Wu S (1999) A comparative study of unit root tests with panel data and a new simple test. Oxford Bull Econ Stat 61:631–652CrossRef

Mankiw G, Reis R (2002) Sticky information versus sticky prices: a proposal to replace the New Keynesian Phillips curve. Quart J Econ 117:1295–1328CrossRef

Müller UK (2009) Comment on unit root testing in practice: dealing with uncertainty over the trend and initial condition. Econom Theory 25:643–648CrossRef

Neuhäuser M (2003) Tests for genetic differentiation. Biom J 8:974–984CrossRef

O’Connell PGJ (1998) The overvaluation of purchasing power parity. J Int Econ 44:1–19CrossRef

Pesaran MH (2007) A simple panel unit root test in the presence of cross-section dependence. J Appl Econom 22:265–312CrossRef

Roy SN (1953) On a heuristic method of test construction and its use in multivariate analysis. Ann Math Stat 24:220–238CrossRef

Sheng X, Yang J (2013a) Truncated product methods for panel unit root tests. Oxford Bull Econ Stat 75:624–636CrossRef

Sheng X, Yang J (2013b) An adaptive truncated product method for combining dependent p values. Econ Lett 119:180–182CrossRef

Simes RJ (1986) An improved Bonferroni procedure for multiple tests of significance. Biometrika 73:751–754CrossRef

Sims CA (2003) Implications of rational inattention. J Monet Econ 50:665–690CrossRef

Stouffer Samuel A, Suchman Edward A, DeVinney Leland C, Star Shirley A, Williams Robin M Jr (1949) Studies in social psychology in world war II: The American soldier, Adjustment during Army Life. vol 1. Princeton University Press, Princeton

Tippett LHC (1931) The method of statistics. Williams & Norgate, London

Westberg M (1985) Combining independent statistical tests. The Statistician 34:287–296CrossRef

Zaykin DV, Zhivotovsky LA, Westfall PH, Weir BS (2002) Truncated product method for combining p-values. Genet Epidemiol 22:170–185CrossRef

Title: Combination of “combinations of p values”
Authors: Lan Cheng
Xuguang Simon Sheng
Publication date: 11-02-2017
Publisher: Springer Berlin Heidelberg
Published in: Empirical Economics / Issue 1/2017
Print ISSN: 0377-7332
Electronic ISSN: 1435-8921
DOI: https://doi.org/10.1007/s00181-017-1230-9

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 1/2017

The role of indicator selection in nowcasting euro-area GDP in pseudo-real time

Demand for household sanitation in India using NFHS-3 data

Measuring uncertainty and assessing its predictive power in the euro area

Forecasting economic activity by Bayesian bridge model averaging

Special issue on forecasting, use of survey data on expectations, and panel data applications: editors’ introduction

The national segmentation of euro area bank balance sheets during the financial crisis