Abstract
In recent years permutation testing methods have increased both in number of applications and in solving complex multivariate problems. When available permutation tests are essentially of an exact nonparametric nature in a conditional context, where conditioning is on the pooled observed data set which is often a set of sufficient statistics in the null hypothesis. Whereas, the reference null distribution of most parametric tests is only known asymptotically. Thus, for most sample sizes of practical interest, the possible lack of efficiency of permutation solutions may be compensated by the lack of approximation of parametric counterparts. There are many complex multivariate problems, quite common in empirical sciences, which are difficult to solve outside the conditional framework and in particular outside the method of nonparametric combination (NPC) of dependent permutation tests. In this paper we review such a method and its main properties along with some new results in experimental and observational situations (robust testing, multi-sided alternatives and testing for survival functions).
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abd-Elfattah, E.F., Butler, R.W.: The weighted log-rank class of permutation tests: P-values and confidence intervals using saddlepoint methods. Biometrika 94, 543–551 (2007)
Basso, D., Pesarin, F., Salmaso, L., Solari, A.: Permutation Tests for Stochastic Ordering and ANOVA: Theory and Applications in R. Lecture Notes in Statistics, vol. 194. Springer, New York (2009)
Basso, D., Pesarin, F.: Exact permutation testing for two-sample survival functions. Technical Report N. 13, Department of Management and Engineering, University of Padua (2011, submitted)
Bertoluzzo, F., Pesarin, F., Salmaso, L.: On multi-sided permutation tests. J. Stat. Plan. Inference (2011, forthcoming)
Birnbaum, A.: Combining independent tests of significance. J. Am. Stat. Assoc. 49, 559–574 (1954)
Birnbaum, A.: Characterizations of complete classes of tests of some multiparametric hypotheses, with applications to likelihood ratio tests. Ann. Math. Stat. 26, 21–36 (1955)
Cox, D.R., Hinkley, D.V.: Theoretical Statistics. Chapman and Hall, London (1974)
David, H.A.: The beginnings of randomization tests. Am. Stat. 62, 70–72 (2008)
Edgington, E.S., Onghena, P.: Randomization Tests, 4th edn. Chapman and Hall/CRC, London (2007)
Goggin, M.L.: The “Too few cases/too many variables” problem in implementation research. West. Polit. Q. 39, 328–347 (1986)
Good, P.: Permutation, Parametric, and Bootstrap Tests of Hypotheses, 3rd edn. Springer, New York (2005)
Hoeffding, W.: The large-sample power of tests based on permutations of observations. Ann. Math. Stat. 23, 169–192 (1952)
Lehmann, E.L.: Consistency and unbiasedness of certain nonparametric tests. Ann. Math. Stat. 22, 165–179 (1951)
Lehmann, E.L.: Parametric versus nonparametrics: two alternative methodologies. J. Nonparametr. Stat. 21, 397–405 (2009)
Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Springer, New York (2005)
Lehmann, E.L., Scheffé, H.: Completeness similar regions, and unbiased estimation. Sankhyā 10, 305–340 (1950)
Lehmann, E.L., Scheffé, H.: Completeness similar regions, and unbiased estimation—part II. Sankhyā 15, 219–236 (1955)
Mehta, C.R., Patel, N.R.: A network algorithm for the exact treatment of the 2×K contingency table. Commun. Stat., Simul. Comput. 9, 649–664 (1980)
Mehta, C.R., Patel, N.R.: A network algorithm for performing Fisher’s exact test in r×c contingency tables. J. Am. Stat. Assoc. 78, 427–434 (1983)
Mielke, P.W., Berry, K.J.: Permutation Methods. A Distance Function Approach, 2nd edn. Springer, New York (2007)
Moder, K., Rasch, D., Kubinger, K.D.: Don’t use the two-sample t-test anymore! In: Ermakov, S.M., Melas, V.B., Pepelyshev, A.N. (eds.) Proceedings of the 6th St. Petersburg Workshop on Simulation, pp. 258–264 (2009)
Pesarin, F.: Multivariate Permutation Tests: With Application in Biostatistics. Wiley, Chichester (2001)
Pesarin, F.: Extending permutation conditional inference to unconditional one. Stat. Methods Appl. 11, 161–173 (2002)
Pesarin, F., Salmaso, L.: Finite-sample consistency of combination-based permutation tests with application to repeated measures designs. J. Nonparametr. Stat. (2009). doi:10.1080/10485250902807407
Pesarin, F., Salmaso, L.: Permutation Tests for Complex Data. Theory, Applications and Software. Wiley Series in Probability and Statistics. Wiley, Chichester (2010)
Pesarin, F., Salmaso, L.: On the weak consistency of permutation tests. J. Stat. Plan. Inference (2011, forthcoming)
Romano, J.P.: On the behaviour of randomization tests without group variance assumption. J. Am. Stat. Assoc. 85, 686–692 (1990)
Scheffé, H.: Statistical inference in the non-parametric case. Ann. Math. Stat. 14, 305–332 (1943)
Strasser, H., Weber, Ch.: The asymptotic theory of permutation statistics. Math. Methods Stat. 8, 220–250 (1999)
Westfall, P.H., Young, S.S.: Resampling-Based Multiple Testing: Examples and Methods for p-Values Adjustment. Wiley, New York (1993)
Zhang, Y., Rosenberger, W.F.: On asymptotic normality of the randomization-based logrank test. J. Nonparametr. Stat. 17, 833–839 (2005)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pesarin, F., Salmaso, L. A review and some new results on permutation testing for multivariate problems. Stat Comput 22, 639–646 (2012). https://doi.org/10.1007/s11222-011-9261-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-011-9261-0