Power Calculations and Critical Values for Two-Stage Nonparametric Testing Regimes

Interim analysis techniques for clinical trials provide improved power with smaller average sample sizes. These techniques crucially require multivariate probability calculations for determining critical values. Most existing techniques rely on multivariate normal approximations to the joint null distribution of test statistics evaluated on potential interim and full data sets. More accurate critical values for nonparametric testing with an interim analysis are given, using a new multivariate Cornish–Fisher expansion. While earlier authors demonstrated that such an expansion is possible, it has never been implemented before this manuscript. Generally, the superior accuracy of power calculations via an Edgeworth series is demonstrated. Example calculations giving sample sizes from desired power are provided. Calculations are implemented in an R package.

The authors thank Yuzo Maruyama for an English summary of Takeuchi (1978) and Todd Kuffner for pointers to references Takemura and Takeuchi (1988) and Takemura and Takeuchi (1988). John Kolassa thanks NSF DMS 1712839 for partial support for this project. The authors thank editors and referees for helpful comments.

Authors and Affiliations

1. Rutgers, The State University of New Jersey, Piscataway, NJ, USA

   John Kolassa & Xinyan Chen

2. Anheuser-Busch Companies, LLC, New York, NY, USA

   Yodit Seifu

3. Bristol-Myers Squibb, Berkeley Heights, NJ, USA

   Dewei Zhong

Corresponding author

Correspondence to John Kolassa .

Editors and Affiliations

1. School of Statistics, Beijing Normal University, Beijing, China

   Mengxi Yi

2. Department of Mathematics and Statistics, University of Jyväskylä, Jyväskylä, Finland

   Klaus Nordhausen

Appendix 1: A Bivariate Recursion for Exact Probabilities

The univariate recursion is constructed by counting the number of data set orderings leading to the statistic value and decomposing them into orderings based on one fewer value from the first group and one fewer value from the second group. The bivariate recursion is similar to the above univariate recursion and is similar to that of Wilding et al. (2011).

Let b(u₁, u₂, m₁, n₁, m₂, n₂) represent the number of orderings of (4) for which U₁ = u₁ and U₂ = u₂. This number is zero if any of the sample sizes m₁, n₁, m₂, n₂ is negative, if either statistic value is negative, or if either statistic value is larger than its maximum value. It is also zero if both additional sample sizes for stage 2 are zero but the second statistic value exceeds the first. If all sample sizes are zero, then the sums in (1) and (3) are empty, and both statistic values are zero; hence b(0, 0, 0, 0, 0, 0) = 1. These end conditions are given by

Otherwise, the number of rearrangements of the data (4) giving rise to statistic values u₁ and u₂ are the sum of four contributions. First, add those with sample sizes m₁ − 1, n₁, m₂, n₂, with an additional value from the first group in the first sample that exceeds all values in the sample, and hence leaves the statistic value unchanged. Second, add those with sample sizes m₁, n₁ − 1, m₂, n₂, with an additional value from the second group in the first sample that exceeds all values in the sample, and hence increases the first statistic by m₁ and increases the second statistic by m₁ + m₂. Third, add those with sample sizes m₁, n₁, m₂ − 1, n₂, with an additional value from the first group in the second sample added that exceeds all values in the sample, and hence leaves the statistic value unchanged. Fourth, add those with sample sizes m₁, n₁, m₂, n₂ − 1, with an additional value from the second group in the second sample added that exceeds all values in the sample, and hence leaves U₁ unchanged, and increases the second statistic by m₁ + m₂. This leads to the following recursion:

Numerical examples in Figs. 1 and 2 exhibit comparisons of probability approximations to bivariate probabilities and quantiles for U = (U₁, U₂), to the exact values.

Appendix 2: A Continuous Example with Nonzero Skewness

Our aim in determining the expansion for c₂ is to apply the techniques to Wilcoxon testing, but the same quantile approximation may be used more generally. Before application to the Wilcoxon statistic, which is somewhat atypical, because the third cumulants are zero, leading to a less dramatic effect, and because the Wilcoxon statistic is discrete, and hence lacks the continuity that the technique was developed for. Instead, we present a more general example consisting of a continuous distribution with nonzero third order cumulants.

Consider Y₁, Y₂, Y₃ independent exponentials. Let U₁ = Y₁ + Y₃, and U₂ = Y₂ + Y₃. Figure 3 compares Edgeworth (E) or Cornish–Fisher (CF), normal (N), and Monte Carlo (MC, taken with 500,000 samples and treated as the truth). Panels are:

1. (a)

   compares difference of E and N upper tail univariate probability approximation from the MC approximation, as a function of the MC approximation.

   

   Exponential example, sample size 1. (a) Absolute error for approximations. (b) Difference in absolute error. (c) True bivariate probability. (d) Second ordinate approximation range

   Full size image

2. (b)

   compares error of E and N upper tail bivariate probability approximation, as a function of the ordinate.

3. (c)

   Gives contours of MC upper tail probabilities.

4. (d)

   Represents CF and N approximation to upper tail, vs. MC value. CF and N values exhibit some dependence on target for first univariate tail, and so are represented as a range.

Note from panel b that the Edgeworth approximation fails to dominate the normal approximation only for a narrow band between the contours marked 0 in the middle of the plot.

Reprints and permissions

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

Policies and ethics