Stopping for efficacy in single-arm phase II clinical trials

2.1. Simon's two-stage design

According to Simon's design [26], $n_1$ patients are recruited in the first stage. Let x be the number of responses in the first stage. If $x\le r_1$, then the trial is stopped for futility, and the drug is identified as ineffective. If more than $r_1$ responses are observed in the first stage, then the study proceeds to the second stage, and $n_2$ ( $n_1+n_2=n$) more patients are recruited. If the total number of responses observed from the two-stages is more than r, then $H_0$ is rejected. Simon's design is indexed by the four numbers $r_1$, r, $n_1$ and n, and is referred as a ‘ $r_1/n_1$ r/n’ design. The probability that the design would not proceed to the second stage or the probability of early termination $(PET)$ is

The probability of not recommending a drug to proceed further is

where $b(.)$ and $B(.)$ are the probability mass function and cumulative distribution function of the binomial distribution, respectively. The expected sample size is then obtained as

Both $\overline{R}(p)$ and $E(\mathrm{N}|p)$ are expressed as a function of the true response rate p. Let α and β be the type I and type II error probabilities, respectively. For predetermined $p_0$, $p_1$, α and β, an acceptable design should satisfy the error constraints $\overline{R}(p_0)\ge 1-\alpha$ and $\overline{R}(p_1)\le \beta$. Let Ω be the set of all such designs. Simon's optimal design is the one in Ω that has the minimum expected sample size $E(\mathrm{N}|p_0)$ under the null response rate. On the other hand, the minimax design is the one that has the smallest $E(\mathrm{N}|p_0)$ among those designs in Ω with the smallest n.

2.2. Mander and Thomson's design

Like Simon's design, Mander and Thompson's design [17] also recruits $n_1$ patients in the first stage and have a similar stopping rule in the second stage. The only exception is that the design stops for efficacy if $x>r_2$, where x is the number of responses in the first stage. The design rejects $H_0$ and proceed to the second stage only if $r_1<x\le r_2$. It recruits $n_2$ patients in the second stage, where $n=n_1+n_2$. This design is indexed by five numbers and will be referred as ‘ $(r_1$ $r_2)/n_1$ r/n’ design. The probability of early termination for this design is

The probability of not rejecting the null hypothesis is given as

Then the expected number of patients is

As before we require the error probabilities to be constrained by $\overline{R}(p)\ge 1-\alpha$ and $\overline{R}(p)\le \beta$. Suppose ${\mathrm{\Omega }}_E$ be the set of all such designs. Mander and Thompson used four optimality criteria and named those designs as:

1. $H_0-optima{l}_E$: $E(\mathrm{N}|p_0)$ is the smallest.

2. $H_0-minima{x}_E$: $E(\mathrm{N}|p_0)$ is the smallest among those designs in ${\mathrm{\Omega }}_E$ with the smallest n.

3. $H_1-optima{l}_E$: $E(\mathrm{N}|p_1)$ is the smallest.

4. $H_1-minima{x}_E$: $E(\mathrm{N}|p_1)$ is the smallest among those designs in ${\mathrm{\Omega }}_E$ with the smallest n.

2.3. Lin and Shih's design

Lin and Shih [16] emphasised the uncertainty that investigators often face while choosing the target response rate. They proposed an adaptive two-stage design with two choices of the target response rates $p_1$ and $p_2$ ( $p_0<p_1<p_2$). This design has a fixed sample size $n_1$ in the first stage, like Simon's design, but the sample size in the second stage depends on the number of responses observed in the first stage. Let x be the number of responses observed in the first stage. Then Lin and Shih's design proceeds as follows:

1. If $x\le s_1$, stop the trial for futility.

2. If $s_1<x\le r_1$, power the study at $(1-{\beta }_1)$ for $p=p_1$ and enter $m_2=m-n_1$ additional patients into the study. Reject the null hypothesis ( $p\le p_0$) if the total number of responses $>s$ out of m patients.

3. If $x>r_1$, power the study at $(1-{\beta }_2)$ for $p=p_2$ and enter $n_2=n-n_1$ additional patients into the study. Reject the null hypothesis ( $p\le p_0$) if the total number of responses $>r$ out of n patients.

This design is indexed by the seven numbers $s_1$, $r_1$, $n_1$, s, m, r and n, and we refer it as ( $s_1/r_1/n_1$) (s/m) (r/n). The probability of terminating the study at the first stage is

The probability of not recommending the drug is

Finally, the expected sample size is given as

For predetermined $p_0$, $p_1$, $p_2$, α, ${\beta }_1$ and ${\beta }_2$, an acceptable design must satisfy the following error constraints

Although not required, it is reasonable to let ${\beta }_1\ge {\beta }_2$ in practice because we would like to have higher power for detecting more improvement of the new therapy ( $p_2$ versus $p_0$). From the feasibility aspect, we need to compromise the power somewhat for less improvement ( $p_1$ versus $p_0$) because the sample size cannot be too large for a phase II study. Let ${\mathrm{\Omega }}^{\prime }$ be the set of designs that satisfy the error constraints. Lin and Shih [16] proposed designs based on the following optimality conditions.

1. Optimality type 1 (O1): $E(N|p_0)$ is the smallest.

2. Optimality type 2 (O2): $max\{\mathrm{E}(N|p_i);i=0,1,2\}$ is the smallest.

3. Optimality type 3 (O3): $max(n,m)$ is the smallest among all fesible solutions and $E(N|p_0)$ is the smallest among such solutions.

4. Optimality type 4 (O4): $max(n,m)$ is the smallest among all fesible solutions and $max\{E(N|p_i);i=0,1,2\}$ is the smallest among such solutions.

O1 and O3 are extensions to Simon's ‘optimal design’ and ‘minimax design’ criteria. That is, if $p_1=p_2$ and ${\beta }_1={\beta }_2$, then $r_1=s_1$, s = r, m = n, and O1 and O3 reduces to Simon's optimal and minimax designs, respectively.

2.4. Proposed design

Like Simon's design, the design by Lin and Shih [16] considers only futility as a reason for early stopping in the first stage. The design has a larger expected sample size if the proposed drug is effective. We propose an adaptive design that considers both futility and efficacy as reasons for early stopping. Although it is easier to set the uninteresting rate $p_0$, the same is not right for $p_1$, the alternate response rate. The challenge in selecting a single alternate response rate can be minimised by the specification of two alternate response rates instead. Therefore, as in Lin and Shih's design, the proposed design also considers two alternate response rates. Based on the response in the first stage, it determines which target response rate is to be tested and engage patients in the second stage accordingly. Let x be the number of responses in the first stage out of the $n_1$ patients enrolled. The proposed design then proceeds as follows:

1. If $x\le s_1$, stop the trial for futility.

2. If $s_1<x\le r_1$, power the study at $(1-{\beta }_1)$ for $p\ge p_1$ and enter $m_2=m-n_1$ additional patients into the study. Reject the null hypothesis ( $p\le p_0$) if the total number of responses $>s$ out of m patients.

3. If $r_1<x\le c_1$, power the study at $(1-{\beta }_2)$ for $p\ge p_2$ and enter $n_2=n-n_1$ additional patients into the study. Reject the null hypothesis ( $p\le p_0$) if the total number of responses $>r$ out of n patients.

4. If $c_1<x\le c_2$, stop for efficacy in the first stage and reject the null hypothesis for $H_1$: $p\ge p_1$.

5. If $x>c_2$, stop for efficacy and reject the null hypothesis for $H_1$: $p\ge p_2$.

This design is indexed by the nine parameters $s_1$, $r_1$, $c_1$, $c_2$, $n_1$, s, m, r and n, where $0\le s_1<r_1<c_1<c_2\le n_1$. It is referred as ( $s_1/r_1/c_1/c_2/n_1$) (s/m) (r/n). The probability of early termination in the proposed design is

Then the probability of not rejecting $H_0$ is

Finally, the expected sample size is

where $w_1=PET(p)$ is the probability that the design would stop at the first stage, $w_2=\{B(n_1,p,r_1)-B(n_1,p,s_1\}$ is the probability that the design would proceed to the second stage and test for the target response rate $p_1$ and $w_3=\{B(n_1,p,c_1)-B(n_1,p,r_1)\}$ is the probability that the design would proceed to the second stage and test for the target response rate $p_2$. Note that $w_1+w_2+w_3=1$. For a predetermined level of ϕ= $(p_0,p_1,p_2,\alpha ,{\beta }_1,{\beta }_2)$, we find the optimal designs under the four optimality criteria proposed by Lin and Shih [16], and these designs satisfy the error constrains in (7). From Equation (8), we can see that the probability of not rejecting $H_0$ does not depend on $c_2$. So the value of $c_2$ will be determined based on the following hypothesis

2.5. Algorithm

Now we discuss the algorithm to find the optimal solutions. For specified values of $p_0$, $p_1$, $p_2$, α, ${\beta }_1$ and ${\beta }_2$, at first feasible values of m and n are calculated. The range of m is calculated as 0.85 to 1.5 times the sample size of the one-stage design for testing $p_0$ versus $p_1$ at the significance level α and power $(1-{\beta }_1)$. This strategy is similar to that was used by Simon [26] in his algorithm. To ensure that the range's values are integers, the lower value is taken as the largest integer that is less than or equal to the previously calculated lower value. Similarly, the range's upper value is taken as the smallest integer that is greater than or equal to the previously calculated upper value of the range. Similarly, feasible range of n is calculated from the sample size of the design for testing $p_0$ versus $p_2$ at the significance level α and power $(1-{\beta }_2)$.

For each combination of m, n and $n_1$ in the range $\{1,\min (m,n)-1\}$, at first $c_2$ is calculated as the $[\{1-({\beta }_1-{\beta }_2)\}$th quantile of $n_1$ $-1]$. Following this, $s_1$ is taken in the range $(0,$ $c_2)$, $r_1$ in the range $(s_1+1,$ $c_2)$, $c_1$ in the range $(r_1+1,$ $c_2)$, s in the range $(s_1+1,$ $m)$, and r in the range $(r_1+1,$ $m)$. Then we find the designs with parameters $s_1$, $r_1$, $c_1$, $c_2$, $n_1$, s, m, r, and n those satisfy the error constraints associated with ${\beta }_1$ and ${\beta }_2$. Then we check which remaining designs fulfill the error constraint associated with α and call them ‘set of feasible designs’. For every feasible design, the expected sample sizes $E(\mathrm{N}|p_0)$, $E(\mathrm{N}|p_1)$, and $E(\mathrm{N}|p_2)$ are calculated.

To find the designs of optimality type 1, we arrange the designs in order of $E(\mathrm{N}|p_0)$. For optimality type 2, a vector of $\max \{E(\mathrm{N}|p_i);i=0,1,2\}$ is calculated and the designs are ordered accordingly. For finding the design of optimality type 3 and 4, first a column containing $\max (m,n)$ is calculated. For optimality 3, the designs are first ordered according to $E(\mathrm{N}|p_0)$ and then $\max (m,n)$. Finally, for optimality type 4, the ordering is done first according to $\max \{E(\mathrm{N}|p_i);i=0,1,2\}$ and then $\max (m,n)$. In every optimality criterion, the first ten ordered designs are kept and the first design is called the optimality design under that optimality criterion. All the designs are obtained numerically using a self-written code in R through parallel computation.

3.1. Proposed design

We have considered $p_0$ from 0.05 to 0.50 with regular interval 0.05 and the differences $(p_1-p_0)$ and $(p_2-p_0)$ are kept fixed as 0.15 and 0.20, respectively. Level of significance α is taken as $5\mathrm{\%}$ and $10\mathrm{\%}$ and the maximum type II error ${\beta }_1$ and ${\beta }_2$ are kept fixed as $20\mathrm{\%}$ and $10\mathrm{\%}$ for the target response rate $p_1$ and $p_2$, respectively. The designs for $\alpha =0.05$ and $\alpha =0.10$ are shown in Tables 1 and 2, respectively.

Let us assume that the maximum uninteresting response rate ( $p_0$) is $5\mathrm{\%}$, and we are not sure whether the target response rate is $20\mathrm{\%}$ or $25\mathrm{\%}$. If the maximum type II errors are considered as $20\mathrm{\%}$ and $10\mathrm{\%}$ for the target response rates $20\mathrm{\%}$ and $25\mathrm{\%}$, then our proposed design for $\alpha =0.05$ is (0/1/2/3/10) (3/28) (4/38) under the optimality criterion 1 (O1): see Table 1. That is, at the first stage, 10 patients would be recruited. If none of the patients respond, then the design would be stopped for futility. If one patient responds, the design will proceed to the second stage to test the target response rate as $20\mathrm{\%}$ at $20\mathrm{\%}$ maximum type II error rate and 28−10 = 18 more patients would be recruited. If more than 3 respondents are observed at the second stage, the null hypothesis would be rejected for an alternate response rate $20\mathrm{\%}$. If 2 respondents are observed at the first stage, the design will proceed to the second stage, and 38−10 = 28 more patients would be recruited to test for the target response rate as $25\mathrm{\%}$ at $10\mathrm{\%}$ maximum type II error rate. If more than 4 respondents are observed, the null hypothesis would be rejected for an alternate response rate $25\mathrm{\%}$. Otherwise, the drug would be identified as ineffective, and the trial would be stopped. However, if three respondents are observed in the first stage, then the design would be stopped for efficacy, and the null hypothesis would be rejected for the target response rate $20\mathrm{\%}$. Finally, if the number of respondents observed at the first stage is more than 3, the design would be stopped, and the null hypothesis would be rejected for the target response rate $25\mathrm{\%}$.

The expected sample sizes are 17.76, 23.29 and 21.26, if the true response rates are 0.05, 0.20 and 0.25, respectively. Note that the expected sample size is much higher if $p_1$ is the true response rate than if $p_0$ or $p_2$ is the true response rate. This trend is found for almost every designs except few exceptions. The reason behind is that the proposed design is more unlikely to proceed to the second stage if the drug is either futile or highly effective. If $p_2$ is true, then we can say that the drug is more effective than it was at $p_1$. The optimality criterion for O2 design is to minimise max $\{E(\mathrm{N}|p_i);i=0,1,2\}$ and in every case $E(\mathrm{N}|p_1)$ is the highest. So we can say that our proposed O2 design have minimum $E(\mathrm{N}|p_1)$ among all the designs fulfilling the error constraints. Although not presented in the paper, for every set of parameter values, the first ten designs under each optimality criterion are computed. For ϕ $=(0.25,0.40,0.45,0.05,0.20,0.10)$, the O2 design is (9/10/13/16/34) (29/87) (21/61) with 45.58, 46.97 and 41.37 as the expected sample sizes under $p_0$, $p_1$ and $p_2$, respectively. Though not presented here, the second design under same optimality criterion is (9/11/13/16/34) (23/68) (22/64), where the expected sample sizes are 44.1, 47.17 and 41.73. The O2 design has $max(m,n)=87$ and for the second design, $\max (m,n)=68$. The second design may be more attractive to some investigatorss than the optimal one because of lower max(m,n). Note that the expected sample sizes are very similar in this case.

The O1 designs have some common features that can be observed from Tables 1 and 2. E $(N|p_0)$ is the lowest among the designs under four optimality criteria. It is obvious as the optimality criterion for these designs ensures the smallest expected sample size under the null hypothesis. The maximum difference in the expected sample sizes between O2 and O1 is observed for $\varphi =(0.25,0.40,0.45,0.05,0.20,0.10)$, which is 45.58−38.31 = 7.27. This difference is higher for designs at $5\mathrm{\%}$ significance level than that of the designs at $10\mathrm{\%}$ significance level. For the same values of design parameters but at $10\mathrm{\%}$ significance level, the difference is 33.78−31.06 = 2.72. It is because the sample size in the first stage $n_1$ and total sample sizes m and n are higher at the smaller significance level. Also, the probability of early termination if the null hypothesis is true, $PET(p_0)$, is the highest, and the sample size in the first stage ( $n_1$) is the lowest for designs under optimality criterion 1. If $PET(p_0)$ is high, then the design is less likely to proceed to the second stage when the drug is ineffective. In Equation (9), we have expressed the expected sample size as an weighted sum of $n_1$, m and n. For O1 designs, the $E(\mathrm{N}|p_0)$ is lowest because $w_1$ or $PET(p_0)$ is highest and the expected sample size is dominated by $n_1$. Generally, $\max (m,n)$ for O1 designs is higher than the other three designs.

O2 designs generally have larger $n_1$ than those for the O1 designs but smaller than those for O3 and O4 designs. Only two exceptions are observed in Tables 1 and 2. For $\varphi =(0.10,0.25,0.30,0.05,0.20,0.10)$, $n_1$ for O2 design is 17, which is smaller than $n_1$ for O1 design $(=18)$. For this set of parameters, $PET(p_0)$ for O1 design is 0.740, which is the highest for the designs we have computed. Another exception is observed for $\varphi =(0.40,0.55,0.60,0.10,0.20,0.10)$, where $n_1$ for O2 is 30, which is larger than $n_1$ for O3 $(=27)$. Moreover, the O2 designs have highest $min\{PET(p_i);i=0,1,2\}$ and smallest max $\{E(N|p_i);i=0,1,2\}$ among the four optimal designs. Between the O3 and O4 designs, generally O3 designs have smaller $n_1$ except the cases where it is the same for both the designs. Tables 1 and 2 show that in many cases the sample size in the first stage are the same under both conditions and in some cases the two designs coincide. For $10\mathrm{\%}$ significance level in Table 2, we see that in the first three cases, O3 and O4 designs coincide. At $5\mathrm{\%}$ significance level, for the first and third cases, these two designs coincide. Among these two, the O3 designs have larger $PET(p_0)$ while O4 have larger $min\{PET(p_i);i=0,1,2\}$.

3.2. Comparison with Lin and Shih's design

We now compare the proposed design with the Lin and Shih's (LS) design. It is seen that the expected sample sizes are notably smaller for the proposed design if the target response rates ( $p_1$ or $p_2$) are true. The difference is higher if $p_2$ is the true response rate. For $\varphi =$ $(0.05$, 0.20, 0.25, 0.10, 0.20, $0.10)$, the LS design under the optimality criterion O1 is $(0/2/9)$ $(3/31)$ $(5/43)$ and $E(\mathrm{N}|p_0)$, $E(\mathrm{N}|p_1)$ and $E(\mathrm{N}|p_2)$ are 17.23, 31.19 and 34.14, respectively. The difference in $E(\mathrm{N}|p_1)$ for O1 design is 31.19−23.29 = 7.90 while the difference in $E(\mathrm{N}|p_2)$ is 34.14−21.26 = 12.88. The highest differences between $E(\mathrm{N}|p_1)$ and $E(\mathrm{N}|p_2)$ are observed in O1 design for the set $\varphi =$ (0.45, 0.60, 0.65, 0.05, 0.20, $0.10)$, which are 72.38−60.32 = 12.06 and 76.75−54.06 = 22.69, respectively. The lowest differences are observed for O4 designs for $\varphi =$ (0.40, 0.55, 0.60, 0.05, 0.20, $0.10)$, which are 59.29−57.96 = 1.33 and 53.98−51.80 = 2.18, respectively.

If $p_0$ is the true response rate, the expected sample size in LS design is generally smaller than that of the proposed design, but the increment is tiny. For $\varphi =$ $(0.05$, 0.20, 0.25, 0.10, 0.20, $0.10)$, $E(\mathrm{N}|p_0)$ is 17.23, which is smaller than the proposed design( $=17.76$). It happened since the probabilities of early termination under the null hypothesis $PET(p_0)$ for LS and proposed designs are high. But $PET(p_1)$ and $PET(p_2)$ for LS design is very low, which means that these designs have a very high chance of proceeding to the second stage, and therefore results in larger expected sample sizes. In case of the proposed design, $PET(p_1)$ and $PET(p_2)$ are notably larger than LS design.

Figure 1 shows $PET(p)$ and $E(N|p)$ versus p for $\varphi =$ $(0.05$, 0.20, 0.25, 0.05, 0.20, $0.10)$. We see that PET decreases with the increment of the true response rate for LS design. However, for the proposed design, PET starts increasing after reaching a minimum value. Note that the curves for O3 and O4 designs are not shown since those designs' main goal is to minimise the maximum sample size. For LS O1 design, the expected sample size is a monotonic non-decreasing curve, and eventually, it proceeds to max(m,n). For LS O2 design, there is no common trend because of the optimality criterion, but the curve also proceeds to max(m,n). For our proposed design, the expected sample size curve starts to increase as the true response rate increases. However, after a certain point, it starts decreasing and eventually reaches the sample size at the first stage ( $n_1$). As stated earlier, $E(\mathrm{N}|p_0)$ is smaller for the LS design in this case. The expected sample sizes for LS and proposed designs are very similar if the true response rate is close to $p_0$ under optimality criterion 1. After a little increment in p, the proposed design's expected sample size seems to be much lower than that of the LS design.