Modified Simon’s minimax and optimal two-stage designs for single-arm phase II cancer clinical trials

Simon’s and the admissible two-stage designs

Suppose that p₀ and p₁ are the success rates under the null and alternative hypotheses, respectively. For given type I and II error rates of α and β, Simon’s minimax two-stage design is the design, (r₁, n₁, r, n), which minimizes the total sample size n. If multiple solutions, (r₁, n₁, r, n), exist, the design with the minimal expected sample size under the null hypothesis,

$$E{N}_0=n_1\text{+(1}\text{ }\text{-}\text{ }PE{T}_0\text{)}×\text{(}n\text{ }\text{-}\text{ }{n}_1\text{),}$$

is selected as the minimax two-stage design. Herein, the PET₀ is the probability of early termination under p₀ after the first stage;

$$PE{T}_0\text{ }\text{=}\text{ }\text{B(}{r}_{\text{1}}\text{ }\text{|}{p}_0\text{,}\text{ }{n}_1\text{),}$$

where B(·|p, m) is the cumulative distribution function for the binomial distribution with success probability of p and number of trials, m, respectively. Likewise, Simon’s optimal two-stage design is the design which minimizes the EN₀ with the same constraints used for the minimax design. The optimal design is a two-stage design for which the PET₀ should be as high as possible and n₁ as small as possible. Accordingly, the probability of early termination under p₁ (PET₁) which corresponds to the type II error spent at the first stage, could be undesirably high, especially for β = 0.2.

The admissible two-stage design by Jung et al. [14] is the design which minimizes the Bayes loss or risk function,

$$q×n\text{ }+\text{ }\left(\text{1}\text{ }\text{-}\text{ }q\right)E{N}_0,\text{ }q\text{ }∈\text{ }\left[0,\text{ }\text{1}\right],$$

with the same constraints as used in Simon’s design. Simon’s minimax and optimal designs are equal to the admissible two-stage designs with q = 1 and q = 0, respectively. Thus, no additional admissible design exists if the difference in n between two Simon’s designs is less than or equal to 1.

As these Simon’s designs and the admissible designs do not take into account the balance in the sample size and type II error between two stages, the severe imbalance in the sample size or in type II error is often observed. For example, with design parameters (p₀, p₁, α, β) = (0.7, 0.9, 0.05, 0.2), 23 of 26 (88%) and 6 of 27 (22%) subjects will be evaluated in the first stage by Simon’s minimax and optimal designs, respectively, and no additional admissible two-stage design is available. The type II errors spent in the first stage by Simon’s minimax and optimal designs are 19.3% and 11.4%. For Simon’s minimax design with design parameter (p₀, p₁, α, β) = (0.5, 0.65, 0.05, 0.2), 66 out of 68 subjects (97%) will be evaluated in the first stage, while Simon’s optimal design requires additional 15 subjects. The type II error spent at the first stage by Simon’s minimax and optimal designs are as high as 18.9% and 14.3%. Other examples will be discussed in Section 3.3.

Modified minimax and optimal two-stage designs

We propose the modified minimax two-stage design for single-arm phase II clinical trials which is the solution, (r₁, n₁, r, n), to an integer optimization problem expressed by

minimize n

$$\text{subjects}\text{ }\text{to}\text{ }\text{ }PE{T}_1\text{ }=\text{ }B(r_1|p_1,\text{ }{n}_1)\text{ }≤\text{ }\mathrm{Î\mu }\text{ }≤\text{ }\text{ }\text{β},$$(1)

$${\mathrm{λ}}_{\text{1}}n≤n_1≤\text{ }{\mathrm{λ}}_{\text{2}}n,\text{ }0\text{ }\text{<}\text{ }{\mathrm{λ}}_{\text{1}}\text{ }<\text{ }{\mathrm{λ}}_{\text{2}}\text{ }\text{<}\text{ }\text{1},$$(2)

Type I error ≤ α and Type II error ≤ β.

The aforementioned two drawbacks of Simon’s design can be addressed by considering two additional constraints (1) and (2). With appropriate values of λ₁, λ₂, and ε, the pre-selected range of subjects will be evaluated in the first stage and the probability of falsely declaring futility spent at the first stage will be less than or equal to ε ≤ β. As ε, a maximally allowed type II error at the first stage, gets close to β, the impact of constraint (1) becomes diminished. Likewise, the modified optimal two-stage design is the solution which minimizes EN₀ with the same constraints. Note that the modified two-stage design matches Simon’s design if it satisfies equation (1) and (2). Investigators may choose different values of λ₁, λ₂, and ε, depending on their purpose. λ₁ = 1/4 and λ₂ = 1/2, for instance, could be selected if one wants to conduct the interim analysis with 25% to 50% of the planned information for whether the second stage is open. The optimal timing for interim analyses for the confirmative clinical trials has been examined by Lawrence Gould [15] and Togo and Iwasaki [16]. Lawrence Gould claimed that the interim analysis for futility for randomized two-arm ‘proof of concept’ trials be carried out after accumulating at least 40% of the planned observations. As Lawrence Gould pointed out, if the interim analysis for futility is carried out with too little data, it is not conclusive enough to support the decision. Little benefit will be gained if the interim analysis is conducted with too much data. In this paper, λ₁ = 1/3, λ₂ = 2/3 and ε = 0.1 are selected to provide practical boundary so that 33% to 67% of subjects will be evaluated in the first stage to make decision with the reasonable amount of data and the PET₁ is controlled under 0.1. For β ≤ 0.1, constraint (1) makes no impact on searching for the solution. With constraint (1), the modified design, however, guarantees that when β is chosen to be > 0.1, the probability of falsely declaring futility after the first stage is controlled to be at most 10%. For β =0.2, a common choice, the modified design is well balanced in terms of type II error as well as sample sizes between two stages. Simon’s and the admissible design were computed through Dr. Anastasia Ivanova’s website [17].

Comparisons with Simon’s and the admissible design

Firstly the total sample size of the modified design with γ₁ = 1/3, γ₂ = 2/3, and ε = 0.1 is compared with Simon’s design for Δ = p₁ - p₀ = 0.15 (16 cases) and 0.2 (15 cases) in Figure 1. The top panels of Figure 1A and 1B show the number of additional subjects required for the modified minimax design while the bottom panels indicate those for the modified optimal design. Overall, 66 of 93 (71%) have the same total sample size to Simon’s design (10 (11%) have different first stage numbers), with the remaining 27 cases (29%) needing at most 3 additional subjects. For the modified optimal design, the results differ dramatically by β. For β = 0.1, 56/62 cases (90%) have the same total sample size, while 3 cases each require more (1 to 3 subjects) or fewer (2 to 9 subjects). For β = 0.2, only 2 cases (6%) have the same total sample size, while 81% (25/31) of cases save 1 to 13 subjects, and 13% (4/31) require 1-3 additional cases. Thus, for β = 0.2, dramatic improvements over the Simon design can be achieved.

The further comparisons are conducted and summarized in Supplementary Figures 1 and 2. The number in parenthesis for (α, β) = (0.05, 0.2) denotes the difference in n, compared with the modified optimal design. In cases that there is no difference in n, we investigate if the sample size of the first stage n₁ and the early stopping rule for futility r₁ are identical; “=” indicates that the designs are identical but “≠” shows that they are not identical even though the total sample sizes are the same. For example, for (p₀, p₁, α, β) = (0.3, 0.5, 0.1, 0.1), the modified minimax design is not identical to Simon’s minimax even though the total sample sizes are the same; (r₁, n₁, r, n) = (6, 26, 15, 39) for the modified minimax against (7, 28, 15, 39) for Simon’s minimax. For (p₀, p₁, α, β) = (0.05, 0.25, 0.05, 0.2), three designs, the modified minimax and optimal and Simon’s optimal design ((r₁, n₁, r, n) = (0, 9, 2, 17)), are identical, and one more subject is required in n, compared to Simon’s minimax design, (r₁, n₁, r, n) = (0, 12, 2, 16).

Figure 2 illustrates the characteristics of Simon’s designs for (α, β) = (0.05, 0.2). The left and right panels show the ratio of n₁ to n and the Type II error rate spent after the first stage (PET₁), respectively. Top and bottom panels show Simon’s minimax and optimal designs, respectively. The PET₁ of Simon’s minimax design is greater than 0.1 in 10 of 31 cases (32%) and less than n/3 subjects will be investigated in either the first or the second stage in 11 of 31 cases (35%). The PET₁ of Simon’s optimal design for (α, β) = (0.05, 0.2) is greater than 0.1 except for two cases, p₁ - p₀ = (0.05, 0.25) and (0.8, 0.95). With (α, β) = (0.1, 0.1) and (0.05, 0.1), all PET₁s of Simon’s minimax and optimal design considered satisfy constraint (1) (plots are omitted) and thus the modified designs are not identical to Simon’s designs if <n/3 subjects are evaluated in either the first or the second stage; 24 of 62 (39%) for Simon’s minimax and 9 of 62 (15%) for Simon’s optimal design. The EN₀ of the modified minimax design is smaller than or equal to Simon’s minimax except for 4 cases (plots are omitted) while the EN₀ of the modified optimal design increases by 0.04 to 3.36. As the EN₀ is highly attributed to the sample size in first stage, n₁, the large difference in EN₀ between the modified and Simon’s design can be found when the ratio of n₁ to n is too large or too small.

Examples

The characteristics of the modified design are compared in detail with the other two designs in Table 1 for four cases. With (p₀, p₁, α, β) = (0.35, 0.55, 0.1, 0.1), 86% of subjects will be evaluated in the first stage by Simon’s minimax design while 48% will be evaluated in the first stage by the modified minimax design. The modified minimax design is identical to an admissible design and requires two additional subjects in n. The EN₀ of the modified minimax, however, decreases by 5.2. The modified optimal design is the same as Simon’s optimal design.

With (p₀, p₁, α, β) = (0.7, 0.9, 0.05, 0.2), the sample size of each stage for both Simon’s minimax and optimal design is seriously imbalanced (88% and 22% in the first stage) and the PET₁s of them are as high as 19% and 11%. No additional admissible design is available. The modified minimax and optimal design provides investigators with a novel design, (r₁, n₁, r, n) = (8, 11, 23, 28) which requires 1 or 2 additional subjects if the second stage is open. The PET₁ of this design decreases to 9% (approximately 10% and 2% lower than Simon’s optimal and minimax) and 39% of subjects will be evaluated in the first stage.

With (p₀, p₁, α, β) = (0.8, 0.95, 0.1, 0.1), Simon’s minimax design is identical to Simon’s optimal design and 7 of 31 (23%) subjects will be evaluated in the first stage and no additional admissible design is available. Similarly, the modified minimax design is optimal in term of EN₀ in those satisfying constraint (1) and (2), and 16 out of 31 (52%) subjects will be evaluated in the first stage. When compared to Simon’s optimal design, the EN₀ of the modified design increases by 0.5, which seems ignorable. In fact, the PET₀ of the modified design is much higher than that of Simon’s design (0.648 vs. 0.423) and the sample size of the modified design is much better balanced.

With (p₀, p₁, α, β) = (0.5, 0.65, 0.05, 0.2), the sample size of each stage for Simon’s minimax design is severely imbalanced (97% in first stage) and the PET₁s of Simon’s designs are as high as 19% and 14% for Simon’s minimax and Simon’s optimal designs. The sample size of each stage for the modified design is well balanced and the PET₁s are controlled to be below 10%. The total sample size of the modified optimal design decreases by 8, compared with Simon’s optimal but the EN₀ of the modified optimal design increases by 1.7. The modified minimax design is identical to one of 4 other admissible designs.