IIb or not IIb?
AIDS vaccine trial sponsors weigh the merits of intermediate-size efficacy trials
By Emily Bass*
“To be, or not to be: that is the question,” says young prince Hamlet in the Shakespeare play that bears his name. Over the past year, a punning variation on this question has surfaced in the AIDS vaccine field as trial designers and statisticians have debated the merits and shortcomings of intermediate size trials—often known as Phase IIb trials—in advancing the search for an AIDS vaccine.
It’s more than a superficial allusion. When the young prince poses his question he is wrestling with the meaning of life. Likewise, the scientific question—IIb or not IIb—reflects soul-searching on the part of a field that faces challenging decisions about how to best invest its limited resources, and how to determine which AIDS vaccine candidates should be advanced into large-scale efficacy trials.
In the traditional clinical trials sequence AIDS vaccines move from small Phase I safety studies to Phase II trials, which gather additional information about safety, dosing and immunogenicity in a few hundred people, directly to Phase III efficacy trials that are designed to meet regulatory standards for product licensure.
If there were known correlates of protection for AIDS vaccines then immunogenicity data from Phase II trials could provide some hints about efficacy. But in the absence of such a correlate AIDS vaccine developers have little indication of whether or not a vaccine is likely to be effective before they decide whether to embark on a Phase III trial. This is a weighty decision; although the precise size of all trials depends on the local incidence rate, Phase IIIs usually enroll many thousands of people—a massive undertaking that requires lengthy preparation and large investments of financial and human resources.
Enter Phase IIb trials, which are smaller than Phase IIIs but significantly larger than Phase IIs. Their intermediate size gives them statistical power to provide valuable preliminary information about vaccine efficacy, though they are generally not geared towards securing product licensure unless the data show very strong evidence of efficacy. Instead, Phase IIb trials are designed to answer key research questions about one or more vaccine candidates, with the expectation that these data will lead to more trials or help guide basic scientific research in the future.
Phase IIb trials are starting to appeal to many AIDS vaccine trial sponsors. “We don’t know what immunogenicity measures mean or if they correlate with protection,” says HVTN statistician Steve Self (SCHARP, Seattle,). “We’re doing the best we can do with Phase I and II trials, but we’re still groping in the dark. Phase IIbs would give us some sense of whether there is any protective effect for these vaccines—and that would be beneficial.”
Although there are no confirmed plans for Phase IIb AIDS vaccine trials at present, IAVI is actively considering Phase IIb trials, as is the US HIV Vaccine Trials Network (HVTN), which is currently developing plans for a large-scale trial of Merck’s adenovirus candidate. “A Phase IIb test of concept trial is the kind of design that we’re discussing,” says Self. The HIV Prevention Trials Network (HPTN) is also poised to launch an intermediate size trial of two candidate microbicides before the end of 2004.
Figure 1. Trial size depends on primary hypothesis^
| Trial Objective | VEs > 30% | VEs > 0% | VEp > 1 log VL |
| Type of trial | Phase III | Medium phase IIB | Small phase IIB |
| # endpoints* | 250 | 100 | 40 |
| Sample size (2% incidence/yr) | 12,500 | 5,000 | 2,000 |
| Power for VL | 0.5 log VL | .75 log VL | 1.0 log VL |
| Power to analyze subgroups+ | Formal for 1, exploratory for 2 | Exploratory for 1 | None |
| VEs = percent reduction in susceptibility to HIV infection VEp = percent reduction in the cumulative risk of progression to AIDS or death following HIV infection. ∆VL is sometimes used as a surrogate number for VEp VL = viral load * endpoints = HIV infections + Subgroup analyses could include comparisons of vaccine effects by gender, race or genetic subtype of the infecting virus ^ Adapted from Susan Buchbinder, plenary address presented at the 11th Conference on Retroviruses and Opportunistic Infections, San Francisco, CA, February 8-11, 2004 [link] |
|||
The new interest in Phase IIbs marks a shift in thinking for the AIDS vaccine field, which so far has launched only three efficacy trials, all of which were classic Phase III studies geared towards licensure. “We can think of efficacy trials [Phase IIbs and Phase IIIs] as the penultimate step in the process of developing vaccines for licensure and widespread use, or we can think of them as experiments that can provide us with valuable information to be used at early stages in the development process,” said Susan Buchbinder in her plenary presentation (available here) at the recent Retrovirus conference outlining the types of questions that could be answered by Phase III and Phase IIb trials (see figure 1).
As Buchbinder and others explain it, the question of IIb or not IIb reaches beyond the pros and cons of adding an intermediate step to the vaccine development process and touches on the way that the field defines its expectations for efficacy trials—to itself, to potential volunteers, and to the world.
What is a Phase IIb trial?
Phase IIb trials are also known as “proof of concept” trials, intermediate-size- or probe-efficacy trials, or—to their detractors—underpowered Phase III trials. As these names suggest, this type of trial is often used to gather preliminary information on the efficacy of experimental candidates. “Overall, the major question being asked is: Does it work?” says IAVI Medical Affairs Director Pat Fast.
Although the question is simple, the answers—in some cases—are not. As the number of volunteers is reduced, so is the number of HIV infections that are likely to occur during the course of the trial. And as the number of these infections decreases, so does the precision of the trial’s estimates of vaccine efficacy.
One key difference between a Phase III and a Phase IIb AIDS vaccine trial is the width of the confidence intervals—the range of uncertainty—around the “point estimate” of the vaccine’s efficacy in preventing new HIV infections. This type of efficacy is also known as VEs, or vaccine effect on susceptibility. Broadly speaking, most Phase III trials can estimate VEs within +/- 15%. A Phase IIb trial of less than half that size—with less than half the number of HIV infections—has broader confidence intervals of +/- 30%.
Given these broad confidence intervals, a Phase IIb trial would only be able to say with certainty whether a vaccine was an outright success or a complete failure. For example, if a Phase IIb yielded a VEs point estimate of 60% (+/- 30%), the lowest possible value for VEs would be at least 30%. Since 30% is widely accepted as a minimum threshold for VEs, this would likely be considered a clear positive answer about efficacy. Likewise, if a Phase IIb trial found that a candidate had a VEs of zero, it would be safe to assume that efficacy was less than 30%—and that the candidate does not warrant further testing in trials.
In most cases a product that showed moderate efficacy in a Phase IIb would be tested again in a confirmatory efficacy trial before seeking licensure; but if the efficacy was very high, a Phase IIb trial could possibly be used as the basis for a licensure application to regulatory authorities.
Point estimates that fell into the middle range—over 30 and under 60—would be harder to interpret. “If you’ve got a point estimate of 40% with similarly wide confidence intervals, then that’s a lot more ambiguous,” says Fast. In this instance, sponsors would have to make judgment calls about whether to continue testing the product, abandon it, or to attempt to improve the vaccine design.
In addition to measuring VEs, a Phase IIb AIDS vaccine trial could also be used to detect vaccine-induced reductions in viral load (DVL), a measurement that will likely be used as a surrogate marker for VEp, or vaccine effect on disease progression. VEp measurements are particularly important for the current generation of candidates, which primarily induce cellular immune responses and will be evaluated for their ability to improve overall health and prolong life in vaccine recipients who subsequently become infected with HIV.
It could take many years to make a direct estimate of VEp. One recent article (J. Infect. Dis. 2003; 188:179-93) by HVTN researchers proposed a definition of VEp as the percent reduction in the cumulative risk of progressing to AIDS or death 5 to 7 years after infection. In contrast, viral load set point is established within months of HIV infection in people who do not begin ARVs immediately. Early viral load levels have been linked to long-term disease progression, which is why DVL could be used as an early indicator of VEp.
Phase IIb trials can make more precise measurements of DVL than they can of VEs. This is because VEs calculations are based on the “binary” variable of HIV status — a person is either HIV infected or uninfected at the end of the trial period. In contrast, DVL is calculated based on the “continuous” variable of viral load, which can take on a range of values. In statistical analysis, there are generally wider confidence intervals for calculations based on binary variables than those based on continuous variables. “A yes or no [binary] outcome is a lot less statistical information than a number like viral load,” explains Wasima Rida (Statistics Collaborative, Washington, DC).
In her presentation, Buchbinder estimated that 5,000- and 2,000-person Phase IIb trials in a population with a 2% incidence rate could measure vaccine-induced reductions in viral load of 0.75 and 1.0 log10 respectively at about three months after infection. In contrast, a 12,500 person Phase III trial in the same population would have high statistical power to detect a DVL effect of 0.5 log10.
For now, this added level of precision may not be necessary. Although DVL is a plausible surrogate marker for VEp, there are still many unknowns including how much of a DVL effect is needed and how long it would have to last to achieve a clinical or public health benefit. Given these uncertainties, it might make sense to do a smaller Phase IIb that can detect a more pronounced viral load effect and provide some information on its clinical benefit before proceeding to a large-scale trial that will pick up smaller virus load effects which could have less clinical relevance.
Intermediate-size trials in other fields
In weighing the risks and benefits of intermediate-size trials, many AIDS vaccine researchers turn to other vaccine fields. One example of a best case scenario comes from Merck’s human papillomavirus (HPV) vaccine research project.
In 2002 Merck announced positive results from a trial of a vaccine candidate targeting HPV 16. (There are over 100 strains of HPV, some of which can cause genital warts and cervical cancer.) Since the company ultimately hoped to license a polyvalent vaccine against several strains of HPV, this intermediate-size trial was used as “proof of concept” for the basic vaccine design. The trial followed roughly 1500 volunteers for an average of a year and a half following the final immunization, and monitored them for persistent HPV 16 infection—a surrogate measure of the vaccine’s ability to protect against cervical cancer, which may not develop until many years after initial HPV infection.
The trial showed 100% protection against persistent HPV 16 infection in vaccine recipients. Based on these encouraging data, the company has gone on to launch a full-scale Phase III trial designed to secure licensure for the polyvalent vaccine that targets strains 6, 11, 16 and 18. The trial will include long-term follow up so that researchers can directly assess the vaccine’s efficacy in preventing cervical cancer.
Asked about the worst-case scenario, many scientists utter a single word: Patarroyo. This was the name of the Colombian scientist whose malaria vaccine, SPf66, was tested in a series of trials in Africa and South America. These trials yielded varying, often ambiguous, results—most notably the first African trial that tested the vaccine in nearly 600 Tanzanian children. Although the trial was conducted in a region with high rates of malaria there were not enough cases of symptomatic malaria (the primary endpoint for the trial) to make statistically precise estimates of the vaccine’s effects. The vaccine was found to be 31% effective in preventing a first episode of clinical malaria but the 95% confidence interval for efficacy ranged from 0% to 52%.
The ambiguity of the results led to a stalemate as to how to proceed. “We agree on the need for further research, but raise the question of whether this should be further field studies or, as we would recommend, more detailed pre-clinical studies,” wrote malaria vaccine researchers Adrian Hill (Oxford University, Oxford) and Sarah Gilbert (Wellcome Trust, Oxford) in a commentary published at the time. Although a 31% efficacious malaria vaccine might have been beneficial—particularly in areas of high endemicity—the debate over the SPf66 trial data effectively stalled the development of the Patarroyo candidate, which has still not been tested in a definitive trial.
Looking ahead, the AIDS vaccine field will likely gain insights from a 3,100-woman multicenter international Phase II/IIb microbicide trial to be launched in 2004 by the HPTN. HPTN 035 is a 4-arm trial comparing two candidate microbicides, PRO 2000 and Buffergel, with a condom-only and a gel-only arm as controls. The study will collect intensive safety data in 800 women followed for three months each—nesting a more traditional Phase II type study into the expanded design. The trial has been designed so that it could be the basis for a licensure application to the FDA if the estimated VEs for either candidate exceeds 43.6%.
Why now?
In the AIDS vaccine field, discussions of Phase IIb dates back to 1994 when the US National Institute of Allergy and Infectious Diseases (NIAID) decided not to proceed with a Phase III trial of two gp120 candidates designed to elicit neutralizing antibodies against HIV. This decision prompted a NIAID-sponsored meeting to consider whether and how smaller, less expensive intermediate size trials could be used to advance AIDS vaccine research. (A summary of this meeting appeared in J. Acquir. Immune Defic. Syndr. 16(3):195-203, 1997).
Wasima Rida co-organized the meeting and co-authored the J. Acquir. Immune Defic. Syndr. summary report that remains the only peer-reviewed publication on Phase IIbs and AIDS vaccine trials to date. During the 1995 discussion, Rida says, “I really didn’t know which side I fell on. I was afraid that the field could end up with a Patarroyo situation.”
The potential for this type of confusion is a powerful deterrent to some AIDS vaccine researchers. US Vaccine Research Center head Gary Nabel warns that these trials could yield indeterminate results that add to, rather than allay, confusion. “If you have to double the trial size to get statistical significance, then double the trial size,” he says.
Many trial sponsors are now weighing the risk that Nabel identifies against the financial and human risks that come with investing in a Phase III trial. VaxGen’s two Phase III trials cost as much as US$300 million; the current prime-boost trial in Thailand will cost over US$100 million. Ed Tramont, head of the Division of AIDS at NIAID, predicts that by 2009 the US government-funded networks will need an additional $239.9 million over and above projected government spending on large-scale AIDS vaccine trials.
“It truly is a matter of dealing with resources around a substantial amount of scientific uncertainty,” says Self. Much of this uncertainty has to do with the field’s current focus on CTL-eliciting candidates that will be evaluated for their ability to affect disease progression. Since such a vaccine has never been developed before, vaccine developers will have to learn as they go exactly what types of vaccine effects are acceptable to regulatory authorities—and to communities where vaccines may be used.
“We know from the Food and Drug Administration (FDA) exactly what is useful and licensable for VEs,” says statistician Ira Longini (Emory University, Atlanta). “But as far as reducing viral load, we don’t know what we’re looking for, so the trials need to be more exploratory. We need a variety of trials that are large enough to sort out what makes an efficacious, licensable vaccine.”
Phase III trials are too large and costly to conduct simply to learn more about the characteristics of a candidate. Instead, these trials are geared towards licensure, which means that trial sponsors must set and adhere to well-defined hypotheses and criteria for success. “It’s hard to build a hypothesis for a Phase III trial when you only have a sense of what you may see, or want to see, from a candidate vaccine,” says Emilio Emini, Senior Vice President for Vaccine Development at IAVI.
Emini suggests that the field use Phase IIb trials as a chance to “loosen the statistical criteria”—turning the trial into an investigative research study, rather than a quest for licensure. “In a probe efficacy [Phase IIb] trial, you’re looking for anything at all,” he says. “Given that we’re assessing vaccines with unknown biological effects and with an uncertain magnitude of effect, one could argue that it is the better part of valor to do Phase IIb trials before moving on to Phase IIIs.”
Buchbinder agrees. “Rather than asking, ‘Do we have a vaccine for licensure?’ we need to be asking better questions,” she says, adding that the results of these trials could influence design of future candidates as well as Phase III trial decisions.
Phase IIb trials can also be useful in speeding evaluation of vaccine candidates that have yet to complete the costly, time-consuming steps of “process development”, since sponsors generally wait to launch Phase III trials until this process development has been completed. “If you don’t have your final candidate—if you still have to do process development for example—then it saves time and money to get an answer from a Phase IIb, even if it is not the final answer,” says Fast.
New challenges and opportunities
Although Phase IIb trials could be smaller and cheaper than Phase IIIs, they are not necessarily less work than Phase III trials. If anything, Phase IIb trials could require the field to be more selective in the questions it poses, and more thorough in its education efforts to prepare communities for ambiguous results. “We need to think carefully about what we will do depending on the outcome,” says Rida. “We need to think about what we will do if we fall into a ‘gray area’—how do we respond to the community; what do we do next?”
Positive findings would also have to be carefully presented, since a candidate that showed moderate levels of protection in a Phase IIb would still have to be tested in a full-scale efficacy trial to confirm the initial data and gather additional information on vaccine effects in a more diverse population. Trial sponsors would have to work with communities and ethical review boards to explain the need for further research on, rather than licensure of, a vaccine candidate that appeared effective.
Phase IIbs will be closely considered at an April meeting on AIDS vaccine trial endpoints in Washington, D.C. The NIAID-sponsored event will bring together an international group of regulators, scientists and statisticians from the major AIDS vaccine trials networks to discuss different trial designs and their possible outcomes.
For now, proponents of Phase IIb trials say that the benefit of these trials in terms of giving the field some efficacy data on existing candidates far outweigh the risk that this data will be ambiguous and add to confusion in the field. “For the field to continue to debate in a data-free zone is necessarily bad,” says Self. “I would much rather continue the debate in the zone of ambiguous data—that’s where gutsy scientific decisions can be made.”
------------
*Emily Bass is senior writer of the IAVI Report