Barcelona Sessions Spark Full Discussion of Partially Effective Vaccines
By Patricia Kahn, Ph.D.*
Is the AIDS vaccine glass half full or half empty? With a host of animal studies on candidates that fail to protect against infection but delay or prevent disease, it can be difficult to tell—especially since it’s not known whether results from animal studies are predictive of what will be seen in humans.
But some of the most talked-about presentations in Barcelona clearly reflected a “glass half full” outlook as they highlighted the potential benefits of partially-effective vaccines that, while far from perfect, could still have an enormous public health impact on the AIDS epidemic.
The potential usefulness of partially effective vaccines was driven home most recently by a World Bank report issued in March entitled The Epidemiological Impact of HIV/AIDS Vaccines in Developing Countries, which provided a meta-analysis of several mathematical models that simulate the impact of partially-effective vaccines. There are many variables in these scenarios, including what proportion of the population is immunized, how long vaccine protection lasts, and what stage the epidemic has reached in a given region. The report concluded that there is a compelling case for seeking these imperfect tools: Hypothetically, a vaccine with 50% efficacy over 10 years given to 65% of all adults could reduce HIV incidence by 25 to 60%. And other studies have shown that even a 30% efficacious vaccine could have a significant impact on reducing new HIV infections in certain contexts.
.jpg)
Partial efficacy is a difficult concept to define—especially when it comes to vaccines, which are widely perceived to provide more or less complete protection. As the discussion in Barcelona and the World Bank report revealed, there are subtle distinctions that will have to be part of public health messages around experimental and licensed vaccines. In fact, the term “partially effective” can refer to two different vaccine effects. First, it can be used to describe vaccines that protect only some of the people who are immunized, rather than nearly everyone. Alternatively, it can describe vaccines that do not protect against infection but work instead by delaying or reducing disease in those who become infected.
In practice, it may be very difficult with HIV vaccines to make clear distinctions between these different categories. The current crop of candidates that induce cellular immune responses recognize and kill already-infected cells, and it is considered unlikely that they will efficiently block infection. But it could turn out that they protect at least some people from detectable infection, due to the interplay between vaccine, genetics, route and frequency of exposure and other variables. It’s also possible that sexual exposure to HIV in a vaccinated person could result in a local infection that does not spread systemically, and is therefore never detectable in the bloodstream by viral load measurements. Or a local infection could be completely eliminated by the death of infected cells. People who are infected and clear or contain the infection would not, technically, be completely protected from infection, although it would appear this way by most standard measures.
As difficult as it is to define and tease apart these different vaccine effects at a hypothetical level, it’s even more daunting to design trials that detect them. HIV-positive individuals can remain healthy and asymptomatic for five years or more—longer than the duration of current (and planned) Phase III trials. To evaluate vaccines for licensure, planners therefore have to rely on surrogate markers such as viral load, rather than on clinical symptoms of disease. They’ll also have to reach consensus with regulatory agencies on how much of a decrease in viral load, sustained for how long, is significant in terms of health benefits and reducing transmission risk. And even with viral load as an accepted surrogate, they will have to do long-term follow-up studies to determine whether viral load effects translate into changes in disease progression in vaccinees as compared to matched controls. Duration of immunity is another key consideration, since immune defenses could wane over time, as they do with many vaccines. And vaccinees may be exposed to different subtypes or new strains of HIV that are not recognized by vaccine-induced responses, leading to a change in the degree of protection afforded by immunization.
Math Models Help Plan Ahead
Faced with these real world challenges, it’s not surprising that some of the most discussed presentations in Barcelona were trial simulations. Roy Anderson, Geoff Garnett and colleagues (Imperial College, London) are developing models that look at shifts in key variables—such as HIV incidence in the trial population, number of volunteers, and viral load reductions—depending on the level of efficacy a trial seeks to detect (Abstract #WeOrC1309). The goal, Anderson says, is to design trials that will give clear answers to questions about how well a vaccine protects against both infection and disease progression.
Among other things, this requires choosing realistic endpoints and efficacy thresholds from the outset and then powering trials to detect them. Aiming too high can lead to ambiguous results—which could result in overlooking potentially useful candidates. That’s what happened with a malaria vaccine, Spf66, tested in children in Kilombero, Tanzania in 1994. The Kilombero trial was powered to detect a vaccine with 50% efficacy. But when the study data was unblinded, researchers found what appeared to be 31% efficacy—a level that could nevertheless have a profound impact on malaria in this hard-hit area. However, the result was ambiguous, since it was below what the trial was powered to detect and yielded wide confidence intervals (range of uncertainty). In other words, the vaccine could actually be significantly more or less effective than 31%.
The ongoing VaxGen trials in Thailand and the US illustrate the interplay between efficacy thresholds, HIV incidence in the cohort (numbers of new infections every year) and the numbers of volunteers needed to measure protection against infection. (VaxGen’s vaccine is the only one in the clinical trials pipeline aimed at inducing antibody responses that might block infection.) To detect a minimum of 30% efficacy, the Thai trial (based on an incidence of 4 new infections per 100 person-years) had to enroll about 2,500 volunteers, while the US trial, with a lower expected incidence of 1.5% annually, recruited over twice that number (5,400 individuals). And in the upcoming Phase III trial prime-boost trial (canarypox with gp120) in Thailand, which is powered to detect 50% efficacy, the much lower incidence rates in the study population will require enrollment of 16,000 volunteers.
Since surrogate endpoints like viral load will most likely be used to make decisions about vaccine licensure, Anderson says that long-term, post-licensing studies are crucial. Gathering five or even eight years of post-immunization follow-up data is a monumental task. But without it, he warns, there won’t be enough information about duration and nature of effect to make strategic decisions about who should get highest priority for vaccination and when—i.e., high-risk groups, versus general population or adolescents. It will be an ongoing decision-making process, since public health officials will likely make one set of decisions following licensure, and then review and re-evaluate them as follow-up data on long-term health and protection effects come in.
One Trial, Two Goals
Building on the complexity of evaluating vaccines based on surrogate markers, Michael Hudgens of the HIV Vaccine Trials Network (HVTN) also presented modeling work on the thorny task of designing trials that yield conclusive data about a vaccine’s efficacy in preventing both infection (termed VEs) and disease (VEp) (#WeOrD1298). The HVTN model, developed by Steve Self (HVTN) and others, is mainly concerned with avoiding a statistical pitfall called “selection bias,” which could interfere with interpretation of VEp results. In the case of AIDS vaccines, the bias has to do with unforeseen interactions affecting efficacy and the ways in which these interactions could skew data analysis. For example, if a vaccine worked only in people with strong immune systems, then vaccinees who became infected in an efficacy trial would represent mostly individuals with weaker immune systems; in contrast, infections in the unvaccinated group would include individuals with weak and strong immunity.
In this scenario, when statisticians analyzed VEp data from all infected volunteers, they would be comparing individuals with varying immune strength (from the unvaccinated group) with a pool of people having weaker immune systems. And if infected vaccinees appeared to progress more rapidly, or to have higher viral load setpoints than the infected controls, this could be wrongly attributed to the vaccine, rather than to bias introduced by fact that vaccine protection failed only in people with weak immunity overall.
Another example involves strain virulence: a trial could end up comparing a group of infected vaccinees infected mostly with highly virulent HIV strains to controls infected with HIV strains of varying pathogenicity.
So far, the HVTN team has developed a framework for estimating how big a trial would need to be to avoid errors in interpretation of VEp data. This depends, to a large extent, on the vaccine’s VEs: the better a vaccine is at preventing infection, the fewer individuals this leaves for the VEp analysis. For example, in a 2000 person trial of a vaccine which prevents infection 50% of the time (VEs = 0.5), there would have to be 45 infections in the control arm, and 23 in the experimental arm to say with certainty that a 0.5 log drop in viral load among infected vaccinees was due to the vaccine. If VEs is increased to 80%, cohort size would have to be increased by about 50% to measure the same effect on VEp. Knowing the HIV incidence in a cohort, the estimated VEs of the vaccine and the minimal level of VEp the trial aims to detect, planners can begin to get a sense of how to design efficacy trials that yield conclusive data on vaccine effects.
The HVTN work is one piece of the puzzle. At Emory University, Ira Longini and colleagues have done a series of studies that look at other, related trial design aspects, such as how to gather information about vaccine effect on infectiousness (they suggest recruiting sexual partners of trial participants) and building studies that look at heterogeneity in type and frequency of HIV exposure among participants, thereby reducing the risk of erroneous assumptions about vaccine efficacy.
Forecasting A Bumpy Ride
The models presented in Barcelona leave open as many questions as they answer. “We’re only just beginning to look at whether we need to design different strata [of trial data analyses], by gender and clade of circulating virus, to get independent looks at vaccine effects in different settings,” says the HVTN’s Steve Self, who lists genetic background, age and HLA type as other potential variables that could affect responses to vaccines.
It may also be difficult to use existing natural history data to set viral load goals for vaccine trials, since there is wide variability in viral kinetics over time—a factor that could complicate decisions about VEp goals. And eventually, the approval of low-to-moderate efficacy AIDS vaccines could have a dramatic effect on the design (and size) of subsequent vaccine trials, since these products may replace the placebo arm of the trial—and raise the bar for the level of efficacy new candidates will need to achieve.
It won’t only be clinical researchers who must come to terms with these issues, but also the policy makers and advocates who will plan vaccine deployment strategies. Some efforts are already underway to begin forecasting the demand for vaccines with partial efficacy, issues that were discussed in a presentation on a qualitative study co-sponsored by the WHO, UNAIDS and IAVI (#WeOrD1297) (to be covered in the Oct/Nov 2002 issue of the IAVI Report).
Ultimately, these mathematical tools will be only partial guides—ones that may help the AIDS vaccine field steer clear of a Kilombero-like trial, but which can still not ensure perfect clarity. “A math model is a blunt tool,” warned Jorge Beloqui, a mathematician and AIDS vaccine advocate who co-chaired a bridging session where models were discussed. Yet despite these limitations, the simulations presented at Barcelona gave a sense of the work that needs to be done—and, perhaps, a preview of Bangkok, 2004.
------------
*Patricia Kahn, Ph.D., is editor of the IAVI Report.