Moving Targets

Establishing accurate estimates of HIV incidence rates are critical to the success of prevention trials

By Kristen Jill Kresge

Contestants on the classic American game show The Price Is Right compete for the chance to advance to the final challenge—the showcase showdown—where the key to winning a whole host of prizes is to come as close as possible to guessing the actual retail cost of the items, without overbidding. A contestant automatically loses if they overestimate the dollar value.

Now HIV researchers, just like the game-show contestants, are learning that when it comes to estimating HIV incidence rates, there can be serious consequences to guessing too high. Recently two HIV prevention trials of microbicides were stopped prematurely because the observed incidence was so much lower than anticipated that the data safety monitoring board (DSMB) determined there would be too few HIV infections to conclusively show if the intervention was effective.

This, along with trends showing that HIV incidence is declining in several countries (Figure 1), including Kenya, Zimbabwe, Malawi, Ethiopia, Haiti, Nigeria, Rwanda, and perhaps India, has made many trial sponsors and funding agencies sensitive to the accuracy of HIV incidence estimates. The importance of determining HIV incidence was one of the major themes at a pair of meetings sponsored by the Bill & Melinda Gates Foundation on the methodological challenges of HIV prevention trials held earlier this year by the esteemed Institute of Medicine (IOM). Recommendations on how best to estimate incidence will likely be included in the report the IOM will release this fall (see Advisory panel considers complexities of HIV prevention trials, IAVI Report 11, 1, 2007, andOptimizing HIV prevention research, IAVI Report 11, 2, 2007).

An HIV incidence rate is the number of people who are newly infected with the virus in a given period of time. Good epidemiological data in a specific population, including HIV incidence, help researchers track the spread of local epidemics and are a requirement for designing trials to test whether or not an intervention, such as a vaccine, is effective at preventing HIV transmission. "In order to undertake an AIDS vaccine trial, you need to know the incidence," says Omu Anzala of the Kenyan AIDS Vaccine Initiative (KAVI) in Nairobi.

But accurately determining HIV incidence can be difficult because of the substantial lag time between when a person is first infected and when they develop symptoms of the disease. Measuring HIV incidence requires testing people who may be without signs or symptoms of infection, typically in a research study that follows uninfected volunteers over time. There are also several logistical challenges to determining incidence rates; these longitudinal cohort studies are expensive and time consuming, incidence rates can often fluctuate, and many of the shortcut methods that have been developed to estimate incidence by detecting recently HIV-infected individuals fail to work universally. There are also questions about how far in advance of efficacy trials to start studying incidence. Still, most researchers agree that conducting longitudinal incidence studies are critical. "The feasibility studies to determine true HIV incidence are extremely important," says Gita Ramjee of the Medical Research Council in South Africa. "They allow you to build capacity so that your Phase III trials are successful."

The "power" of incidence

There are many reasons to study HIV prevalence and incidence in different populations. Prevalence refers to the number of people in a population infected with HIV at any given time. While HIV prevalence can be determined more easily, this doesn't provide a clear picture of the current dynamics of an epidemic. Even as incidence drops, HIV prevalence may continue to rise due to the lapse between initial HIV infection and mortality. Some experts contend that the dramatic drop in HIV prevalence in countries like Uganda was due at least partly to increasing death rates of those infected early on in the epidemic.

Incidence measurements have several advantages. They can show how the epidemic is changing within certain groups, the speed at which HIV is spreading in light of current sexual or drug-use behaviors, and the effectiveness of available HIV prevention technologies.

Accurate estimates of HIV incidence are also indispensable to the design of HIV prevention trials. Statisticians "power" a study based on the number of people they predict will become HIV-infected during the course of the study, and this prediction is based on the HIV incidence in that population. This determines, among other parameters, how many volunteers must be included in the trial. If the actual incidence ends up being much lower than predicted, it can profoundly affect the outcome of the trial.

Benôit Masse of the Statistical Center for HIV/AIDS Research & Prevention (SCHARP) in Seattle says that even small differences between the predicted and observed HIV incidence can have an enormous impact. He presented an example at an IOM meeting showing that in a trial where statisticians assume an HIV incidence rate of 5% and a rate of only 4% is observed, the total number of volunteers would have to be increased by 25% to preserve the statistical power of the trial. This affects the length and cost of the trial. In the worst case, if the HIV incidence is much lower than expected the trial may be determined futile because it would be unable to provide conclusive results on the efficacy of the intervention and, depending on their charter, could be stopped by the DSMB.

"If you underestimate, that's OK. You just don't want to overestimate," says Zeda Rosenberg, chief executive officer of the International Partnership for Microbicides (IPM), a non-profit microbicide research and advocacy group.

Measuring incidence

For this reason it is critical to start a trial with the most accurate incidence estimates possible within the specific population where a study will occur. Most often incidence data is reported from antenatal clinics because almost all pregnant women are tested for HIV infection so that health officials can intervene to protect infants. This is the only setting where semi-mandatory HIV testing occurs. But this data fails to capture HIV incidence in other high-risk groups, including injection drug users (IDUs), men who have sex with men (MSM), and commercial sex workers.

Collecting incidence rates among the general adult population and within certain sub-groups is substantially more complicated. The gold standard method is the prospective cohort study where researchers follow large groups of uninfected individuals for long periods of time, testing them regularly—typically at three-month intervals—for HIV infection to determine the rate of seroconversion. Incidence rates are usually reported as the percentage of people infected in a single year.

But prospective studies are time-consuming, labor-intensive, and expensive, and add substantially to the already complex process of running a clinical trial. Consequently some sponsors may rely on previously published incidence data when designing their study. But this approach can be risky. CONRAD, a US-based reproductive health and HIV/AIDS prevention organization, and Family Health International (FHI), a non-profit organization in North Carolina, based their Phase III efficacy trial of the microbicide candidate cellulose sulfate in Nigeria on incidence data collected several years prior to the start of their study. Although this data suggested a 4% annual incidence in the population they intended to enroll—women who were considered at high risk of heterosexual HIV transmission—the actual observed incidence during the study was half that. As a result the DSMB advised investigators that they would have to increase recruitment significantly. When researchers determined this wasn't feasible because of difficulty working in the country, they had to close the Nigerian trial sites and start anew in South Africa where incidence rates are much higher. The discrepancy between predicted and actual incidence illuminates a conundrum facing HIV prevention researchers. "We don't want people to get infected, but without [infection] events you can't make any conclusion about the product," says Doug Taylor, director of biostatistics at FHI.

Another Phase III trial of the microbicide candidate SAVVY conducted by FHI, which was powered based on older published incidence data, was stopped prematurely by the DSMB in Ghana because of lower-than-expected HIV incidence. The other branch of this trial in Nigeria was stopped a year later, also for futility.

Figure 1. Estimated Incidence Rates for 8 African Countries Over a 10-Year Period

 

View larger image

Where there's a will

Several other methods also exist for estimating HIV incidence. One method is to use mathematical models to predict incidence based on existing prevalence data, but the accuracy of this approach is dubious.

Over the past 10 years several immunological assays have also been developed to differentiate between recent and chronic HIV infections in an effort to estimate incidence. By sampling a large number of individuals researchers can extrapolate the HIV incidence based on the number of people surveyed who appear to be recently infected. This eliminates the need to repeatedly test people over time and relies on the presence of certain antigens or antibodies that appear within a defined window period of acute HIV infection.

One approach is to use a test that detects the plasma levels of p24, the HIV capsid protein that reaches peak titers during acute HIV infection. Once HIV-specific antibodies are induced they bind the p24 antigen, thereby making it undetectable. This assay, however, is not sufficiently sensitive to reliably detect the narrow window period between p24 antigen positivity and the presence of antibodies and therefore isn't useful for calculating incidence.

Another approach is to use a combination of two ELISA tests of differing sensitivity for HIV-specific antibodies, known as the Serological Testing Algorithm for Recent HIV Seroconversion (STARHS) or "detuned" assay. If antibodies are detectable by the more-sensitive assay, the ELISA is repeated with a purposely less-sensitive assay to see if antibodies are still detectable. The theory is that only chronically HIV-infected individuals who have developed a much more robust and polyclonal immune response to the virus will have a high enough titer of antibodies to still be detectable. But in practice the assay doesn't work so well. "Our finding was that this was just not true," says Salim Karim, director of the Centre for the AIDS Programme of Research in South Africa. He found that STARHS substantially overestimated HIV incidence at sites in South Africa.

A third method for detecting recent infection is known as the BED capture enzyme immunoassay, so-named by its developers at the US Centers for Disease Control and Prevention (CDC) because it was originally developed based on the B, E, and D clades of HIV. The premise of the BED assay is that the immune system ramps up production of HIV-specific antibodies over time and these responses, IgG in particular, evolve from having a weaker to a stronger avidity for HIV. The BED approach uses an ELISA assay to measure the proportion of the total IgG levels that are HIV-specific. This ratio is then compared with a set of predefined parameters to determine if an infection is classified as recent or not.

With the BED assay, incidence estimates could be calculated after conducting a survey of only around 1000 people. "But the assay has tended to overestimate HIV incidence in African situations," says Anatoli Kamali of the Medical Research Council in Entebbe, Uganda (see Putting the BED to bed, below).

Putting the BED to bed

The BED immunoassay seemed to provide promising results when originally tested in the US in a cohort of women in Atlanta. Researchers reported there was "excellent agreement when estimating incidence," based on this approach (J. Acquir. Immune. Defic. Syndr. 44, 196, 2007). Samples collected during the AIDSVAX AIDS vaccine trial were also evaluated with the BED assay and there was strong agreement between the actual and BED-estimated incidence (AIDS Res. Hum. Retroviruses 22, 945, 2006). But when this assay was tested in Uganda, Kenya, Zambia, and Rwanda, researchers found that the BED assay drastically overestimated the HIV incidence. In Masaka, Uganda the incidence as determined by BED was 6%, while prospective data from that same population showed an incidence of 1.7% the year before the BED assay was used and 1.4% the year after (AIDS 21, 403, 2007). Similar findings were also reported by researchers working in Cote D'Ivoire (J. Acquir. Immune. Defic. Syndr. 45, 115, 2007). In this study the authors tested both the "detuned" and the BED assay methods and concluded that neither could be used routinely to estimate HIV incidence in the country. One problem with the BED assay is that it can't distinguish a recently infected individual from someone in late-stage HIV infection whose immune system has begun to fail, and therefore has waning levels of HIV-specific antibodies. In many cases the BED assay was incorrectly classifying individuals who had already met the clinical diagnosis of AIDS and were taking antiretroviral therapy as recent infections. "We found that our BED estimates were so bad you might as well guess," says Matt Price, clinical program manager at IAVI. The BED assay was tested by IAVI with several collaborators, including the Uganda Virus Research Institute, the Kenya AIDS Vaccine Initiative, the Zambia Emory HIV Research Project, and Project San Francisco, and they found that "it hugely overestimates incidence," says Price. In an effort to fix this, the assay's developers created a mathematical correction equation. But even with this fudge factor, the estimates were much higher than actual HIV incidence in Africa. "With these overestimates, this becomes a problematic tool to use in preparation for AIDS vaccine trials," adds Price. Another complication with this approach is that both the type and breadth of antibodies can vary greatly within a population and even within individuals. In general, total IgG levels are elevated in African populations and other factors like poor nutrition are also connected to immunological dysfunction (J. Acquir. Immune. Defic. Syndr. 45, 115, 2007). These results prompted the Joint United Nations Programme on HIV/AIDS (UNAIDS) reference group on estimates, modeling, and projections to recommend that the BED immunoassay not be used for routine surveillance, measuring absolute HIV incidence, or even monitoring trends in incidence. Another approach currently being used in studies being conducted by the Center for HIV/AIDS Vaccine Immunology is to test individuals for HIV-specific antibodies with a standard ELISA and then test those who are antibody-negative with a very sensitive reverse transcriptase PCR assay. "This tells you the people who have virus but haven't yet made antibodies to HIV," says Karim. But these are expensive tests and since there are so few people who will fall into this stringent category of early infection, very large numbers of people must be tested. There is widespread agreement among researchers that for HIV prevention trials, there is no substitute for prospective, longitudinal cohort studies to measure incidence. "The more costly and time-consuming methods are more reliable," says Karim. "There's nothing like doing the follow-up studies." Still it can be difficult to drum up support for these preparatory studies. "This is really expensive work," says Price.

 

The Missing Piece

A handful of countries around the world have aggressively monitored HIV incidence for many years as a way to track their own epidemic's progress. Thailand, a country lauded for its early and progressive response to HIV/AIDS, began a national surveillance program in 1984 and has been determining annual incidence rates ever since. Early on in the epidemic there was also a national effort in Thailand to determine new cases of HIV infection among particularly high-risk groups, like commercial sex workers, injection drug users, and men who have sex with men. This allowed Thai officials to detect the first wave of the epidemic in these individuals, says Rerks-Ngarm. "Knowing what the real situation was like was the most important thing we could do to solve the problem," he says. This led to the requirement that all of the country's sex workers use condoms to limit the spread of HIV. In Uganda, another place where early HIV prevention efforts were credited with stunting an exploding HIV/AIDS epidemic, public health officials started collecting HIV incidence data in 1989. From 1990 until around 2000, the HIV incidence in the general population hovered around 1%, says Kamali. "This is good, reliable data on incidence," he adds. This relatively stable incidence level, compared to other African countries, was attributed to the government's endorsement of the ABC approach (abstinence, be faithful, use condoms). But since 2000, there seems to be a slight increase in HIV incidence within the general population, according to Kamali. In many other countries there is very little current data on HIV incidence. Throughout Asia, for example, reliable HIV incidence data are scarce. Recently India revised its estimates on the number of HIV-infected people in the country based on declines in HIV prevalence among commercial sex workers and within the general population in some of the southern regions of the country (see Vaccine Briefs). Although there is very limited incidence data in the country, UNAIDS concludes that based on the revised prevalence data there is probably also a decline in the number of new HIV infections. Even in South Africa, which is home to the world's largest HIV/AIDS epidemic, incidence data is limited. In 2005 researchers from the Human Sciences Research Council determined incidence rates in 16,000 South Africans and found that the total number of new infections during the year was 571,000 (South African Medical Journal 97, 194, 2007). The highest incidence rate of 5.6% was observed in women between the ages of 20 and 29. But since the BED immunoassay was used to collect this national incidence data, Karim warns that the absolute numbers should be "regarded as tentative" (South African Medical Journal 97, 190, 2007).

Peripheral benefits

Even if immunological assays could more accurately and cheaply estimate HIV incidence, they fail to replicate the conditions of a clinical trial, where individuals are receiving regular counseling, education on their risk behaviors and HIV prevention, and continuous access to condoms. "They look at incidence in populations that are not exposed to behavioral interventions, which could, and likely will, lower HIV incidence," says Price, whereas all participants in a longitudinal study to determine HIV incidence receive risk-reduction counseling, just like their counterparts in clinical trials.

HIV incidence tends to be even lower among volunteers in clinical trials than in the general population. "Every time you start working in a community the incidence drops," says Anzala, highlighting another conundrum of this work—it is impossible to measure incidence without affecting it.

Incidence studies prior to a clinical trial also provide an opportunity for researchers to cultivate relationships with the community members and leaders, to start educational programs that will aid enrollment in future trials, and help establish both the infrastructure and technical know-how among people working at the clinical trial site. "There's no point undertaking a clinical trial in an area where you have no community support," Ramjee says.

In the process of conducting longitudinal HIV incidence studies at several sites in South and East Africa, investigators are able to refine both their recruitment and retention skills, other aspects that will be critical to the success of a future trial. "The traditional way of looking at incidence lets you decide if it is really a suitable community for doing a vaccine trial," says Anzala.

There is also valuable social science research that can be conducted during incidence studies. Researchers can study trends in sexual behaviors within a population, what is putting individuals most at risk for HIV infection, and pregnancy rates among female volunteers that can help determine condom use. "Invariably you obtain a lot of scientific data," says Kamali.

Beware of falling incidence

Another complicating factor is that HIV incidence can change rapidly, often declining due to effective prevention campaigns, the recent proliferation of antiretroviral (ARV) treatment programs in some developing countries, and also due to more accurate methodologies (Figure 1). Ronald Brookmeyer, a biostatistician from Johns Hopkins University who was a member of the IOM panel evaluating HIV prevention trials, referred to incidence rates as a "moving target".

Thailand once had one of the most rapidly expanding epidemics in the world, but now HIV incidence seems to have trailed off outside high-risk groups. When the first AIDS vaccine efficacy trial with the AIDSVAX candidate was conducted in Thailand, preparative cohort studies showed incidence rates of as high as 6%, but during the trial incidence was only 3.4%. Since then incidence has dropped even further.

The only other Phase III AIDS vaccine trial, which is evaluating the efficacy of a combination of Sanofi Pasteur's canarypox candidate and AIDSVAX, is now ongoing in Thailand and Supachai Rerks-Ngarm, a principal investigator at the Thai Ministry of Public Health, reports that at his sites the incidence is low but still within the statistical limits of the study.

When the CDC started a Phase III trial in Thailand to test the efficacy of antiretroviral pre-exposure prophylaxis (PrEP) for preventing HIV transmission, they chose to enroll a cohort of only IDUs because of a higher incidence rate in these individuals. Even so, the predicted annual incidence is only 2%.

Identifying the right cohort

Even as the HIV incidence rates drop in many areas there are still a staggeringly high number of new HIV infections occurring globally—last year alone 4.3 million people were newly infected. Researchers are now focusing more on pockets or sub-groups of individuals where HIV transmission rates still tend to be very high. "You can go anywhere and if you find the right populations, you can have a high enough incidence," says Karim. But the problem with working in exclusively high-risk populations is first identifying them and then working to recruit and retain them in long-term prevention studies. Many research groups are gaining experience in these areas by conducting prospective incidence studies in these high-risk populations in preparation for AIDS vaccine efficacy trials.

Kamali and others in several African countries are now working with cohorts of HIV discordant couples, where only one partner is HIV infected. In Uganda, Kamali's group in cooperation with IAVI has established a cohort of about 500 discordant couples and has observed an incidence rate of around 4%, nearly four times that seen in the general population. Susan Allen, an HIV/AIDS researcher from Emory University in Atlanta, was one of the pioneers of working with discordant couples. At three sites in Zambia affiliated with her program, the Rwanda Zambia HIV Research Group, the transmission rates among discordant couples range from 3% to 9%, even with access to the best behavioral interventions.

"We are not just watching people get infected," says Kamali. "We are giving them everything that is available for HIV prevention and even with that comprehensive package we still observe, unfortunately, a high HIV incidence."

Anzala is conducting an HIV incidence study in Kangemi, Kenya involving 701 individuals, including discordant couples, commercial sex workers, and MSM. This work is also part of IAVI's efforts to establish incidence information at trial sites in preparation for large-scale AIDS vaccine trials. Both this cohort and Kamali's discordant couple cohort in Uganda will be participating in the upcoming Phase IIb AIDS vaccine trial known as PAVE 100. This trial will evaluate the safety and preliminary efficacy of the combination of DNA and adenovirus serotype 5 vaccine candidates developed by the Vaccine Research Center at the National Institutes of Allergy and Infectious Diseases.

Other groups including the US Military HIV Research Program are conducting incidence studies in preparation for AIDS vaccine trials. According to Rosenberg, IPM plans to conduct their own incidence studies before starting efficacy trials with microbicide candidates in women who are at high-risk for HIV infection.

In South Africa, which has the largest number of HIV-infected individuals, the HIV prevalence and incidence are generally so high that it is often unnecessary to recruit only high-risk volunteers. "I'm not saying that all the work should be done in South Africa, but you put out the fire where the fire is raging," says Ramjee.

Lingering questions

One question still puzzling researchers is when is the right time to start incidence studies. This information is obviously important to have before beginning Phase IIb proof-of-concept trials, but if it is still years before an AIDS vaccine candidate is ready for a Phase III efficacy trial, it is likely that the work will need to be repeated. "This is a Catch 22," says Anzala. "If you start too early, you risk that the community will be burned out. But if you wait until a vaccine is available it takes a lot of time. When you start Phase I trials you should to some extent start to prepare for Phase III," he adds.

Also as male circumcision, improved access to ARVs, and possibly other yet unproven HIV prevention modalities become available, incidence may decline even in high-risk populations. Although this would be an outstanding accomplishment, it will make it even more difficult to conduct efficacy trials in the future. "It's a trade-off," says Ramjee.