17
$\begingroup$

In the last few weeks, Pfizer/BionTech, Moderna and AstraZeneca have each released preliminary estimates of the efficacy of their SARS-COV-2 vaccines.

But what do their respective efficacy percentages actually mean? Is Moderna's vaccine 95% effective in any person? Or is the success of provoking the desired immune response limited to 95% of the population?

Put very simply: is it 100% effective in 95% of the population, or 95% effective in 100% of the population?

Will we just have to find out the hard way like the participants in trials whether we are immune, or will protocols to screen for efficacy on an individual level be derived from trial results before the vaccination campaign starts? So those who in whom the immune response does not get triggered as desired can be vaccinated with one of the other vaccines, to try again?

| improve this question | |
$\endgroup$
  • 3
    $\begingroup$ Summarizing, it means 95% fewer infections than the placebo group. How is not yet explained. Whether 95% become immune, or everyone becomes 95% tougher to infect. More importantly, the main purpose of corona vaccines is the old herd immunity. Like extinguishing a fire by hosing down all of the surrounding wood. See below for excellent detail and further reading. $\endgroup$ – user345360 yesterday
  • $\begingroup$ How could this even be distinguished? $\endgroup$ – kutschkem 43 mins ago
16
$\begingroup$

Vaccine efficacy

Pfizer's target measures for efficacy (see the study on clinicaltrials.gov) seem to be:

Confirmed COVID-19 in Phase 2/3 participants without evidence of infection before vaccination

Confirmed COVID-19 in Phase 2/3 participants with and without evidence of infection before vaccination

From Pfizer's study plan (VE = vaccine efficacy):

VE will be estimated by 100 × (1 – IRR), where IRR is the calculated ratio of confirmed COVID-19 illness per 1000 person-years follow-up in the active vaccine group to the corresponding illness rate in the placebo group from 7 days after the second dose. VE will be analyzed using a beta-binomial model.

(note: they also have other time windows and checkpoints in their analysis plan; which one is reported will shift from press release to press release as they get more data. If you are interested in the details the study plan also describes the planned interim analyses, and some of the press releases discuss deviations they've made from their original interim plans in consultation with regulatory agencies)

This measure is called relative risk, and can be written like this:

%InfectedPerTimeplacebo = NInfectedplacebo / (NStudiedplacebo * AverageFollowUpTime)

%InfectedPerTimevaccine = NInfectedvaccine / (NStudiedvaccine * AverageFollowUpTime)

Efficacy aka VE = %Infectedvaccine / %Infectedplacebo

If you assume risks are the same in the placebo and vaccine groups (they "should" be, but might vary if, for example, people who experience vaccine side effects change their behavior) and the average follow-up time is the same (they should be approximately the same, because they are giving the vaccine and placebo to patients enrolled at the same time), this ratio would tell you that a vaccine efficacy of 95% means that if you took 20 people who would have had a positive test after the placebo, you would only expect 1 of them to test positive if they instead got the vaccine.

Population vs. individual statistics

Put very simply: is it 100% effective in 95% of the population, or 95% effective in 100% of the population?

These are population-based measures. They can't say anything about efficacy in particular individuals by these outcome measures. For example, there is no way to know from a study like this whether the vaccine is 100% effective in 95% of the population, or if it raises the infective dose in everyone by some amount which causes the number of people exposed to this critical viral dose in their environment to decrease by 95% (or some other effect with the same end result).

Other approaches like challenge studies, where vaccinated individuals (or animal models) are intentionally exposed to a certain dose of the virus, can help understand the individual effects of vaccination, as can indirect measures of immune response like antibody titers. These approaches have other drawbacks, however (safety, translating animal results to humans, translating a given immune response to an infection chance, etc).

Beyond just 'vaccine efficacy': disease severity, real-world efficacy

These particular outcomes also say nothing about disease severity. It could be that the people who do test positive despite getting the vaccine get just as sick as the sickest people who don't (interpretation would be that the vaccine protects mostly against mild illness). It could also be the reverse, and that people who get the vaccine and still test positive have a milder illness than they would have otherwise. Efficacy defined by this relative risk ratio does not say anything about this, it only compares positive vs negative rates.

An additional note: these numbers report efficacy in the trial environment. "Real-world" efficacy might depend on other factors such as differences in the people enrolling in trials vs the general population (both in terms of things like age and preexisting conditions as well as behavior and exposure risks), failure to administer the vaccine properly (including improper storage), failure to complete both doses in timely fashion, etc. This measure of efficacy also refers only to "primary" efficacy. One would expect that if enough people were vaccinated, the effect on the population could far exceed the primary efficacy, because not only do vaccinated individuals have a lower chance of infection, but everyone else in the population also has a lower risk if there are fewer people available to transmit the infection.

| improve this answer | |
$\endgroup$
  • 4
    $\begingroup$ Should also note that (as with flu vaccines, which are only around 40-60% effective for the individual) a major purpose of the vaccine is not to provide 100% protection to the person that receives it. It's to reduce the transmission rate so that the virus doesn't spread. $\endgroup$ – jamesqf yesterday
  • 3
    $\begingroup$ @jamesqf Meant to include that in my final paragraph but it slipped my mind; included now. $\endgroup$ – Bryan Krause♦ yesterday
  • $\begingroup$ "Efficacy aka VE = %Infected_vaccine / %Infected_placebo" -- did you mean IRR here? $\endgroup$ – aland 5 hours ago
5
$\begingroup$

It means protection against the virus brought to you by the vaccination. In the trial, around 45.000 people participate. Among them, 50% are vaccinated with the trial vaccine, the other half receives a placebo. Additionally you exclude people from the trial which had COVID-19 before. Since infecting people with a potentially deadly disease is ethically not possible, you watch both groups for the occurance of disease.

I only have the numbers for the Biontech/Pfizer vaccine, but the principle is the same for the other ones as well. In this trial 170 cases of COVID-19 where seen, 168 in the placebo group and only 2 in the vaccine group. If the vaccine would do nothing, you would expect similar numbers in both groups, meaning these 2 cases represent a reduction (or efficacy) of about 95%.

With the vaccine only 5% will catch the disease, while there is no protection for the unvaccinated persons. For the numbers see the press release from Pfizer and the database entry on clinical trials for this trial.

| improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ To further clarify the statement that "with the vaccine, only 5% will catch the disease", this means that 5% of vaccinated people who would have otherwise caught the disease will become infected, not that 5% of all vaccinated individuals will catch the disease (only about 0.01% of vaccinated people did). $\endgroup$ – Nuclear Hoagie yesterday
  • 4
    $\begingroup$ @NuclearHoagie Exactly. Without the vaccine you would expect similar numbers of infections in both groups, which is reduced by 95%. $\endgroup$ – Chris♦ yesterday
2
$\begingroup$

Vaccine efficacy is the percent change in the percent of individuals who test positive in the vaccinated group versus the placebo group in a trial. The CDC website explains it here

So a 95% efficacy means the vaccinated group had 95% less percentage of people test positive than in the placebo group. So if the placebo group had a 20% positive rate, the vaccinated group had a 1% positive rate (5% of 20%). If this 95% efficacy is deemed to be statistically significant, it likely means that the positive rate in the vaccinated portion of the population will be 5% of the positive rate in the unvaccinated population.

Normally, if 100% of the population takes a vaccine, a 60% vaccine efficacy is considered enough to cause enough decrease in positivity rate to stop the spread of COVID-19 so 95% is really good. This paper goes into what the vaccine efficacy needs to be if less than 100% of the population takes the vaccine.

| improve this answer | |
New contributor
mihirb is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
  • $\begingroup$ I'm sure you'll have seen the recent results of the Oxford vaccine by now! The headlines stated 70% efficacy at first, with the intended regimen of two full doses two week(?) apart, but found (by accident, apparently) that if the first dose was a half dose, the efficacy rose to 90%. $\endgroup$ – drkvogel yesterday
  • $\begingroup$ More intriguingly, Professor Jonathan Van-Tam, Deputy Chief Medical Officer for England, said today that of the 24,000 (I think) trial participants who were given the vaccine in either regimen, none of them needed to be admitted to hospital. Does this, I wonder, mean that although the efficacy in terms of not contracting the virus at all was 70% or 90%, the people that did get the virus had milder symptoms than usual, and thus the efficacy in terms of preventing hospitalisation and death is closer to 100%? What would the name for that be? $\endgroup$ – drkvogel yesterday
-2
$\begingroup$

is it 100% effective in 95% of the population, or 95% effective in 100% of the population

If every person takes the vaccine once, these situations are experimentally equivalent. They're only different if people who the vaccine fails for take it a second time.

It is possible for a vaccine to fail the first time a person takes it but succeed the second time - in fact, this is the one of the reasons that some vaccines recommend multiple "booster shots". So the answer is closer to your second scenario.

| improve this answer | |
New contributor
BlueRaja - Danny Pflughoeft is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
  • 2
    $\begingroup$ "If every person takes the vaccine once, these situations are equivalent" - they are not. They are exchangeable at a population level, and for public health that's probably close enough, but not equivalent from the perspective of biology. You could have a vaccine that is 100% effective in 95% but is 0% effective in the other 5% no matter how many times you vaccinate. Efficacy here is not really about the vaccine itself, it's about the individuals' responses to it. $\endgroup$ – Bryan Krause♦ yesterday
  • $\begingroup$ @BryanKrause: ...Yes, that's the point of the post. The very next sentence was "they're only different if people who the vaccine fails for take it a second time." $\endgroup$ – BlueRaja - Danny Pflughoeft yesterday
  • 1
    $\begingroup$ No, they are still not the same. The other alternative is that in 100% of the people, the efficacy is 95%. Not that it randomly chooses 19/20 people to protect, but that it protects everyone quite a bit. Imagine it puts a suit of armor on everyone. It protects them all equally unless the virus hits them in the weak point. $\endgroup$ – Bryan Krause♦ yesterday
  • $\begingroup$ @BryanKrause: ...yep, which is what I said in the second paragraph. My point was that the two scenarios are mathematically indistinguishable (they lead to exactly the same results) if each person is only given the vaccine once. Where they actually differ is if we give the vaccine more than once. Which in fact we sometimes do, so we're able to distinguish them. $\endgroup$ – BlueRaja - Danny Pflughoeft yesterday
  • 1
    $\begingroup$ They are also not the same if you give the vaccine once. As I've said, it does not necessarily matter for public health, but it does for biology (and I remind you, we're on Biology.SE). $\endgroup$ – Bryan Krause♦ yesterday

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.