Researchers poised to answer questions about vaccines in real world

Answers to these questions would guide major health policy decisions and influence the personal vaccination choices of millions. The stakes could hardly have been higher. The nation was desperate to get back to work, school, and play. Variants of the coronavirus were appearing on the scene. More than half a million people in the United States had already died from COVID-19, and the number of deaths worldwide exceeded 2.7 million.

“The Kaiser Permanente Southern California Department of Research & Evaluation had been doing postlicensure vaccine safety and effectiveness research for many years before the pandemic. So, we made a natural segue into studying the safety and effectiveness of the COVID-19 vaccines,” said Lina Sy, MPH, who is the scientific program manager for vaccine safety and effectiveness research at Research & Evaluation.

“The main difference with COVID-19 vaccines was the urgency. We had to quickly do the analyses and prepare manuscripts or presentations so the information could be used to help policy or regulatory decision-making in real time,” said Sy.

To help provide the answers needed to guide opinion, policy, and regulation, Kaiser Permanente Southern California conducted multiple large, high-impact, multipartner studies in 2021.

Are the mRNA vaccines effective in the real world?

Research & Evaluation epidemiologist Sara Tartof, PhD, MPH, led an essential early study of the Pfizer-BioNTech vaccine. She and her team looked at its effectiveness, including its protective power against the delta variant. It was clear that delta was dominating infections, which were rising. Many feared that the higher infection rate was due to delta escaping the vaccine. If that were the case, vaccine makers and public health officials would need to seek new interventions. Others suspected that the infection rate was simply a product of the vaccine’s waning effectiveness over time after injection. In that case, a booster shot of the same vaccine would be the best way to strengthen protection.

Dr. Tartof’s illuminating study, which appeared in The Lancet (preprint August 2021, published October 2021) had a major impact. It resolved the question for the Pfizer-BioNTech vaccine. It found that the vaccine remained effective against hospitalization up to 6 months after the second shot. The study also found that decreased effectiveness against infection was likely due to waning rather than due to delta escaping vaccine protection.

Data from the study were cited as demonstrating the validity of the waning hypothesis at the FDA’s Vaccines and Related Biological Products Advisory Committee (VRBPAC) meeting. It was also cited at the CDC’s Advisory Committee on Immunization Practices (ACIP) meeting. Booster dose recommendations were subsequently made for the Pfizer-BioNTech vaccine in September 2021.

“We could see that the vaccines were working, but we also realized that we had to treat vaccine effectiveness as a moving target, changing over time as well as from variant to variant,” Dr. Tartof said.

While Dr. Tartof and her team were assessing the Pfizer vaccine and its effectiveness, Hung Fu Tseng, PhD, MPH, a senior scientist with Research & Evaluation, was leading a team of researchers looking at the effectiveness of the Moderna COVID-19 vaccine. Their first study was published in November 2021 in The Lancet Regional Health – Americas (preprint September 2021).

Katia Bruxvoort, PhD, MPH, an adjunct investigator with Research & Evaluation, was the first author on the first of these Moderna studies. Research & Evaluation researchers found that 2 doses of the Moderna vaccine were effective against asymptomatic and symptomatic COVID-19. They also found that 2 doses provided protection against hospitalization and death for up to 5 months after the second dose.

The study was much larger than studies that led to the Moderna vaccine’s Emergency Use Authorization. It included 350,000 recipients who had received 2 doses of the Moderna COVID-19 vaccine between December 2020 and March 2021, and the same number of unvaccinated members.

“The racially and ethnically diverse study population also had a more representative range of underlying conditions, including chronic diseases, immunocompromising conditions, and autoimmune conditions,” said Dr. Bruxvoort. People in the Moderna vaccine clinical trial, which was smaller, were mostly younger and healthier.

“This was one of the first studies showing that the Moderna vaccine worked in the real world,” said Dr. Tseng, who was the senior author. “It certainly boosted the confidence of the public and of physicians using the vaccine to protect patients against infection. Long-term follow-up will continue to evaluate the durability of that protection.”

The study was cited in discussions about booster doses at both the VRBPAC and ACIP meetings in October 2021.

Do the mRNA vaccines protect against emerging variants? If so, for how long?

“Whereas other systems had to use proxies such as time periods when certain variants were predominant, our whole genome sequencing work allowed us to precisely ascertain the effectiveness of each vaccine against the different variants,” said Sy. “Kaiser Permanente Southern California studies of both mRNA vaccines’ effectiveness were instrumental in showing waning protection over time after 2 doses. The studies also demonstrated the potential need for boosters.”

Another study, published December 2021 in The BMJ, showed that 2 doses of the Moderna vaccine were still highly effective against all SARS-CoV-2 variants. However, their effectiveness against infection by the delta variant diminished with time after the second shot. It fell from 94% in the first 2 months to 80% after 6 months. Even after 6 months, though, protection against hospitalization remained high at 98%. This study was also led by Dr. Bruxvoort.

At the time of The BMJ publication, the omicron variant had not yet been found in Southern California. But just a few months later, in early 2022, Research & Evaluation authors published another paper in Nature Medicine. It showed that the Moderna vaccine provided strong protection against infection by the delta variant.

But it was less effective at protecting individuals from infection by omicron, the variant that came to dominate at the very end of the year. Individuals in the study were more vulnerable to infection by omicron. However, after 3 doses of the Moderna vaccine, they still had strong protection against hospitalization and death.

The study published in Nature Medicine, led by Dr. Tseng, was pivotal. It showed that “third doses may be needed sooner than 6 months after the second dose of the Moderna COVID-19 vaccine for adequate protection against omicron infection,” said Dr. Tseng. “It also revealed that 3 doses may not be sufficient to protect people who are immunocompromised from omicron infection.”

Data from the study were cited at the FDA’s April 2022 VRBPAC meeting on COVID-19 vaccine booster doses. Its reach was also international. It was cited by the World Health Organization, Australian policymakers, the European Medicines Agency, and the European Centre for Disease Prevention and Control. It provided evidence of gradual waning of immunity against omicron infection.

Later, research published in The Lancet Respiratory Medicine and then supplemented with additional analysis showed that a third dose of the Pfizer COVID-19 vaccine provides strong protection, roughly 80% in the first few months against hospital admissions and emergency department visits caused by the delta and omicron variants. However, against omicron, this protection wanes over time for immunocompromised individuals — even after a third dose.

“A third dose of Pfizer BioNTech COVID-19 vaccine significantly improves protection against omicron, although while that protection stays high for immunocompetent patients, it seems to wane for immunocompromised patients after 3 months,” said Dr. Tartof.

Are the mRNA vaccines safe?

Research & Evaluation’s contribution to the COVID vaccine safety studies was an extension of its membership in the Vaccine Safety Datalink (VSD) program. The VSD is a collaboration between the CDC’s Immunization Safety Office and 9 health care organizations, the largest of which is Kaiser Permanente Southern California.

The mRNA vaccines were authorized for use based on safety data from robust clinical studies. However, “many people still hesitated to get vaccinated because of fears that the new vaccines were causing blood clots, anaphylaxis, and other serious conditions,” said Stanley Xu, PhD, MS, an Research & Evaluation senior research scientist biostatistician. “Toward the end of spring 2021, vaccination rates began to slow significantly. Safety was one of the main reasons given for people’s reluctance to get vaccinated,” said Dr. Xu.

“There were reports of death following vaccination, but these were not necessarily caused by vaccination. Without an epidemiologic study using a comparison group, it was unclear whether death rates following COVID-19 vaccination were higher than background rates.”

Dr. Xu led a study that relied on data from the VSD to compare 6.4 million vaccinated and 4.6 million unvaccinated Americans. It aimed to determine whether any of the 3 COVID-19 vaccines authorized in the United States caused an increased risk of death.

“Kaiser Permanente Southern California has been studying vaccine safety with the VSD for 30 years,” Dr. Xu said. “So, we were well prepared for this when the pandemic hit. We began preparing to do safety studies as soon as the FDA authorized the vaccines,” he said.

The study examined all 3 vaccines in use in the United States. It looked at the mRNA vaccines, each of which required 2 doses at the time, and a single dose viral vector vaccine. The study was published in the CDC’s Morbidity and Mortality Weekly Report in October 2021.

“The results of the analysis were reassuring,” said Dr. Xu. “It showed that the vaccines are safe. People who received COVID-19 vaccines in the United States had a lower death rate than those who didn’t. That’s the case even if you don’t count COVID deaths,” he said. “I like to think it helped a lot of people with their choice to get vaccinated.”

How quickly do new variants spread, and how are they distributed across the population?

Another study, published in CDC’s MMWR in October 2021, demonstrated how quickly and ably researchers could respond to pressing epidemiologic questions. This study was led by Debbie Malden, DPhil, MSc. Dr. Malden is a CDC Epidemic Intelligence Service Officer stationed at Research & Evaluation. Her unique role made it possible to design the study, run the analysis, and publish the report quickly.

This study used a large set of sequencing data, combined with Kaiser Permanente Southern California’s extensive medical records, to describe the distribution of SARS-CoV-2 variants circulating in the Kaiser Permanente Southern California population.

The Kaiser Permanente Southern California study team used sequencing data from almost 7,000 SARS-CoV-2–positive specimens that were linked to members’ medical records to establish the distribution of variants and hospitalization rates. The investigators found that the weekly percentage of all infections attributed to the delta variant rapidly increased from 0% to 95% between mid-April and late July 2021. This demonstrated the ability of new variants — and delta in particular — to quickly spread over a large geographic area.

The study also found that SARS-CoV-2 infections attributed to the delta variant were more common among younger members and among non-Hispanic Black members. In addition, the study showed that unvaccinated patients infected with delta were more likely to get sick enough to require hospitalization than unvaccinated patients infected with other variants.

This study showed how important it was to monitor variants for new mutations, and the findings reinforced the continued value of vaccinations, social distancing, and wearing masks to prevent the spread of delta.

Kaiser Permanente Southern California’s resources presented a unique opportunity to do this study. The combination of available sequencing data and epidemiologic data from members’ electronic health records represents a unique data source that could be used to answer high-priority questions about hospitalization rates and variant distribution.

Addressing the most pressing questions

Together, these studies provided essential answers about vaccine effectiveness against hospitalization and death, the duration of protection, vaccine safety, and the threats posed by variants as they emerged on the scene. Those answers enabled regulators and policy makers to make informed public health decisions, particularly about booster dose recommendations in response to variants and waning vaccine effectiveness.

Over the course of the COVID-19 pandemic, studies from Kaiser Permanente Southern California have enlightened and informed policy makers and the public on the best ways to move toward a safer future. Scientists know, though, that their mission continues. As the pandemic wanes and surges, as new variants emerge, and as public knowledge and opinion shift, the research by Kaiser Permanente Southern California will continue to address the most pressing questions.