Appearance of new variants raises questions about vaccine effectiveness

This variant, alpha, seemed to be much more transmissible than the original form. Some feared the virus was developing new characteristics that could move the world back into lockdown.

“We were starting up studies to evaluate the effectiveness of the new vaccines,” said Sara Tartof, PhD, MPH, an infectious disease epidemiologist and research scientist with the Kaiser Permanente Southern California Department of Research & Evaluation. “But a super critical question was, how might vaccine effectiveness change for different variants?”

Kaiser Permanente Southern California senior scientist and vaccine researcher Hung Fu Tseng, PhD, MPH, said, “The key was linking the variant data back to the clinical data to find out whether the variants were more virulent or more fatal.”

“We needed to know which variants we were seeing among COVID-positive vaccinated individuals and which variants were among COVID-positive unvaccinated individuals,” said Katia Bruxvoort, PhD, MPH. She is an epidemiologist and was then a post- doctoral research fellow working with both Dr. Tseng and Dr. Tartof. “Then we would see how the severity or complications of COVID might be different for different variants.”

Dr. Tseng said that one way to determine that would be to continuously sequence samples found infecting a wide cross section of the population.

Large, diverse, integrated health system ideal for variants study

In January 2021, the researchers and the vaccine makers, recognizing the value of Kaiser Permanente Southern California’s unique potential to track the evolution of coronavirus into new forms, began to converge around the idea of the COVID-19 variants study. The size and diversity of the Southern California region’s patient population and the centralization of Kaiser Permanente’s labs made Kaiser Permanente Southern California an ideal location for this kind of study. So did the depth of the electronic health records and the great number of members of all ages that received one of the mRNA vaccines.

Most vaccinated members had avoided infection, but some had not. Some of those who were infected had been hospitalized. And some of them had died. Some members had chosen not to — or were unable to — be vaccinated. Comparing different groups’ outcomes to the variants associated with them would yield valuable knowledge to vaccine makers, policy makers, and clinicians.

The 2 major mRNA vaccine makers, Pfizer-BioNTech and Moderna, each reached out to Kaiser Permanente Southern California separately to explore using Kaiser Permanente Southern California’s COVID-19 specimens to track and identify emerging variants that might evade, or at least pose new challenges to, their vaccines.

Harpreet Takhar, MPH, a senior research support manager at Research & Evaluation for the vaccine research teams, was observing a similar request from the researchers and from industry, all pointing to the value of a Kaiser Permanente Southern California COVID-19 variants study. “We could all work together and generate data that everyone could use,” he said. “As far as I knew, nothing like this had been done; however, it made sense in terms of efficiency.”

Two separate studies, one common aim It was a simple and elegant plan. But to make it work, all the stakeholders would have to collaborate in unprecedented ways, working hand in hand to sequence the samples, while keeping some elements of the data from the 2 parallel studies — 1 for each of the major mRNA vaccines — separate.

Each study team would be doing its own research, but they would be sharing a lot. Dr. Tartof would be the principal investigator for Pfizer-BioNTech. Dr. Tseng would be the principal investigator for Moderna. The studies would share what data they could, but would keep sensitive and proprietary information separate.

Additionally, the Research & Evaluation lab would have to be radically scaled up and would have to expand at a second location at the Chino Hills lab, where most of the COVID-19 testing occurred. At first there were only about 1,100 positive samples per week. If there were surges, however — and there soon would
be — research lab staff could be processing many thousands of biospecimens a day.

The team brought in Research Lab Manager Michael Aragones, MD, to design, jump-start, and oversee the lab workflow. Dr. Aragones describes his job as “making lab research projects work. Scientists come to me with a study idea, and I figure out how to make it happen.”

Devising laboratory solutions in real time

Dr. Aragones described one of his biggest and most important constraints in making research projects work: “Nothing research does can get in the way of the clinical lab’s operations — nothing!”

“So, I look for a point of insertion where we won’t adversely affect clinical operations. And when we need biospecimens, like we do for the variants study, the best approach may be to collect the specimens once the clinic is finished with them, when they would otherwise have been tossed in the biohazard bin. One person’s castoff is another’s treasure!”

“The clinical side had begun setting the positive specimens aside for us, and we were not going to waste them. So, we had only a week to get the
research lab operation up and processing samples and sending them out,” said Dr. Aragones.

While the Central Business Office negotiated an agreement with the vaccine makers and Dr. Tartof and Dr. Tseng laid out strategies for their studies, Dr. Aragones went to see Robert Rabot and Agnes Bautista in the clinical lab in Chino Hills. Dr. Aragones had worked for their team previously. Now his former colleagues offered to carve out a space for research in their area. They also offered to lend the study teams the equipment needed to get started.

“It required working very fast,” said Dr. Tartof. “Hiring and training people fast. Working out processes — I can’t even tell you the number of processes there were. We had to label, aliquot, and store specimens at just the right temperatures. There are so many details when you’re working with specimens.”

Dr. Aragones said, “Positive samples of different types (some nasal swabs and others saliva) were arriving in different media and in different-size tubes. That made processing, storage, and shipping a challenge. And we were also dealing with a nationwide shortage of tubes and containers.”

Dr. Tartof said, “We had to build this study while we were running. That’s a big difference from clinical trials. We didn’t have the established protocols as there would be in a clinical trial. We pretty much had to invent them. There was always a new challenge to overcome. And I think we’ve met every single one.”

Once underway, study shifts into overdrive

The study got underway in March 2021. After obtaining Institutional Review Board approval, Research & Evaluation employees swept through the clinical labs emptying the bins containing each day’s positive samples after they had been read and entered and were considered completed by the clinical team.

Then they brought the samples back to their new Chino Hills workstation. There they de-identified them, aliquoted them, and stored them at the appropriate temperature until they could be shipped to the outside lab contracted to do the sequencing itself. Eventually, housing the tubes of samples required securing an entire freezer trailer that was brought in and parked in the lot outside of the Chino Hills lab.

The third-party lab obtained the sequences and identified the variant causing each infection. They aggregated sequencing data from the samples taken from members who received a Pfizer-BioNTech vaccine but had still been infected. They sent those data to the Kaiser Permanente Southern California study team focused on the Pfizer vaccine for further analysis. Aggregated sequencing data from those who received the Moderna vaccine, but had still been infected, were sent to the Kaiser Permanente Southern California Moderna vaccine study team. Sequencing data from those who were unvaccinated were shared with both Kaiser Permanente Southern California study teams.

With the emergence of the delta variant in June 2021, the volume of samples rose exponentially. The team ramped up to process as many as 5,700 samples a week. “Our team of research associates worked 6 days a week for 7 weeks to deal with the rise in samples,” said Dr. Aragones. “And during omicron, we were handling more than 13,000 samples a week.”

Innovative team effort bears groundbreaking, high-impact results

“At the end of the first phase of the sequencing study, even those research associates who were exhausted working on the delta variant over the summer
agreed to continue to work on the next phase of the sequencing study,” said Takhar.

“Everyone understands how important this work is. That’s what helps the lab research associates who are often working overtime shifts. They know their work is making a difference, and I am very grateful for the commitment of our skilled research associates and dedicated project managers to this study’s success,” he said.

Key publications from the studies have explored the effectiveness of vaccines against several specific variants. They have also studied the severity of the disease associated with them. These studies were published in some of the world’s most influential medical journals. They were used by decision-makers at the U.S. FDA and CDC and their international counterparts. They helped determine policy, particularly around the need for booster doses to preserve vaccine effectiveness against variants.

“I feel fortunate to be able to use our system in such an elegant, beautiful, helpful, and unprecedented way,” said Dr. Tartof. “It would be hard to overstate the importance and influence of the studies this work has already made possible, or of those to come.”