How Science Competitions Fuel Biology Breakthroughs

John Moult remembers the time he was floored by an algorithm. Moult, one of the organizers of the Critical Assessment of Protein Structure Prediction (CASP) competitions and a computational biologist at the University of Maryland, was running the 14^th iteration of the event—and in the middle of 2020, no less.

CASP is a biannual competition that gathers researchers from around the world to focus on one scientific problem: how to accurately predict protein structure. Since the competition started in 1994, scientists have gathered every two years to share results and compare tactics. But it was in 2020—as COVID-19 shut down the world—when researchers from Google DeepMind submitted jaw-dropping results from their AlphaFold model for CASP14.¹ “It was such a large improvement and so close to the experimental limit, that it was just astonishing,” said Moult.

Besides CASP, several other science competitions have cropped up over the past few years. These include prizes focused on finding solutions for things like virtual cells, longevity, Alzheimer’s disease, and beyond. Besides these competitions, researchers are also flocking to different strategies for funding their research—particularly to drive forward projects considered to be “high-risk” undertakings.

Continue reading below…

During a tumultuous time for science, researchers hope that these competitions and funding methods will provide different strategies to do science—strategies that use more community-driven, fast-paced innovation to tackle big problems. “We can come together and see what is working and what is not on an equal footing as much as possible,” said Hani Goodarzi, a computational biologist at the Arc Institute and one of the leaders of the Virtual Cell Challenge. “That should really quickly feed back to the community.”

The Arc Institute is running the Virtual Cell Challenge.

Raymond Rudolph

CASP, AlphaFold, and the Structure of a Breakthrough

Science is inherently tied to competition. Scientists compete for money, positions, and recognition. It’s baked into the ways by which many people first become interested in science—through exposure to things like high school science fairs, synthetic biology competitions, or science olympiads. People do science projects, present their results, and by a specific rubric, judges determine which projects are the best.

When Moult first started CASP, he was intrigued by this concept of critical assessment. Specifically, he and the other organizers wanted to quantify if certain methods worked better than others. “I’m very interested in ways of doing critical assessment in science in a positive sense, not to be a policeman, but to try and advance the field,” Moult said.

Predicting the structure of a protein was a good problem to apply this philosophy to in part because the endpoint is relatively clear-cut: The predicted structure either matches the experimentally determined thing, or it does not. For a long time, protein crystallographers were the only people who could generate ground-truth protein structures. They do this by first crystallizing the protein and then exposing those crystals to X-rays, which creates a series of patterns that can be resolved into single atoms.

But protein crystallography is a difficult science, and generating a single structure can take years. “Either you have the answer, or you have nothing,” Moult said. Getting some sort of computational method that could accurately predict protein structure would be a true boon for the field—and would save people years of painstaking work.

Each CASP competition starts when organizers post sequences of protein structures not yet known to the public. Researchers can then use their method of choice to model the protein structure based on its sequence. After submitting their predicted structures, independent assessors evaluate how close each prediction is to the experimentally determined protein structure once it becomes available. The predictions are scored on a scale from 0-100 using a metric called Global Distance Test (GDT), which looks at the physical similarity between the predicted structure and experimental structure. Scores in the 20 to 30s represent predictions that are nothing like the experimental structure while 100 means that it is exactly the same.

Continue reading below…

In CASP competitions over the years, scientists threw a bunch of potential solutions at the structure prediction problem—some of which stuck a little harder than others. Several strategies included looking at contact between specific amino acids, which then led into the introduction of deep learning methods.

Having a very standardized approach to assess each predictive method helped steer the field away from strategies that didn’t work well. “There were these enthusiasms we had—which if we hadn’t shown very clearly were duds—we would have stayed at for years,” Moult said. “It does allow you to focus on the more promising ideas.”

Alphafold, which came out of the CASP competition, can predict protein structures.

Wikimedia Commons

Until 2020, the GDT scores generally oscillated between the 30s to 50s. In 2020, Google DeepMind’s AlphaFold started continuously generating predicted structures that scored in the 90s, even for the most difficult class of proteins. Their approach utilized a system of neural networks that had been trained on the Protein Data Bank, a public repository of over 200,000 experimentally determined biological structures.

Despite CASP not providing any monetary prizes, the sheer clout gained from such scores was immense and later led to the Nobel Prize. “After you struggle with this stuff, in my case for 50 years, it’s kind of satisfying to see someone solve the problem,” Moult said.

The Virtual Cell Challenge and A New Wave of Science Competitions

Luckily for scientists, there is no dearth of problems on which to fixate—particularly in biology. While the protein structure prediction problem is now largely solved, Moult is interested in topics like predicting RNA structure, protein ensembles, and ligand interactions—which are important in drug design.

Other scientists are interested in a more macro-level problem such as creating a virtual cell, which is a simulation of a living cell and how it might respond to external or internal cues. While scientists are debating how viable the concept of a virtual cell is and how close they may be to creating one, a group of researchers at the Arc Institute wanted to see if they could focus research efforts toward that goal. “What you saw with CASP is that it really formalized the protein structure prediction task,” said Yusuf Roohani, a computer scientist at the Arc Institute. “We wanted to see if we could bring that same progress to the field of virtual cells.”

Continue reading below…

To that end, Roohani, Goodarzi, and others decided to launch the Virtual Cell Challenge in June 2025. This year, participants aim to create computational models that predict a cell’s response to genetic perturbations—essentially trying to figure out how, on a transcriptomic level, a cell reacts to the silencing of a certain gene. These perturbations can be accomplished through a technique called “perturb-seq,” where hundreds or thousands of genes are systematically knocked down by way of CRISPR interference. Using single-cell RNA sequencing, researchers can assess the effects of each perturbation on the cell’s transcriptome.

Such a challenge requires several datasets, so the Arc Institute team created a dataset on which participants could train their models by perturbing 150 genes in human embryonic cells and sequencing them. They then released a second set of 50 gene perturbations to help participants determine how well their models performed, the statistics of which sit on a live leaderboard. The challenge organizers also held onto another dataset with 100 perturbations, which they will use to determine final model rankings later this year.

Like Moult and others did with CASP, the scientists thought deeply about how to determine which models do better than others—and what sorts of biological data are needed for that metric. “Having this competition-like structure encourages the field towards standardization, which will ultimately help the machine learning models,” said Abhinav Adduri, a research scientist at the Arc Institute.

It’s tricky, though, because it’s not just a niche group of academic researchers who are obsessed with these types of scientific questions. Cells are deeply complex, and GPUs are very expensive. As a result, “a lot of the science and research actually happens in industry, in AI research labs,” said Goodarzi. To encourage participation, the team made it voluntary for researchers to submit potentially proprietary information like model code. “You want to kind of minimize the friction for everyone to come in,” he said.

Researchers Compete for Non-Traditional Funding

For these types of competitions—and more broadly for research as a whole—there is usually an underbelly of pressure to be the first, to be the best, or to win. As technologies have advanced in recent years, researchers sometimes generate results or think of ideas faster than the traditional publishing and grant funding mechanisms can keep up. To publish a paper, researchers submit their results to a scientific journal for peer review. They revise, then resubmit, and then repeat this grueling cycle until the paper is finally published—which could take years. Grant applications can be a similar slog.

Continue reading below…

“The intersection of AI and biology just moves too fast for the traditional kind of publication cycle,” said Goodarzi. And in traditional publications, “everyone is the hero of their own story,” he added. “As a result, we don’t really learn from each other because it isn’t easy to really know what is working and what is not.”

The need for speed is a sentiment shared by Martin Borch Jensen, a biologist who runs Impetus Grants—a new funding mechanism where scientists submit a grant application for up to half a million dollars and hear back within three weeks. Jensen started Impetus Grants to streamline scientists’ time from ideation to actual funding, specifically for the field of longevity. Besides the amount of time it takes for a grant to be funded, traditional funding institutions often require lots of preliminary data—which necessitates pre-existing resources. Additionally, they may reject certain contrarian ideas because review panels think that they have a low margin of success. Impetus Grants instead focuses on the potential impact of the research in the grant application.

With legal upheaval in many of the large institutes that parcel out these grants—like the National Institutes of Health—scientists are looking for other ways to fund their research and discuss their ideas. “Academics are broke and resourceful,” Jensen said.

For Jensen, creating Impetus Grants also meant dealing with the possibility that some of the big ideas funded might not work. Therefore, he decided to partner with the journal GeroScience to offer scientists a platform through which to publish their negative results. “If negative results are not published, then others don’t learn from them,” he said. “That just slows things down.”

In a similar fashion to these science competitions, Impetus Grants is part of a new wave of competitive funding mechanisms that prioritize speed and ambition—Fast Grants, another example, provided funding for COVID-19 focused projects. There is no lack of applicants, either. Jensen noted that there were a few hundred applications in the first funding round, which jumped up to one thousand in the second round.

Continue reading below…

Winning, Losing, and Just Hanging Out

Setting up a science competition is hard—and so is creating a new, faster way to fund research. But besides promoting community science and enabling people to more openly discuss successes and failures, the researchers who are setting up these contests hope that competitions can foster dialogue between scientists from different fields. “One positive outcome of creating this competition is that we have these excited engineers, machine learning researchers, and others who are learning about this problem,” said Roohani about creating virtual cells.

In some ways, competitions also serve as a venue for scientists to play around with their colleagues. A computational model called “Kwisatz Haderach” (a reference from Dune) topped the Virtual Cell Challenge leaderboard for some time in August. At CASP, Moult remembers the excitement—that seemed almost theatric in nature—permeating the conference hall during the competitions that were in-person.

It’s not entirely about pursuit of glory, either. Scientists keep writing grants and papers. The Virtual Cell Challenge leaderboard keeps populating models. Even CASP continues, with the problem of protein structure prediction largely solved. “I don’t know what’s going to happen next, and that’s what makes it so interesting,” Moult said. “It’s more fun when you don’t see it coming—that’s when you get the sense that anything can happen.”

Continue reading below…

_{Images are for reference only.Images and contents gathered automatic from google or 3rd party sources.All rights on the images and contents are with their legal original owners.}