I was working on a biological data analysis problem centered on biological sequences, with two assays that capture related properties of these sequences. One was relatively easy to run, but its readout was only an indirect view of the quantity we actually cared about. The other was harder to obtain, but it measured the target property more directly.
The first assay produced a per-position measurement along each sequence, whereas the second produced a single sequence-level measurement. The goal was not only to assess whether the two assays were correlated, but also to determine whether features derived from the first assay’s positional signal could be summarized into a single score that served as a reliable proxy for the second assay. In practice, that meant asking which aspects of the positional signal were worth extracting, and how they should be combined. If that worked, the first assay could potentially be used as a more accessible surrogate for the second in future work.
Before starting the analysis, I selected a diverse and representative library of sequences on which to run both assays.
flowchart TD
A["Transcripts of meetings with biology experts"]
B["Project goals and initial ideas"]
D["Workspace setup and data spec"]
F["Internal skill creator"]
G["Agent Skill"]
A --> G
B --> G
D --> G
F --> G
My goal was to build an agent skill that contained enough context and instructions to carry out the analysis reproducibly, rather than relying on a one-off prompt. For a broad introduction to Agent Skills and the SKILL.md format, see the Agent Skills overview.
I started by asking biologists to walk me through the wet-lab assays, the experimental details, the important caveats, and the scientific questions we were actually trying to answer. I recorded the conversations and used the transcripts as the starting point for the skill’s biological/wet-lab context.
From there, I layered in my own notes on the project goals, the structure I wanted the analysis to follow, some initial ideas of methods to explore, and practical instructions about how the workspace should be organized. That included what intermediate artifacts to save (such as scripts, pre-processed data), which interpretation files to maintain and how to keep a running journal of the work. I also included clear guidance on expected data formats, preprocessing constraints, and paths to the data files. I used all of that material, along with our internal skill creator (that just explains the format the SKILL.md file should have, along with our standard coding practices, some information about folder structure and other practical useful information) to generate the first version of the skill using an LLM. Open-source skill creators such as Anthropic’s skill creator are also available.
That skill encoded not only what to do, but how to do it in a way that was inspectable and reproducible: engineer candidate features from the more accessible assay into sequence-level scores, study the correlation with the scores from the second assay, and document each step well enough that the full workflow could be rerun and reviewed.
In this project, I happened to run the skill-based analysis with Claude Opus 4.6, but the skill is model-agnostic and can be used with other LLMs as well. The first version of the skill was already useful. It could take the project from raw data to preprocessing, visualizations, baseline analyses and interpretations so much faster than I would have done by hand. But that first pass also exposed the main limitation: fast execution is not the same thing as scientific exploration. I noticed that the agents strictly applied the methods I had described as vague ideas to try, and nothing else. Zooming in on the interpretation files, I also saw hypotheses presented as general truths, a lack of supporting evidence for a lot of claims, and questionable biological interpretations.
What allowed me to get closer to real scientific exploration was adding two sections in the skill that forced the agent to behave more like a careful collaborator. First, scientific rigor guidelines that required it to present uncertain statements as hypotheses rather than truths, identify alternative explanations when doing so, and always explicitly link any statement to the associated plot/data. I also required it to ground all interpretations in the biological reality of the assays and the uncertainty associated with the measurements.
Second, I added an explicit iteration cycle. After each full pass, the agent had to review the current state of the analysis, identify biologically meaningful new directions to extract features and scores from the per-position signals. Then, it had to implement everything as reproducible analyses, and generate the corresponding plots and interpretations. Most importantly, I instructed it to keep iterating on candidate score construction methods until the resulting score reached a predefined minimum Pearson correlation threshold with the second assay based on all sequences.
In practice, that meant the workflow was not trying to defend the first reasonable answer. It was exploring the space of plausible sequence-level scores derived from the positional signal until the relationship with the target readout became strong enough to be genuinely useful. That was the turning point: the workflow stopped being just an automation layer and became a structured, open-ended search process.
After a few rounds of iteration, the workflow converged on two scores that made sense from a biological perspective, and that each correlated with the second assay on their own. The most interesting part is that in its final cycle, the agent itself proposed combining these scores as predictors in a dual linear regression rather than treating them as separate alternatives. That suggestion only made sense because it first checked that the two scores were not themselves strongly correlated, so they had a chance of contributing complementary information rather than duplicating the same signal.
Framed by the iterative workflow encoded in the skill, that final combined model ended up giving a much stronger relationship with the target readout than the earlier baseline analyses, and the whole process: from raw data to preprocessing, exploration, and interpretation, with the refinement I had to do on the skill, took two or three days rather than the weeks this kind of work can easily consume.
What this project made clear to me is that in scientific work, rigor instructions matter just as much as analysis instructions. An agent can follow a computational plan, but that does not automatically make the result scientifically sound: the standards for interpretation, documentation, and follow-up have to be specified just as explicitly.
More broadly, having an agent skill that structures domain knowledge: wet-lab experimental design, folder organization, data specifications and pre-processing, interpretation rules, and iteration triggers, makes that knowledge reusable in a way that scattered notes or verbal context never really are.
It also reinforced a point that feels obvious in retrospect: agents are far more useful when given a workflow than when given a prompt. In the end, the real gain was not full automation, but a much faster loop for supervised exploration, where the hardest part was not generating code, but turning tacit scientific judgment into something explicit enough to be executed repeatedly.
]]>Impact in applying Machine Learning for biology comes from models grounded in biological reality: how experiments work, how data are produced, and the biological mechanisms the readout reflects. That is the difference between a technically impressive model and one that is pertinent, robust, and can inform decisions and guide the next experiments.
In my day‑to‑day work on mRNA and protein design, the wins have come from pairing modeling skills with practical biology: understanding assays, making preprocessing choices that reflect the experiment, and picking model architectures that match the mechanisms I’m trying to capture. After that, communicating results with expectations set by the problem’s biological complexity and the variability of the measurements.
This post highlights four places where biological insight is essential and makes modeling efforts actionable: assay understanding, data processing, modeling, and analysis/communication of results.
A biological assay is an experiment that turns a biological question into a measurable signal. Some examples include a fluorescence readout for gene expression or a binding assay signal reflecting the affinity between a protein and the molecule it recognizes. Understanding the biology behind that signal clarifies what information is present, what it reflects, and how to interpret the scale and units so modeling choices align with the phenomenon of interest.
A few examples show why biological understanding matters here. First, within the same setup, repeated measurements of the same sample don’t perfectly agree. Instrument limits and handling can add randomness that limits how consistently a model can match replicate measurements. A separate concern arises across runs: systematic context changes such as new reagent lots, altered plate layouts, or shifted incubation times, can move the entire distribution in ways that look like biology but are actually shifts caused by differences in how the experiment was run. Recognizing this distinction is key for preprocessing and interpretation.
Understanding biology also helps you design experiments so that modeling captures the right signal and can be acted on. Sometimes there isn’t an assay for the property you care about; understanding the mechanism lets you co‑develop a readout with wet‑lab scientists that reflects the phenomenon and yields data in a model‑friendly format. Collaborating early ensures the readout covers the range that matters, includes clear positives and negatives, and has enough replicates to estimate variation rather than guess it later. With the biology in view, assay design can align with the modeling goal, yielding models that are pertinent and impactful.
Data processing starts with what the assay measures and the biological level at which the signal lives, because that determines how to normalize, what to filter, and which comparisons make sense. If your dataset spans multiple transcripts, you might want to normalize within each transcript first so differences in RNAs are not mistaken for position‑level effects. You might then filter or weight using replicate variability so unstable measurements contribute less or are removed. Understand and acting upon the biology behind the assay ensures you move from raw numbers to data that reflects the biological problem you care about.
Some biological literacy about common tools helps you ingest raw outputs and interpret results without reinventing the wheel. Familiarity with biological data formats like FASTA and tools for sequence search and alignment such as BLAST and Bowtie helps you place assay signals in their proper biological context. These tools let you map raw data back to known sequences, genes, or motifs, making it easier to interpret what the assay is measuring. Together, these tools connect raw assay outputs to meaningful sequence information relevant to your task.
Biological understanding can also simplify the learning problem by matching the data representation to the mechanism. If the property varies at the level of local motifs, frame the task as window‑to‑value rather than full‑sequence‑to‑profile: fragment sequences into windows and predict a single value per window instead of forcing a long‑range sequence‑to‑sequence model. Conversely, if long‑range interactions are plausible, preserve the full context or add features that summarize global structure rather than fragmenting the input.
Encoding choices should follow the biology of the task. For RNA, for instance, use nucleotides for motif‑level effects, codons when translation dynamics matter, and amino acids when modeling protein‑level properties. Train, validation and test splits should also reflect biological relatedness, not just random rows. In that same example, splitting by proteins can prevent leakage and produce robust evaluations.
In practice, understanding the biology lets you align normalization, filtering, preprocessing, and encoding with what the assay can truly tell you, producing cleaner targets and representations that support pertinent models - and those are the models that can ultimately be acted upon.
It all starts with the biology. If there are concrete features known to drive the property (motifs, k‑mer counts, specific structural elements), featurize your inputs accordingly and use simple, auditable models such as GLMs or EBMs. If the mechanism is unclear or likely non‑linear, encode sequences (one‑hot, embeddings, etc.) and consider models that can discover patterns in these more complex representations.
As discussed earlier, understanding the biology of the assay clarifies its uncertainty. That uncertainty should be reflected rather than ignored. You can account for replicate variability in the loss or sample weights, and you have options to expose uncertainty when useful: use confidence intervals, ensemble models to estimate prediction uncertainty, or Bayesian models when you want uncertainty that can be propagated into downstream decisions.
Model architectures should also be based on the biological context. When effects are local, windowed setups or CNNs with kernel sizes aligned to expected motif lengths can be efficient; when interactions span longer ranges, use architectures that carry broader context, such as transformers or hybrids that mix local pattern detectors with global summaries, instead of forcing a purely local view. These are illustrative choices rather than hard rules: recurrent models, structure‑aware or graph models, and additive models all have places where they shine, with associated trade‑offs.
Bake in biological constraints up front so the model explores relevant regions. If certain motifs must be avoided, add masks or hard rules; if codon usage or structural constraints apply, encode them as constraints or penalties rather than filtering after the fact. This reduces wasteful exploration and keeps outputs usable without heavy post hoc cleanup.
Design evaluations in the frame of the next question you need to answer, not just what is easy to score. Use the biology‑aware split strategy discussed above to avoid leakage, and choose metrics that match the decision you will take, such as ranking for selection, probabilities you can threshold for go or no‑go decisions, or regression metrics when you will compare predictions to a numeric readout.
For a deeper dive on biological sequence modeling, see this piece.
Biological data are noisy and context‑dependent, so set expectations to match that reality and emphasize what is robust across repeats or conditions. Lower correlations can still be useful when the signal is subtle or when the practical goal is to rank the best candidates rather than predict exact values. Use visuals to make uncertainty and trade-offs visible at a glance, not to chase a perfect score; a good figure should clarify what is reliable and what is tentative. Focus on biologically meaningful insights, like motifs that consistently help or constraints that prevent failures, instead of chasing marginal metric gains that may not translate to biological language.
Formal training in biology helps, but it is not required. With curiosity and steady practice, anyone can learn and turn models into decisions that move the science forward.
]]>When establishing computational approaches to design biological sequences, one often faces two fundamental choices. How to predict the sequence properties you care about, and how to search the vast sequence space for candidates that are optimal with respect to these properties.
Layer 1: Property Prediction Models
At the core is the need to predict the properties or functionalities that the designed sequences should exhibit. These properties may include stability, binding affinity or translational efficiency. Property prediction models can be used to estimate how well a candidate sequence meets the design objectives. They take as input a candidate sequence and output a score or prediction reflecting the desired property. The optimization step can then aim to maximize or minimize that property, target specific values, a specific range etc. These models can be data-driven empirical models (often referred to as black-box models), or they can be derived from physical first principles (mechanistic models). The accuracy of this predictive layer significantly influences the downstream search for optimal sequences.
Layer 2: Sequence Optimization Algorithms
After getting property prediction models, the second challenge is to navigate the astronomical sequence space to identify candidates that maximize the target properties. Consider the problem of designing mRNA sequences encoding the SARS-CoV-2 spike protein. This protein has 1,273 amino acids, and each amino acid can be encoded by different codons, called synonymous codons. For the SARS-CoV-2 spike protein, there are around $2.4 * 10^{632}$ candidate mRNA sequences (Zhang et al., 2023), which is many orders of magnitude greater than the numbers of particles in the universe. Exhaustive enumeration is impossible, so we rely on sequence optimization algorithms to efficiently explore and exploit this space. Just like for the first layer, there are multiple ways to tackle these combinatorial optimization problems, but one important choice that has to be made and that I decided to explore in this article, is whether to choose an end-to-end differentiable optimization routine or a sampling-based one.
In the remainder of this article, I unpack these choices. Section II clarifies what “differentiable” really means in the context of biological sequence modelling, and why it matters. Section III dissects the choice of black-box models versus mechanistic models for Layer 1, property predictors. Section IV turns to Layer 2 and explores the tradeoffs between end-to-end differentiable optimization and sampling-based optimization. Finally, Section V summarizes the trade-offs into a decision guide.
At its core, differentiability refers to the existence of derivatives (gradients) of functions with respect to its inputs. Consider a function $f: \mathbb{R}^{N \times V} \to \mathbb{R}$ that maps an input matrix (e.g., a one hot encoded biological sequence) to a scalar output (e.g., predicted property score). The function is differentiable at $X \in \mathbb{R}^{N \times V}$ if $\nabla f(X)$, the gradient of $f$ at $X$, exists. This gradient tells us how infinitesimal changes in each component $X_{i,j}$ affect the output. For example, in neural networks, $f$ is a composition of differentiable layers, enabling back-propagation to efficiently compute $\nabla f(X)$. In our case, biological sequences are fundamentally discrete: nucleotides or amino acids are categorical variables, not continuous numbers. To apply differentiable optimization methods, sequences are often encoded. A simple yet very used example is to one-hot-encode sequences, using in the above definition N as the sequence length and V as the vocabulary size (4 for DNA or RNA sequences).
Differentiability is orthogonal to whether a model is mechanistic (rule/physics‑based) or black-box (data-driven). Mechanistic models can be expressed with continuous states and differentiable update rules, so gradients propagate end to end and enable gradient‑based optimization. They can also contain non-differentiable operations, preventing back-propagation. Black-box data‑driven predictors such as neural networks are inherently differentiable, but others such as decision trees and random forests are not.
To avoid conflation, it is helpful to separate the pertinence of differentiability in our two axes:
Black box property predictors take datasets of sequences and associated experimentally measured properties and learn a function $f(\mathbf{X})$ that maps sequences to the property of interest. The modeling toolbox is broad, ranging from linear regressions and shallow neural networks to Random Forests, CNNs and Transformers. Deep domain expertise is not strictly required to build these models, but biological understanding meaningfully improves outcomes. It informs how to process and normalize biological experimental assays to get ML ready data, and how to represent and encode sequences. When building deep neural network models, biological understanding guides architectural choices: if local motifs dominate, a CNN may suffice, for long-range dependencies, attention mechanisms or Transformers are more appropriate.
This setup is flexible enough to accommodate many tasks seen across the literature, such as predicting mRNA degradation with a hybrid CNN–RNN architecture as done with the Saluki model (Agarwal & Kelley, 2022), or modeling translation efficiency with the RiboNN model (Zheng et al., 2024). The same pattern extends beyond these examples: whenever there is available data that links sequences to properties, one can use it to train a custom ML model. In practice, this often means starting from a straightforward baseline model such as a Random Forest, then iterating on data processing, sequence representation and model architecture until accuracy and calibration needs are met for downstream design.
Given adequate data, black box predictors can achieve strong held‑out accuracy, reflecting their ability to learn complex sequence-property relationships. These relationships might not be explicit, meaning that they could not be translated into heuristics or a mechanistic algorithm. When implemented in a differentiable way, they integrate seamlessly into gradient‑based pipelines, and when not, they still serve as fast scorers for sampling‑based search. Moreover, because these models can be trained quickly as new measurements arrive, they naturally support iterative improvements: train a model on current data, generate specific candidates with the model (targeting extreme property values for instance), screen them experimentally, and feed the new data back into the model. This active-learning cadence steadily improves the predictor in the regions that matter for the design objective. As candidates accumulate, their shared patterns can also reveal informative signal, enabling hypothesis generation and possible new biological insights.
There are, however, important trade-offs. While the models can be retrained quickly, incorporating new measurements still depends on experimental timescales, which can range from weeks to months. Another dominant constraint is coverage: biological sequence spaces are astronomically large, and models are almost always trained on a thin, biased subspace of that space. As a result, generalization beyond the measured subspace is difficult; uncertainty should be quantified, and used to govern exploration and risk. Interpretability is also limited compared to explicit, rule-based frameworks; even with attribution maps or attention analyses, justifying specific design decisions can be difficult. In a biotechnology/pharmaceutical industry setting, the black-box nature of these models can complicate their explanation and acceptance by stakeholders.
Mechanistic property predictors tackle sequence–property tasks from a fundamentally different angle. Rather than learning end-to-end mappings from continuous sequence representations, these models explicitly characterize the mechanisms that drive property values. They encode hypothesized causal structure (e.g., thermodynamic energies, kinetic schemes, hard constraints, dynamic‑programming recurrences) to produce predictions whose components are transparent and auditable. Some implementations are fully differentiable and thus can participate in end‑to-end differentiable optimization pipelines; others rely on non-differentiable operations and are better paired with sampling‑based optimization.
Mechanistic approaches span diverse formalisms such as thermodynamic free-energy calculations, dynamic-programming algorithms that optimize over discrete structures or differential equation models solved with differentiable integrators such as Diffrax (Kidger, P. 2022). Some known dynamic programming examples include the Zuker–Stiegler dynamic programming algorithm for RNA secondary structure prediction (Zuker & Stiegler, 1981), which computes minimum free energy through explicit thermodynamic parameters. LinearDesign (Zhang et al., 2023) builds on this foundation with a different dynamic programming algorithm that takes as input a protein sequence and searches for optimal RNAs based on secondary structure and the codon-adaptation index (CAI) to reward preferred codons.
A first benefit of mechanistic models is their interpretability. Models built on decades of biochemical research offer trustworthy predictions where every energy contribution or score component can be traced back to well-understood molecular interactions. For example, physics-based differential equation models offer high interpretability because of the possible mapping between parameters and physical concepts (such as decay rate constants that explain signal behavior over time). This interpretability advantage is particularly valuable in regulated industries where model decisions must be justified. Another very important benefit is the ability to use biological knowledge to create algorithms that can generalize well across the sequence space, whereas black-box data driven approaches are limited by the subspace represented by the available data. Additionally, these models do not need data to be trained on; the mechanism itself acts as prior knowledge that bounds sequence to properties even when examples are scarce.
However, these approaches face important constraints. Mechanistic models require deep domain expertise to construct and validate, though once built they can often be applied immediately without extensive training data. If parts of the underlying mechanisms behind the properties of a sequence are not well-understood, they can’t be incorporated in the modeling.
Another drawback that can occur for both mechanistic and black-box models is that only differentiable implementations can be back‑propagated through. Non-differentiable models cannot be integrated into end-to-end differentiable optimization pipelines, limiting their compatibility with gradient-based design algorithms that require backpropagation through the property predictor.
End-to-end differentiable optimization routines leverage gradients to navigate sequence space systematically, steering updates toward candidates with improved predicted properties. They require property predictors to be differentiable. In the simplest setting, a sequence is encoded into a continuous embedding and standard gradient descent adjusts each position so that a differentiable property predictor’s score improves. The same principle scales to more expressive generators: diffusion models, Transformers, or VAEs learn to emit whole sequences and are trained end-to-end with the predictor’s loss as a guiding signal. In every case, back-propagation supplies an exact direction in which to nudge the sequence or the generator’s parameters, turning an otherwise intractable combinatorial problem into a tractable continuous optimization problem. One example is RiboCode (Li, Y. et al. 2024), a model that optimizes mRNAs for both translational efficiency and RNA structure.
The chief advantage is purposeful navigation. Because each update follows the objective’s gradient, the search can converge on high-scoring regions in far fewer evaluations than random or heuristic methods. The machinery is also modular: any differentiable property model, whether black-box or mechanistic, alone or in a weighted combination can be “plugged in” without altering the optimization logic, letting practitioners tune multiple objectives simply by adding terms to the loss.
These gains come with familiar trade-offs. Like the predictors they rely on, gradient-driven methods are hard to interpret: the calculus explains why a step improves the numerical loss, but not the biology behind property improvement. As is typical in non‑convex optimization, gradient methods can converge to local optima, and after training the fixed weights tend to sample a narrow neighborhood of that solution. That means that sequences would have low diversity. There aren’t straightforward fixes but one can experiment with injecting stochasticity through for instance noise or temperature scaling.
Despite these caveats, end-to-end differentiable optimization routines have become widely used in modern sequence-engineering pipelines, offering a scalable, mathematically principled route through the vast design space.
Sampling-based optimization routines explore sequence space without gradients. The only feedback they need is a scalar score: be that a thermodynamic energy, a wet-lab measurement, the output of a differentiable model treated as a black box, or a combination of many such numbers. At the simplest end sit rule-based heuristics that pick codons or motifs position-by-position; at the other extreme are full population-based genetic algorithms such as AdaLead (Sinai, S. et al., 2020), which mutates and recombines sequences to climb a fitness landscape, or reinforcement-learning schemes like DyNA-PPO (Angermueller, C. et al.) that refine a policy through repeated actions with well-defined states and rewards.
Because every move is hand-specified, these methods offer explicit control. Biologists can embed hard constraints (such as specific sites to avoid) and design interpretable operators e.g., “swap synonymous codons”, that reflect known molecular rules. Their sampling nature naturally supports diverse exploration: populations, mutations and crossovers help escape the local minima that often trap gradient descent. Finally, by depending only on a numerical reward, they can mix-and-match both differentiable and non-differentiable property predictors.
The trade-offs are real. Hand-crafted moves give no guarantee of optimality; the search may never stumble on remote high-fitness subspaces. Performance is hyper-parameter sensitive: mutation rate, population size and reward weights can each tilt the outcome and usually require empirical tuning. And because there is no convergence guarantee, it is hard to diagnose stagnation: a flat fitness curve might mean a global optimum, or that exploration got stuck on a plateau. For reliable deployment, one often needs to run many seeds and monitor diversity metrics.
Use the checklist below to decide which flavor of method fits your data, constraints and goals, and keep in mind that the two philosophies can always be mixed.
Layer 1: Predictor choice
Black‑box predictors are suitable when:
Mechanistic predictors are suitable when:
Layer 2: Optimization routine choice
End-to-end differentiable optimization routines are suitable when:
Sampling‑based optimization routines are suitable when:
Choose carefully. Resist the reflex to directly fit a neural network or code heuristic-based algorithms. Explore and understand the data you have, the constraints you face, and the confidence you need. The “right” pipeline is the one that consistently yields sequences with the properties that matter to your project, and does so with a level of certainty that lets you act on the results. Sometimes that means gradients all the way, sometimes pure rules, and often a blend that evolves as new data arrive.
I’m grateful to Eric J. Ma for encouraging me to start this blog, and for his thoughtful review and feedback of this post which significantly improved the clarity and organization of the final piece.