Generative modeling of intrinsically disordered protein regions by reinforcing sparse autoencoder features
Intrinsically disordered protein regions (IDRs) are essential to cellular function but cannot be designed with structure-based methods because they lack stable folds. This work introduces IDiom, a protein language model pretrained on 54 million predicted IDRs, and RL-SAE, a reinforcement learning method that rewards the activation of interpretable sparse-autoencoder features to control sequence patterns. Together they enable composable, function-targeted generation of disordered sequences, though the results remain purely computational.
IDiom and RL-SAE present a compelling and well-engineered pipeline for computational IDR design. The base model generates diverse, low-identity sequences that recapitulate natural disorder statistics, while RL-SAE robustly installs interpretable features that steer predicted function. Nevertheless, the study is undermined by evaluation circularity—ProtGPS is evaluated on sequences drawn from the same data used to train ProtGPS and to select SAE targets—and by a complete absence of experimental validation. As the authors acknowledge, "the evaluations in this paper are computational and experimental validation is needed to confirm whether the predicted properties translate into biological function." (Limitations)
The strongest evidence is the sheer gap between RL-SAE and its baselines: the authors show that "across the 8 sets, sequences generated after RL-SAE activate on average 90% of the targeted top-30 features, compared to 24% for activation steering." (Section 5.2.2) Moreover, the enriched SAE features themselves correspond to recognizable biological grammars—such as FG/GLFG repeats for the nuclear pore complex and $\psi$KxE SUMOylation motifs for repression domains—and RL-SAE installs these motifs with far higher fidelity than supervised fine-tuning or steering. The composability demonstration further supports that the model can negotiate incompatible patterning requirements, such as segregating acidic activation blocks from basic chromosome-association blocks. (Section 5.2.3)
A critical flaw is the circular evaluation loop for compartment association: "Since the IDR sequences used for compartment association feature enrichment are extracted from the same full-length sequences used to train ProtGPS, there is circularity in using ProtGPS as a downstream evaluator." (Section 5.2.2) Although DeepLoc 2.1 is offered as an orthogonal check, it only provides coarse nuclear-versus-cytoplasmic labels and cannot fully resolve this issue.
Beyond circularity, the paper offers no wet-lab validation, and the authors concede that "our top-N feature selection is simplistic and may omit functionally relevant features or include redundant ones." (Limitations) The reward $R(y)=R_{S}(y)-[(L(y)-L_{\mathrm{target}})/L_{\mathrm{target}}]^{2}-[(H(y)-H_{\mathrm{target}})/H_{\mathrm{target}}]^{2}$ also treats feature activation as independent binary events, ignoring possible contextual or epistatic dependencies among SAE features.
The internal comparisons are generally fair within the paper’s own framework: RL-SAE outperforms activation steering and SFT on feature coverage and on most oracle scores. For example, "SFT raises the score above the base model, but SFT scores remain below those of the natural sequences for all six compartments, potentially due to the limited dataset size," whereas RL-SAE often surpasses natural sequences in mean ProtGPS score. (Section 5.2.2) However, the study does not benchmark against other recent IDR-specific generative models or modern alignment methods outside the SAE–RL paradigm, and the reliance on ProtGPS for both feature selection and evaluation means the reported improvements may partly reflect dataset memorization rather than generalizable design. The DeepLoc 2.1 results are reassuring but too coarse to fully compensate.
Reproduction is well-supported in principle: the authors state that "Code can be found at: https://github.com/rotskoff-group/idiom" and provide HuggingFace checkpoints. Appendices B and G further report pretraining, SAE, and GRPO hyperparameters. However, exact reproduction depends on re-curating IDiom-DB from the AlphaFold Database via several external predictors—MMseqs2, SignalP 6.0, and Metapredict V3—and on proprietary oracles such as ProtGPS and PADDLE that may not be freely available or bit-exact across versions. The reliance on a 302M-parameter model further raises the barrier to entry for independent groups with limited GPU resources.
Intrinsically disordered protein regions (IDRs) play central roles in cellular processes such as transcriptional regulation, signal transduction, and subcellular localization, yet their functional design remains challenging. Structure-based design methods do not readily apply to IDRs, and existing protein language models are trained on full-length protein sequences, thus learning a prior that is biased towards folded domains. Here, we present IDiom, an autoregressive protein language model trained on IDiom-DB, a dataset of 54 million predicted IDRs curated from the AlphaFold Database. IDiom generates diverse sequences that recapitulate the composition, patterning, motifs, and predicted disorder of natural IDRs. To control function-associated sequence patterns, we also introduce reinforcement learning with sparse autoencoder features (RL-SAE), a post-training method that rewards the generation of sequences that activate specified feature sets. Across eight IDR design tasks, RL-SAE sequences activate, on average, 90% of 30 targeted features, compared to 24% for activation steering. We demonstrate that RL-SAE improves the predicted subcellular localization and transcriptional activity of generated IDRs compared to steering and supervised fine-tuning, and enables features associated with distinct biological functions to be combined within individual sequences. Thus, IDiom and RL-SAE enable interpretable and composable IDR design through explicit control of function-associated sequence features. More broadly, RL-SAE could extend to other protein design settings where interpretable features provide useful design targets. Code is available at https://github.com/rotskoff-group/idiom.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.