{"publication":{"abstract":"Auditing the evolutionary information encoded by MSA Pairformer is a critical step in understanding the behavior of protein language models (pLMs) that leverage external context in the form of MSAs. In this study, we bridge MSA Pairformer with classical phylogenetics to quantify how the model's internal sequence weighting maps onto inferred phylogenetic trees. We show that while MSA Pairformer effectively encodes evolutionary relatedness, and does so in specific layers, this signal is a subtle optimization rather than a load-bearing pillar for contact prediction accuracy. This work is intended for biologists and model developers seeking to interpret how evolutionary history is represented within pLMs. It provides a framework for evaluating the \"phylogenetic operating range\" of MSA-based models and highlights the need for future architectures to selectively gauge input signal reliability.","body":"# View the notebook\n\n::::::div{.info-box}\nThe **full pub** is available [here](https://arcadia-science.github.io/2025-phylogenetic-analysis-of-msa-pairformer).\n\nThe **source code** to generate it is available in [this GitHub repo](https://github.com/Arcadia-Science/2025-phylogenetic-analysis-of-msa-pairformer) (DOI: [10.5281/zenodo.18423956](https://doi.org/10.5281/zenodo.18423956)).\n::::::\n\nIn the future, we hope to host [notebook pubs](https://dx.doi.org/10.57844/arcadia-ca21-23bb) directly on our publishing platform. Until that’s possible, we’ll create stubs like this with key metadata like the DOI, author roles, citation information, and an external link to the pub itself.","contributors":[{"user_id":4278,"role":"Critical feedback","first_name":"Rishi","last_name":"De-Kayne","avatar":"avatars/users/user_4278_1783705269155.jpg","in_byline":0,"priority":null,"affiliations":[]},{"user_id":4278,"role":"Editing","first_name":"Rishi","last_name":"De-Kayne","avatar":"avatars/users/user_4278_1783705269155.jpg","in_byline":0,"priority":null,"affiliations":[]},{"user_id":4228,"role":"Critical feedback","first_name":"James R.","last_name":"Golden","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/James-Arcadia-headshot.png","in_byline":0,"priority":null,"affiliations":[]},{"user_id":4228,"role":"Supervision","first_name":"James R.","last_name":"Golden","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/James-Arcadia-headshot.png","in_byline":1,"priority":null,"affiliations":[]},{"user_id":4228,"role":"Validation","first_name":"James R.","last_name":"Golden","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/James-Arcadia-headshot.png","in_byline":0,"priority":null,"affiliations":[]},{"user_id":4229,"role":"Conceptualization","first_name":"Evan","last_name":"Kiefl","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Evan-Arcadia-headshot.png","in_byline":1,"priority":null,"affiliations":[]},{"user_id":4229,"role":"Formal analysis","first_name":"Evan","last_name":"Kiefl","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Evan-Arcadia-headshot.png","in_byline":1,"priority":null,"affiliations":[]},{"user_id":4229,"role":"Investigation","first_name":"Evan","last_name":"Kiefl","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Evan-Arcadia-headshot.png","in_byline":1,"priority":null,"affiliations":[]},{"user_id":4229,"role":"Methodology","first_name":"Evan","last_name":"Kiefl","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Evan-Arcadia-headshot.png","in_byline":1,"priority":null,"affiliations":[]},{"user_id":4229,"role":"Software","first_name":"Evan","last_name":"Kiefl","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Evan-Arcadia-headshot.png","in_byline":1,"priority":null,"affiliations":[]},{"user_id":4229,"role":"Visualization","first_name":"Evan","last_name":"Kiefl","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Evan-Arcadia-headshot.png","in_byline":0,"priority":null,"affiliations":[]},{"user_id":4229,"role":"Writing","first_name":"Evan","last_name":"Kiefl","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Evan-Arcadia-headshot.png","in_byline":1,"priority":null,"affiliations":[]},{"user_id":4213,"role":"Critical feedback","first_name":"Austin H.","last_name":"Patton","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Austin-Arcadia-headshot.png","in_byline":0,"priority":null,"affiliations":[]},{"user_id":4213,"role":"Editing","first_name":"Austin H.","last_name":"Patton","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Austin-Arcadia-headshot.png","in_byline":0,"priority":null,"affiliations":[]},{"user_id":4222,"role":"Critical feedback","first_name":"Ryan","last_name":"York","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Ryan-Arcadia-headshot.png","in_byline":0,"priority":null,"affiliations":[]},{"user_id":4222,"role":"Supervision","first_name":"Ryan","last_name":"York","avatar":"https://thestacks-01.s3.amazonaws.com/users/avatars/Ryan-Arcadia-headshot.png","in_byline":1,"priority":null,"affiliations":[]}],"created_at":"2026-01-30T18:49:31.000Z","doi":"10.57844/arcadia-ppdz-asaz","version_doi":null,"feedback_form_embed_src":null,"id":116,"license":"CC BY","linked_assets":[{"type":"Code","url":"https://github.com/Arcadia-Science/2025-phylogenetic-analysis-of-msa-pairformer","name":""},{"type":"Data","url":"  https://doi.org/10.5281/zenodo.18318397","name":""}],"orgs":[{"org_id":2,"priority":null,"slug":"arcadia-science","name":"Arcadia Science","avatar":"avatars/orgs/org_2_1766013565.jpg"}],"pdf_url":"https://thestacks-01.s3.us-west-2.amazonaws.com/publications/result-phylo-analysis-msa-pairformer/result-phylo-analysis-msa-pairformer_v2.pdf","references":[],"slug":"result-phylo-analysis-msa-pairformer","social_posts_count":null,"social_posts_embed_src":"https://publishing-tools.arcadiascience.com/twitter/274217294577868942","state":"PUBLISHED","subtitle":"​​We characterize how MSA Pairformer encodes phylogenetic relationships. Sequence weights correlate with evolutionary distance, with distinct layers specializing as phylogenetic filters. Yet uniform averaging often outperforms learned weights for contact prediction.","tags":["result","feedback requested"],"title":"MSA-based pLMs encode evolutionary distance but don’t reliably exploit it","version_desc":"Removed social feed from main text body","version_number":2,"versions":[{"id":435,"version_number":2,"version_desc":"Removed social feed from main text body","doi":null,"created_at":"2026-05-07T18:06:43.000Z"},{"id":403,"version_number":1,"version_desc":"","doi":null,"created_at":"2026-01-30T18:49:31.000Z"}]}}