lsantuari / balm-paper

Fork of the BALM repository from Burbach et al., 2023

Improving antibody language models with native pairing

To determine whether and the extent to which training with natively paired antibody sequence data can improve antibody-specific language models (LMs), we trained two baseline antibody language model (BALM) variants: BALM-paired, which is trained using only natively paired training data, and BALM-unpaied, which is trained using the same antibody sequences but without pairing information. Additionally, we performed full fine-tuning of the state-of-the-art general protein LM ESM-2 using the same natively paired dataset used to train BALM-paired. The Jupyter notebooks in this repository contain all code necessary to re-train each of these models from scratch:

• BALM-paired: downloads training data (if necessary) and trains BALM-paired.
• BALM-unpaired: downloads training data (if necessary) and trains BALM-unpaired.
• ESM-2 fine-tuning: downloads training data (if necessary) and performs full fine-tuning of ESM-2.

pre-trained models

Weights for each of the aforementioned models can be downloaded from Zenodo.

how should I cite BALM?

BALM has been published as a preprint on arXiv, and can be cited as:

Burbach SM, Briney B. Improving antibody language models with native pairing.
arXiv [q-bio.BM]. 2023. Available: http://arxiv.org/abs/2308.14300

The current version of the BALM dataset (v2023.08.17) can be cited as:

Burbach SM, Briney B. Improving antibody language models with native pairing (v2023.08.17) [Data set].
Zenodo. 2023. https://doi.org/10.5281/zenodo.8253367