Mapping gene networks requires large amounts of transcriptomic data to learn the connections between genes, which impedes discoveries in settings with limited data, including rare diseases and diseases affecting clinically inaccessible tissues. Recently, transfer learning has revolutionized fields such as natural language understanding and computer vision by leveraging deep learning models pretrained on large-scale general datasets that can then be fine-tuned toward a vast array of downstream tasks with limited task-specific data. Here, we developed a context-aware, attention-based deep learning model, Geneformer, pretrained on a large-scale corpus of about 30 million single-cell transcriptomes to enable context-specific predictions in settings with limited data in network biology. During pretraining, Geneformer gained a fundamental understanding of network dynamics, encoding network hierarchy in the attention weights of the model in a completely self-supervised manner. Fine-tuning toward a diverse panel of downstream tasks relevant to chromatin and network dynamics using limited task-specific data demonstrated that Geneformer consistently boosted predictive accuracy. Applied to disease modeling with limited patient data, Geneformer identified candidate therapeutic targets for cardiomyopathy. Overall, Geneformer represents a pretrained deep learning model from which fine-tuning toward a broad range of downstream applications can be pursued to accelerate discovery of key network regulators and candidate therapeutic targets.
About the speaker
About the speaker
Christina Theodoris, MD, PhD, is an Assistant Investigator at Gladstone Institutes and the University of California, San Francisco (UCSF). In her undergraduate research at the California Institute of Technology, she worked in the Eric Davidson Lab studying gene regulatory networks in early sea urchin development. In her MD/PhD research at UCSF, mentored by Deepak Srivastava and co-mentored by Katherine Pollard and Benoit Bruneau, she developed an innovative network-based approach to therapeutic design leveraging machine learning and iPS cell disease modeling, which ultimately identified a candidate molecule for the treatment and prevention of cardiac valve disease, currently under further development toward clinical trials. As a postdoctoral fellow at the Broad Institute of MIT, Harvard, and the Dana-Farber Cancer Institute, co-mentored by X. Shirley Liu and Patrick Ellinor, she developed a foundational deep learning model pretrained on large-scale single cell transcriptomic data to enable context-specific predictions in settings with limited data in network biology through transfer learning. The Theodoris Lab develops machine learning models that leverage the unprecedented volume of transcriptomic and epigenomic data now available to gain a fundamental understanding of network dynamics that can be democratized to a vast array of downstream applications, accelerating the discovery of network-correcting therapies for human disease.
Christina Theodoris, MD, PhD, is an Assistant Investigator at Gladstone Institutes and the University of California, San Francisco (UCSF). In her undergraduate research at the California Institute of Technology, she worked in the Eric Davidson Lab studying gene regulatory networks in early sea urchin development. In her MD/PhD research at UCSF, mentored by Deepak Srivastava and co-mentored by Katherine Pollard and Benoit Bruneau, she developed an innovative network-based approach to therapeutic design leveraging machine learning and iPS cell disease modeling, which ultimately identified a candidate molecule for the treatment and prevention of cardiac valve disease, currently under further development toward clinical trials. As a postdoctoral fellow at the Broad Institute of MIT, Harvard, and the Dana-Farber Cancer Institute, co-mentored by X. Shirley Liu and Patrick Ellinor, she developed a foundational deep learning model pretrained on large-scale single cell transcriptomic data to enable context-specific predictions in settings with limited data in network biology through transfer learning. The Theodoris Lab develops machine learning models that leverage the unprecedented volume of transcriptomic and epigenomic data now available to gain a fundamental understanding of network dynamics that can be democratized to a vast array of downstream applications, accelerating the discovery of network-correcting therapies for human disease.
