Context Dependent: Multimodal Architectures for Social Grounding of Language Models
Visiting speaker
Oren Tsur
Assistant Professor, Ben-Gurion University
Past Talk
In-person talk
Friday
May 31, 2024
Watch video
1:00 pm
EST
Virtual
177 Huntington Ave.
11th floor
Devon House
58 St Katharine's Way
London E1W 1LP, UK
Online
Register here
State-of-the-Art Natural Language Processing (NLP) systems are trained on massive collections of data. Traditionally, NLP models are uni-modal: one form of data, e.g., textual data, is used for training. However, recent trends focus on multimodality, utilizing multiple forms of data in order to improve the system’s performance on classic tasks as well as broadening the capabilities of AI systems. Image and code are the two common modalities that are used in training popular tools such as OpenAI’s GPT and Google's Gemini, among other LLMs.. Language, however, is not merely a collection of stand-alone texts, nor texts merely grounded in image or aligned with code. Language is primarily used for communication between speakers in some social settings. The meaning (semantic, pragmatic) of a specific utterance is best understood by interlocutors that share some common ground and are aware of the context in which the communication takes place. In this talk I will demonstrate the benefits of the multi-modal framework through three unique tasks: conversational stance detection, the detection of hate mongers, and through modeling distributed large-scale coordinated campaigns.
About the speaker
About the speaker
Dr. Oren Tsur is an Assistant Professor (Senior Lecturer) at the Department of Software and Information Systems Engineering at Ben Gurion University in Israel where he heads the NLP and Social dynamics Lab (NASLAB), and serve as the director of the newly founded Interdisciplinary Center for the Study of Digital Politics and Strategy (DPS@BGU). Oren’s work combines Machine Learning, Natural Language Processing (NLP), Social Dynamics, and Complex Networks. Specifically, Oren’s work varies from sentiment analysis to modeling speakers’ language preferences, hates-speech detection, community dynamics, and adversarial influence campaigns. Oren serves as an editor and Senior Program Committee member in venues like ACL, EMNLP, WSDM and ICWSM and as a reviewer for journals ranging from TACL to PNAS and Nature. Oren’s work was published in top NLP and Web Science venues. His work/s on sarcasm detection was listed in the “top 50 inventions of the year” in Time Magazine’s Special technology issue. Academic homepage: https://www.naslab.ise.bgu.ac.il/orentsur
Dr. Oren Tsur is an Assistant Professor (Senior Lecturer) at the Department of Software and Information Systems Engineering at Ben Gurion University in Israel where he heads the NLP and Social dynamics Lab (NASLAB), and serve as the director of the newly founded Interdisciplinary Center for the Study of Digital Politics and Strategy (DPS@BGU). Oren’s work combines Machine Learning, Natural Language Processing (NLP), Social Dynamics, and Complex Networks. Specifically, Oren’s work varies from sentiment analysis to modeling speakers’ language preferences, hates-speech detection, community dynamics, and adversarial influence campaigns. Oren serves as an editor and Senior Program Committee member in venues like ACL, EMNLP, WSDM and ICWSM and as a reviewer for journals ranging from TACL to PNAS and Nature. Oren’s work was published in top NLP and Web Science venues. His work/s on sarcasm detection was listed in the “top 50 inventions of the year” in Time Magazine’s Special technology issue. Academic homepage: https://www.naslab.ise.bgu.ac.il/orentsur