EMNLP 2024

Introduction If you have ever asked a Large Language Model (LLM) like ChatGPT about a controversial political topic, you have likely encountered a very specific type of response. It might be a bland refusal to answer, a “both-sides” hedge that says nothing of substance, or—as recent research has increasingly shown—a response that subtly (or overtly) leans toward a specific socio-political worldview. Most off-the-shelf LLMs exhibit what researchers call “normative stances.” They tend to reflect the biases present in their training data or the specific “safety” tuning applied by their creators. Often, this results in models that exhibit progressive, liberal, and pro-environmental biases. While these are not inherently negative traits, they pose a problem for the utility of AI in a democratic society. If a voter uses an AI to understand the political landscape, but the AI can only speak in the voice of a liberal progressive, that voter is getting a distorted view of reality. ...

The Confidence Trap Imagine you ask a friend, “Who won the championship?” If your friend is a tennis fanatic, they might immediately say, “Novak Djokovic.” If they love golf, they might say, “Scottie Scheffler.” But if they know a little bit about everything, they will pause and ask you: “Which sport and which year are you talking about?” That pause is intelligence. It is the recognition of ambiguity. Large Language Models (LLMs) are notoriously bad at this pause. Trained to predict the next likely token, they often prioritize fluency over accuracy. When faced with a vague query like “Who won the championship?”, an LLM is statistically likely to pick the most popular entity in its training data and present it as an absolute fact. It falls into a “confidence trap,” hallucinating a specific answer to a general question. ...

Imagine a friend telling you, “I’m fine.” Depending on their tone, pitch, and speed, they could mean they are genuinely happy, indifferent, or potentially furious. For a long time, AI has treated speech emotion as a classification task—simply categorizing that audio clip into buckets like “Sad,” “Happy,” or “Angry.” But human emotion is rarely that simple. It is nuanced, mixed, and evolving. A simple label fails to capture the complexity of a voice that is “trembling with excitement” or “speaking quickly with a veiled tone of dissatisfaction.” ...

Introduction: The Black Box of Academic Publishing If you are a student or researcher, you likely know the anxiety that comes after clicking the “Submit” button on a conference paper. For the next few months, your work enters a “black box.” Inside, anonymous reviewers judge your methods, debate your findings, and ultimately decide the fate of your research. Peer review is the cornerstone of scientific integrity, yet it is notoriously fraught with challenges. It suffers from high variance (the “luck of the draw” with reviewers), potential biases against novice authors, and the opaque motives of the reviewers themselves. We know these problems exist, but studying them scientifically is incredibly difficult. Privacy concerns prevent us from seeing who reviewed what, and the sheer number of variables—from the reviewer’s mood to the Area Chair’s leadership style—makes it nearly impossible to isolate specific causes for a rejection. ...

Introduction Imagine showing a picture of a fluffy, greyish-white dog to a state-of-the-art AI. The model immediately springs into action, describing the dog’s pointed ears, its curly tail, and the texture of its fur. It might even tell you that this is a loyal companion breed. But when you ask, “What specific breed is this?”, the model confidently replies: “It is a Samoyed.” The problem? It’s actually a Keeshond. This scenario highlights a critical gap in modern Artificial Intelligence. While Large Vision-Language Models (LVLMs)—like LLaVA, GPT-4V, or Gemini—demonstrate incredible reasoning and descriptive capabilities, they often stumble on what seems like a simple task: fine-grained object classification. They can explain the context of an image but fail to identify the specific entity within it. ...

Large Language Models (LLMs) like GPT-4 and Llama 3 have revolutionized how we interact with text. They can summarize novels, write code, and even pass bar exams. However, when it comes to specific medical diagnostics—such as detecting dementia from speech patterns—these powerful models often hit a wall. Despite their vast training data, standard prompting strategies (like asking the model “Does this person have dementia?”) often yield results that are barely better than a coin flip. The reason? The linguistic markers of dementia are subtle, inconsistent, and poorly defined, making it difficult for an LLM to leverage its internal knowledge effectively. ...

Introduction Imagine you have trained a state-of-the-art speech recognition model. In the quiet confines of your laboratory, it performs flawlessly. It transcribes every word with near-perfect accuracy. Then, you deploy it into the real world. Suddenly, the model faces the hum of an air conditioner, the unique cadence of a non-native speaker, or perhaps someone humming a tune while they speak. Performance plummets. This phenomenon is known as domain shift—the mismatch between the clean data the model was trained on and the messy, “wild” data it encounters during deployment. ...

Humans are fundamentally social creatures. Our history, culture, and survival have depended on our ability to interpret a raised eyebrow, understand a pause in conversation, or sense the mood in a room. We call this Social Intelligence. As Artificial Intelligence becomes more integrated into our daily lives—from healthcare robots to education assistants and customer service chatbots—the demand for these systems to understand us socially is growing. We don’t just want AI that can calculate or retrieve information; we want agents that can empathize, collaborate, and adhere to social norms. ...

Introduction In the world of Natural Language Processing (NLP), determining whether two sentences mean the same thing is a cornerstone task. Known as Semantic Textual Similarity (STS), this capability powers everything from search engines and recommendation systems to plagiarism detection and clustering. For years, the industry has oscillated between two major paradigms. On one side, we have Sentence-BERT, a reliable architecture that encodes sentences independently. On the other, we have the modern heavyweights of Contrastive Learning (like SimCSE), which have pushed state-of-the-art performance to new heights. ...

If you have ever asked a Large Language Model (LLM) like ChatGPT to solve a complex math problem, you might have noticed a fascinating quirk. Sometimes, the model gets the right answer for the wrong reasons. Other times, it starts perfectly, makes a single logical slip in the middle, and spirals into a hallucination. This inconsistency stems from how these models process reasoning. They generate text token by token, and once a mistake is made, it’s hard for the model to recover. To fix this, researchers have been developing “Verifiers”—auxiliary models designed to check the LLM’s work. ...

Large Language Models (LLMs) have transformed how we seek information. Instead of browsing through ten different search results, we now get a concise summary generated instantly. But there is a well-known catch: hallucinations. LLMs can sound incredibly confident while inventing facts entirely. To solve this, researchers and engineers have been pushing for attributed text generation. This is essentially asking the model to show its work—generating text with inline citations (like [1], [2]) pointing to specific evidence documents. When a model attributes its claims, users can verify the information, making the system trustworthy. ...

Imagine asking a powerful AI, “Who is the President of the United States?” The answer seems simple, but for an AI processing millions of documents ingested from the internet, it is anything but. One document from 2008 might say Barack Obama. Another from 2024 says Joe Biden. A historical text might discuss the powers of the “POTUS” generally. When an AI encounters this, it usually forces a single answer, potentially hallucinating certainty where none exists. ...

Imagine you are chatting with a friend about movies. You ask, “Who directed Inception?” Your friend replies, “Christopher Nolan.” You then ask, “What else did he direct?” Your friend instantly knows “he” refers to Nolan. But if you type “What else did he direct?” into a standard search engine, it fails miserably. It lacks the context. This is the fundamental challenge of Conversational Question Answering (CQA). To bridge the gap between human conversation and search engines, we use Query Rewriting (QR). A QR model translates “What else did he direct?” into “What movies did Christopher Nolan direct?” ...

Introduction In the world of Natural Language Processing (NLP), Deep Neural Networks (DNNs) are the reigning champions. They power everything from sentiment analysis on e-commerce sites to toxic comment detection on social media. However, these models have a significant Achilles’ heel: they are brittle. A slight, often imperceptible change to an input sentence—known as an adversarial attack—can cause a state-of-the-art model to completely misclassify the text. While there has been a massive amount of research into breaking English models, the security of models processing Chinese—the second most popular language on the internet with over a billion users—has been surprisingly underestimated. ...

Introduction Language is rarely neutral. When we write or speak about different social groups—whether defined by nationality, race, or gender—we often rely on subtle associations that frame how those groups are perceived. These associations are what we call social stereotypes. For years, natural language processing (NLP) researchers have tried to quantify these biases. Early attempts were groundbreaking but somewhat blunt, often relying on static word embeddings to show, for example, that “man” is to “computer programmer” as “woman” is to “homemaker.” While useful for identifying broad societal biases, these methods struggle with nuance. A social group like “Canadians” or “Chinese” isn’t stereotyped in the exact same way across every context. The stereotypes applied in a political discussion differ vastly from those in a discussion about sports or economics. ...

When you ask a professor a simple question like “What is 2 + 2?”, you expect a simple answer: “4.” But if you ask, “How does a neural network learn?”, you expect a detailed, step-by-step explanation. If the professor gave a twenty-minute lecture on number theory just to answer “2 + 2,” you’d be confused. Conversely, if they answered the deep learning question with a single word, you’d learn nothing. ...

Introduction Imagine you have trained a state-of-the-art AI model to classify text. It works perfectly on your test data. Then, a malicious actor changes a single word in an input sentence—swapping “bad” with “not good”—and suddenly, your model’s prediction flips completely. This is an adversarial attack, and it is one of the biggest vulnerabilities in modern Natural Language Processing (NLP). To fix this, researchers typically use Adversarial Training (AT), where they force the model to learn from these tricky examples during training. However, this comes with a heavy price: ...

In the traditional world of machine learning, there is a golden rule that almost always holds true: if you want a model to perform better on a specific topic, you train it on data from that topic. If you want a neural network to recognize cats, you show it more cats. If you want a language model to understand biology, you train it on biology papers. But in the era of Large Language Models (LLMs), this intuitive logic is beginning to fracture. ...

The Moral Mirror: How LLMs Can Be Prompted to Justify Sexism Large Language Models (LLMs) are often described as the sum of human knowledge found on the internet. They have read our encyclopedias, our codebases, and our novels. But they have also read our comment sections, our arguments, and our biases. While significant effort goes into “aligning” these models to be helpful, honest, and harmless, the underlying training data still contains a spectrum of human values—ranging from progressive ideals to regressive prejudices. ...

Introduction The rapid evolution of Large Language Models (LLMs) like Llama-2 and RoBERTa has revolutionized natural language processing. However, adapting these massive models to specific tasks—a process known as fine-tuning—presents a significant computational barrier. As model sizes balloon into the billions of parameters, the GPU memory required to train them via standard methods becomes prohibitively expensive. The culprit is backpropagation. In traditional First-Order (FO) optimization (like SGD or AdamW), the training process requires storing intermediate activation values for every layer to calculate the gradient chain rule. For a multi-billion parameter model, this “memory wall” often requires clusters of enterprise-grade GPUs. ...