EMNLP 2024

Have you ever searched for a “camping stove” online, added it to your cart, and then been flooded with recommendations for… more camping stoves? While modern e-commerce recommendation systems are incredibly powerful, they often suffer from a fundamental misunderstanding of why a user is shopping. They excel at identifying product similarity (“You liked this stove, here is another stove”) or co-buying patterns (“People who bought this stove also bought this fuel canister”). However, they struggle to capture the broader User Intent. ...

Decoding the French Flow: A Data-Efficient Approach to Pronunciation Learning If you have ever tried to learn French, you have likely encountered a specific frustration. You learn a word, you memorize how to pronounce it, and then you hear a native speaker say it in a sentence, and it sounds completely different. This is not just a problem for language learners; it is a massive headache for Text-to-Speech (TTS) systems. While modern TTS systems are incredibly advanced, achieving natural-sounding speech in French requires mastering complex phonological rules where words blend into one another. ...

When we interact with Large Language Models (LLMs) like ChatGPT or Llama, we often view them as magic boxes: we input a prompt, and a coherent response appears. However, under the hood, these models are fundamentally next-token predictors. They output a probability distribution over a vocabulary of thousands of tokens. The process of converting these probabilities into the fluent text we read is called decoding. For a long time, the community operated on “folk wisdom” derived from older, task-specific models (like those used for translation in 2018). But do these rules apply to the massive, general-purpose LLMs of today? A recent paper, “A Thorough Examination of Decoding Methods in the Era of LLMs,” provides a comprehensive answer. ...

Introduction We are living in the golden age of Large Language Models (LLMs). Every week, a new model drops—claiming to be faster, smarter, and more capable than its predecessors. We see charts showing massive bars, higher numbers, and claims of “state-of-the-art” performance on benchmarks like MMLU or HumanEval. But here is the uncomfortable question: Can we actually trust these numbers? Evaluating an LLM is not like compiling code; it is not a binary pass/fail. It is a complex, nuanced process fraught with invisible pitfalls. If you change the way you phrase a question slightly, the model’s score might plummet. If the model was accidentally trained on the test questions (data contamination), it’s cheating without knowing it. If the code used to grade the model’s answers is buggy, the leaderboard ranking is meaningless. ...

Introduction Since the arrival of Transformers and Large Language Models (LLMs) like GPT-4 and LLaMA, a singular question has dominated the field of Natural Language Processing (NLP): Do these models actually reason, or are they just sophisticated parrots? We know LLMs are incredibly proficient at language. They can write poetry, summarize emails, and even generate code. But proficiency in language does not automatically equate to proficiency in logic. When a human solves a math problem or a logic puzzle, they are (usually) applying strict deductive rules. When an LLM does it, are they doing the same? Or are they acting as “soft reasoners”—emulating the appearance of reasoning based on statistical patterns and surface-level semantics? ...

Introduction In recent years, the field of Natural Language Processing (NLP) has undergone a paradigm shift. We have moved from training specific models for specific tasks to utilizing massive, general-purpose Large Language Models (LLMs) like GPT-4 and Llama. What makes these models truly revolutionary is not just their size, but their ability to learn a new task simply by looking at a few examples, without any update to their internal parameters. This phenomenon is known as In-Context Learning (ICL). ...

If you have ever interacted with a customer service chatbot, you have likely hit a wall. You ask a question, perhaps phrased slightly differently than the bot expects, or about a topic that feels relevant but is technically “new,” and you get the dreaded response: “I’m sorry, I didn’t understand that.” This limitation stems from a fundamental design choice in traditional conversational AI: the static ontology. Most systems are built on a pre-defined list of things they can understand. If a user’s request falls outside that list, the system fails. But the real world is dynamic. New products launch, new terminologies emerge (think “social distancing” or specific vaccine brand names in 2020), and user needs evolve rapidly. ...

In the current era of Natural Language Processing (NLP), Large Language Models (LLMs) often feel like magic. You feed in a sentence, and out comes a translation, a summary, or a poem. However, despite their prowess, these neural models remain “black boxes.” They rely on statistical probabilities rather than explicit understanding, which can lead to hallucinations, logical inconsistencies, or a lack of interpretability. This brings us to a pivotal question: How do we inject structure, logic, and explicit meaning into these systems? ...

In the modern digital landscape, Artificial Intelligence is no longer just a futuristic concept; it is an active gatekeeper. Algorithms decide which comments on social media are flagged as “hate speech,” which customer service tickets are prioritized based on “sentiment,” and sometimes even which resumes get passed to a human recruiter. But what if these gatekeepers are xenophobic? What if a simple change of a name—from “John” to “Ahmed” or “Santiago”—drastically alters how an AI interprets the exact same sentence? ...

Have you ever typed a message in a hurry, hit send, and then realized your phone’s autocorrect turned a heartfelt compliment into complete nonsense? In English, spelling errors are usually about letter arrangements. In Chinese, however, the problem is far more complex due to the nature of the language. Because Chinese input methods rely heavily on Pinyin (phonetic typing), a simple slip of the finger or a similar-sounding syllable can result in a completely different character with a radically different meaning. ...

The race for larger context windows in Large Language Models (LLMs) is one of the most exciting developments in AI. We have moved rapidly from models that could barely remember a few paragraphs to systems like GPT-4 and Gemini 1.5 that can process entire novels, codebases, or legal contracts in a single prompt. However, this capability comes with a massive computational cost. The bottleneck is often memory. Specifically, the Key-Value (KV) Cache. ...

Introduction Imagine you are watching a three-minute video of someone assembling a complex piece of furniture. At the end, I ask you: “What was the very first tool the person picked up?” To answer this, you have to remember the beginning of the video, understand the sequence of actions, and identify the object. For humans, this is trivial. For AI, it is notoriously difficult. While computer vision has mastered short clips (5–10 seconds), understanding “long-range” videos (minutes or hours) remains a significant hurdle. Traditional methods try to cram massive amounts of visual data into memory banks or complex spatiotemporal graphs, often hitting computational bottlenecks. ...

Vocabulary acquisition is often the bane of a student’s existence. Whether preparing for the GRE, learning a new language, or mastering medical terminology, the sheer volume of new terms can be overwhelming. Cognitive science has long offered a solution: keyword mnemonics. These are memorable verbal links that connect a new, complex term to a simpler, familiar keyword, followed by an explanation that bridges the two. For example, to learn the word Benevolent (meaning kind), you might link it to Benefit. The explanation? “A boss who gives their employees benefits is kind—or benevolent.” ...

Large Language Models (LLMs) like GPT-4 and Claude 3 have dazzled the world with their ability to write code, compose poetry, and solve complex problems. When we see an LLM answer a classic riddle or a logic puzzle correctly, it is tempting to attribute human-like reasoning capabilities to the machine. We assume the model “understands” the logic. But what if that understanding is brittle? What if the model isn’t solving the logic puzzle, but rather recognizing the specific words—the tokens—used in the puzzle? ...

Introduction In the evolving landscape of Artificial Intelligence, we often find ourselves managing two distinct types of “brains.” on one side, we have Knowledge Graphs (KGs). These are structured, logical databases that map the world into entities (nodes) and relations (edges). They are precise and factual but often brittle; if a connection is missing, the system fails to see the relationship. On the other side, we have Large Language Models (LLMs) like GPT-4 or Llama 3. These possess vast, general-world knowledge and can generate human-like text, but they can be prone to “hallucinations” and are computationally expensive to update or fine-tune. ...

Large Language Models (LLMs) like GPT-4 or LLaMA are often described as having “emergent abilities”—capabilities that appear as the models scale in size. One of the most fascinating, and controversial, of these behaviors is memorization. Memorization occurs when an LLM generates content verbatim from its training data. On one hand, this allows models to act as vast knowledge bases, recalling historical facts or coding syntax. On the other hand, it poses significant privacy and copyright risks. If a model was trained on sensitive personal data or copyrighted books, elicitng that exact text is a major vulnerability. ...

If you read the sentence “The dog bit the man,” you know exactly who is in trouble. If you swap the words to “The man bit the dog,” the meaning flips entirely. This is because English relies heavily on word order to convey meaning. To understand the sentence, you need to know not just what words are present, but where they are sitting. But what if you spoke a language where “The dog bit the man” and “Man dog bit the” meant the exact same thing? ...

In the fast-moving world of Artificial Intelligence, a Large Language Model (LLM) is often only as good as its most recent data. We all know the frustration of asking a chatbot about a recent event, only to be told, “My knowledge cutoff is…” To keep models relevant, engineers must perform version updates. As new data continuously emerges, models need to ingest it. However, this creates a massive logistical and financial headache. Do you retrain the whole model from scratch every time (insanely expensive)? Or do you just train on the new data (computationally cheap, but often degrades performance)? ...

Introduction Imagine you are building a digital life assistant. A user types: “I want to buy a vehicle suited for weekend field adventures.” Your system, trained on broad categories, successfully identifies the intent as “Buy Vehicle.” Based on this, it recommends a sleek, high-speed roadster. The user is frustrated. A roadster is terrible for “field adventures.” The system failed because it only understood the coarse-grained category (Vehicle) but missed the fine-grained nuance (Off-road Vehicle). ...

Introduction In the world of Natural Language Processing (NLP), Named-Entity Recognition (NER) is a cornerstone task. We typically ask models to read a sentence and highlight specific items—persons, organizations, locations, or medical symptoms. For years, the standard approach has been to treat these entities as solid blocks of text. If you see “New York City,” you draw a box around three consecutive words. But human language is rarely so tidy. Especially in specialized fields like biomedicine, entities often break apart, wrap around other words, or share components. Consider the phrase: “The patient suffered pain in the left and right knees.” ...