EMNLP 2024

Introduction In the current landscape of Artificial Intelligence, we are running into a bottleneck: high-quality human-generated data is becoming scarce and expensive. To circumvent this, the industry has turned to synthetic data—text generated by Large Language Models (LLMs) to train other LLMs. It is an appealing solution that promises infinite data at a fraction of the cost. However, this solution treats datasets as static commodities. We tend to assume that if a “Teacher” model (like GPT-4 or a large LLaMa model) generates data, the “Student” model will simply learn to be smarter. But learning is not just about facts and reasoning capabilities; it is also about style, bias, toxicity, and preference. When a student model trains on synthetic data, it inherits a complex web of latent characteristics from the teacher. ...

If you have ever played with a “base” language model—one fresh out of pre-training—you know it can be a bit of a wild card. It might ramble, complete your sentence instead of answering your question, or output something unsafe. To turn these raw computational engines into helpful assistants like ChatGPT or Llama-Instruct, we need alignment. Alignment is the process of fine-tuning a model to follow instructions and adhere to human preferences. While we have general ideas about how this works—usually a mix of Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF)—the specific “secret sauce” used by big labs (like OpenAI or Meta) is often kept closed-source. ...

Introduction Imagine you are trying to teach a Large Language Model (LLM) about the world. You train it on data up to 2023. In 2024, the Prime Minister of a country changes. You teach the model this new fact. In 2025, a new scientific element is discovered. You teach it that too. Here is the problem: In current deep learning architectures, when you teach the model the new fact, it has a nasty habit of forgetting the old one—or worse, its general reasoning capabilities start to degrade. This is known as Catastrophic Forgetting. ...

Introduction The rise of Large Language Models (LLMs) like GPT-4 and Llama has revolutionized how we interact with technology. From writing code to summarizing legal documents, these models seem capable of almost anything. However, for industries dealing with highly sensitive information—such as healthcare and finance—utilizing these powerful tools presents a massive dilemma. Hospitals want to train models on patient records to assist doctors, and banks want to analyze transaction histories to detect fraud. But this data is strictly private. You cannot simply upload patient history to a public API without violating privacy laws (like HIPAA or GDPR) and risking data leaks. ...

Large Language Models (LLMs) like GPT-4 and Llama 2 have revolutionized how we interact with technology. They can summarize documents, write code, and answer complex questions. But they have a well-known “kryptonite”: Hallucinations. An LLM hallucination occurs when the model generates content that sounds plausible but is factually incorrect or unfaithful to the source material. For a student writing a paper or a developer building a chatbot, this is a critical reliability issue. ...

Introduction: The “Yes-Man” Problem in AI Imagine you are a student taking a history exam. You encounter a question about a specific event you never studied and have zero knowledge about. In a multiple-choice setting, you might guess. But in an essay setting, if you are forced to write an answer, you might try to sound confident, fabricating details that sound plausible but are entirely fictional. Large Language Models (LLMs) face a strikingly similar dilemma during their training, specifically during the alignment phase. We train these models to be helpful, harmless, and honest. However, recent research suggests that the way we fine-tune these models often inadvertently encourages them to hallucinate. ...

Introduction Mental health counseling is a domain where every word matters. In a typical session, a therapist must balance two critical tasks: actively listening to the client to build a therapeutic bond, and meticulously documenting the session for future reference. This documentation, known as a “counseling note” or summary, is essential for tracking progress and ensuring continuity of care. However, the cognitive load of taking notes can distract the therapist, potentially weakening the connection with the client. ...

Beyond Cut and Paste: Editing LLMs with Knowledge Graphs Imagine you are reading a biography of LeBron James generated by a Large Language Model (LLM). The model correctly states, “LeBron James plays for the Los Angeles Lakers.” But when you ask, “Does LeBron James work in Los Angeles?” the model hesitates or, worse, confidently replies, “No, he works in Miami.” This inconsistency highlights a critical flaw in current AI systems. LLMs store vast amounts of world knowledge, but that knowledge can become outdated or remain factually incorrect. While we have methods to “edit” specific facts—like updating a database entry—LLMs struggle with the ripple effects of those edits. Changing a team affiliation should logically change the city of employment, the teammates, and the home stadium. ...

Imagine you ask a trusted friend, “Who won the most FIFA World Cup championships?” You expect them to say Brazil. But before they answer, you hand them a stack of news clippings. Some clippings confirm it’s Brazil, while others falsely claim it’s Germany or Argentina. Suddenly, your friend is conflicted. Do they rely on what they know to be true (Brazil), or do they trust the documents you just gave them? ...

Introduction We have all experienced it: you ask a Large Language Model (LLM) a specific, detailed question—perhaps about a medical condition or a historical event—and the answer comes back sounding incredibly confident. The grammar is perfect, the tone is professional, but the content is… slightly off. Maybe it misses a crucial detail, hallucinates a date, or presents arguments in a confusing order. Despite their massive pre-training on the internet, LLMs still struggle with knowledge-intensive tasks. They are excellent at mimicking the style of an expert but often fail to retrieve the specific substance required for complex queries. This leads to three main problems: ...

Introduction In the world of online gaming, there is a psychological phenomenon known as the “Proteus Effect.” It suggests that the appearance of a user’s digital avatar influences their behavior. If a player is given a tall, attractive avatar, they tend to act more confidently; if they are given an aggressive-looking warrior, they might act more confrontationally. But as we enter the era of Multi-modal Large Language Models (LLMs)—AI that can see as well as read—a fascinating question arises: Does the Proteus Effect apply to AI? ...

Introduction We live in an era where Artificial Intelligence is reshaping education. From homework helpers to interactive storytelling, Large Language Models (LLMs) are increasingly becoming a part of children’s daily lives. According to UNICEF, one in three internet users globally is a child. Yet, the very models designed to interact with them—ChatGPT, Llama, and others—are fundamentally not built for them. The problem lies in the data. Modern LLMs are trained on massive scrapes of the open internet. They learn the language of adults: complex sentence structures, nuanced debates, and, unfortunately, the toxicity and bias prevalent in online discourse. When we try to adapt these models for children using Supervised Fine-Tuning (SFT), we hit another roadblock: the annotators. The people labeling data to “teach” the AI how to behave are almost exclusively adults aged 18-35. ...

If you have ever played with a Large Language Model (LLM) like ChatGPT or Claude, you know that the magic doesn’t just lie in the model’s ability to predict the next word. It lies in the model’s ability to follow your instructions, answer your questions, and act as a helpful assistant. This capability is achieved through a process called Supervised Fine-Tuning (SFT). But here is the catch: SFT requires massive datasets of high-quality conversations—specifically pairs of (instruction, response). Curating these datasets by hand is incredibly expensive and slow. To solve this, researchers have turned to using LLMs to generate their own training data, a technique known as bootstrapping. ...

Introduction Large Language Models (LLMs) have revolutionized how we interact with information. However, they suffer from a well-known flaw: hallucination. When asked about specific, factual data—like the number of citations a researcher has or the specific rail network of a small town—LLMs often guess convincingly rather than answering accurately. To solve this, researchers link LLMs to external Knowledge Bases (KBs). Instead of answering directly, the LLM acts as a translator. It converts a natural language question (e.g., “Who is taller, LeBron James Jr. or his father?”) into a logical program (e.g., Find(LeBron James Jr.) -> Relate(Father) -> ...). This process is called Program Induction (PI). ...

If you have ever learned a new language, crammed for a medical board exam, or memorized trivia, you are likely familiar with Spaced Repetition Systems (SRS) like Anki or SuperMemo. These tools are the gold standard for efficient studying. They work by scheduling flashcards at the exact moment you are about to forget them, maximizing the efficiency of your memory. However, standard SRS algorithms have a significant blind spot: they are illiterate. ...

Imagine you have just launched a new streaming service. A new user signs up. You know their age and location, but you have zero data on what movies they actually like. What do you recommend? If you recommend a romantic comedy to a horror fan, they might churn immediately. This is the classic Cold Start Problem in recommender systems. The algorithm needs data to learn preferences, but it needs to make good recommendations to get that data. Traditionally, the system has to “explore” (make random guesses) before it can “exploit” (make smart choices), leading to a poor user experience in the early stages. ...

In the world of Artificial Intelligence, Knowledge Graphs (KGs) act as the structured memory for machines. They store vast amounts of data in the form of triples—(Head Entity, Relation, Tail Entity)—such as (Paris, is_capital_of, France). These graphs power everything from search engine sidebars to recommendation systems and question-answering bots. However, there is a fundamental problem: Knowledge Graphs are rarely complete. Real-world data is messy, and relationships are often missing. This has given rise to the field of Knowledge Graph Completion (KGC), which uses algorithms to predict missing links, like inferring (? , operates_system, iOS) implies Apple. ...

If you have ever worked in data science, you know the “80/20 rule”: you spend 80% of your time cleaning and preparing data, and only 20% actually analyzing it or building models. Data Preprocessing (DP) is the unglamorous backbone of the data pipeline. It involves fixing spelling errors, filling in missing values, matching records from different databases, and standardizing formats. Traditionally, this has been handled by a fragmented ecosystem of specialized tools—one algorithm for error detection, a completely different one for entity matching, and so on. ...

Large Language Models (LLMs) like GPT-4 and Claude 3 are designed to be helpful, but they are also designed to be safe. If you ask these models to write a guide on how to create malware or build a bomb, they are trained to refuse. This safety training, often achieved through Reinforcement Learning from Human Feedback (RLHF), acts as a firewall around the model’s vast knowledge. However, security researchers are constantly searching for cracks in this firewall. While most safety training focuses heavily on English, a new vulnerability has emerged in the linguistic “blind spots” of these models. ...

Imagine a virtual town populated entirely by AI agents. They wake up, go to work, gossip at the coffee shop, and negotiate prices at the market. It sounds like science fiction—specifically, like Westworld or The Sims powered by supercomputers—but recent advances in Large Language Models (LLMs) have brought us tantalizingly close to this reality. Researchers and developers are increasingly using LLMs to simulate complex social interactions. These simulations are used for everything from training customer service bots to modeling economic theories and testing social science hypotheses. The assumption is simple: if an LLM can write a convincing dialogue between two people, it can simulate a society. ...