Papers

The Trojan Horse in Your Pixels: How Image Adapters Enable a New Wave of AI Jailbreaking The rapid rise of Text-to-Image Diffusion Models (T2I-DMs) like Stable Diffusion, Midjourney, and DALL-E has revolutionized digital creativity. We can now conjure elaborate worlds with a simple sentence. However, with great power comes the inevitable security struggle: Jailbreaking. Jailbreaking, in the context of AI, refers to bypassing a model’s safety filters to generate prohibited content—usually NSFW (Not Safe For Work), violent, or illegal imagery. Until now, this was largely a game of “linguistic gymnastics,” where attackers tried to trick the model using clever text prompts. ...

Introduction In the rapidly evolving world of computer vision and computer graphics, Implicit Neural Representations (INRs) have become a cornerstone technology. Whether you are looking at Neural Radiance Fields (NeRFs) for 3D scene reconstruction or novel compression methods for images, INRs—which represent signals as continuous functions parameterized by neural networks—are everywhere. However, there is a fundamental tension in this field. On one side, we have fully implicit methods (like SIREN), which are compact but slow to train and suffer from “spectral bias” (struggling to learn high-frequency details). On the other side, we have grid-based (or hybrid) representations (like Instant NGP), which are incredibly fast and scalable but rely on discrete feature grids. ...

Stop Guessing, Start Learning: Automating PEFT Hyperparameters with Meta-Learning In the era of massive foundation models like GPT-4, CLIP, and Stable Diffusion, the paradigm of machine learning has shifted. We rarely train models from scratch anymore. Instead, we take a pre-trained giant and “fine-tune” it for our specific needs. But what happens when your specific need is incredibly niche, data-scarce, or imbalanced? Consider Remote Sensing (RS)—analyzing satellite imagery to detect specific types of ships or terrain. Training a foundation model from scratch for this is impossibly expensive. Full fine-tuning (updating all parameters) is computationally heavy and often leads to overfitting, especially on “tail classes” (rare objects that don’t appear often in the data). ...

Introduction The race to bridge the gap between 2D images and 3D content creation is moving at breakneck speed. We have seen massive leaps in diffusion models that can conjure images from thin air, and naturally, researchers are applying these principles to the third dimension. However, generating a high-quality, production-ready 3D asset from a single image remains a formidable challenge. Current state-of-the-art methods generally fall into two camps: optimization-based methods (which are slow) and Large Reconstruction Models (LRMs) that predict 3D representations like NeRFs or Gaussians directly. While LRMs are fast, they often struggle when converting those volumetric representations into clean, editable meshes. Furthermore, “native” 3D diffusion models—which attempt to learn the distribution of 3D shapes directly—often produce overly simple, symmetric blobs that lack the sharp geometric details of the input image. ...

Introduction For decades, the field of computer vision has chased a specific dream: taking a handful of flat, 2D photographs and instantly converting them into a perfect, navigable 3D world. This process, known as photogrammetry, is the backbone of modern 3D content creation, mapping, and special effects. However, the traditional road to 3D reconstruction is bumpy. It usually involves a fragmented pipeline of disparate algorithms—one to figure out where the cameras were looking, another to estimate depth, and yet another to stitch it all together. ...

In the world of computer graphics, creating a 3D model is only half the battle. The shape—or geometry—gives an object its form, but the material gives it its soul. Is the object made of shiny gold, dull wood, or rusted iron? How does light bounce off its scratches? For years, creating these physically accurate materials has been a tedious bottleneck. Artists often use complex software like Substance 3D Painter to manually paint textures. While recent AI advancements have automated 3D geometry generation, they often fail at the next step: generating high-quality materials. Most AI models simply “paint” colors onto a shape, baking in lighting and shadows that make the object look fake when moved to a new environment. ...

Solving the Invisible Hand: How ManiVideo Masters 3D Occlusion in Video Generation If you have ever tried to draw a hand, you know the struggle. Getting the proportions right is hard enough, but the real nightmare begins when the fingers start curling, overlapping, and gripping objects. Suddenly, parts of the hand disappear behind the object, or behind other fingers. Now, imagine teaching an AI to not just draw this interaction, but to generate a temporally consistent video of it. ...

The transition from a black-and-white sketch to a fully colored character is one of the most labor-intensive steps in animation and comic production. For decades, artists have manually filled in colors, ensuring that a character’s hair, eyes, and clothing remain consistent across thousands of frames. While automated tools have attempted to speed up this process, they often stumble when presented with a simple reality of animation: characters move. When a character turns their head, zooms in, or changes pose, the geometry of the line art changes drastically compared to the reference sheet. Traditional colorization algorithms frequently fail here, painting outside the lines or confusing the color of a shirt with the color of the skin. ...

Decoding the Wild: How Multimodal AI is Revolutionizing Wildlife Monitoring in the Swiss Alps Imagine attempting to document the daily lives of elusive mountain creatures—Red Deer, Wolves, or Mountain Hares—without ever stepping foot in the forest. For decades, ecologists have relied on camera traps to act as their eyes in the wild. These motion-activated sensors capture millions of images and videos, offering an unprecedented glimpse into biodiversity. However, a new problem has emerged: we have too much data. With modern camera traps capable of recording high-definition video for weeks on end, researchers are drowning in footage. Manually annotating this data to understand not just what animal is present, but what it is doing (behavior), is a Herculean task. ...

Introduction Imagine trying to track a friend in a crowded video. Sometimes you know what they look like (a visual reference), and sometimes you only know a description, like “the person wearing a red hat.” Now, imagine the video is long. Your friend changes pose, walks behind a tree, or takes off the hat. To keep tracking them effectively, you need memory. You need to remember their history to predict where they are now. ...

Imagine training a self-driving car algorithm entirely inside the video game Grand Theft Auto V. The roads look realistic, the lighting is perfect, and the weather is controlled. Now, take that same car and drop it onto a rainy street in London at night. Does it crash? This scenario represents the core challenge of Domain Generalized Semantic Segmentation (DGSS). We want models that learn from a “source” domain (like a simulation or a sunny dataset) and perform flawlessly in “target” domains (real-world, bad weather, night) without ever seeing them during training. ...

In the rapidly expanding worlds of video games, VR, and the metaverse, 3D content creation is booming. We have incredible tools to generate static 3D models from text or images, resulting in millions of digital assets. However, a significant bottleneck remains: movement. A static 3D model is essentially a digital statue. To make it move—to make it run, jump, or dance—it must undergo two complex processes: rigging (building a digital skeleton) and skinning (defining how the surface moves with that skeleton). Traditionally, this is the domain of skilled technical artists, taking hours of manual labor per character. Even existing automated tools often struggle with non-standard body shapes or characters that aren’t standing in a perfect “T-pose.” ...

Imagine dumping a folder of random photos—taken with different cameras, from different angles, with no metadata—into a system and getting a perfect, dense 3D model out the other side. This is the “Holy Grail” of geometric computer vision: unconstrained Structure-from-Motion (SfM). Recently, a method called DUSt3R (Dense Unconstrained Stereo 3D Reconstruction) made waves by solving this problem without the complex, handcrafted pipelines of traditional photogrammetry. It treated 3D reconstruction as a simple regression task. However, DUSt3R had a significant Achilles’ heel: it was fundamentally a pairwise method. It looked at two images at a time. If you wanted to reconstruct a scene from 100 images, you faced a combinatorial explosion of pairs to process and a messy global alignment problem to stitch them together. ...

Introduction Imagine you are driving down a busy highway. You see a car merging from the right, a truck braking in front of you, and a pedestrian waiting on a corner. Instantly, your brain maps these objects in 3D space, assigns them importance, and formulates a plan: “Slow down for the truck, watch the merging car.” You don’t think in raw GPS coordinates or pixel values. You think in terms of objects and relationships. ...

Introduction In the era of AI-Generated Content (AIGC), the volume of visual media being created and shared online is exploding. From social media feeds to generative art platforms, the flow of images is endless. But with this creativity comes a significant risk: the proliferation of harmful content, ranging from graphic violence to explicit material. For years, the standard solution has been human moderation. Platforms hire thousands of people to look at disturbing images and label them as “safe” or “unsafe” based on a rulebook (a safety constitution). This approach has two massive problems: it is expensive and slow to scale, and it takes a heavy psychological toll on the human annotators. ...

Imagine you are watching a soccer game. If a player runs behind a referee, you don’t panic and assume the player has vanished from existence. Your brain uses context, trajectory, and perhaps a view from a different angle (if you were watching a multi-camera broadcast) to predict exactly where that player will emerge. Computer vision systems, however, often struggle with this exact scenario. In the world of Visual Object Tracking (VOT), losing sight of an object due to occlusion (being blocked by another object) is a primary cause of failure. Traditional trackers rely on a single camera view. If the target walks behind a pillar, the tracker is blind. ...

The dream of computer vision is simple yet incredibly difficult to achieve: take a handful of photos of an object or a scene, and instantly generate a perfect, photorealistic 3D model. In recent years, we have seen an explosion in “neural rendering” techniques. Methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have revolutionized our ability to synthesize novel views. They can take a set of images and allow you to look at the scene from a new angle with startling clarity. However, there is a catch. While these methods produce beautiful images, the underlying 3D geometry they create is often messy, noisy, or blurry. They are designed to fool the eye, not to build a solid mesh. ...

Introduction “In computer vision, there is only one problem: correspondence, correspondence, correspondence.” This famous quote by Takeo Kanade highlights a fundamental truth about how machines “see.” Whether a robot is navigating a room, an AI is editing a photo, or a system is tracking a moving car, the core task is almost always the same: identifying which pixel in Image A corresponds to which pixel in Image B. However, historically, we haven’t treated this as one problem. We have fragmented it into three distinct domains: ...

Visual Simultaneous Localization and Mapping (SLAM) is often considered the “Holy Grail” of spatial intelligence. Ideally, we want a robot or a pair of AR glasses to open its “eyes” (cameras), look at a scene, and immediately understand where it is and what the world looks like in 3D—without any manual setup. However, the reality of SLAM has traditionally been finicky. It usually requires strict hardware expertise, careful camera calibration (those checkerboard patterns), and reliable feature extractors. While “sparse” SLAM (tracking points) works well, “dense” SLAM (reconstructing the whole surface) remains computationally heavy and prone to drift. ...

Imagine showing an AI a video of a person boxing. The catch? They are doing it inside a library. A typical Video Large Language Model (Video-LLM) might look at the bookshelves and quiet atmosphere and completely ignore the boxing, describing the scene as “students reading.” Or, it might see the boxing motion and hallucinate a “boxing ring” in the background, ignoring the books entirely. This phenomenon is known as Action-Scene Hallucination. It occurs when a model relies too heavily on the context of the scene to guess the action, or uses the action to incorrectly infer the scene. ...