Papers

Introduction Imagine trying to draw a precise blueprint of a house, but all you have is a grainy, satellite-like scan taken from a plane flying overhead. Some parts of the roof are missing, trees are blocking the walls, and the data is just a collection of scattered dots. This is the reality of reconstructing 3D building models from airborne LiDAR (Light Detection and Ranging) point clouds. Building reconstruction is a cornerstone technology for smart cities, autonomous driving, and Virtual Reality/Augmented Reality (VR/AR). While we have become good at capturing data, turning that raw, noisy data into clean, lightweight “wireframes” (skeletal representations of geometry) remains a massive challenge. ...

Introduction Imagine driving down a highway at night in pouring rain. Your eyes strain to distinguish between a parked car on the shoulder and a shadow, or between a distant streetlight and an oncoming vehicle. Now, imagine you are a computer algorithm trying to do the same thing. In autonomous driving, the vehicle’s “brain” typically relies on a Bird’s-Eye-View (BEV) representation. This is a top-down, grid-like map of the surroundings generated from onboard cameras and LiDAR sensors. This map is the foundation for everything the car does next: detecting objects, predicting movement, and planning a path. ...

Introduction Imagine you are standing in the middle of a room, holding a camera, and taking a \(360^{\circ}\) panoramic photo. Now, walk into the next room and take another one. Can you reconstruct the entire floor plan of the house—accurate to the centimeter—just from those two photos? This is the problem of Wide-Baseline Floor Plan Reconstruction, and it is notoriously difficult. Unlike a video feed where camera frames are millimeters apart, wide-baseline images are taken far apart (often in different rooms). The visual overlap is small, and traditional computer vision algorithms struggle to stitch these “islands” of data together into a coherent map. ...

For the last few years, the recipe for building a Vision-Language Model (VLM) has been relatively static. If you wanted a model that understood how images and text relate—like OpenAI’s CLIP—you needed to collect a massive dataset of hundreds of millions of image-text pairs and train two neural networks (one for vision, one for text) from scratch. This process is computationally expensive, data-hungry, and often results in models that are “jacks of all trades, masters of none.” The vision encoder might be decent, and the text encoder might be passable, but neither is state-of-the-art compared to models dedicated to a single modality. ...

From Chaos to Code: Transforming Sparse Point Clouds into Structured 3D Buildings with ArcPro Imagine flying a drone over a city to map it. The drone captures thousands of images, and through photogrammetry, you generate a 3D representation of the scene. What you get back, however, is rarely a pristine, CAD-ready model. Instead, you get a “point cloud”—a chaotic swarm of millions of floating dots. If the scan is high-quality, the dots are dense, and you can see the surfaces clearly. But in the real world, data is often messy. Aerial scans can be sparse (containing very few points), noisy (points are in the wrong place), or incomplete (entire walls might be missing due to occlusion). ...

Introduction In the world of Computer Vision, things are surprisingly orderly. Whether you are training a model on ImageNet or your own collection of vacation photos, the data usually looks the same: standard RGB images, captured by standard cameras, often resized to a standard resolution (like \(224 \times 224\)). This uniformity has allowed models like ResNet and Vision Transformers (ViTs) to become powerful, general-purpose engines. But if you look at the planet from space, that order collapses into chaos. ...

Introduction In the world of medical diagnostics, there is rarely a single, indisputable truth. When three different radiologists look at a CT scan of a lung nodule or an MRI of a tumor, they will likely draw three slightly different boundaries around the lesion. This isn’t an error; it is the inherent ambiguity of medical imaging caused by blurred edges, low contrast, and complex anatomy. However, traditional Deep Learning models treat segmentation as a deterministic task. They are trained to output a single “correct” mask. This creates a disconnect between AI outputs and clinical reality. Furthermore, training these models requires massive datasets with pixel-perfect annotations, which are incredibly expensive and time-consuming to obtain. ...

If you have bought a high-end smartphone in the last few years, you have likely benefited from the rapid evolution of image sensors. The quest for instantaneous autofocus has driven hardware engineers to move from standard sensors to Dual-Pixel (DP) sensors, and more recently, to Quad Photodiode (QPD) sensors. While QPD sensors are designed primarily to make autofocus lightning-fast, they hide a secondary potential: depth estimation. Just as our two eyes allow us to perceive depth through stereo vision, the sub-pixels in a QPD sensor can theoretically function as tiny, multi-view cameras. However, extracting accurate depth (or disparity) from these sensors is notoriously difficult due to physical limitations like uneven lighting and microscopic distances between pixels. ...

Introduction In the world of computer vision and signal processing, noise is the enemy. Whether it’s grainy low-light photographs, medical imaging artifacts, or signal degradation in fiber optic cables, “denoising” is a fundamental step in making data usable. Traditionally, we rely on electronic chips (CPUs and GPUs) to clean up these images. We run heavy algorithms—from classical Weiner filtering to modern Convolutional Neural Networks (CNNs)—to guess what the clean image should look like. While effective, this approach hits a hard wall: latency and power consumption. Electronic computing involves moving electrons through transistors, which generates heat and takes time. When you need to process data in real-time, such as in high-speed fiber optic communications, electronic chips often become the bottleneck. ...

Introduction Imagine showing an AI a photo of a bustling street festival. If the festival is Mardi Gras in New Orleans, most top-tier AI models will instantly recognize the beads, the floats, and the context. But what if that photo depicts Mela Chiraghan in Pakistan or a traditional Angampora martial arts display in Sri Lanka? This is where the cracks in modern Artificial Intelligence begin to show. While Large Multimodal Models (LMMs)—systems that can see images and process text simultaneously—have made incredible leaps in capability, they possess a significant blind spot: the majority of the world’s cultures and languages. ...

Why AI Forgets: Solving Catastrophic Forgetting in Medical Imaging Artificial Intelligence has made massive strides in medical diagnostics, particularly in the analysis of pathology slides. However, there is a hidden problem in the deployment of these systems: they are static. In the fast-moving world of medicine, new diseases are discovered, new subtypes are classified, and scanning equipment is upgraded. Ideally, we want an AI model that learns continuously, adapting to new data without losing its ability to recognize previous conditions. This is the realm of Continual Learning (CL). But when researchers apply standard CL techniques to pathology, they run into a wall known as catastrophic forgetting. The model learns the new task but completely forgets the old one. ...

Imagine asking an AI to watch a two-hour movie and then asking, “What was the number on the jersey of the man in the background at the very end?” or “How did the protagonist’s relationship with her sister evolve from the first scene to the last?” For most current multimodal AI models, this is an impossible task. While models like GPT-4V or VideoLLaMA are impressive at analyzing short clips (typically 5 to 15 seconds), they hit a hard limit when the video stretches into minutes or hours. This limit is known as the Memory Wall. As a video gets longer, the number of visual “tokens” (pieces of information) the model must hold in its memory grows massively. Eventually, the GPU runs out of memory (OOM), or the model gets overwhelmed by noise and forgets the context. ...

Introduction: The Invisible World and the Iron Triangle Human vision is trichromatic; we perceive the world through a mix of red, green, and blue. However, the physical world is far richer. Every material interacts with light across a continuous spectrum of wavelengths, creating a unique “fingerprint” invisible to the naked eye. Hyperspectral Imaging (HSI) is the technology that allows us to see these fingerprints. By capturing hundreds of spectral bands instead of just three, HSI can distinguish between real and fake plants, detect diseases in tissue, or classify minerals in real-time. ...

Introduction In the rapidly evolving world of Generative AI, creating a 3D object from a single 2D image is something of a “Holy Grail.” We have seen tremendous progress with models that can turn a picture of a cat into a 3D mesh in seconds. However, if you look closely at the results of most current state-of-the-art models, you will notice a flaw: they look great from the original camera angle, but they often fail to react realistically to light. ...

Introduction Imagine you are playing a high-stakes match of virtual reality table tennis against a friend halfway across the world. You swing your controller, expecting your avatar to mirror the movement instantly. But there’s a catch: the internet connection fluctuates. Your swing data travels through a Wide Area Network (WAN), encountering unpredictable delays before reaching the game server or your opponent’s display. In the world of computer vision and robotics, this is known as the latency problem. Whether it is a surrogate robot replicating a human’s movements or a metaverse avatar interacting with a virtual environment, time delays caused by network transmission and algorithm execution are inevitable. ...

Fashion is an intrinsic part of human culture, serving as a shield against the elements and a canvas for self-expression. However, the backend of the fashion industry—specifically the creation of sewing patterns—remains a surprisingly manual and technical bottleneck. While generative AI has revolutionized 2D image creation (think Midjourney or DALL-E), generating manufacturable garments is a different beast entirely. A sewing pattern isn’t just a picture of a dress; it is a complex set of 2D panels with precise geometric relationships that must stitch together to form a 3D shape. To date, AI models for fashion have been “single-modal,” meaning they could perhaps turn a 3D scan into a pattern, or text into a pattern, but they lacked the flexibility to understand images, text, and geometry simultaneously. ...

In the rapidly evolving world of 3D computer vision, self-supervised pre-training has become the golden standard. Whether you are building perception systems for autonomous vehicles or analyzing 3D medical scans, the recipe for success usually involves taking a massive, unlabeled dataset, pre-training a Deep Neural Network (DNN) on it, and then fine-tuning it for your specific task. We know that it works. Pre-training consistently boosts performance. But why does it work? ...

Imagine you are watching a video of a cat playing with a toy. In a standard video, you are a passive observer, locked into the camera angle the videographer chose. Now, imagine you could pause that video at any second, grab the screen, and rotate the camera around the frozen cat to see the toy from the back. Then, you press play, and the video continues from that new angle. ...

The demand for high-quality 3D assets is exploding. From the immersive worlds of video games and virtual reality to the practical applications of architectural visualization and film production, the need for detailed, realistic 3D models is higher than ever. Traditionally, creating these assets has been a labor-intensive bottleneck, requiring skilled artists to sculpt geometry, paint textures, and tune material properties manually. In recent years, Generative AI has promised to automate this pipeline. We’ve seen models that can turn text into 3D shapes or turn a single image into a rotating mesh. However, a significant gap remains between what AI generates and what professional graphics engines actually need. Most current AI models produce “baked” assets—meshes with color painted directly onto the vertices. They often look like plastic toys or clay models, lacking the complex material properties (like how shiny metal is versus how matte rubber is) required for photorealistic rendering. ...

Introduction In the rapidly evolving world of computer vision, the quest to reconstruct reality inside a computer has seen massive leaps in just a few years. We started with photogrammetry, moved to the revolutionary Neural Radiance Fields (NeRFs), and most recently arrived at 3D Gaussian Splatting (3DGS). 3DGS changed the game by allowing for real-time rendering and fast training speeds that NeRFs struggled to achieve. It represents a scene not as a continuous volume, but as millions of discrete 3D Gaussian “blobs.” While this works incredibly well for organic, fuzzy structures, it hits a wall when dealing with the man-made world. Look around you—walls, tables, screens, and buildings are defined by sharp edges and flat surfaces. Gaussians, by their nature, are soft, round, and diffuse. Trying to represent a sharp cube with round blobs is like trying to build a Lego house out of water balloons; you need an excessive amount of them to approximate the flat sides, and it’s still never quite perfect. ...