In deep learning, building more powerful neural networks has traditionally followed two paths: making them deeper or making them wider.

The VGG architecture demonstrated the impact of depth, stacking many simple, repeated layers to great effect. ResNet introduced residual connections, enabling extremely deep networks to be trained without falling prey to the dreaded vanishing gradients. Meanwhile, Google’s Inception family charted a different course toward width, creating multi-branch modules with carefully designed parallel paths, each with specialized convolution filters.

But what if there’s another way?
What if, rather than only scaling depth or width, we could explore a new, third dimension in neural network design?

This is the core idea in “Aggregated Residual Transformations for Deep Neural Networks” by researchers from UC San Diego and Facebook AI Research, who introduce the ResNeXt architecture. ResNeXt builds on the “split-transform-merge” concept of Inception, but marries it to the simplicity and scalability of ResNet — introducing a new dimension called cardinality.

The surprising finding? Increasing cardinality — the number of parallel transformations within a block — can be more effective for improving accuracy than merely going deeper or wider.

In this article, we’ll unpack what cardinality means, how ResNeXt works, and why it represents a shift in how we think about scaling neural networks.

Background: From Feature Engineering to Network Engineering

The evolution of computer vision models has been a story of moving from handcrafted “feature engineering” (e.g., SIFT, HOG) to automated “network engineering,” where features are learned directly from data.

VGG and ResNet — Simplicity Meets Depth

The VGG network championed the idea of stacking repeated convolutional blocks, typically 3×3 layers, to create very deep architectures with a uniform, easy-to-configure design.

ResNet took this further by introducing residual (shortcut) connections, allowing information and gradients to flow more easily through the network. This made it possible to train extremely deep models with hundreds of layers.
A standard ResNet bottleneck block uses 1×1 convolutions to first reduce, and later restore, the number of channels so that the expensive 3×3 convolution works on a reduced representation.

Inception — Split, Transform, Merge

In contrast, the Inception modules explored width.
An Inception block:

  1. Splits the input into several lower-dimensional feature maps via 1×1 convolutions.
  2. Transforms each branch with filters of different sizes (3×3, 5×5, etc.).
  3. Merges the outputs via concatenation.

This design improves representational power for relatively low computational cost.
But it comes at a price: every branch is hand-crafted, with customized filter sizes and numbers, creating complexity and limiting portability to new tasks without manual tuning.

The question: Can we combine the simplicity of ResNet/VGG with the power of Inception’s split-transform-merge design?

The ResNeXt Idea: Cardinality

The ResNeXt innovation starts by revisiting the most fundamental unit in a neural network: the neuron.

A simple neuron performs a weighted sum of its inputs:

\[ \sum_{i=1}^{D} w_i x_i \]

A simple neuron performing an inner product operation, viewed as splitting, transforming (scaling), and aggregating its inputs.

Figure 2. A simple neuron’s operation can be seen as splitting each input \(x_i\), transforming it via multiplication by weight \(w_i\), and aggregating results via summation.

From this perspective, the neuron’s work involves:

  1. Splitting: Separating the input vector \( \mathbf{x} \) into smaller components.
  2. Transforming: Applying a function (scaling in the simple case).
  3. Aggregating: Summing the transformed results.

Aggregated Transformations

ResNeXt generalizes this notion:
Instead of each transformation being a simple scale, what if each transformation \( \mathcal{T}_i \) was itself a small neural network? Aggregating several such transformations leads to:

\[ \mathcal{F}(\mathbf{x}) = \sum_{i=1}^C \mathcal{T}_i(\mathbf{x}) \]

Here:

  • \( C \) = cardinality = number of parallel transformations.
  • Each transformation \( \mathcal{T}_i \) shares the same architecture (topology).

In practice, each \( \mathcal{T}_i \) is a bottleneck residual block:
1×1 conv → 3×3 conv → 1×1 conv, as in ResNet.

The aggregated transformation becomes the residual function:

\[ \mathbf{y} = \mathbf{x} + \sum_{i=1}^C \mathcal{T}_i(\mathbf{x}) \]

Figure 1. Standard ResNet bottleneck block (left) vs. ResNeXt block (right) with cardinality 32. Both have similar computational complexity.

Figure 1. Left: ResNet bottleneck (C=1, width=64). Right: ResNeXt block (C=32, width per path=4). The total compute is kept comparable while exploring higher cardinality.

Three Equivalent Views of ResNeXt Blocks

One of the most elegant aspects of ResNeXt is that a block can be expressed in three equivalent forms:

The three equivalent formulations of a ResNeXt block: (a) split-transform-sum, (b) early concatenation, and (c) grouped convolution.

Figure 3. Equivalent formulations of the same aggregated transformation.

  1. Split-Transform-Sum (Fig. 3a): Conceptually simple — split the input into \(C\) paths, transform each, sum the outputs.
  2. Early Concatenation (Fig. 3b): Similar to Inception-ResNet — transformations are concatenated, then merged via a 1×1 convolution.
  3. Grouped Convolution (Fig. 3c): Most efficient — one convolution layer with multiple groups, each acting independently on a subset of input channels. In ResNeXt, the 3×3 convolution is grouped with cardinality \(C\).

Grouped convolution, originating in AlexNet as a hardware workaround, here becomes a clean architecture tool. The beauty: the entire network can be built by stacking identical grouped-convolution-based blocks, following ResNet’s rules for downsampling.

Controlling for Complexity

To fairly measure the effect of cardinality, researchers maintained similar computational complexity and parameter count across comparisons.

Number of parameters in a ResNeXt bottleneck block:

\[ \text{Params} \approx C \cdot (256 \cdot d + 3\cdot 3 \cdot d \cdot d + d \cdot 256) \]
  • \(C\) = cardinality
  • \(d\) = bottleneck width (channels per path)

By inversely adjusting \(C\) and \(d\), they preserved FLOPs/params.
Example: ResNet bottleneck (C=1, d=64) ≈ ResNeXt bottleneck (C=32, d=4).

Trade-off between cardinality and bottleneck width to maintain similar block complexity.

Table 2. Width \(d\) decreases as cardinality \(C\) increases to maintain roughly constant complexity.

Experiments: Cardinality vs. Width/Depth

Cardinality at Constant Complexity

Under equal FLOPs, increasing cardinality reduced top-1 error:

Results on ImageNet-1K comparing ResNet to ResNeXt at equal complexity. Top-1 error improves as cardinality rises.

Table 3. Higher cardinality yields consistent accuracy gains without increasing compute.

Training curves for 50-layer (left) and 101-layer (right) models. ResNeXt (orange) outperforms ResNet (blue) at same complexity.

Figure 5. ResNeXt achieves lower training and validation error, pointing to stronger representation learning — not just regularization.

Example:

  • ResNet-50 (1×64d): 23.9% error
  • ResNeXt-50 (32×4d): 22.2% error

Doubling the Compute Budget

Given ~2× FLOPs of ResNet-101, how best to spend it?

  1. Go deeper (ResNet-200)
  2. Go wider (increase bottleneck width in ResNet-101)
  3. Increase cardinality (e.g., ResNeXt-101 64×4d)

Doubling complexity of ResNet-101: increasing cardinality shows largest gains.

Table 4. Cardinality provides the biggest drop in error compared to depth or width.

Result:

  • Depth gain (ResNet-200): ~0.3% improvement
  • Width gain: ~0.7% improvement
  • Cardinality gain (ResNeXt-101 64×4d): 1.6% improvement

Generalization to Other Tasks

The ResNeXt advantage extends across datasets:

  • ImageNet-5K: Larger datasets show even bigger accuracy gaps favoring ResNeXt.
  • CIFAR-10:
    On CIFAR-10, increasing cardinality (orange) is more parameter-efficient for lowering error than increasing width (blue).

Figure 7. Cardinality scales accuracy more efficiently per parameter than width.

  • COCO Object Detection: Using ResNeXt as backbone improves Faster R-CNN detection metrics over ResNet at equal complexity.

State-of-the-Art Results

ResNeXt achieves 4.4% single-crop top-5 error on ImageNet-1K with a \(320\times320\) input, outperforming ResNet, Inception-v3/v4, and Inception-ResNet-v2 — with a far simpler architecture.

ResNeXt achieved a new state-of-the-art single-crop top-5 error of 4.4% on ImageNet-1K.

Table 5. ResNeXt vs. state-of-the-art architectures on ImageNet-1K.

Conclusion & Implications

The ResNeXt work introduced cardinality as a third architectural dimension alongside depth and width:

  1. A New Lever for Accuracy: Cardinality — number of parallel transformations in a block — unlocks accuracy gains more efficiently than depth/width scaling.
  2. Efficiency: Maintains simplicity while improving representational power. Grouped convolution makes implementation easy and scalable.
  3. Generality: Benefits transfer across datasets and tasks, from image classification to object detection.

ResNeXt’s recipe is straightforward: take a proven architecture like ResNet, replace its bottleneck blocks with grouped-convolution blocks of higher cardinality. This single change can yield substantial accuracy gains without the overhead of complex, hand-crafted modules.

The idea has since influenced a range of state-of-the-art architectures — cementing ResNeXt as a milestone in neural network design.