Skip to content

YOLOX

YOLOX (Megvii, 2021) is a single-pass object detection model. It consists of a CSPDarknet backbone with Focus at the input, a YOLOPAFPN neck, and an anchor-free detection head with separate regression and classification branches. At three scales (strides 8, 16, and 32), it produces boxes, objectness scores, and class probabilities. It uses checkpoints for the 80 COCO classes.

The supported sizes are Small, Medium, Large, and XLarge. YOLOX-Nano uses depthwise convolution (groups = in_channels), so it is currently excluded; support is planned.

References: Repository, Paper, Official checkpoints

Configuration

namespace yolox = model::experimental::yolox;

auto cfg = yolox::getConfigCOCO(yolox::Tier::LARGE);

yolox::Net net({
    .device        = device,
    .wDefaultDtype = Dtype::Fp16,
}, cfg, {
    .imageSize     = 640,
    .confThreshold = 0.25f,
    .iouThreshold  = 0.45f,
});

NeuralNetConfig

YOLOX uses the defaults. Both conv weights and BN parameters are stored in wDefaultDtype (fp16). The weight loader computes BN scale and bias on the CPU, so there is no separate precision setting. For details, see Using vAi > NeuralNet.

ModelConfig

Tier determines the base channel width (baseChannels) and repeat count (baseDepth), and getConfigCOCO(tier) adds 80 classes to create the ModelConfig.

Tier baseChannels baseDepth Checkpoint
SMALL 32 1 yolox_s.pth
MEDIUM 48 2 yolox_m.pth
LARGE 64 3 yolox_l.pth
XLARGE 80 4 yolox_x.pth

Starting at the stem, the backbone channel widths are 1, 2, 4, 8, and 16 times baseChannels. The CSP repeat counts in dark2 through dark5 are 1, 3, 3, and 1 times baseDepth, respectively, and the neck's CSP repeat count is baseDepth. All COCO checkpoints use numClasses 80.

RuntimeConfig

These settings specify the input resolution and postprocessing thresholds. The training resolution of 640 is the best match, but other values are accepted.

Field Default
imageSize 640
confThreshold 0.25
iouThreshold 0.45

vAi graph

In vAi, a model is a DAG built by connecting nodes with -. For details, see Using vAi > Node and NodeGroup.

The model-specific node::FocusConv handles the stem's Focus operation, while the remaining convolutions use the built-in Convolution. For Conv-BN-SiLU blocks, the weight loader computes BN scale and bias on the CPU and fills those slots, and the node uses the filled slots and the SiLU option. Predictions in the detection head use 1x1 convolutions with bias.

H and W: input resolution, N: number of detection positions across all scales (8,400 at 640x640).

Model architecture

input("in0") - backbone;
backbone/"out_p3" - "in_p3"/neck/"out_p3" - "in_p3"/head;
backbone/"out_p4" - "in_p4"/neck/"out_p4" - "in_p4"/head;
backbone/"out_p5" - "in_p5"/neck/"out_p5" - "in_p5"/head;

for (const char* scale : {"p3", "p4", "p5"})
    for (const char* output : {"regobj", "cls"})
    {
        const std::string slot = std::string(output) + "_" + scale;
        head/slot - slot/detector;
    }
detector - output("out0");  // [N, 5 + numClasses]

The output has shape [8400, 85]. Each row contains cx, cy, width, height, objectness, class[0..79], with box coordinates in pixels after applying the grid and stride. Objectness and class values are probabilities after sigmoid. NMS runs outside the graph in postprocessing (models/experimental/yolox/processing.h). The output contains the same values as the batch-one output [1, 8400, 85] of the comparison frameworks.

NodeGroup

These node groups are defined for the YOLOX implementation. The four top-level groups are Backbone, Neck, Head, and DetectionDecoder, with FocusModule, Bottleneck, CSPLayer, SPPBottleneck, and stage-specific groups nested inside them.

FocusModule

The official Focus operation splits the input into even and odd rows and columns, concatenates them along the channel axis, and applies a 3x3 conv. vAi handles this with a single node::FocusConv. It computes Conv-BN-SiLU directly from the original input without creating an intermediate Focus tensor, converting [C, H, W] to [Co, H/2, W/2].

Bottleneck

This block connects a 1x1 conv and a 3x3 conv. When shortcut is enabled and the input and output channel counts match, the second conv adds the input as its residual.

input - conv1 - conv2;
input - "residual"/conv2;  // shortcut with matching channels only

CSPLayer

The input splits into two 1x1 conv branches. The first passes through a chain of Bottleneck groups, then concatenates with the second along the channel axis before a 1x1 conv sets the output channel count.

input - conv1 - bottlenecks[0];
for (uint32_t i = 1; i < repeats; ++i)
    bottlenecks[i - 1] - bottlenecks[i];
bottlenecks.back() - "in0"/concat;
input - conv2     - "in1"/concat;
concat - conv3;

SPPBottleneck

The official SPP's parallel 5x5, 9x9, and 13x13 max pooling is computed as three consecutive 5x5 pooling operations. The model-specific node::SPPF handles the pooling and concatenation of all four tensors, including the input, in one kernel.

conv1 - sppf - conv2;

Backbone, Neck, Head

Group Description Slots
Backbone Focus stem followed by four stride-2 conv + CSP stages (dark2 through dark5), with SPP added in the last stage in0 -> out_p3, out_p4, out_p5
Neck YOLOPAFPN: top-down FPN (upsample + concat + CSP), then bottom-up PAN (stride-2 conv + concat + CSP) in_p3..5 -> out_p3..5
Head A 1x1 stem per scale followed by two branches, regression/objectness and classification (two 3x3 convs + 1x1 pred) in_p3..5 -> regobj_p3..5 [5, Hₚ, Wₚ], cls_p3..5 [numClasses, Hₚ, Wₚ]
DetectionDecoder Reads all three scales' outputs directly and applies grid/stride decoding and sigmoid regobj_p3..5, cls_p3..5 -> out0 [N, 5 + numClasses]

A single five-channel conv computes the four regression channels and one objectness channel. node::Decode reads regobj and cls directly, so it does not need a [85, Hₚ, Wₚ] concatenation buffer for each scale.

Model-specific nodes

These operations are needed for YOLOX. Their implementations are in models/experimental/yolox/nodes/ and shaders/.

Node Description Kernel
node::FocusConv Fuses Focus and the stem's Conv-BN-SiLU focus-conv.comp, focus-conv-cm.comp
node::SPPF Three stages of max pooling and concatenation in one kernel sppf.comp
node::Decode Decodes regression/objectness and class outputs into [N, 5 + numClasses] decode.comp

The FP16 cooperative-matrix path in node::FocusConv transforms the weights once at initialization using focus-conv-transform.comp.

Nodes used

Convolution, NearestUpsample, Concat, Identity

Checkpoints

Place each tier's checkpoint in its own folder under data/ckpts/yolox/megvii/. The demo input image is in data/images/.

data/
├── ckpts/yolox/megvii/
│   ├── large/
│   │   ├── model.pth             Megvii original
│   │   └── model.safetensors     Conversion output: the file read by the loader
│   └── ...
└── images/
    └── dog.jpg                   Demo input

Download the .pth files distributed by Megvii, then convert them to .safetensors.

1. Download the required checkpoint from the official release and save it as data/ckpts/yolox/megvii/<small|medium|large|xlarge>/model.pth. For Large:

mkdir -p data/ckpts/yolox/megvii/large
curl -L -o data/ckpts/yolox/megvii/large/model.pth \
  https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_l.pth

2. Convert the file. The converter reads the checkpoint's model state dict, so it does not require the YOLOX package.

pip install torch safetensors
python models/experimental/yolox/megvii-weights-convertor.py                # all available files
python models/experimental/yolox/megvii-weights-convertor.py small large    # select tiers

This creates model.safetensors next to each model.pth.

Load Weight

The code that maps checkpoint tensors to graph slots is in models/experimental/yolox/megvii-weights.h. Loading takes one line at the call site.

loadWeights<"megvii">(net);

The specialization constructs the path from net.cfg.tier (VAI_DATA_DIR "/ckpts/yolox/megvii/" + tierName(net.cfg.tier)), so the caller only selects the tier. For distribution tags, see Supported models > Loading weights.

The official state dict keys backbone.backbone, backbone, and head map to the backbone, neck, and head members, respectively. Conv weights and BN parameters are read at the net's wDefaultDtype, while the head's regression and objectness weights and biases are combined into regObjPred.

Running

Directory structure

models/experimental/yolox/
├── net.h                       Model and NodeGroup definitions
├── nodes/                      Model-specific nodes (FocusConv, SPPF, Decode)
├── shaders/                    Kernels for model-specific nodes
├── megvii-weights.h             Megvii checkpoint mapping
├── megvii-weights-convertor.py   .pth -> .safetensors conversion
├── demo.cpp                    demo-yolox: detect objects in one image and measure timing
├── processing.h                Preprocessing and postprocessing (letterbox, NMS)
└── CMakeLists.txt

data/
├── ckpts/yolox/megvii/
│   └── <tier>/model.safetensors      Tier: small, medium, large, xlarge
└── images/
    └── dog.jpg

Build and run

cmake -B build
cmake --build build --config RelWithDebInfo --target demo-yolox

# Run
./build/bin/demo-yolox

The demo runs detection on data/images/dog.jpg with the LARGE tier and fp16, then prints a profile by kernel type. Change the tier and inference settings at the top of demo.cpp.

Preprocessing converts the RGB image loader's output to BGR CHW, places it from the top-left corner while preserving its aspect ratio, and fills the remaining area with 114. Input values are floats in the range 0-255, without 1/255 normalization.