YOLOX¶
YOLOX (Megvii, 2021) is a single-pass object detection model. It consists of a CSPDarknet backbone with Focus at the input, a YOLOPAFPN neck, and an anchor-free detection head with separate regression and classification branches. At three scales (strides 8, 16, and 32), it produces boxes, objectness scores, and class probabilities. It uses checkpoints for the 80 COCO classes.
The supported sizes are Small, Medium, Large, and XLarge. YOLOX-Nano uses depthwise convolution (groups = in_channels), so it is currently excluded; support is planned.
References: Repository, Paper, Official checkpoints
Configuration¶
namespace yolox = model::experimental::yolox;
auto cfg = yolox::getConfigCOCO(yolox::Tier::LARGE);
yolox::Net net({
.device = device,
.wDefaultDtype = Dtype::Fp16,
}, cfg, {
.imageSize = 640,
.confThreshold = 0.25f,
.iouThreshold = 0.45f,
});
NeuralNetConfig¶
YOLOX uses the defaults. Both conv weights and BN parameters are stored in wDefaultDtype (fp16).
The weight loader computes BN scale and bias on the CPU, so there is no separate precision setting.
For details, see Using vAi > NeuralNet.
ModelConfig¶
Tier determines the base channel width (baseChannels) and repeat count (baseDepth),
and getConfigCOCO(tier) adds 80 classes to create the ModelConfig.
Tier |
baseChannels |
baseDepth |
Checkpoint |
|---|---|---|---|
SMALL |
32 | 1 | yolox_s.pth |
MEDIUM |
48 | 2 | yolox_m.pth |
LARGE |
64 | 3 | yolox_l.pth |
XLARGE |
80 | 4 | yolox_x.pth |
Starting at the stem, the backbone channel widths are 1, 2, 4, 8, and 16 times baseChannels.
The CSP repeat counts in dark2 through dark5 are 1, 3, 3, and 1 times baseDepth, respectively,
and the neck's CSP repeat count is baseDepth. All COCO checkpoints use numClasses 80.
RuntimeConfig¶
These settings specify the input resolution and postprocessing thresholds. The training resolution of 640 is the best match, but other values are accepted.
| Field | Default |
|---|---|
imageSize |
640 |
confThreshold |
0.25 |
iouThreshold |
0.45 |
vAi graph¶
In vAi, a model is a DAG built by connecting nodes with -. For details, see Using vAi > Node and NodeGroup.
The model-specific node::FocusConv handles the stem's Focus operation, while the remaining convolutions use the built-in
Convolution. For Conv-BN-SiLU blocks,
the weight loader computes BN scale and bias on the CPU and fills those slots, and the node uses
the filled slots and the SiLU option. Predictions in the detection head use 1x1 convolutions with bias.
H and W: input resolution, N: number of detection positions across all scales (8,400 at 640x640).
Model architecture¶
input("in0") - backbone;
backbone/"out_p3" - "in_p3"/neck/"out_p3" - "in_p3"/head;
backbone/"out_p4" - "in_p4"/neck/"out_p4" - "in_p4"/head;
backbone/"out_p5" - "in_p5"/neck/"out_p5" - "in_p5"/head;
for (const char* scale : {"p3", "p4", "p5"})
for (const char* output : {"regobj", "cls"})
{
const std::string slot = std::string(output) + "_" + scale;
head/slot - slot/detector;
}
detector - output("out0"); // [N, 5 + numClasses]
The output has shape [8400, 85]. Each row contains cx, cy, width, height, objectness, class[0..79],
with box coordinates in pixels after applying the grid and stride. Objectness and class values are probabilities after sigmoid.
NMS runs outside the graph in postprocessing (models/experimental/yolox/processing.h).
The output contains the same values as the batch-one output [1, 8400, 85] of the comparison frameworks.
NodeGroup¶
These node groups are defined for the YOLOX implementation. The four top-level groups are Backbone, Neck, Head,
and DetectionDecoder, with FocusModule, Bottleneck, CSPLayer,
SPPBottleneck, and stage-specific groups nested inside them.
FocusModule¶
The official Focus operation splits the input into even and odd rows and columns, concatenates them along the channel axis, and applies a 3x3 conv.
vAi handles this with a single node::FocusConv. It computes Conv-BN-SiLU directly from the original input
without creating an intermediate Focus tensor, converting [C, H, W] to [Co, H/2, W/2].
Bottleneck¶
This block connects a 1x1 conv and a 3x3 conv. When shortcut is enabled and the input and output channel counts match,
the second conv adds the input as its residual.
CSPLayer¶
The input splits into two 1x1 conv branches. The first passes through a chain of Bottleneck groups,
then concatenates with the second along the channel axis before a 1x1 conv sets the output channel count.
input - conv1 - bottlenecks[0];
for (uint32_t i = 1; i < repeats; ++i)
bottlenecks[i - 1] - bottlenecks[i];
bottlenecks.back() - "in0"/concat;
input - conv2 - "in1"/concat;
concat - conv3;
SPPBottleneck¶
The official SPP's parallel 5x5, 9x9, and 13x13 max pooling is computed as three consecutive 5x5 pooling operations.
The model-specific node::SPPF handles the pooling and concatenation of all four tensors, including the input, in one kernel.
Backbone, Neck, Head¶
| Group | Description | Slots |
|---|---|---|
Backbone |
Focus stem followed by four stride-2 conv + CSP stages (dark2 through dark5), with SPP added in the last stage |
in0 -> out_p3, out_p4, out_p5 |
Neck |
YOLOPAFPN: top-down FPN (upsample + concat + CSP), then bottom-up PAN (stride-2 conv + concat + CSP) | in_p3..5 -> out_p3..5 |
Head |
A 1x1 stem per scale followed by two branches, regression/objectness and classification (two 3x3 convs + 1x1 pred) | in_p3..5 -> regobj_p3..5 [5, Hₚ, Wₚ], cls_p3..5 [numClasses, Hₚ, Wₚ] |
DetectionDecoder |
Reads all three scales' outputs directly and applies grid/stride decoding and sigmoid | regobj_p3..5, cls_p3..5 -> out0 [N, 5 + numClasses] |
A single five-channel conv computes the four regression channels and one objectness channel.
node::Decode reads regobj and cls directly, so it does not need a [85, Hₚ, Wₚ] concatenation buffer for each scale.
Model-specific nodes¶
These operations are needed for YOLOX. Their implementations are in models/experimental/yolox/nodes/ and
shaders/.
| Node | Description | Kernel |
|---|---|---|
node::FocusConv |
Fuses Focus and the stem's Conv-BN-SiLU | focus-conv.comp, focus-conv-cm.comp |
node::SPPF |
Three stages of max pooling and concatenation in one kernel | sppf.comp |
node::Decode |
Decodes regression/objectness and class outputs into [N, 5 + numClasses] |
decode.comp |
The FP16 cooperative-matrix path in node::FocusConv transforms the weights once at initialization
using focus-conv-transform.comp.
Nodes used¶
Convolution,
NearestUpsample,
Concat,
Identity
Checkpoints¶
Place each tier's checkpoint in its own folder under data/ckpts/yolox/megvii/.
The demo input image is in data/images/.
data/
├── ckpts/yolox/megvii/
│ ├── large/
│ │ ├── model.pth Megvii original
│ │ └── model.safetensors Conversion output: the file read by the loader
│ └── ...
└── images/
└── dog.jpg Demo input
Download the .pth files distributed by Megvii, then convert them to .safetensors.
1. Download the required checkpoint from the official release
and save it as data/ckpts/yolox/megvii/<small|medium|large|xlarge>/model.pth. For Large:
mkdir -p data/ckpts/yolox/megvii/large
curl -L -o data/ckpts/yolox/megvii/large/model.pth \
https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_l.pth
2. Convert the file. The converter reads the checkpoint's model state dict, so it does not require the YOLOX package.
pip install torch safetensors
python models/experimental/yolox/megvii-weights-convertor.py # all available files
python models/experimental/yolox/megvii-weights-convertor.py small large # select tiers
This creates model.safetensors next to each model.pth.
Load Weight¶
The code that maps checkpoint tensors to graph slots is in
models/experimental/yolox/megvii-weights.h.
Loading takes one line at the call site.
The specialization constructs the path from net.cfg.tier (VAI_DATA_DIR "/ckpts/yolox/megvii/" + tierName(net.cfg.tier)),
so the caller only selects the tier. For distribution tags, see Supported models > Loading weights.
The official state dict keys backbone.backbone, backbone, and head map to the backbone, neck, and head members, respectively.
Conv weights and BN parameters are read at the net's wDefaultDtype, while the head's regression and objectness weights and biases
are combined into regObjPred.
Running¶
Directory structure¶
models/experimental/yolox/
├── net.h Model and NodeGroup definitions
├── nodes/ Model-specific nodes (FocusConv, SPPF, Decode)
├── shaders/ Kernels for model-specific nodes
├── megvii-weights.h Megvii checkpoint mapping
├── megvii-weights-convertor.py .pth -> .safetensors conversion
├── demo.cpp demo-yolox: detect objects in one image and measure timing
├── processing.h Preprocessing and postprocessing (letterbox, NMS)
└── CMakeLists.txt
data/
├── ckpts/yolox/megvii/
│ └── <tier>/model.safetensors Tier: small, medium, large, xlarge
└── images/
└── dog.jpg
Build and run¶
cmake -B build
cmake --build build --config RelWithDebInfo --target demo-yolox
# Run
./build/bin/demo-yolox
The demo runs detection on data/images/dog.jpg with the LARGE tier and fp16, then prints a profile by kernel type.
Change the tier and inference settings at the top of demo.cpp.
Preprocessing converts the RGB image loader's output to BGR CHW, places it from the top-left corner while preserving its aspect ratio,
and fills the remaining area with 114. Input values are floats in the range 0-255, without 1/255 normalization.