Skip to content

Supported models

These models are implemented in vAi.

models/<impl>/<model>/
├── net.h                        Model and NodeGroup definitions
├── <distribution>-weights.h     Distribution checkpoint -> graph slot mapping (loadWeights<"distribution">)
├── nodes/                       Model-specific nodes (if any)
├── shaders/                     Kernels for model-specific nodes (if any)
├── demo.cpp                     demo-<model> - model inference and timing
└── CMakeLists.txt

data/ckpts/<model>/<distribution>/   Checkpoints
data/tokenizers/<model>/             Tokenizer
data/images/                         Input images

Loading weights

// GPT-2 example
#include "net.h"
#include "openai-weights.h"

namespace gpt2 = model::production::gpt2;

gpt2::Net net({ .device = device, .wDefaultDtype = Dtype::Fp16 },
              gpt2::ModelConfig::_124M());

loadWeights<"openai">(net);

vai/engine/weights.h only declares loadWeights<Name, Model>; each model provides specializations for its distributions.

For new models, see Writing custom models.

Model list

Vision

Model Model sizes
YOLOX SMALL(s), MEDIUM(m), LARGE(l), XLARGE(x)

LLM

Model Supported model sizes
GPT-2 124M, 355M, 774M, 1.5B
Llama 3 3.2-1B, 3.2-3B
Gemma 3 270M, 1B, 4B, 12B (4B and 12B also take images)