Supported models¶
These models are implemented in vAi.
models/<impl>/<model>/
├── net.h Model and NodeGroup definitions
├── <distribution>-weights.h Distribution checkpoint -> graph slot mapping (loadWeights<"distribution">)
├── nodes/ Model-specific nodes (if any)
├── shaders/ Kernels for model-specific nodes (if any)
├── demo.cpp demo-<model> - model inference and timing
└── CMakeLists.txt
data/ckpts/<model>/<distribution>/ Checkpoints
data/tokenizers/<model>/ Tokenizer
data/images/ Input images
Loading weights¶
// GPT-2 example
#include "net.h"
#include "openai-weights.h"
namespace gpt2 = model::production::gpt2;
gpt2::Net net({ .device = device, .wDefaultDtype = Dtype::Fp16 },
gpt2::ModelConfig::_124M());
loadWeights<"openai">(net);
vai/engine/weights.h only declares loadWeights<Name, Model>; each model provides specializations for its distributions.
For new models, see Writing custom models.
Model list¶
Vision¶
| Model | Model sizes |
|---|---|
| YOLOX | SMALL(s), MEDIUM(m), LARGE(l), XLARGE(x) |
LLM¶
| Model | Supported model sizes |
|---|---|
| GPT-2 | 124M, 355M, 774M, 1.5B |
| Llama 3 | 3.2-1B, 3.2-3B |
| Gemma 3 | 270M, 1B, 4B, 12B (4B and 12B also take images) |