ModelRein's six architecture adapters cover every major open-weight model family. Pick any model from Hugging Face Hub, point the server at its repo ID, and all 27 ModelRein primitives (sampling, logit bias, residual-stream steering, attention-head masking, layer routing, cross-model pipelines) work against it. Below: the current support matrix.
Transformer decoder-only LLMs. Full ModelRein surface: sampling, logit bias, EOS ramps, residual steering (any layer), attention head masking (any layer, any head), layer routing (skip / repeat), introspection.
| Family | Sizes | Vendor | ModelRein capabilities | Status |
|---|---|---|---|---|
| Qwen 2.5 / Qwen 3 | 0.5B – 72B | Alibaba | generatesteerhead_masklayer_routelogit_bias | validated |
| Llama 3 / 3.1 / 3.2 / 3.3 | 1B – 405B | Meta | generatesteerhead_masklayer_routelogit_bias | validated |
| DeepSeek V3 / R1 | 7B – 685B | DeepSeek AI | generatesteerhead_masklayer_route | validated |
| Mistral / Mixtral | 7B – 8×22B | Mistral AI | generatesteerhead_masklayer_route | validated |
| Gemma 2 / Gemma 3 | 2B – 27B | generatesteerhead_masklayer_route | validated | |
| GLM 4 | 9B – 32B | Zhipu AI | generatesteerhead_mask | validated |
| Phi 3 / 3.5 / 4 | 1.3B – 14B | Microsoft | generatesteerhead_mask | validated |
| Yi 1.5 | 6B – 34B | 01.AI | generatesteerlayer_route | tested |
| Falcon 3 | 1B – 10B | TII | generatesteer | tested |
| OLMo 2 | 1B – 13B | AI2 | generatesteer | tested |
Any AutoModelForCausalLM | any | Hugging Face | generatesteerhead_masklayer_route | by introspection |
Stable Diffusion lineage. ModelRein exposes residual edits at each UNet block, cross-attention head control, timestep interventions, VAE latent probes.
| Model | Resolution | Vendor | ModelRein capabilities | Status |
|---|---|---|---|---|
| Stable Diffusion 1.5 | 512 | Runway | imagexattn_masktimestep | validated |
| Stable Diffusion 2.x | 768 | Stability AI | imagexattn_masktimestep | validated |
Diffusion transformers. Fastest adapter: SDXL-turbo generates a 1024×576 scene in ~0.6 s on an RTX PRO 6000, drives the parallax videos and game-sprite pipeline.
| Model | Resolution | Vendor | ModelRein capabilities | Status |
|---|---|---|---|---|
| SDXL-turbo | 1024 | Stability AI | imagexattn_masktimestep | validated |
| SDXL base / refiner | 1024 | Stability AI | imagexattn_mask | validated |
| PixArt-Sigma | 1024 / 2K | Huawei Noah's Ark | imagexattn_mask | validated |
| FLUX.1-dev / schnell | 1024 | Black Forest Labs | imagexattn_mask | tested |
Vision-language models and captioners. Used in the compositional pipeline to describe SDXL renders, SAM regions, and video keyframes.
| Model | Sizes | Vendor | ModelRein capabilities | Status |
|---|---|---|---|---|
| Qwen2-VL | 2B / 7B / 72B | Alibaba | captiongenerateprojector | validated |
| LLaVA 1.5 / 1.6 | 7B / 13B / 34B | Microsoft/UW | captiongenerate | validated |
| BLIP / BLIP-2 | base / large | Salesforce | caption | validated · heavily used |
| InternVL 2.5 | 1B – 78B | Shanghai AI Lab | captiongenerate | tested |
Not language models, but ModelRein wraps them as pipeline tasks so a single call can chain render → depth → segment → caption.
| Model | Task | Vendor | Capabilities | Status |
|---|---|---|---|---|
| Depth-Anything-V2 | monocular depth | ByteDance / TikTok | depth | validated · heavily used |
| SAM / SAM-ViT-Base | segmentation | Meta AI | segment | validated · heavily used |
| RMBG-2.0 / 1.4 | background removal | Bria AI | matte | optional |
| Model | Task | Vendor | Capabilities | Status |
|---|---|---|---|---|
| MusicGen small / medium | text → music | Meta | musicconditioning | validated · heavily used |
| Whisper base / large | speech → text | OpenAI (open-weight) | transcribe | validated |
| AudioLDM 2 | text → audio | UMusic / HKUST | audio | tested |
| Stable Audio Open | text → stems | Stability AI | audio | tested |
| Model | Resolution | Vendor | Capabilities | Status |
|---|---|---|---|---|
| CogVideoX-2b / 5b | 720×480 · 6s | Zhipu AI / THUDM | videotimestep | validated |
| Zeroscope v2 | 576×320 | cerspense | video | tested |
| ModelScope T2V | 256×256 | Damo-ViLab | video | tested |
If the model loads via AutoModelForCausalLM.from_pretrained(repo_id) or DiffusionPipeline.from_pretrained(repo_id), no new code is required — ModelRein introspects it at load time. Just add the repo ID to the server's allow-list and all primitives work. For architectures that don't match any of the six adapters, implement the adapter interface (one Python file, ~150 lines) and open a PR against the reference server.