Step 1 — Read config.json, pick an adapter
When you load a Hugging Face model, the first file the server reads is config.json. The architectures field is a list; the first entry tells ModelRein which of its six adapters applies:
| architecture | ModelRein adapter |
|---|---|
LlamaForCausalLM, Qwen2ForCausalLM, MistralForCausalLM, … | causal_lm |
StableDiffusionPipeline, UNet2DConditionModel | unet |
StableDiffusionXLPipeline, PixArtAlphaPipeline, FluxPipeline | dit |
Qwen2VLForConditionalGeneration, LlavaForConditionalGeneration, BlipForConditionalGeneration | vlm |
MusicgenForConditionalGeneration, AudioLDM2Pipeline | audio_diffusion |
CogVideoXPipeline, ModelScopeT2VPipeline | video_dit |
If the architecture doesn't match any adapter, ModelRein returns a 400 at load with the message "no adapter for this architecture" and a pointer to the adapter interface. If it does match, we move to step 2.
Step 2 — Probe the model's shape
Different models have different layer counts, head counts, head dims. ModelRein reads them off the live model object rather than guessing from the architecture class:
# inside the causal_lm adapter
num_layers = len(model.model.layers) # Qwen path
# ... or ...
num_layers = len(model.transformer.h) # GPT-NeoX path
# ... or ...
num_layers = model.config.num_hidden_layers # universal fallback
num_heads = model.config.num_attention_heads
head_dim = model.config.head_dim or (hidden_size // num_heads)
Each adapter knows a handful of path variations. The runtime probe is the authoritative source — config.json is cross-checked but not trusted on its own, because some repos ship with stale configs.
Step 3 — Enumerate controllable surfaces
GET /lmcp/surfaces?model={repo_id} returns the live list of control points the server has detected on this model:
{
"model": "Qwen/Qwen2.5-1.5B-Instruct",
"arch_family": "causal_lm",
"n_surfaces": 112,
"by_kind": {
"residual_stream": 28,
"attention_output": 28,
"mlp_output": 28,
"logits": 1,
"embedding": 1,
"attention_heads": 28 * 12 // one per (layer, head)
}
}
This is what every subsequent primitive refers to by (kind, layer, head) tuple. The agent never has to know how a specific model wires those surfaces internally.
Step 4 — Hook the forward pass via nnsight
The actual activation reach-in is handled by nnsight. It opens a tracing context around the model's forward pass, and inside that context you can read or write any tensor by name. ModelRein's primitives are thin wrappers that build the nnsight trace, apply the intervention, and extract the result:
with model.generate(input_ids, max_new_tokens=80) as tracer:
# Steering: additive injection at the residual stream of layer L
for interv in interventions:
hs = model.model.layers[interv.layer].output[0]
hs[:] = hs + load_vec(interv.vec_id) * interv.adaptive_fraction
# Head mask: scale (layer, head) attention output
for mask in head_masks:
a_out = model.model.layers[mask.layer].self_attn.output[0]
# ... reshape by (batch, heads, seq, head_dim), scale one head, flatten
...
# Layer skip: route around a specific layer
for L in skip_layers:
model.model.layers[L].output = model.model.layers[L].input
The same code shape works across every causal-LM family. The underlying tensor paths differ per model but nnsight abstracts them.
Step 5 — Auto-strength calibration
One of ModelRein's design decisions is that intervention strength is expressed as an adaptive_fraction, not an absolute number. "0.5" means "half the typical residual-stream norm at this layer." The server measures the residual norm on a few probe inputs the first time you load the model, caches it, and uses it to translate the fraction to the right absolute scale for this particular model.
This means adaptive_fraction=1.0 produces comparable steering strength on Qwen 1.5B and on Llama 70B even though their absolute residual norms differ by an order of magnitude. The agent doesn't have to re-tune per model.
What happens the first time you load a new model
- Hugging Face download if not cached.
- Model instantiated via
transformers.from_pretrainedordiffusers.from_pretrained. config.jsonread, architecture matched to adapter.- Runtime probe for layer / head / dim counts.
- Residual-norm calibration on 2 probe prompts.
- Surface list emitted; model becomes queryable.
Total time: dominated by the download. Once the weights are local, it's a few seconds. After that, all 27 ModelRein primitives work against the model exactly like they do for every other model.
What this doesn't do (and shouldn't)
- Fine-tune. No gradient updates ever. ModelRein is purely inference-time.
- Change the model's output distribution for other users. Interventions are per-request.
- Guess the right layer for your task. It can offer a separability-based pick via
compute_vector_auto, but the authoritative test is behavioural — see the agent walkthrough.
Related: Controlling Qwen, Llama, DeepSeek with one agent · The full model matrix · What is ModelRein?
