The task
Make an open-source LLM reliably respond in Chinese to English prompts, without changing the prompt itself. We'll do it on Qwen 2.5, Llama 3.1, and DeepSeek V3 using the exact same agent code.
The naive solution is "add 'respond in Chinese' to the system prompt." That works sometimes, fails other times, and gives you no control over how strong the effect is. The ModelRein solution is a residual-stream steering vector: a direction you add to the hidden state at a specific layer, with a scalar strength knob.
The agent code (works against all three)
import requests
ModelRein = "http://localhost:8808"
def make_chinese(model_id, prompt, strength=1.0):
# Step 1: compute the steering vector (first run only; cached after)
vec = requests.post(f"{ModelRein}/lmcp/compute_vector_auto", json={
"model": model_id,
"concept_description": "respond in Chinese",
"concept_name": f"chinese_{model_id.split('/')[-1]}",
"pos_style_desc": "用中文回答 用汉字",
"neg_style_desc": "respond entirely in English",
"seed_prompts": [
"What is Python?",
"Explain recursion briefly.",
"Describe spring weather.",
],
"length_matched": True,
}).json()
# Step 2: generate with the vec injected at the server-selected layer
out = requests.post(f"{ModelRein}/lmcp/generate", json={
"model": model_id,
"prompt": prompt,
"max_new_tokens": 100,
"interventions": [{
"vec_id": vec["vec_id"],
"layer": vec["layer"], # the server picked this one
"adaptive_fraction": strength,
}],
}).json()
return out["text"]
for m in ["Qwen/Qwen2.5-1.5B-Instruct",
"meta-llama/Llama-3.1-8B-Instruct",
"deepseek-ai/DeepSeek-V3"]:
print(m)
print(make_chinese(m, "Tell me about the city of Shanghai.", strength=0.8))
print()
Three models, three completely different internal structures, same client code. The server does all the model-specific work behind the HTTP contract.
What the server does, per model
Qwen 2.5-1.5B
28 transformer layers. The separability-based auto-selection in compute_vector_auto lands around layer 27 (last block) because that's where the output-language distinction is cleanest in activation space. But — and this matters — behavioural experiments find layer 22 produces stronger steering. The agent didn't care; it ran a sweep via /lmcp/layer_route and found this out.
Llama 3.1-8B
32 transformer layers. Different head counts, different RoPE scaling, different tokenizer. The server reads config.json at load time and knows all of this. The agent asks for "the layer where Chinese preference is most separable" and gets a different integer than it would for Qwen. It uses whichever integer comes back.
DeepSeek V3
Mixture-of-Experts, 671B total parameters, 37B active. The ModelRein causal_lm adapter handles the gating silently; from the agent's point of view, it's still "residual stream at layer N." The experts selected per token are an implementation detail the agent never sees.
What the agent doesn't do
- Doesn't import
transformers. - Doesn't know what a hidden size is.
- Doesn't know which layer worked for which model.
- Doesn't have an if/else for Qwen vs Llama vs DeepSeek.
- Doesn't ship any model-specific code.
This is the point of a control protocol. The agent is model-agnostic because the contract is.
Pushing further: a strength sweep
Suppose you want to find the sweet spot for strength on a new model. Same agent code, just loop:
for af in [0.0, 0.3, 0.6, 1.0, 1.5, 2.0]:
out = requests.post(f"{ModelRein}/lmcp/generate", json={
"model": "Qwen/Qwen2.5-1.5B-Instruct",
"prompt": "Describe a river.",
"max_new_tokens": 100,
"interventions": [{
"vec_id": vec_id, "layer": layer, "adaptive_fraction": af
}],
}).json()
print(f"af={af:.1f} zh_ratio={chinese_ratio(out['text']):.3f}")
On the main page we show this exact dial as demo D5. Below ~0.3 nothing happens; above ~1.5 the model starts repeating. The sweet spot sits around 1.0. Different models have different sweet spots. The agent finds it by running the sweep; it doesn't know in advance.
Mixing concepts
Need "respond in Chinese AND use formal language AND keep it short"? Compute three vectors, inject all three. Our demo D6 shows this with (formal + emoji + medieval) composed on the same prompt. Three completely different style vectors added together.
out = requests.post(f"{ModelRein}/lmcp/generate", json={
"model": "Qwen/Qwen2.5-1.5B-Instruct",
"prompt": "Tell me about an autumn morning.",
"max_new_tokens": 90,
"interventions": [
{"vec_id": chinese_vec, "layer": 22, "adaptive_fraction": 1.0},
{"vec_id": formal_vec, "layer": 1, "adaptive_fraction": 0.8},
{"vec_id": short_vec, "layer": 15, "adaptive_fraction": 0.6},
],
"eos_bias_ramp": {"start_pos": 30, "end_pos": 50, "start_bias": 0, "end_bias": 25},
}).json()
Note the simultaneous use of three different layers and a length-controlling EOS bias ramp. All in one request. All model-agnostic.
When to use which primitive
/lmcp/compute_vector_auto— behavioural concepts that have a text-level pos/neg distinction. "Respond in Chinese", "be more formal", "refuse unsafe requests"./lmcp/head_mask— specific attention patterns you want to kill or scale. ITI-style edits./lmcp/layer_route— causal interventions, Mixture-of-Depths, speed experiments./lmcp/logit_bias— length control, forced tokens, forbidden tokens./lmcp/pipeline— chaining multiple models in one wall-clock call.
Related: How ModelRein works with any Hugging Face model · The full model matrix
