Skip to content

model: add Nanbeige4.2 (Looped Transformer) support - #100

Closed
Andgihat wants to merge 1 commit into
Anbeeld:mainfrom
Andgihat:nanbeige-arch
Closed

model: add Nanbeige4.2 (Looped Transformer) support#100
Andgihat wants to merge 1 commit into
Anbeeld:mainfrom
Andgihat:nanbeige-arch

Conversation

@Andgihat

Copy link
Copy Markdown

Adds support for the Nanbeige4.2 architecture (Nanbeige/Nanbeige4.2-3B, Apache-2.0) — a Looped Transformer: the physical decoder layers are reused num_loops times, so effective depth grows without adding parameters (22 physical layers × num_loops=2 → 44 effective).

Rebased onto current main, so the diff is a single clean commit.

What's included

  • New arch LLM_ARCH_NANBEIGE + KV keys nanbeige.num_loops / nanbeige.skip_loop_final_norm.
  • Graph builder src/models/nanbeige.cpp: standard RMSNorm + GQA + SwiGLU decoder with the loop unrolled over the logical (n_layer_all) layers. Physical weights are shared across loops via pointer copies (layers[i + j*n_phys] = layers[i]); each logical slot keeps its own KV index. An optional loop-boundary output_norm is applied between passes unless skip_loop_final_norm is set.
  • gguf-py constants + writer keys and a conversion/nanbeige.py converter.
  • Pure graph/arch code — no new ggml kernels.

load_arch_hparams expands the logical layer count via the existing n_layer (physical) / n_layer_all (logical) fields and replicates the per-layer arrays (n_head, n_head_kv, n_ff, is_swa_impl, is_recr_impl) across the loop copies, so the graph loop, KV cache and out-ids all size correctly with no changes to the base hparams API.

Verification

Built for Windows/CUDA 12.8 (sm_120) on top of current main and compared token-for-token against the authors' reference branch Nanbeige/llama.cpp@nanbeige42 on the same BF16 GGUF with greedy decoding — outputs are identical. Runs correctly with KVarN KV-cache (e.g. -ctk kvarn3 -ctv kvarn3) and long context.

GGUF quants (imatrix + KLD-guided mixed precision) are available at Andgihat/Nanbeige4.2-3B-GGUF.

@Anbeeld

Anbeeld commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Does this upstream merge achieves the same thing? ggml-org@b77d646

@Andgihat

Copy link
Copy Markdown
Author

I'll take a look now

@Andgihat

Copy link
Copy Markdown
Author

Yes — b77d6467 (upstream ggml-org#25994) is the same implementation, merged by the model authors themselves. Same loop mechanism (logical-layer expansion + weight sharing layers[i + j*n_phys] = layers[i]), same per-layer array replication, same output_norm loop boundary gated by skip_loop_final_norm, and the exact same KV keys (nanbeige.num_loops / nanbeige.skip_loop_final_norm). No custom kernels on either side, so it also works with KVarN once master is merged in.

So this PR is redundant — merging upstream master is the cleaner path. Closing it. The GGUF quants at Andgihat/Nanbeige4.2-3B-GGUF are key-compatible and load on stock llama.cpp too.

@Andgihat Andgihat closed this Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants