The model registry.

NVIDIA Cosmos-Reason2 8B — video reasoning VLM for Physical AI, post-trained on Qwen3-VL-8B-Instruct. Takes video or images plus text and answers in text, with spatial-temporal understanding and object detection. 256K context. Needs the mmproj projector for the vision path; NVIDIA's own repo is gated and requires HF_TOKEN, this GGUF conversion is not.

DeepSeek V3 MoE — 685B total, 37B active, 128K context. Native Multi-Token-Prediction head (n=4, ~80% accept rate, ~1.8× decode speedup per DeepSeek tech report). Retired by upstream after 2026-07-24 in favor of DeepSeek V4.

DeepSeek V4 Flash 0731 — 284B total / 13B active MoE; 1M context. Hybrid Compressed Sparse Attention (CSA) + Heavily Compressed Attention (HCA). Quantization-aware-trained: routed experts (96% of the model) natively MXFP4, the rest FP8/BF16. Outperforms V4-Pro Preview. MTP head built into the model file.

Diffusion-generation Gemma 4 26B-A4B. Generates 256-token canvases by parallel denoising rather than autoregressive sampling. Unsloth: 2000+ t/s on RTX 6000. Requires Unsloth Studio or llama.cpp PR #24423+.

High-performance instruction-tuned model from Google

Google's compact instruction-tuned model

Tiny Gemma model for ultra-lightweight on-device inference

Google's largest Gemma model with exceptional capabilities

Extended context Gemma model for chat applications

Google's mid-tier dense Gemma 4 model (128K context). MTP-enabled — pairs with `gemma4-12b-mtp-draft` for 1.5–2.2× throughput on the same hardware (Unsloth: 52 → 162 t/s at Q4 on a 4090).

Google's jointly-trained Multi-Token Prediction head for Gemma 4 12B. Pair with the 12B target via `--spec-type draft-mtp`.

Quantization-Aware-Trained Gemma 4 12B. Higher quality than naive Q4 at the same size. MTP-enabled via `gemma4-12b-mtp-draft`.

Gemma 4 Mixture-of-Experts: 26B total params, 4B active per token (128K context). MTP-enabled — pairs with `gemma4-26b-a4b-mtp-draft`; expect ~1.15–1.2× speedup on MoE targets per Unsloth.

Google's jointly-trained Multi-Token Prediction head for the Gemma 4 26B-A4B Mixture-of-Experts target. Pair via `--spec-type draft-mtp`; Unsloth measures ~1.15–1.2× speedup on MoE targets vs ~1.4–2.2× on dense.

Quantization-Aware-Trained Gemma 4 26B-A4B MoE. Unsloth measures 85.6% MMLU top-1 vs 70.2% on naive Q4 (+15.4 points). MTP-enabled via `gemma4-26b-a4b-mtp-draft`.

Google's largest dense Gemma 4 model (128K context). MTP-enabled — pairs with `gemma4-31b-mtp-draft` for ~2× throughput at 101 t/s on consumer GPUs (Unsloth benchmark).

Google's jointly-trained Multi-Token Prediction head for Gemma 4 31B. Pair with the 31B target via `--spec-type draft-mtp`.

Quantization-Aware-Trained Gemma 4 31B. Higher quality than naive Q4 at the same size. MTP-enabled via `gemma4-31b-mtp-draft`.

Google's compact Gemma 4 multimodal model (text + image, 128K context). MTP-enabled — pairs with `gemma4-e2b-mtp-draft` for 1.5–2.2× throughput.

Google's jointly-trained Multi-Token Prediction head for Gemma 4 E2B. Pair with the E2B target via `--spec-type draft-mtp`.

Quantization-Aware-Trained Gemma 4 E2B. Higher quality than naive Q4 at the same size. MTP-enabled via `gemma4-e2b-mtp-draft`.

Google's efficient Gemma 4 multimodal model (text + image, 128K context). MTP-enabled — pairs with `gemma4-e4b-mtp-draft` for 1.5–2.2× throughput.

Google's jointly-trained Multi-Token Prediction head for Gemma 4 E4B. Pair with the E4B target via `--spec-type draft-mtp`.

Quantization-Aware-Trained Gemma 4 E4B. Higher quality than naive Q4 at the same size. MTP-enabled via `gemma4-e4b-mtp-draft`.

Z.ai GLM-5 — 744B total parameter MoE, 40B active, trained on 28.5T tokens. Routes each token to 8 of 256 experts plus 1 shared across 75 MoE layers, with DeepSeek Sparse Attention over a 198K context. Unsloth dynamic UD-Q4_K_XL GGUF (sharded).

Z.ai GLM-5.1 — next-generation flagship for agentic engineering, class-leading on SWE-Bench Pro; 744B total / 40B active, 8 of 256 experts plus 1 shared across 75 MoE layers; 198K context. `glm_moe_dsa` architecture with Dynamic Sparse Attention.

Z.ai GLM-5.2 — 753B total parameter MoE flagship, 40B active, routing each token to 8 of 256 experts plus 1 shared across 75 MoE layers. Solid 1M-token context with IndexShare sparse-attention (2.9× per-token FLOP reduction at 1M). Improved Multi-Token-Prediction layer increases speculative-decoding accept rate by ~20% over GLM-5.1.

Zhipu AI GLM-4 9B instruction-tuned, 128K context

OpenAI GPT-OSS 120B — open-weights release, native MXFP4

OpenAI GPT-OSS 20B — open-weights release, native MXFP4

IBM Granite 4.0 1B — compact enterprise model

IBM Granite 4.0 350M — ultra-compact for edge deployment

IBM Granite 4.0 H-Small — 32B hybrid for long-context enterprise

IBM Granite 4.0 H-Tiny — hybrid Mamba/Transformer architecture

Thinking Machines Inkling — 975B-total multimodal MoE, 41B active, routing each token to 6 of 256 experts plus 2 shared across 66 layers. Hybrid local/global attention, 1M context. Accepts text, images and 16kHz WAV audio via a hierarchical patch encoder and discrete audio tokens, all projected into one hidden space; output is text. Apache-2.0.

Thinking Machines Inkling Small — 276B-total multimodal MoE, 12B active, 1M context. Accepts text, images and 16kHz WAV audio; output is text. Apache-2.0.

Moonshot AI Kimi K2 MoE — 1T total, 32B active, 128K context

Moonshot AI Kimi K2.5 — 1T total / 32B active MoE; image input support; 256K context. Predecessor to K2.6's hybrid-thinking variant.

Moonshot AI Kimi K2.6 hybrid-thinking MoE — 1T total params, 256K context. Replica-routed on B200-class infrastructure; Unsloth measures >40 t/s on B200. Recommended `UD-Q2_K_XL` (350GB) for size/quality balance.

Moonshot AI Kimi K2.7 Code — code-focused refresh of the K2 series. 1T total / 32B active; 256K context; recent updates target tool-call accuracy on long-horizon coding tasks.

Moonshot AI Kimi K3 — 2.8T total parameters, 104B active, 896 routed experts with 16 selected per token and 2 shared. Kimi Delta Attention plus gated MLA across 93 layers, 1M context, 160K vocabulary, MXFP4 weights and MXFP8 activations from quantization-aware training. Text, image, and video via the MoonViT-V2 encoder. `UD-IQ1_S` (594GB) is the smallest quant; `UD-Q2_K_XL` (861GB) is the size/quality balance point. Both exceed any single machine, so whole-model serving means a pipeline cluster; a lone host runs it as distributed expert extraction instead.

Poolside Laguna S 2.1 — 118B-total agentic-coding MoE, 8B active, with a token-choice router using softplus gating over 256 routed experts plus 1 shared. Grouped-query attention with interleaved full and sliding-window layers, 1M context. Native interleaved thinking between tool calls; preserve reasoning blocks across turns. Requires llama.cpp b10087 or newer.

MiniMax M2.7 — current frontier MiniMax MoE; Lightning Attention with 1M context. Unsloth dynamic UD-Q4_K_XL GGUF (sharded).

MiniMax M3 — ~428B total / ~23B active MoE with native multimodal training. MiniMax Sparse Attention (MSA) delivers 9× prefill and 15× decode speedups vs M2 at 1M context. Note: GGUF builds currently fall back to dense attention; sparse attention not yet supported in llama.cpp.

High-performance Ministral 3 for complex reasoning

Compact Ministral 3 for lightweight tasks

Versatile Ministral 3 for general-purpose tasks

Mistral AI's classic 7B instruction model

Extended-context Mistral model built with NVIDIA

Mistral's latest Small 3.2 model for demanding workloads

Mistral Small 3.1 — improved reasoning over 3.0 baseline

Speculative drafter for Mistral Small 3.1/3.2 — vocab-matched, 6-language fine-tune.

Mistral Small 3.2 — latest 3-series point release

Multimodal (text + image) 30B dense causal transformer with a frozen ViT-G/14 perception encoder (mmproj-kquant.gguf), 128K context. K-Quant build sized to fit 24GB VRAM; paired with its DFlash speculative-decoding drafter (dflash-kquant.gguf) for ~2x decode.

DFlash block-diffusion speculative drafter for Muse-Glimmer-30B. Pair via `--spec-type draft-dflash` for ~2x decode.

NVIDIA Nemotron 3 Nano Omni — 30B total / 3B active hybrid reasoning MoE, 256K context. Accepts audio, video, text, images and documents; output is text. Needs a llama.cpp-compatible backend: the vision path uses a separate mmproj file that Ollama does not load.

NVIDIA Nemotron 3 Ultra — 550B total / 55B active hybrid Transformer-Mamba MoE with Latent MoE and a built-in Multi-Token-Prediction head; up to 1M context.

Hybrid Mamba-2 MoE — 30B total, 3.5B active, 128K context

Hybrid Mamba-2 + Attention edge model, 256K context

NVIDIA Nemotron 3.5 Lightning — 30B total / 3B active hybrid Mamba-2 + Attention MoE. Thinking + instruct, 256K native context extensible to ~1M. Single-file GGUF, single-node serving.

Deep Reinforce Ornith 1.0 35B — agentic-coding MoE with 256 routed experts, 8 selected per token and 1 shared, over 40 layers. 256K context, vision input via the bundled projector. Designed for single-GPU deployment.

Deep Reinforce Ornith 1.0 397B — flagship agentic-coding MoE with 512 routed experts, 10 selected per token and 1 shared, over 60 layers. 256K context. Leads open-weight coding benchmarks at its size; MIT licensed with no regional limitation.

Deep Reinforce Ornith 1.0 9B — dense agentic-coding model post-trained on Qwen 3.5, 256K context, vision input via the bundled projector. Smallest member of the Ornith family.

Microsoft Phi-4 — strong reasoning at 14B

Compact Phi-4 Mini with 128K context

Compact Phi-4 Mini fine-tuned for reasoning tasks

Phi-4 fine-tuned for chain-of-thought reasoning

Qwen-AgentWorld 35B-A3B — agentic MoE, 3B active per token

Compact model optimized for edge deployment

Versatile model for various language tasks

Premium model with extended context support

Mixture-of-Experts with 3B active params for efficient scaling

Top-tier model with 128K context window

Well-balanced model for production use

Extended context model for long-form tasks

Code-focused MoE — 30B total, 3B active, 256K context

Qwen3-Coder-Next — agentic coding, long-context repository work

Qwen3-VL 8B — vision-language instruct, images on the chat surface

Compact multilingual model for efficient on-device inference

Qwen 3.5 0.8B with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth measures ~1.5-2× speedup over the non-MTP baseline.

Qwen 3.5 large MoE — 122B total, 10B active per token. Replica-routed on high-VRAM provider tiers only; Unsloth ships an MTP variant in `unsloth/Qwen3.5-122B-A10B-MTP-GGUF` for compatible runtimes.

Qwen 3.5 122B-A10B (MoE) with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth measures ~1.5-2× speedup over the non-MTP baseline.

Flagship Qwen 3.5 model

Qwen 3.5 27B with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth measures ~1.5-2× speedup over the non-MTP baseline.

Efficient small model for chat and text generation

Qwen 3.5 2B with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth measures ~1.5-2× speedup over the non-MTP baseline.

Mixture-of-Experts with only 3B active params — fast inference at 35B quality

Qwen 3.5 35B-A3B (MoE) with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth measures ~1.5-2× speedup over the non-MTP baseline.

Qwen 3.5 frontier MoE — 397B total, 17B active per token. Multi-GPU replicas only; Unsloth ships an MTP variant in `unsloth/Qwen3.5-397B-A17B-MTP-GGUF` for compatible runtimes.

Qwen 3.5 397B-A17B (MoE) with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth measures ~1.5-2× speedup over the non-MTP baseline.

Mid-size model with strong reasoning and coding performance

Qwen 3.5 4B with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth measures ~1.5-2× speedup over the non-MTP baseline.

High-performance model for complex language understanding

Qwen 3.5 9B with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth measures ~1.5-2× speedup over the non-MTP baseline.

Qwen 3.6 27B — flagship dense model with 128K context

Qwen 3.6 27B Fable-Fusion — community fine-tune of Qwen3.6-27B tuned for long-form fiction and creative writing, with multi-token-prediction tensors carried in-file at Q8_0 so it drafts against itself. 256K native context, YaRN-extensible. Uncensored: the base model's refusal behaviour is deliberately removed.

Qwen 3.6 27B with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth: 160 t/s on RTX 6000.

Qwen 3.6 MoE — 35B total, ~3B active per token

Qwen 3.6 35B-A3B MoE with built-in Multi-Token-Prediction head. Single-file MTP GGUF — no separate drafter needed. Unsloth: 240 t/s on RTX 6000.

Qwen 3.8 flagship MoE — 2.4T total, 95B active per token, thinking-only. 256K native context extensible to ~1M. Multi-node MoE-sharded serving only (10-shard UD-Q1_0 GGUF; first shard is the load entry). Built-in single MTP layer for self-speculative decoding.

Qwen 3.8 27B dense, multimodal, MTP-trained. Strong agentic/competitive coding (SWE-Bench Pro, LiveCodeBench). Hybrid Gated-DeltaNet + attention, 262K context.

DFlash-2 block-diffusion speculative drafter for Qwen3.8-27B. Drafts a whole block per pass and keeps the top candidates at every position. Pair via `--spec-type draft-dflash`.

Qwen 3.8 27B dense at 3-bit, sized to fit a 16 GB consumer GPU. Measured 62.0 tok/s on an RTX 5070 Ti against 19.6 tok/s for the 4-bit build on a GB10 — decode is bandwidth-bound, and this build reads 13.44 GB per token.

Qwen3.8-Flash-Next — agentic coding and long-horizon tool use. MTP-enabled — pairs with `qwen3.8-flash-next-mtp-draft` for lossless speculative decode.

Qwen's jointly-trained Multi-Token Prediction head for Qwen3.8-Flash-Next, exported as a standalone GGUF sidecar (Unsloth's main build strips it). Pair with the Flash-Next target via `--spec-type draft-mtp`.

Compact SmolLM2 for on-device AI

SmolLM3 — 11T tokens, dual-mode reasoning

NVIDIA Canary 1B Flash — Conformer AED with translation across 4 languages (en/de/es/fr)

Moonshine Base — on-device English ASR, balanced (raw waveform input)

Moonshine Tiny — on-device English ASR, edge-tier (raw waveform input)

NVIDIA Parakeet TDT 0.6B v3 — multilingual TDT transducer (25 langs, 128-mel)

Distil-Whisper large-v3 — multilingual distilled ASR (128-mel)

Distil-Whisper medium.en — higher-accuracy distilled English ASR (80-mel)

Distil-Whisper small.en — distilled English ASR (80-mel)

OpenAI Whisper Large-v3-turbo — flagship multilingual ASR (128-mel)

D-FINE large — high-accuracy DETR baseline

D-FINE medium — mid-tier DETR baseline

D-FINE small — efficient DETR variant

RF-DETR 2x-large — flagship >60 AP on COCO (ICLR 2026)

RF-DETR base — real-time DETR baseline (COCO)

RF-DETR large — high-accuracy real-time DETR (2026 refresh)

RF-DETR medium — mid-tier real-time DETR

RF-DETR nano — fastest real-time DETR, edge-tier

RF-DETR small — balanced speed/accuracy

BAAI BGE-M3 — multilingual multi-granularity retrieval encoder

Google EmbeddingGemma 300M — Matryoshka edge embeddings, fp32-only

ModernBERT-embed base — 8192-context mean-pooled retrieval encoder, Matryoshka 256

ModernBERT-embed large — 8192-context mean-pooled retrieval encoder

Qwen3-Embedding 0.6B — multilingual text embeddings, edge-tier

Qwen3-Embedding 4B — mid-tier multilingual text embeddings

Qwen3-Embedding 8B — flagship multilingual text embeddings

Amazon Chronos-2 (full) — multivariate-capable foundation forecaster, 13-quantile head

Amazon Chronos-2 — multivariate-capable foundation forecaster, 13-quantile head

Google TimesFM 2.5 — foundation timeseries forecaster, patch-tokenized decoder

NXAI TiRex 35M — sLSTM zero-shot forecaster with a 9-quantile head

NVIDIA Cosmos3 Edge world foundation model, 4B, image-to-video at 480p/121 frames. Ungated, commercial use permitted. Takes JSON-structured prompts rather than plain text

FLUX.2 dev — 32B flagship text-to-image and editing

FLUX.2 klein 4B text-to-image and editing, runs on consumer GPUs

FLUX.2 klein 9B — distilled few-step generation and editing

FLUX.2 klein 9B, Q4_K_M — runs beside a video model on one box

FLUX.2 klein 9B, Q8_0 — near-lossless at under half the bf16 footprint

FLUX.2 klein 9B KV — few-step generation and editing

FLUX.2 klein base 4B — undistilled text-to-image and editing

FLUX.2 klein base 9B — undistilled generation and editing

Tencent Hunyuan3D 2.1 image-to-3D with PBR texture synthesis. Ungated on the Hub but non-commercial: enrollment refuses it until the operator sets --accept-non-commercial. Loads through `hy3dgen`

LTX-2.3 22B distilled text-to-video, 8-step schedule, synchronized audio branch

LTX-2.5 22B dev text-to-video, the full trainable transformer with a synchronized audio branch. Unverified: catalogued from the model card and the pinned diffusers signature, never run here.

LTX-2.5 22B dev text-to-video, comfy-int8 transformer with the matching int8 text encoder. Same weights repo as the bf16 entry; the smaller serving set is what lets it share a node with the LLM tier. Unverified: catalogued from the files on disk, never rendered here.

LTX-2.5 22B distilled text-to-video with a synchronized audio branch. Unverified: catalogued from the model card, never run here.

LTX-2.5 22B distilled text-to-video, nvfp4 transformer. Lightricks ships nvfp4 for the distilled transformer only, so there is no dev equivalent. Fixed 8-step, CFG 1.0 schedule inherited from the distilled checkpoint; raising either fights the distillation. Unverified: catalogued from the files on disk, never rendered here.

MiniMax H3 omni-modal text-to-video with native stereo audio, 768p short edge, 4-15s. Licence excludes the EU, UK, South Korea and the USA, including outputs

MiniMax H3 first-and-last-frame to video, Q4_K. Licence excludes the EU, UK, South Korea and the USA, including outputs

MiniMax H3 text-to-video, FP8 transformer, 768p short edge, 4-15s. Licence excludes the EU, UK, South Korea and the USA, including outputs

MiniMax H3 reference-image-to-video, FP8. Licence excludes the EU, UK, South Korea and the USA, including outputs

MiniMax H3 reference-image-to-video, Q4_K. Licence excludes the EU, UK, South Korea and the USA, including outputs

MiniMax-Music3 2B text-to-music: 32 kHz stereo, up to five minutes, prompt plus optional lyrics. Unverified: catalogued from the model card, never run here.

Qwen-Image 2512 text-to-image MMDiT with strong text rendering

Qwen-Image-Edit instruction-driven image editing, multi-reference

Qwen-Image distilled to four steps, guidance disabled

Qwen-Image 2512 text-to-image, Q5_K_M GGUF transformer

Microsoft TRELLIS.2 image-to-3D. Produces a GLB mesh with PBR materials including transparency, at up to 1536³. Loads through the `trellis2` package, not diffusers

Wan 2.1 first-last-frame interpolation; bridges two stills into motion

Wan 2.2 image-to-video mixture-of-experts, 480P and 720P

Wan 2.2 text-to-video mixture-of-experts, 480P and 720P

Wan 2.2 hybrid text/image-to-video at 720P 24fps on a single 24 GB GPU

Z-Image Turbo few-step text-to-image, 9 steps without guidance

EdgeSAM — ultra-compact 9.6M-param segmentation (research-only)

MobileSAM (community ONNX export) — compact mobile-optimized

Meta SAM 2 base (community ONNX export) — previous-gen flagship

Meta SAM 2 large (community ONNX export) — high-fidelity

Meta SAM 3 ViT-H — text-promptable open-vocabulary segmenter (community ONNX)

Smaller Qwen3-TTS for nodes where speech shares the machine

Ten-language speech synthesis from raw text; no phonemizer required

Voice cloning from ~3s of reference audio and its transcript; requires an explicit operator opt-in

OpenAI CLIP ViT-B/32 — compact image encoder, 512-dim embeddings

OpenAI CLIP ViT-L/14 — large image encoder, 768-dim embeddings

Meta DINOv3 ViT-B/16 — flagship self-supervised features, base-tier

Meta DINOv3 ViT-L/16 — large self-supervised features

Meta DINOv3 ViT-S/16 — next-gen self-supervised features, edge-tier

Google SigLIP base — sigmoid-loss image-text encoder

Google SigLIP2 base — improved multilingual image-text encoder

Google SigLIP2 large — high-fidelity multilingual image-text encoder

Google SigLIP2 SO400M-384 — flagship encoder, top zero-shot accuracy