# CATALOG.md — the RSI model catalog > Every open-weights model that can run a robot today, license-verified for commercial hosted serving. > Verified 2026-07-19 by live fetch of HF model pages / LICENSE files (10-agent sweep, 463 lookups). > Verdicts: **CLEAN** = permissive (Apache/MIT/BSD/CC-BY/OpenMDW) — serve without strings. **CONDITIONS** = commercial serving allowed with named strings (flow-down terms, guardrail clauses, MAU caps). **NO** = research-only; do not serve. ## Action (VLAs & policies) ### ✅ Clean — serve without strings | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Octo (base/small)** | UC Berkeley RAIL | 93M / 27M | MIT | rail-berkeley/octo-base-1.5 MIT tag confirmed, not gated; clean permissive chain. | | **ACT (Action Chunking Transformer)** | Stanford ALOHA (Zhao et al.) / LeRobot | ~80M (train-per-task) | Original repo MIT | tonyzhaozh/act confirmed MIT (LICENSE + footer); policies are typically trained from scratch so no weight-license passthrough. | | **Diffusion Policy** | Columbia/TRI (Chi et al.) / LeRobot | ~90-260M (train-per-task) | MIT (real-stanford/diffusion_policy) | real-stanford/diffusion_policy confirmed MIT ('released under the MIT license'); clean chain. | | **RDT-1B** | Tsinghua TSAIL | 1.2B diffusion transformer | MIT | MIT tag confirmed; card states 'all the code, pre-trained model weights, and data are licensed under the MIT license'; not gated. | | **RDT-2 (RDT2-VQ / RDT2-FM)** | Tsinghua TSAIL | 8B (Qwen2.5-VL-7B-Instruct backbone + RV | Apache-2.0 | Apache-2.0 tags confirmed on both RDT2-VQ (page) and RDT2-FM (API tags quoted), not gated; Qwen2.5-VL-7B backbone is Apache — clean chain. | | **MolmoAct-7B-D** | Allen Institute for AI (AI2) | 7B (Qwen2.5-7B + SigLIP2; 8.1B F32 shard | Apache-2.0 | Apache-2.0 tag + card confirmed, not gated; card's 'intended for research and educational use' is non-binding intent language against a binding Apache grant. | | **V-JEPA 2-AC** | Meta FAIR | ~1.2B (ViT-g/16 encoder + action-conditi | MIT (majority) with portions Apache-2.0 | facebookresearch/vjepa2 confirmed 'majority... licensed under MIT' with 3 utility files Apache-2.0; no NC restriction anywhere — clean for hosted serving. | | **LingBot-VLA (4B / v2-6B)** | Robbyant (Ant Group embodied-AI arm) | 4B (v1, Jan 2026); 6B (v2, Jul 2026) | lingbot-vla-v2-6b card: Apache-2.0 (verified) | lingbot-vla-v2-6b card confirmed 'licensed under the Apache-2.0 License', not gated; lingbot-vla-4b confirmed to have NO license tag — serve v2-6b, avoid the untagged 4b. | | **GigaBrain-0 / 0.1** | GigaAI (open-gigaai) | 3.5B | Apache-2.0 | Apache-2.0 confirmed on GigaBrain-0-3.5B-Base (page) and GigaBrain-0.1-3.5B-Base (API tags quoted); not gated. | | **RynnVLA-001-7B** | Alibaba DAMO Academy | 7B | Apache-2.0 | RynnVLA-001-7B-Base Apache-2.0 tag confirmed, not gated; RynnVLA-002 confirmed via API to still have NO license tag — hold 002 until tagged, serve 001. | | **X-VLA-0.9B** | 2toINF (THU/Shanghai) | 0.9B (Florence-2-large base + soft-promp | Apache-2.0 | X-VLA-Pt Apache-2.0 tag + license text confirmed, not gated (other 4 checkpoints not individually fetched); Florence-2 base is MIT — clean chain. | | **FastWAM** | Hugging Face LeRobot | 6B (initialized from Wan2.2 video-diffus | Apache-2.0 | lerobot/fastwam_base Apache-2.0 tag confirmed, not gated; Wan2.2 base is Apache-2.0 — clean chain. | | **Unitree RL locomotion policies (unitree_rl_gym)** | Unitree Robotics | <10M per policy (motion.pt per robot) | BSD 3-Clause | BSD 3-Clause confirmed on the repo; pretrained policies ship in-repo under it — permissive, standard notice/no-endorsement conditions only. |
Serving details - **Octo (base/small)** — Transformer diffusion-head generalist policy on 800k OXE episodes — single-arm + bimanual; classic pre-VLA baseline · GPU: 2-4GB VRAM · Serving: JAX/Flax repo only; no server, no vLLM; LeRobot has no official port · Released 2024-05 · https://huggingface.co/rail-berkeley/octo-base-1.5 - **ACT (Action Chunking Transformer)** — Classic imitation baseline for bimanual fine manipulation (ALOHA); trained per-task on ~50 demos — the workhorse of hobby/lab arms · GPU: 2-4GB VRAM; Jetson OK · Serving: LeRobot native (train/eval/deploy); the most-forked policy on HF · Released 2023-04 · https://github.com/tonyzhaozh/act - **Diffusion Policy** — Diffusion visuomotor policy baseline — single-arm precision tasks; standard comparison point for every VLA paper · GPU: 2-4GB VRAM · Serving: LeRobot native; original repo scripts · Released 2023-03 · https://github.com/real-stanford/diffusion_policy - **RDT-1B** — Bimanual manipulation diffusion VLA (ALOHA dual-arm), pretrained on 1M+ episodes; strong few-shot fine-tuning · GPU: 8-16GB inference; 24GB+ fine-tune · Serving: Repo scripts (train/inference); no server, no vLLM · Released 2024-10 · https://huggingface.co/robotics-diffusion-transformer/rdt-1b - **RDT-2 (RDT2-VQ / RDT2-FM)** — Zero-shot cross-embodiment bimanual VLA trained on 10k+ hours UMI in-the-wild data; 24-step action chunks from binocular wrist cams; UR5e/Franka demos · GPU: ≥16GB VRAM + ≥32GB RAM (RTX 4090 documented) · Serving: HF transformers + repo scripts; 13k downloads/mo on VQ; no vLLM · Released 2025-09 · https://huggingface.co/robotics-diffusion-transformer/RDT2-VQ - **MolmoAct-7B-D** — Action-reasoning model: depth-aware perception → editable visual trace plan → actions; single-arm (Franka, LIBERO); steerable/interpretable · GPU: 16-24GB bf16 · Serving: HF transformers; AI2 released full training/eval code; no official server · Released 2025-08 · https://huggingface.co/allenai/MolmoAct-7B-D-0812 - **V-JEPA 2-AC** — Action-conditioned video world model for zero-shot Franka arm planning (reach/grasp/pick-place via CEM/MPC energy minimization) — world-model-as-policy, not a direct VLA · GPU: 16GB+ (planning loop is compute-heavy per step) · Serving: HF transformers (vjepa2 supported) + planning code in repo; no server · Released 2025-06 · https://github.com/facebookresearch/vjepa2 - **LingBot-VLA (4B / v2-6B)** — Dual-arm manipulation VLA pretrained on 60k hours (robot + human ego video) across 20 robot configs; RoboTwin post-trains; sibling world-model + video stack (lingbot-world, lingbot-video) · GPU: 16-24GB bf16 (6B) · Serving: Repo inference; community LeRobot port exists (RobotWin fine-tune, 4k downloads); no vLLM · Released 2026-01 (4B), 2026-07 (v2-6B) · https://huggingface.co/robbyant/lingbot-vla-v2-6b - **GigaBrain-0 / 0.1** — VLA trained predominantly on world-model-generated data (RGB-D, embodied CoT); bimanual + mobile manipulation, dexterous long-horizon tasks · GPU: ~12GB bf16 · Serving: Repo (diffusers-based); no server/vLLM · Released 2025-11 (0), 2026-02 (0.1) · https://huggingface.co/open-gigaai/GigaBrain-0-3.5B-Base - **RynnVLA-001-7B** — VLA bootstrapped from generative human-video pretraining; trajectory-aware variant; single-arm LeRobot-style embodiments · GPU: ~24GB bf16 · Serving: Repo scripts; no server · Released 2025-08 (001), 2025-11 (002, untagged) · https://huggingface.co/Alibaba-DAMO-Academy/RynnVLA-001-7B-Base - **X-VLA-0.9B** — Soft-prompted flow-matching VLA; won AgiBot World Challenge 2025; checkpoints for WidowX, Calvin, Google Robot, AgiBot; 10.6k downloads on Pt · GPU: 6-8GB — cheapest capable VLA to serve · Serving: Repo scripts; no server yet · Released 2025-11 · https://huggingface.co/2toINF/X-VLA-Pt - **FastWAM** — World Action Model policy — video world-modeling during training, direct action prediction at inference; dual-camera + 8-dim state → 7-dim actions · GPU: 16-24GB bf16 · Serving: LeRobot native (train/eval/deploy) · Released 2026-06 (weights; paper 2026-03) · https://huggingface.co/lerobot/fastwam_base - **Unitree RL locomotion policies (unitree_rl_gym)** — Pretrained locomotion policies + Isaac Gym/Mujoco training stack for Go2 quadruped, H1/H1-2 and G1 humanoids — the standard open humanoid walking baseline · GPU: CPU/Jetson at runtime; GPU only for training · Serving: Sim2sim/sim2real deploy scripts in repo · Released 2024-2025 (maintained) · https://github.com/unitreerobotics/unitree_rl_gym
### ⚠️ Servable with conditions | Model | Org | Params | License | Notes | |---|---|---|---|---| | **π0 (pi0)** | Physical Intelligence (openpi) | 3.3B (PaliGemma-3B VLM + ~300M flow-matc | Apache-2.0 (openpi code+weights) | openpi Apache-2.0 + LICENSE_GEMMA confirmed; lerobot/pi0_base tagged 'gemma' — hosted serving allowed but Gemma Terms + Prohibited Use Policy ride the PaliGemma backbone and pass through to your service. | | **π0-FAST** | Physical Intelligence (openpi) | 3.3B (same backbone, autoregressive FAST | Apache-2.0 (openpi) | lerobot/pi0fast-base (hyphen; underscore variant 401s) tagged license:gemma, not gated; same Apache-code/Gemma-weights split as pi0 — serve under Gemma Terms + use-policy passthrough. | | **π0.5 (pi05)** | Physical Intelligence (openpi) | ~3.3B (PaliGemma-class + action expert, | Apache-2.0 (openpi) | lerobot/pi05_base tagged 'gemma' confirmed; PI's Apache statement covers their layer but Gemma Terms are the binding extra layer for hosted serving. | | **GR00T N1.7 (N1.7-3B)** | NVIDIA | 3B (Cosmos-Reason2-2B VLM backbone repla | NVIDIA Open Model License Agreement | NVIDIA Open Model License, card states 'ready for commercial/non-commercial use', not gated — serving OK but OML guardrail-preservation, attribution, and termination clauses apply. | | **OpenVLA-7B** | Stanford/Berkeley/TRI (Kim et al.) | 7.5B (Llama-2-7B + DINOv2 + SigLIP) | MIT | MIT tag + card ('all checkpoints... MIT') confirmed, not gated — downgraded from 'yes': Llama-2 Community License (AUP, attribution, 700M-MAU clause) flows through the fine-tuned weights; serving is permitted under those terms. | | **OpenVLA-OFT** | Stanford (Moo Jin Kim) | 7.5B | MIT | Spot-checked moojink/openvla-7b-oft-finetuned-libero-spatial: MIT tag confirmed, not gated (other 4 checkpoints not individually fetched); same Llama-2 Community License passthrough as OpenVLA. | | **CogACT-Base** | Microsoft Research | ~7.6B (Prismatic-7B VLM + DiT action mod | MIT | MIT tag + card ('code, pre-trained model weights... MIT') confirmed; Llama-2 inside the Prismatic backbone confirmed on card — Llama-2 Community License terms flow through; serving permitted under them. | | **SpatialVLA-4B** | IPEC / SJTU + Shanghai AI Lab | 4B (PaliGemma2-3B base) | MIT | MIT tag + card confirmed, finetuned from paligemma2-3b-pt-224 — downgraded from 'yes': Gemma Terms attach to PaliGemma2 derivatives regardless of the MIT tag; serve under Gemma Terms + Prohibited Use Policy. | | **UniVLA** | HKU / OpenDriveLab | 7B (Prismatic-based) | Apache-2.0 | qwbu/univla-7b Apache-2.0 tag confirmed, not gated; Llama-2 via Prismatic backbone → Llama-2 Community License passthrough conditions the serving. | | **Magma-8B** | Microsoft | 8.6B (Llama-3-8B + ConvNeXt-XXL CLIP) | MIT | MIT tag + card confirmed; card confirmed to say 'designed only for research purposes' and 'not intended to be deployed in production' (non-binding vs the MIT grant) — downgraded from 'yes' for the Llama-3 Community License passthrough plus the explicit research-intent flags. | | **GR00T-Dreams (DreamGen artifacts)** | NVIDIA | Pipeline: Cosmos-Predict2-14B video WM f | GitHub code Apache-2.0 (verified) | GitHub code Apache-2.0 confirmed; Cosmos-Predict2 weights / dream-dataset NVIDIA OML layer not independently re-fetched but consistent with NVIDIA's published licensing — treat the pipeline's model artifacts as NVIDIA OML (guardrail + termination clauses). | | **DM05** | Dexmal (Dexbotic) | 6B | HF tag: Gemma (verified) | Dexmal/DM05 tagged 'gemma' confirmed, not gated — commercial serving allowed under Gemma Terms + Prohibited Use Policy; DM0-base's claimed bare 'cc' tag not re-checked, avoid it regardless. | | **Pelican-VLA05** | X-Humanoid (Beijing Humanoid Robot Innovation Center) | ~5B (Qwen3-VL-4B-Instruct frozen backbon | Apache-2.0 checkpoint (card verified) | Card confirmed: checkpoint Apache-2.0, Qwen3-VL weights Apache-2.0, NVIDIA Cosmos tokenizer under NVIDIA Open Model License — serving OK but the tokenizer component binds you to OML guardrail/attribution terms. |
Serving details - **π0 (pi0)** — Generalist flow-matching manipulation VLA — single-arm, bimanual (ALOHA), and mobile manipulators; 50Hz action chunks · GPU: >8GB VRAM inference (RTX 4090 class); LoRA fine-tune >22.5GB; full fine-tune >70GB (A100/H100) · Serving: Official websocket policy server (scripts/serve_policy.py); LeRobot port; PyTorch + JAX; no vLLM · Released 2025-02 (weights), openpi · https://github.com/Physical-Intelligence/openpi - **π0-FAST** — Autoregressive variant of π0 using FAST DCT action tokens — ~5x cheaper training, same embodiments · GPU: >8GB VRAM inference (RTX 4090 class) · Serving: openpi policy server; LeRobot port (2026); no vLLM · Released 2025-02 (weights) · https://github.com/Physical-Intelligence/openpi - **π0.5 (pi05)** — Open-world generalization VLA — long-horizon mobile manipulation in unseen homes (clean kitchen/bedroom); base + DROID + LIBERO checkpoints · GPU: >8GB VRAM inference; full fine-tune >70GB · Serving: openpi policy server (websocket, PyTorch since Sep 2025); LeRobot port; no vLLM · Released 2025-09 (weights in openpi) · https://github.com/Physical-Intelligence/openpi - **GR00T N1.7 (N1.7-3B)** — Flagship cross-embodiment humanoid/manipulator VLA — 21.6M samples, 13 datasets; DROID/LIBERO/SimplerEnv post-trains published · GPU: ~8-10GB VRAM; TensorRT 35.9Hz on H100, 4.6Hz on Jetson Orin · Serving: Isaac GR00T official inference service (PyTorch, torch.compile, TensorRT); LeRobot integration; Jetson deploy · Released 2026-02-25 · https://huggingface.co/nvidia/GR00T-N1.7-3B - **OpenVLA-7B** — Single-arm manipulation VLA trained on 970k OXE episodes (WidowX/Google Robot embodiments); the de-facto academic baseline, 408k downloads · GPU: 16GB bf16 (fits RTX 4090 4-bit/8-bit quantized) · Serving: Official REST deploy server in repo; HF transformers; LoRA fine-tune on 1x A100; no official vLLM · Released 2024-06 · https://huggingface.co/openvla/openvla-7b - **OpenVLA-OFT** — Optimized fine-tuning recipe: parallel decoding + continuous actions + action chunking — 26x faster inference, SOTA LIBERO; ALOHA bimanual recipe included · GPU: 16GB bf16 · Serving: openvla-oft repo (train + deploy scripts); no vLLM · Released 2025-02 · https://github.com/moojink/openvla-oft - **CogACT-Base** — Componentized VLA with diffusion action transformer; single-arm (OXE, SimplerEnv, real WidowX/Franka) · GPU: 16-24GB bf16 · Serving: Repo scripts; HF transformers; no vLLM · Released 2024-11 · https://huggingface.co/CogACT/CogACT-Base - **SpatialVLA-4B** — 3D spatial-representation VLA (Ego3D position encoding, adaptive action grids); single-arm WidowX/Franka; strong SimplerEnv results · GPU: 10-16GB bf16 · Serving: HF transformers; repo fine-tune scripts; no server · Released 2025-01 · https://huggingface.co/IPEC-COMMUNITY/spatialvla-4b-224-pt - **UniVLA** — Task-centric latent-action VLA learned from cross-embodiment + human video without action labels; manipulation + navigation heads · GPU: 16-24GB bf16 · Serving: Repo scripts; no server · Released 2025-05 · https://huggingface.co/qwbu/univla-7b - **Magma-8B** — Agentic multimodal foundation for UI + robot manipulation (Set-of-Mark/Trace-of-Mark grounding); WidowX manipulation, UI navigation · GPU: 20-24GB bf16 · Serving: HF transformers (custom fork required); no server/vLLM · Released 2025-02 · https://huggingface.co/microsoft/Magma-8B - **GR00T-Dreams (DreamGen artifacts)** — Synthetic trajectory generation ('dreams') to train VLAs on new tasks/environments from video world models — training-time asset, not a runtime policy · GPU: H100-class for generation (14B video diffusion); not a serving target · Serving: Batch pipeline in GR00T-Dreams repo (generation + IDM extraction); not a servable policy · Released 2025-06 · https://github.com/NVIDIA/GR00T-Dreams - **DM05** — Open-world robot-control VLA — open-ended instructions, long-horizon tasks, disturbance recovery, multi-embodiment (DOS-W1 built-in; LIBERO + RoboTwin2 post-trains) · GPU: 1x RTX 4090 inference (per card); 8 GPUs recommended for training · Serving: Built-in HTTP inference server (multipart API, port 7891) + Dexbotic toolbox — one of the few with a first-party server · Released 2026-07-09 · https://huggingface.co/Dexmal/DM05 - **Pelican-VLA05** — Unified understanding + future-frame generation + action VLA with bottleneck tokens; RoboTwin humanoid post-train; joint-position action space · GPU: ~16GB bf16 · Serving: GitHub inference package; no server · Released 2026-07 (arXiv 2607.06655) · https://huggingface.co/X-Humanoid/Pelican-VLA05
### ❌ Research-only — do not serve | Model | Org | Params | License | Notes | |---|---|---|---|---| | **GR00T N1 (N1-2B)** | NVIDIA | 2.2B (Eagle-2 VLM + flow-matching DiT) | HF card: 'Nvidia License' linking the NVIDIA OneWay Noncomme | Research/non-commercial only under NVIDIA OneWay NC license. | | **GR00T N1.5 (N1.5-3B)** | NVIDIA | 2.7B | HF card README states 'This model is ready for non-commercia | Non-commercial only (current card text; verified live from raw README). | | **GR00T N1.6 (N1.6-3B)** | NVIDIA | ~3B | NVIDIA OneWay Noncommercial License per current HF card (lin | Non-commercial only. | | **WALL-OSS (FLOW / FAST / 0.5)** | X Square Robot (Shenzhen) | 4.2B (Qwen2.5-VL-3B backbone + MoE actio | GitHub wall-x code Apache-2.0 (verified) | wall-x code Apache-2.0 confirmed, but x-square-robot/wall-oss-flow API shows NO license tag (no grant on weights) and the Qwen2.5-VL-3B backbone is confirmed 'qwen-research' (non-commercial) — downgraded from conditions; do not serve commercially. | | **GO-1 / GO-1-Air (Genie Operator-1)** | AgiBot (Zhiyuan Robotics) | ~3B (InternVL2.5-2B + latent planner + a | CC BY-NC-SA 4.0 | Non-commercial ShareAlike — cannot be metered commercially without a separate AgiBot license. | | **NORA / NORA-1.5** | SUTD declare-lab | 3.8B (Qwen2.5-VL-3B + FAST+ tokenizer; 1 | nora-1.5 HF tag: MIT (verified). Base nora and most fine-tun | nora-1.5 MIT tag confirmed; base nora confirmed to have NO license tag (API); Qwen2.5-VL-3B backbone confirmed 'qwen-research' (non-commercial) — the fine-tuner cannot relicense NC weights via an MIT tag; commercial serving needs a separate Alibaba grant. Downgraded from conditions. | | **EO-1-3B** | EO Robotics / IPEC-COMMUNITY (SJTU + Shanghai AI Lab) | 3B (Qwen2.5-VL-3B based, interleaved vis | MIT | MIT tag confirmed and base Qwen/Qwen2.5-VL-3B-Instruct confirmed on card, but that backbone is confirmed 'qwen-research' (non-commercial) — MIT tag can't clear an NC upstream; tightened from conditions to no absent an Alibaba commercial grant. | | **WorldVLA** | Alibaba DAMO Academy | 7B (Chameleon-7B base) | Apache-2.0 HF tag | Apache-2.0 tag confirmed, but base facebook/chameleon-7b confirmed under 'chameleon-research-license' (license:other, manually gated, research-only) — DAMO cannot relicense Meta's NC weights; tightened from conditions to no. | | **UnifoLM-WMA-0 (Base/Dual)** | Unitree Robotics | Undisclosed (video-diffusion world-model | UNRESOLVED: no HF license tag | Fetched the raw GitHub LICENSE: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 ('for NonCommercial purposes only'); HF repo gated (auto) with no license tag — commercial hosted serving is prohibited, tightened from conditions. | | **UnifoLM-VLA (Base/Libero)** | Unitree Robotics | Undisclosed | CC BY-NC-SA 4.0 | Non-commercial ShareAlike. | ### ❓ Unverified — check before listing | Model | Org | Params | License | Notes | |---|---|---|---|---| | **SmolVLA (450M)** | Hugging Face LeRobot | 450M (SmolVLM2 backbone + flow-matching | Apache-2.0 (HF tag on lerobot/smolvla_base | HF API shows NO license tag/field on lerobot/smolvla_base (tags quoted, gated:false) — claimed Apache-2.0 tag does not exist; code + SmolVLM2 backbone are Apache but the weights repo publishes no grant — get the tag fixed or written confirmation before serving. | **Excluded (closed / NC / no grant):** Figure Helix · Google Gemini Robotics On-Device / 1.5 / ER · ByteDance GR-3 · Physical Intelligence π*0.6 / π-0.6 · Toyota Research Institute LBM (Large Behavior Models) · Covariant RFM-1 · DeepMind RT-2 / RoboCat / ALOHA Unleashed · Tesla Optimus / 1X NEO (Redwood) / Agility / Apptronik controllers · Skild AI Skild Brain · InternVLA-M1 / InternVLA-A1 (Shanghai AI Lab InternRobotics) · NVIDIA GR00T-H / GR00T-H-N1.7 (surgical) · AgiBot EnerVerse-AC / EWMBench · Galaxea G0 (dual-system VLA) · RynnVLA-002 · Dexmal DM0 · NVIDIA Alpamayo-R1 / Alpamayo-1.5 · BAAI RoboBrain 2.0 · Meta V-JEPA 2 (non-AC checkpoints) ## World models ### ✅ Clean — serve without strings | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Cosmos3-Nano (16B) — incl. Nano-Policy-DROID variant** | NVIDIA | 16B (Mixture-of-Transformers, AR + diffu | OpenMDW-1.1 (HF tag and card agree | Verified on both base and DROID variant pages — OpenMDW-1.1 is permissive, ungated, and the card explicitly permits commercial use; hosted serving is fine. | | **Cosmos3-Super (64B) — incl. Super-Text2Image / Super-Image2Video** | NVIDIA | 64B | OpenMDW-1.1 (not gated) | Claim confirmed — OpenMDW-1.1, ungated, explicit commercial-use statement; hosted serving permitted. | | **V-JEPA 2 / V-JEPA 2.1 (+ V-JEPA 2-AC)** | Meta AI | 80M (ViT-B) / 300M (ViT-L) / 600M (ViT-H | MIT per GitHub repo ('majority MIT, portions Apache-2.0') | Permissive either way, but the claim's detail is off: the LICENSE file I fetched is pure MIT with no 'portions Apache-2.0' rider, and the HF tag I checked reads mit, not apache-2.0. | | **LingBot-World-Fast (+ world-base-cam, va-base family)** | Robbyant (Ant Group robotics) | 19B | Apache-2.0 (HF tag, not gated) | v1 line confirmed Apache-2.0, hosted serving fine; the claimed CC BY-NC-SA v2 line could not be located to verify (fetch 401) — treat any v2 checkpoint as unverified and audit before use. | | **Matrix-Game (v1)** | Skywork AI | 17B | MIT (HF tag and card agree | Claim confirmed — MIT, ungated; hosted serving permitted. | | **Matrix-Game-2.0** | Skywork AI | 1.8B | MIT (tag and card agree | Claim confirmed — MIT, ungated; hosted serving permitted. | | **Matrix-Game-3.0** | Skywork AI | 2x14B MoE + distilled 5B deploy variant | Apache-2.0 (not gated) | Claim confirmed — Apache-2.0, ungated; note the license differs from the MIT v1/v2 line but is equally permissive. | | **Vista** | OpenDriveLab | ~2.5B (SVD-based) | Apache-2.0 (entire repo incl. weights | Claim confirmed on both GitHub and HF — no separate weights license found; hosted serving permitted. | | **iVideoGPT (OXE family)** | THU Machine Learning (thuml) | tokenizer 114M–310M + transformer 138M–4 | MIT | Claim confirmed — MIT with no NC weights rider spotted; hosted serving permitted. | | **1X World Model baselines (worldmodel_unet_v0 + GENIE_35M/138M/210M)** | 1X Technologies | 0.9B (UNet diffusion decoder); 35M/138M/ | Apache-2.0 (HF tag) | Claim confirmed on two of the four repos (unet_v0 and GENIE_138M); 35M/210M siblings in the same family assumed identical but not individually fetched. | | **Oasis 500M** | Decart + Etched | 500M (DiT) | MIT | MIT terms permit commercial hosted serving, but access requires accepting Etched/Decart's gate conditions first — review the clickwrap text on acceptance for any extra terms beyond contact sharing. | | **RynnVLA-002 (formerly WorldVLA)** | Alibaba DAMO Academy | not stated on card (v1 WorldVLA used a ~ | Apache-2.0 | Claim confirmed — Apache-2.0 for code and checkpoints, no NC rider found; the 'unless specified otherwise' hedge exists but no exceptions are specified for the released weights. |
Serving details - **Cosmos3-Nano (16B) — incl. Nano-Policy-DROID variant** — Omnimodal world foundation model: generates video, image, audio, text AND robot actions from multimodal input — world sim + embodied policy in one · GPU: 1x H100 80GB (BF16 only; tested H100/GB200) · Serving: vLLM-Omni (official docker vllm/vllm-omni:cosmos3), vLLM, HF Diffusers, SGLang; batch (secs-mins per generation) · Released 2026-05-31 · https://huggingface.co/nvidia/Cosmos3-Nano - **Cosmos3-Super (64B) — incl. Super-Text2Image / Super-Image2Video** — Flagship omnimodal world model — text/image/video/audio/action generation for Physical AI world sim and policy learning · GPU: 8x H100/H200/A100 recommended (~55s per 50-step video); minimum 2x H200 (~3 min/video) · Serving: vLLM-Omni with OpenAI-compatible endpoints, vLLM, Diffusers, SGLang; batch + async jobs · Released 2026-05-31 · https://huggingface.co/nvidia/Cosmos3-Super - **V-JEPA 2 / V-JEPA 2.1 (+ V-JEPA 2-AC)** — Self-supervised video world model in latent space — video understanding/encoding plus action-conditioned variant (V-JEPA 2-AC) for zero-shot robot manipulation planning · GPU: 1x 16–24GB for encoding; AC planning comfortable on 1x A100 · Serving: Native HF Transformers (AutoModel/AutoVideoProcessor) — trivial to serve; encoder is real-time capable · Released 2025-06-25 (V-JEPA 2), 2026-03-16 (V-JEPA 2.1) · https://github.com/facebookresearch/vjepa2 - **LingBot-World-Fast (+ world-base-cam, va-base family)** — REAL-TIME interactive world simulator — image+text+camera/action-conditioned video at 16 FPS, <1s latency, up to ~1 min rollouts; robot world-model line (lingbot-va) shares the stack · GPU: 8x 80GB for real-time long sequences (FSDP + Ulysses); single-GPU offline feasible with offload · Serving: Diffusers integration + official generation scripts; community ComfyUI ports · Released 2026-01-29 · https://huggingface.co/robbyant/lingbot-world-fast - **Matrix-Game (v1)** — Diffusion image-to-world model — keyboard/mouse-conditioned Minecraft world video; batch generation (not real-time) · GPU: 1x H100 80GB (bf16, FlashAttention-3/Apex) · Serving: Custom repo scripts only · Released 2025-05 (HF 2025-06-26) · https://huggingface.co/Skywork/Matrix-Game - **Matrix-Game-2.0** — REAL-TIME interactive world model — 25 FPS streaming few-step autoregressive diffusion, minute-long keyboard/mouse-controlled rollouts · GPU: 1x 24GB (RTX 4090-class) for real-time; runs in Colab · Serving: Diffusers-based pipeline in repo (needs FlashAttention + Apex) · Released 2025-08-18 · https://huggingface.co/Skywork/Matrix-Game-2.0 - **Matrix-Game-3.0** — REAL-TIME memory-augmented interactive world model — 40 FPS at 720p, minute-long consistent rollouts, INT8 + 3-step sampling on the distilled 5B · GPU: 1x H100 for 40 FPS 720p (distilled INT8); multi-GPU + async VAE for full MoE · Serving: Official repo runtime; multi-GPU support; no HF inference provider · Released 2026-03 (HF 2026-04-28) · https://huggingface.co/Skywork/Matrix-Game-3.0 - **Vista** — Driving world model — high-fidelity future prediction with multi-modal action control (steer/speed/command/trajectory/goal); the open successor to GenAD; batch · GPU: 1x 40GB (A100) for sampling; 24GB with reduced settings · Serving: Custom PyTorch repo only · Released 2024-06-06 · https://github.com/OpenDriveLab/Vista - **iVideoGPT (OXE family)** — Compact generic world model — action-free/action-conditioned/goal-conditioned video prediction on Open X-Embodiment for MBRL and robot eval; batch, cheap · GPU: 1x 12–24GB — the lightest real world model in the catalog · Serving: HF-hosted checkpoints, transformers/torch pipeline in repo · Released 2024-05-30 (models), 2024-08-31 (training code) · https://github.com/thuml/iVideoGPT - **1X World Model baselines (worldmodel_unet_v0 + GENIE_35M/138M/210M)** — Humanoid (EVE) egocentric world-model baselines from the 1X World Model Challenge — action-conditioned future prediction on 1X robot data; batch · GPU: 1x 8–16GB · Serving: Plain PyTorch/HF; challenge repo scripts · Released 2024-07-30 (data updated 2025-04) · https://huggingface.co/1x-technologies/worldmodel_unet_v0 - **Oasis 500M** — REAL-TIME interactive world model — keyboard-input → autoregressive gameplay (Minecraft-style) frame generation; the open small version of Decart's larger closed model · GPU: 1x 24GB; H100 for full-frame-rate interactivity · Serving: Open inference repo (decartAI/open-oasis) · Released 2024-11 · https://huggingface.co/Etched/oasis-500m - **RynnVLA-002 (formerly WorldVLA)** — Unified autoregressive action-world model — one model both predicts future frames (world model) and emits robot actions (VLA); 97.4% on LIBERO · GPU: 1x A100 80GB recommended; 40GB feasible for inference · Serving: HF checkpoints + repo training/eval code incl. real-world LeRobot experiment support · Released 2025-11-10 (WorldVLA v1: 2025-06) · https://github.com/alibaba-damo-academy/WorldVLA
### ⚠️ Servable with conditions | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Cosmos-Predict2.5 (2B / 14B)** | NVIDIA | 2.06B and 14B (flow-based diffusion tran | NVIDIA Open Model License, Apache-2.0 code (HF tag nvidia-op | Commercially usable per card, but gated access plus auto-termination if any safety guardrail is bypassed/disabled — keep guardrails intact in the serving stack. | | **Cosmos-Transfer2.5 (2B)** | NVIDIA | 2.36B | NVIDIA Open Model License + Apache-2.0 (gated) | Same NOML terms verified: commercial use allowed, gated repo, rights auto-terminate on guardrail circumvention. | | **Cosmos-Predict1 (7B/14B diffusion, 4B/12B autoregressive)** | NVIDIA | 4B–14B | NVIDIA Open Model License (gated | Claim confirmed — hosted serving allowed under NOML with the guardrail auto-termination clause; gated behind contact-info agreement. | | **Cosmos-Predict2 (0.6B/2B/14B, Text2Image + Video2World)** | NVIDIA | 0.6B / 2B / 14B | NVIDIA Open Model License (gated) | Claim confirmed — commercially usable, gated, guardrail auto-termination clause applies. | | **Cosmos-Transfer1 (7B, + Sample-AV multiview)** | NVIDIA | 7B | NVIDIA Open Model License (gated | Claim confirmed — commercial serving permitted with guardrails intact; gated repo. | | **Cosmos-Reason1-7B** | NVIDIA | 8B total (675M vision encoder + 7.07B LM | NVIDIA Open Model License + Apache-2.0 (HF tag nvidia-open-m | Confirmed ungated but still NOML-governed — commercially usable with the guardrail auto-termination clause, so 'conditions' not 'yes'. | | **Cosmos-Reason2 (2B / 8B)** | NVIDIA | 2B and 8.77B | NVIDIA Open Model License + Apache-2.0 (GATED | Claim confirmed — perpetual royalty-free commercial license incl. sale/offer-for-sale, but revocable and auto-terminates on guardrail/DRM circumvention. | | **OmniDreams (omni-dreams-models)** | NVIDIA | 2B (Cosmos-Predict2.5-based) | NVIDIA Open Model License (gated) | Claim confirmed — standard NOML gated repo; commercial hosted serving allowed subject to guardrail and AUP terms. | | **Hunyuan-GameCraft-1.0** | Tencent | not disclosed (HunyuanVideo-scale DiT); | Tencent Hunyuan Community License (LICENSE file in GitHub re | Claim confirmed — hosted serving allowed under 100M MAU with attribution, but you cannot serve EU/UK/South Korea users at all and outputs can't train other models. | | **HunyuanWorld-Voyager** | Tencent | not disclosed (HunyuanVideo-I2V-based) | tencent-hunyuanworld-voyager-community (HF tag | Claim direction right but materially understated — this variant's MAU threshold is 1M, 100x tighter than GameCraft's; above that you need a negotiated Tencent license, and EU/UK/SK are excluded territories. |
Serving details - **Cosmos-Predict2.5 (2B / 14B)** — Unified Text2World/Image2World/Video2World prediction — 720p 16fps future video for robot sim, eval, synthetic data · GPU: 2B: 32.5GB VRAM (1x L40S works but ~43 min/video; H100 ~4 min); 14B: 80GB-class · Serving: Official cosmos-predict2.5 runtime (PyTorch + Transformer Engine); no vLLM; batch · Released 2025-10-06 · https://huggingface.co/nvidia/Cosmos-Predict2.5-2B - **Cosmos-Transfer2.5 (2B)** — Sim2Real / ControlNet-style world transfer — 720p video from up to 4 control inputs (edge/depth/seg/blur); multiview AV variant · GPU: 65.4GB VRAM → 1x H100 80GB (B200 ~286s/video) · Serving: Official cosmos-transfer2.5 repo only; batch · Released 2025-10-06 · https://huggingface.co/nvidia/Cosmos-Transfer2.5-2B - **Cosmos-Predict1 (7B/14B diffusion, 4B/12B autoregressive)** — Gen-1 world foundation models — physics-aware 5s Video2World/Text2World prediction (120 frames, 12–40 fps) · GPU: 7B: 27.3GB (max offload) to 76.5GB → 1x A100/H100 80GB; ~383s/video on H100 · Serving: Official cosmos-predict1 repo; batch · Released 2025-01-06 · https://huggingface.co/nvidia/Cosmos-Predict1-7B-Video2World - **Cosmos-Predict2 (0.6B/2B/14B, Text2Image + Video2World)** — Gen-2 world prediction (superseded by 2.5 but still served/finetuned widely, e.g. GR00T-Dreams robot-data pipelines build on it) · GPU: 2B: ~32GB; 14B: 80GB-class · Serving: Official cosmos-predict2 repo, Diffusers support for T2I; batch · Released 2025-06-17 · https://huggingface.co/nvidia/Cosmos-Predict2-2B-Video2World - **Cosmos-Transfer1 (7B, + Sample-AV multiview)** — World-to-world transfer: photoreal video from sim renders via edge/depth/seg/keypoint controls — the sim2real synthetic-data workhorse · GPU: 1x H100 80GB (tested H100/A100/GB200) · Serving: Official cosmos-transfer1 repo; batch · Released 2025-03-17 · https://huggingface.co/nvidia/Cosmos-Transfer1-7B - **Cosmos-Reason1-7B** — Physical-AI reasoning VLM — spatial-temporal / embodied common-sense reasoning over robot video, used as world-model critic and sim-data judge · GPU: 1x 40GB (A100) comfortable; 24GB feasible via vLLM BF16 · Serving: vLLM (official), SGLang, Docker Model Runner — easiest-to-serve Cosmos member · Released 2025-05-17 (GitHub), latest HF update 2025-08-01 · https://huggingface.co/nvidia/Cosmos-Reason1-7B - **Cosmos-Reason2 (2B / 8B)** — Next-gen physical reasoning VLM — physics/common-sense chain-of-thought, spatial grounding with coordinates, 256K context robot video QA · GPU: 8B: 32GB minimum (tested H100/A100); 2B: 1x 24GB · Serving: HF Transformers (BF16); vLLM not yet documented on card · Released 2025-12-19 (improved 2026-03-10) · https://huggingface.co/nvidia/Cosmos-Reason2-8B - **OmniDreams (omni-dreams-models)** — Action-conditioned neural sensor sim for driving — multi-camera photoreal video from sim state + actions for closed-loop AV policy eval; REAL-TIME capable · GPU: 1x H100; ~52 FPS at 720p on GB300 with flashdreams TensorRT runtime · Serving: flashdreams (TensorRT + CUDA Graphs) official runtime; real-time interactive · Released 2026-06-02 · https://huggingface.co/nvidia/omni-dreams-models - **Hunyuan-GameCraft-1.0** — Interactive game-world video generation — keyboard/mouse → continuous camera-space gameplay video; interactive-conditioned but not real-time · GPU: 24GB minimum (very slow); 80GB recommended · Serving: Custom repo scripts; single-GPU low-VRAM mode · Released 2025-08-14 · https://github.com/Tencent-Hunyuan/Hunyuan-GameCraft-1.0 - **HunyuanWorld-Voyager** — Camera-path-controllable RGB-D world video from a single image — world-consistent point-cloud sequences for 3D scene reconstruction / sim environments; batch · GPU: 60–80GB class (HunyuanVideo-scale); not stated on card · Serving: Custom repo; no inference providers · Released 2025-06-04 (paper; weights Sep 2025) · https://huggingface.co/tencent/HunyuanWorld-Voyager
### ❌ Research-only — do not serve | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Genie Envisioner (GE-Base / GE-Act / GE-Sim)** | AgiBot | not stated on card (GE-Base builds on LT | SPLIT LICENSE | Claim of 'conditions' is wrong — I fetched the actual ltx-video-2b-v0.9 license text: research-only, no commercial use at any scale, hosted-service serving explicitly in scope; plus NC code license. Classic permissive-code/restricted-weights trap. | **Excluded (closed / NC / no grant):** Genie 3 (Google DeepMind) · GAIA-1 / GAIA-2 (Wayve) · 1X production world model (Redwood-era) · DreamerV3 (Hafner) · DreamerV4 (Hafner et al., Oct 2025 paper) · WHAM / Muse 200M & 1.6B (Microsoft) · EnerVerse-AC (AgiBot) · lingbot-world-v2-14b-causal-fast (Robbyant) · DriveDreamer / DriveDreamer-2 (GigaAI) · GenAD (OpenDriveLab, CVPR 2024) · PAN (MBZUAI Institute of Foundation Models) · UniSim (Google) · Decart Oasis 2.0 / MirageLSD · World Labs Marble / Odyssey · Qwen-AgentWorld-35B-A3B · Matrix-3D (Skywork) · Emu3.5 (BAAI) · Hunyuan-GameCraft & HunyuanWorld-Voyager in EU/UK/South Korea ## Language & reasoning (robot brains + voice) ### ✅ Clean — serve without strings | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Qwen3.5 (0.8B–397B, natively multimodal)** | Alibaba Qwen | 0.8B / 2B / 4B / 9B / 27B dense; 35B-A3B | Apache-2.0 (HF tag verified on all sizes, incl. 397B-A17B) | Clean Apache-2.0 on both checked sizes incl. the 397B MoE; ungated; hosted serving unrestricted. | | **Qwen3.6-27B / Qwen3.6-35B-A3B** | Alibaba Qwen | 27B dense; 35B-A3B MoE (multimodal nativ | Apache-2.0 (HF tag verified) | Both repos Apache-2.0, ungated; hosted serving unrestricted. | | **Qwen3-VL (2B/4B/8B/32B, 30B-A3B, 235B-A22B; Instruct+Thinking)** | Alibaba Qwen | 2B, 4B, 8B, 32B dense; 30B-A3B and 235B- | Apache-2.0 | Apache-2.0 on all sizes checked incl. largest MoE and 2B-Thinking; no size carve-outs found. | | **Qwen2.5-VL-7B / 32B-Instruct** | Alibaba Qwen | 7B, 32B (also AWQ variants) | Apache-2.0 (HF tag verified | 7B/32B are clean Apache-2.0; the 3B (research) and 72B (Qwen license) carve-outs are real — don't generalize across sizes. | | **Gemma 4 (E2B / E4B / 12B / 26B-A4B / 31B)** | Google DeepMind | E2B/E4B (MatFormer, on-device, any-to-an | Apache-2.0 | Ungated (gated=false), genuine Apache-2.0; Google's page links a prohibited-use policy alongside but the license text itself is plain Apache — hosted serving permitted. | | **InternVL3.5 (1B–38B dense; MoE up to 241B-A28B)** | OpenGVLab | 1B–38B dense + 20B/30B-class MoE + 241B- | Apache-2.0 across the series (verified tags, incl. 241B-A28B | Uniform Apache-2.0 incl. the largest MoE; no qwen-license leftovers in metadata. | | **Phi-4-multimodal-instruct** | Microsoft | 5.6B (vision + audio + text in, text out | MIT (verified) | Plain MIT, ungated; hosted serving unrestricted. | | **Molmo (MolmoE-1B, 7B-O, 7B-D, 72B)** | Allen Institute for AI | MolmoE-1B: ~7B MoE / ~1B active; 7B-O/7B | Apache-2.0 (verified | Apache-2.0 on checked sizes; note 7B-D/72B sit on Qwen2 backbones but Ai2 ships them under Apache. | | **Molmo2 (4B / 8B / O-7B / VideoPoint-4B)** | Allen Institute for AI | 4B, 8B (Qwen3 backbones), O-7B (fully-op | Apache-2.0 (verified all) | All four repos Apache-2.0, ungated; hosted serving unrestricted. | | **Molmo2-ER (Embodied Reasoning)** | Allen Institute for AI | 4B-class (~5B total; Qwen3-4B + SigLIP2) | Apache-2.0 (verified) | Apache-2.0, ungated; serving unrestricted. | | **Kimi-VL-A3B (Instruct / Thinking-2506)** | Moonshot AI | 16B total MoE, ~3B active (+0.4B vision | MIT (verified) | Plain MIT; hosted serving unrestricted. | | **GLM-4.1V-9B-Thinking** | Zhipu AI (zai-org) | ~9B | MIT (verified) | Plain MIT; serving unrestricted. | | **GLM-4.5V** | Zhipu AI (zai-org) | 106B total MoE, 12B active | MIT (verified) | Plain MIT; serving unrestricted. | | **GLM-4.6V (+ GLM-4.6V-Flash)** | Zhipu AI (zai-org) | MoE flagship (4.5V-class or larger; exac | MIT (verified tags on all three repos incl. FP8) | MIT on all three repos incl. FP8; serving unrestricted. | | **MiniCPM-V-4.5 / V-4.6 (+4.6-Thinking)** | OpenBMB | ~8B (Qwen3-8B + SigLIP2) | Apache-2.0 (verified on all current repos | Current repos are plain Apache-2.0 — the old custom MiniCPM commercial-registration license is gone from these cards. | | **MiniCPM-o-4.5** | OpenBMB | ~9B omni (vision + audio in, speech + te | Apache-2.0 (verified) | Apache-2.0, ungated; serving unrestricted. | | **MiMo-VL-7B (RL / SFT, 2508)** | Xiaomi | 7B | MIT (verified) | Plain MIT; serving unrestricted. | | **ERNIE-4.5-VL-28B-A3B (PT + Thinking; 424B-A47B for cloud)** | Baidu | 28B total MoE / 3B active; 424B-A47B fla | Apache-2.0 (verified on all VL repos) | Apache-2.0, ungated; serving unrestricted. | | **Magma-8B** | Microsoft Research | 8.9B | MIT (verified tag + cardData) | MIT is clean; card describes it as 'designed for research purposes' — an intent disclaimer, not a license restriction on serving. | | **RoboBrain 2.0 (3B / 7B / 32B)** | BAAI | 3B, 7B, 32B (+official FP8/W8A16 7B vari | Apache-2.0 (verified all) | Apache-2.0 so serving is unrestricted; 7B repo has a click-through auto-gate (contact info) not mentioned in the claim. | | **RoboBrain 2.5 (4B / 8B-MT / 8B-NV)** | BAAI | 4B; 8B variants are 9B actual (Qwen3-VL | Apache-2.0 (verified) | Apache-2.0 with an auto contact-info gate, exactly as claimed; serving unrestricted. | | **Hy-Embodied-VLM-1.0** | Tencent (Robotics X / Hy Vision) | ~30B total MoE, ~3B active (Hy3-A3B back | Apache-2.0 (verified) | Apache-2.0, ungated — no Tencent Hunyuan community-license terms on this repo. | | **SpaceQwen3-VL-2B-Thinking** | Remyx AI | 2B (Qwen3-VL-2B base) | Apache-2.0 (verified tag | Clean chain: Apache finetune of an Apache base — unlike the SpaceThinker/SpaceOm siblings on research-licensed Qwen2.5-VL-3B. | | **Whisper large-v3 / large-v3-turbo** | OpenAI | 1.54B / 0.81B | HF tags: apache-2.0 (v3) and MIT (turbo) | Tag mismatch across repos confirmed but both licenses are permissive; serving unrestricted. | | **Parakeet-TDT 0.6B v2 / v3** | NVIDIA | 0.6B | CC-BY-4.0 (verified | Commercial serving fine with attribution; caution stands for newer 2026 parakeet repos that moved to license:other — check each new release. | | **Qwen3-ASR (0.6B / 1.7B)** | Alibaba Qwen | 0.6B, 1.7B (+-hf transformers ports, Jun | Apache-2.0 (verified) | Apache-2.0, ungated; serving unrestricted. | | **Kyutai STT (stt-1b-en_fr, stt-2.6b-en)** | Kyutai | 1B, 2.6B | CC-BY-4.0 (verified) | Commercial serving fine with attribution. | | **Kyutai TTS (tts-1.6b-en_fr, tts-0.75b)** | Kyutai | 1.6B / 0.75B | CC-BY-4.0 (verified) | Commercial serving fine with attribution. | | **Moshi (moshiko / moshika)** | Kyutai | 7.7B Helium backbone + Mimi neural codec | CC-BY-4.0 on the 2024 base checkpoints (verified) | 2024 base checkpoints are CC-BY-4.0 (serve commercially with attribution); the 2026 RL refresh is genuinely NC — the warning in the claim is confirmed, don't serve that one. | | **CSM-1B (Sesame)** | Sesame AI | 1B Llama-style backbone + audio decoder | Apache-2.0 (verified) | Apache-2.0 so paid serving is permitted; auto-gate + Sesame acceptable-use (no voice impersonation) rides alongside — couldn't read the gated README text itself. | | **Voxtral (Mini-3B / Small-24B, 2507)** | Mistral AI | 3B (Mini), 24B (Small) — speech-understa | Apache-2.0 (verified) | Apache-2.0, ungated; serving unrestricted. | | **Voxtral-Mini-4B-Realtime-2602** | Mistral AI | 4B | Apache-2.0 (verified) | Apache-2.0, ungated; serving unrestricted. | | **Qwen3-Omni-30B-A3B (Instruct / Thinking / Captioner)** | Alibaba Qwen | 30B total MoE / 3B active; text+image+au | Apache-2.0 | Author-declared Apache-2.0 in card metadata; treat as Apache but note the missing LICENSE file if you need paper-trail-grade provenance. | | **Qwen2.5-Omni-7B** | Alibaba Qwen | 7B (+Talker; ~10.7B total footprint) | Apache-2.0 (cardData license_name apache-2.0 | LICENSE file is verbatim Apache-2.0; tag/card 'other' is cosmetic. 3B sibling remains research-only — keep it excluded. | | **DeepSeek-V4-Flash / V4-Pro (text-only)** | DeepSeek | MoE (sizes not re-verified this pass; Fl | MIT (verified tags) | Plain MIT; serving unrestricted. | | **SmolVLM2 (256M / 500M / 2.2B)** | Hugging Face TB | 0.256B–2.2B (video-capable) | Apache-2.0 | Apache-2.0, ungated; serving unrestricted. | | **Step-3 (StepFun)** | StepFun | 321B total MoE / 38B active, multimodal | Apache-2.0 (per release | Apache-2.0, ungated; serving unrestricted. |
Serving details - **Qwen3.5 (0.8B–397B, natively multimodal)** — Current flagship open robot-brain family: vision-language planning, spatial reasoning, agentic tool use across the full size ladder · GPU: ≤9B: L40S (4B fits L4); 27B/35B-A3B: 1×80GB BF16 or L40S with official FP8; 122B-A10B: 2×H100; 397B-A17B: 8×H100 · Serving: vLLM day-0 (official FP8/GPTQ-Int4 variants shipped); transformers; SGLang. No LeRobot (not an action model) · Released 2026-02 · https://huggingface.co/Qwen/Qwen3.5-9B - **Qwen3.6-27B / Qwen3.6-35B-A3B** — Newest Qwen multimodal generation (post-3.5) — highest-quality mid-size reasoning VLM as of mid-2026 · GPU: 1×80GB BF16; official FP8 fits L40S (tight KV headroom) · Serving: vLLM (FP8 repos shipped); transformers · Released 2026-04 · https://huggingface.co/Qwen/Qwen3.6-27B - **Qwen3-VL (2B/4B/8B/32B, 30B-A3B, 235B-A22B; Instruct+Thinking)** — Proven robot-brain VLM generation — strong grounding/pointing, video, 256K ctx; substrate of Cosmos-Reason2 and RoboBrain2.5 · GPU: 2B–8B: L4/L40S; 32B: 1×80GB BF16 or L40S FP8; 30B-A3B: L40S FP8 / 1×80GB BF16; 235B-A22B: 8×H100 BF16 or 4×H100 FP8 · Serving: vLLM native (confirmed in supported-models docs); SGLang; transformers; GGUF/llama.cpp · Released 2025-09/10 · https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct - **Qwen2.5-VL-7B / 32B-Instruct** — The most-deployed robot VLM baseline of 2025; grounding + agentic; base of Cosmos-Reason1, RoboBrain2.0, countless fine-tunes · GPU: 7B: L40S; 32B: 1×80GB BF16 or L40S AWQ/FP8 · Serving: vLLM native (mature); transformers; SGLang · Released 2025-01 (32B 2025-03) · https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct - **Gemma 4 (E2B / E4B / 12B / 26B-A4B / 31B)** — First fully-Apache Gemma; omnimodal small models ideal for on-robot brains with audio+vision in one model · GPU: E2B/E4B: L4; 12B: L40S; 26B-A4B/31B: 1×80GB BF16 (official QAT q4_0 GGUF fits L40S/edge) · Serving: transformers day-0 (AutoModelForMultimodalLM); official QAT GGUFs for llama.cpp; vLLM support recent — pin/verify vLLM version at launch · Released 2026-03 (12B 2026-05) · https://huggingface.co/google/gemma-4-E4B-it - **InternVL3.5 (1B–38B dense; MoE up to 241B-A28B)** — Cleaner-licensed successor to InternVL3 with reasoning (Cascade RL) and GUI/embodied agent skills · GPU: ≤14B: L40S; 38B: 1×80GB; 241B-A28B: 8×H100 (4×H100 FP8) · Serving: vLLM (confirmed); LMDeploy; transformers · Released 2025-08 · https://huggingface.co/OpenGVLab/InternVL3_5-8B - **Phi-4-multimodal-instruct** — Compact tri-modal brain — hears speech and sees simultaneously; good single-GPU voice-capable robot head · GPU: L4 / L40S · Serving: vLLM (confirmed); transformers; ONNX Runtime for edge · Released 2025-02 · https://huggingface.co/microsoft/Phi-4-multimodal-instruct - **Molmo (MolmoE-1B, 7B-O, 7B-D, 72B)** — Pointing-native VLM — outputs pixel coordinates for 'point at X' grounding/affordance; robotics community favorite · GPU: ≤7B: L40S; 72B: 2×80GB · Serving: vLLM (Molmo arch supported); transformers · Released 2024-09 · https://huggingface.co/allenai/Molmo-7B-D-0924 - **Molmo2 (4B / 8B / O-7B / VideoPoint-4B)** — Video + multi-image pointing, tracking, temporal reasoning — upgraded open pointing stack for robot perception · GPU: L40S (4B fits L4) · Serving: vLLM + SGLang (per model card); transformers · Released 2025-12 · https://huggingface.co/allenai/Molmo2-8B - **Molmo2-ER (Embodied Reasoning)** — Embodied-perception specialist: scene understanding, pixel-accurate pointing, ego↔exo correspondence, video temporal reasoning; +17pts over Molmo2 base, beats closed models on 9/13 embodied benchmarks · GPU: L4 / L40S · Serving: vLLM + SGLang (card-confirmed); transformers; Docker recipe · Released 2026-05 · https://huggingface.co/allenai/Molmo2-ER - **Kimi-VL-A3B (Instruct / Thinking-2506)** — Cheap-to-serve agentic/reasoning VLM; Thinking-2506 strong on long-horizon reasoning and video · GPU: L40S (weights ~32GB BF16) · Serving: vLLM (supported); transformers · Released 2025-04 (2506 update 2025-06) · https://huggingface.co/moonshotai/Kimi-VL-A3B-Thinking-2506 - **GLM-4.1V-9B-Thinking** — Compact RL-trained reasoning VLM; strong GUI + spatial grounding for its size · GPU: L40S · Serving: vLLM (GLM-V confirmed); transformers · Released 2025-06 · https://huggingface.co/zai-org/GLM-4.1V-9B-Thinking - **GLM-4.5V** — Top-tier open VLM of H2-2025: grounding, GUI agent, video, spatial reasoning with thinking mode · GPU: 2×H100 FP8 (official FP8 repo) or 4×H100 BF16 · Serving: vLLM (confirmed); SGLang; transformers · Released 2025-08 · https://huggingface.co/zai-org/GLM-4.5V - **GLM-4.6V (+ GLM-4.6V-Flash)** — Dec-2025 refresh of the best MIT-licensed VLM line; Flash gives a small-GPU option of the same family · GPU: Flagship: multi-H100 (FP8 repo provided); Flash: L40S-class · Serving: vLLM (GLM-4.5V/4.6V confirmed in docs); transformers · Released 2025-12 · https://huggingface.co/zai-org/GLM-4.6V - **MiniCPM-V-4.5 / V-4.6 (+4.6-Thinking)** — Edge-class VLM with GPT-4o-level claims on vision benchmarks; high-FPS video understanding — good onboard brain · GPU: L40S; int4/GGUF runs on Jetson Orin-class · Serving: vLLM (confirmed); llama.cpp/ollama; transformers · Released V-4.5 2025-08; V-4.6 2026-04 · https://huggingface.co/openbmb/MiniCPM-V-4.6 - **MiniCPM-o-4.5** — Full end-to-end voice+vision loop on a single GPU — live speech conversation while seeing; strong humanoid-voice candidate · GPU: L40S (int4 fits smaller) · Serving: vLLM (MiniCPM-o arch supported); transformers; official web demo stack · Released 2026-02 · https://huggingface.co/openbmb/MiniCPM-o-4_5 - **MiMo-VL-7B (RL / SFT, 2508)** — RL-tuned 7B VLM with strong reasoning + GUI grounding; underrated compact brain · GPU: L40S · Serving: vLLM (Qwen2.5-VL-compatible arch); transformers; GGUF · Released 2025-08 (2508 refresh) · https://huggingface.co/XiaomiMiMo/MiMo-VL-7B-RL-2508 - **ERNIE-4.5-VL-28B-A3B (PT + Thinking; 424B-A47B for cloud)** — Efficient MoE VLM; Thinking variant (Nov 2025) adds visual reasoning — cheap active compute for always-on perception QA · GPU: 28B-A3B: 1×80GB BF16 or L40S W4/FP8; 424B-A47B: 8×H100+ · Serving: vLLM (ERNIE VL MoE supported); FastDeploy; transformers · Released 2025-06 (Thinking 2025-11) · https://huggingface.co/baidu/ERNIE-4.5-VL-28B-A3B-Thinking - **Magma-8B** — Agentic foundation model bridging UI + robot manipulation (Set-of-Mark / Trace-of-Mark action grounding) — pipeline_tag is literally 'robotics' · GPU: L40S · Serving: transformers only (custom MagmaForCausalLM code); no vLLM · Released 2025-02 · https://huggingface.co/microsoft/Magma-8B - **RoboBrain 2.0 (3B / 7B / 32B)** — Embodied brain suite: affordance prediction, pointing, trajectory sketching, long-horizon task planning · GPU: 3B/7B: L40S; 32B: 1×80GB · Serving: vLLM/SGLang per repo (Qwen2.5-VL arch); transformers · Released 2025-06/07 · https://huggingface.co/BAAI/RoboBrain2.0-7B - **RoboBrain 2.5 (4B / 8B-MT / 8B-NV)** — 2026 upgrade: 3D spatial reasoning (depth-aware pointing), keypoint-sequence prediction for manipulation, dense temporal value/progress estimation · GPU: L4 (4B) / L40S (8B) · Serving: vLLM via Qwen3-VL arch; transformers · Released 2026-01/02 · https://huggingface.co/BAAI/RoboBrain2.5-8B-MT - **Hy-Embodied-VLM-1.0** — Brand-new (days old) embodied VLM: action-relevant state understanding, action-transition reasoning, sequential adaptive reasoning; #1 on 19/38 embodied benchmarks · GPU: 1×80GB BF16 (~60GB weights); card recommends TP4–8 for throughput; FP8 would fit L40S when released · Serving: vLLM (card-recommended) + transformers + OpenAI-compatible endpoints · Released 2026-07-15 · https://huggingface.co/tencent/Hy-Embodied-VLM-1.0 - **SpaceQwen3-VL-2B-Thinking** — Tiny spatial-reasoning thinker — quantitative spatial QA on edge GPUs · GPU: L4 · Serving: vLLM (Qwen3-VL arch); transformers; GGUF · Released 2025-10 · https://huggingface.co/remyxai/SpaceQwen3-VL-2B-Thinking - **Whisper large-v3 / large-v3-turbo** — Default open ASR for robot voice input; 99 languages, robust in noise · GPU: L4 (runs on CPU/edge via whisper.cpp) · Serving: vLLM (Whisper supported); faster-whisper; whisper.cpp; transformers · Released 2023-11 (turbo 2024-10) · https://huggingface.co/openai/whisper-large-v3 - **Parakeet-TDT 0.6B v2 / v3** — Fastest production open ASR (v3 = 25 languages) with timestamps/punctuation — ideal low-latency robot ear · GPU: Any 8GB+ GPU (T4/L4) · Serving: NeMo / Riva; ONNX exports; no vLLM · Released 2025-04 / 2025-08 · https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3 - **Qwen3-ASR (0.6B / 1.7B)** — 2026 multilingual ASR with context-biasing (feed it your product vocabulary) — strong Whisper alternative · GPU: L4 · Serving: transformers (official -hf ports); Qwen recipes · Released 2026-01 · https://huggingface.co/Qwen/Qwen3-ASR-1.7B - **Kyutai STT (stt-1b-en_fr, stt-2.6b-en)** — Streaming ASR with semantic VAD built for live voice agents (the unmute.sh stack) — word-level latency · GPU: L4 · Serving: moshi server (Rust) + transformers ports (-trfs repos) + MLX · Released 2025-06 · https://huggingface.co/kyutai/stt-2.6b-en - **Kyutai TTS (tts-1.6b-en_fr, tts-0.75b)** — Streaming low-latency TTS (starts speaking before the text is finished) — pairs with Kyutai STT for a full voice loop · GPU: L4 · Serving: moshi server / PyTorch; no vLLM · Released 2025-06/07 · https://huggingface.co/kyutai/tts-1.6b-en_fr - **Moshi (moshiko / moshika)** — Full-duplex real-time speech-to-speech dialogue (~200ms, listens while talking) — the natural humanoid conversation engine · GPU: L40S BF16; L4 with int8/4 · Serving: Official Rust/PyTorch/MLX servers; no vLLM · Released 2024-09 · https://huggingface.co/kyutai/moshika-pytorch-bf16 - **CSM-1B (Sesame)** — Context-aware conversational TTS with natural prosody — high-quality robot voice output · GPU: L4 · Serving: transformers (CsmForConditionalGeneration) + official repo; no vLLM · Released 2025-03 · https://huggingface.co/sesame/csm-1b - **Voxtral (Mini-3B / Small-24B, 2507)** — ASR + spoken-language understanding + function calling directly from voice — lets a robot act on speech without a separate NLU stage · GPU: Mini: L4/L40S (~9.5GB); Small: 1×80GB (or 2×L40S) · Serving: vLLM (officially recommended by Mistral); transformers · Released 2025-07 · https://huggingface.co/mistralai/Voxtral-Mini-3B-2507 - **Voxtral-Mini-4B-Realtime-2602** — Streaming realtime speech model (Jan 2026) for live agents — sub-second interactive voice loop · GPU: L4/L40S · Serving: vLLM realtime path per Mistral release; official recipes · Released 2026-01 · https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 - **Qwen3-Omni-30B-A3B (Instruct / Thinking / Captioner)** — Strongest open end-to-end omni brain: ~211ms voice latency, sees while it converses — top humanoid voice-loop candidate · GPU: 1×80GB BF16 (~66GB with vision+talker); FP8/quant to fit L40S is tight · Serving: vLLM (qwen3_omni_moe arch, vLLM ≥0.11) + transformers · Released 2025-09 · https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct - **Qwen2.5-Omni-7B** — Smaller end-to-end voice+vision omni model — single-L40S full loop · GPU: L40S · Serving: transformers (full speech-out); vLLM audio-in partial · Released 2025-03 · https://huggingface.co/Qwen/Qwen2.5-Omni-7B - **DeepSeek-V4-Flash / V4-Pro (text-only)** — Frontier text-only reasoning (Apr 2026) for planning/impasse-handling behind a separate perception stack — include if the platform wants text brains too · GPU: Multi-H100 class (Flash smaller); FP8 recipes standard for DeepSeek · Serving: vLLM + SGLang (DeepSeek archs first-class) · Released 2026-04 · https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash - **SmolVLM2 (256M / 500M / 2.2B)** — Tiny video VLMs for on-robot low-power perception QA / captioning telemetry · GPU: CPU/Jetson/L4 · Serving: transformers; ONNX/WebGPU; llama.cpp · Released 2025-02 · https://huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct - **Step-3 (StepFun)** — Large open multimodal reasoner (mid-2025) — cloud-tier alternative brain · GPU: 8×H100 · Serving: vLLM/SGLang · Released 2025-07 · https://huggingface.co/stepfun-ai/step3
### ⚠️ Servable with conditions | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Qwen2.5-VL-72B-Instruct** | Alibaba Qwen | 72B | Qwen License (HF tag license:other / 'qwen') | LICENSE verbatim: commercial use OK, but >100M MAU products must request a license from Alibaba — hosted serving fine below that. | | **Gemma 3 (4B / 12B / 27B vision; 1B text)** | Google DeepMind | 1B (text-only), 4B, 12B, 27B (vision) | Gemma Terms of Use (HF tag 'gemma' | Commercial/hosted serving allowed under Gemma Terms, but Prohibited Use Policy binds and must flow down to your API users; manual gate acceptance required. | | **Gemma 3n (E2B / E4B)** | Google DeepMind | E2B (~5B raw, 2B effective), E4B (~8B ra | Gemma Terms of Use (tag 'gemma') | Same Gemma Terms as Gemma 3: serving allowed, prohibited-use flow-down + gated acceptance. | | **PaliGemma 2 (3B / 10B / 28B; pt + mix, 224–896px)** | Google DeepMind | 3B, 10B, 28B | Gemma Terms of Use (tag 'gemma') | Gemma Terms: commercial serving permitted subject to Prohibited Use Policy and downstream flow-down. | | **Llama 4 Scout / Maverick** | Meta | Scout 109B total (17B active, 16 experts | Llama 4 Community License (HF tag license:other | Serving permitted, but: >700M-MAU licensees need Meta's grant, 'Built with Llama' display + 'Llama' model-name prefix on derivatives, AUP incorporated; no EU-domicile exclusion found in Scout's LICENSE. | | **InternVL3 (1B–78B)** | OpenGVLab (Shanghai AI Lab) | 1B, 2B, 8B, 9B, 14B, 38B, 78B | MIXED per size (verified tags): 1B/2B/8B/14B/38B Apache-2.0 | Per-size mess confirmed — 78B inherits Qwen-72B 100M-MAU terms, and even the 'Apache' 8B card self-contradicts (qwen license_name leftover); audit each size before serving. | | **DeepSeek-VL2 (Tiny / Small / Base)** | DeepSeek | Tiny 3.4B (A1B), Small 16B (A2.8B), Base | Code MIT | Commercial serving explicitly supported, but Attachment-A use restrictions must flow down to your API users and DeepSeek reserves a remote-restriction right. | | **Kimi-K2.5 (vision)** | Moonshot AI | 1.06T total MoE (~32B active); natively | Modified MIT (cardData license_name 'modified-mit' | MIT + one rider verified verbatim: products >100M MAU or >$20M/mo revenue must prominently display 'Kimi K2.5' in the UI — serving otherwise unrestricted. | | **Nemotron Nano 12B v2 VL** | NVIDIA | 12.6B hybrid Mamba-Transformer (video + | NVIDIA Open Model License (tag license:other) | Commercial + hosted serving explicitly permitted, but guardrail-bypass auto-terminates the license, Trustworthy AI terms bind, and litigation terminates rights. | | **Nemotron-3-Nano-Omni-30B-A3B-Reasoning** | NVIDIA | 30B total MoE / ~3B active; omni (any-to | NVIDIA Open Model License (tag license:other) | 'Works are commercially usable', derivatives allowed, no prohibited-use list — but it's a bespoke NVIDIA agreement (attribution + litigation-termination), not OSI; read before shipping. | | **Cosmos-Reason1-7B** | NVIDIA | 8.1B (Qwen2.5-VL-7B substrate) | NVIDIA Open Model License (tag license:other) | Commercial serving permitted under NVIDIA OML; guardrail-bypass auto-termination + Trustworthy AI obligations apply. | | **Cosmos-Reason2 (2B / 8B / 32B)** | NVIDIA | 2B, 8B (8.77B, Qwen3-VL substrates), 32B | NVIDIA Open Model License (verified in card: 'Models are com | NVIDIA OML = commercially usable incl. serving, with guardrail/Trustworthy-AI clauses; auto-gate (contact sharing) confirmed. | | **RoboPoint v1 (7B / 13B)** | UW (wentao-yuan) | 7B/13B (Vicuna-1.5 and Llama-2 bases; me | HF tag says Apache-2.0 | Tag/reality split confirmed: weights are Llama-2 derivatives, so Llama 2 Community License (AUP, 700M-MAU clause, no-training-other-LLMs) realistically governs serving despite the Apache tag. | | **ECoT — Embodied Chain-of-Thought (ecot-openvla-7b-bridge / -oxe; steerable-policy 2026 variants)** | Embodied-CoT (Stanford/Berkeley) | 7B (OpenVLA/Prismatic + Llama-2-7B base) | MIT tag on bridge + steerable checkpoints (oxe untagged) | MIT-tagged, but downstream of Llama-2 Community License; verify appetite before commercial serving | | **SpaceLLaVA (13B)** | Remyx AI | 13B (LLaVA-1.5 / Vicuna base); -lite Mob | Apache-2.0 tag | Apache tag can't launder the LLaVA-1.5/Vicuna (Llama-2) lineage; treat Llama 2 Community License as governing for paid serving. | | **LFM2-VL (450M / 1.6B)** | Liquid AI | 0.45B, 1.6B | LFM Open License v1.0 (not re-verified this pass | Verified clause: commercial rights are conditioned on staying under $10M annual revenue — above the Threshold, commercial use (incl. paid serving) is simply not licensed; NC/research exempt. |
Serving details - **Qwen2.5-VL-72B-Instruct** — Large-tier VLM brain for cloud-side planning/impasse escalation · GPU: 2×80GB BF16; 4×L40S with AWQ · Serving: vLLM native; transformers · Released 2025-01 · https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct - **Gemma 3 (4B / 12B / 27B vision; 1B text)** — Solid mid-2025 VLM brain generation, 128K ctx, wide ecosystem · GPU: 4B/12B: L40S; 27B: 1×80GB or L40S int4 · Serving: vLLM native (confirmed); transformers; ollama/llama.cpp · Released 2025-03 · https://huggingface.co/google/gemma-3-12b-it - **Gemma 3n (E2B / E4B)** — On-device omnimodal brain (speech in!) for edge/onboard compute — superseded by Apache-licensed Gemma 4 E-series · GPU: L4 / Jetson-class; trivial on L40S · Serving: transformers; vLLM (added v0.10+); LiteRT/MediaPipe for edge · Released 2025-06 · https://huggingface.co/google/gemma-3n-E4B-it - **PaliGemma 2 (3B / 10B / 28B; pt + mix, 224–896px)** — Detection/segmentation/captioning transfer workhorse — best as a fine-tune base for perception heads, not a chat brain · GPU: 3B: L4; 10B: L40S; 28B: 1×80GB · Serving: vLLM (confirmed); transformers; big_vision (JAX) · Released 2024-11 · https://huggingface.co/google/paligemma2-3b-mix-448 - **Llama 4 Scout / Maverick** — Long-context multimodal generalist brains; Scout's 10M ctx useful for full-shift episodic memory · GPU: Scout: 1×H100 int4 / 4×H100 BF16; Maverick: 8×H100 node · Serving: vLLM (confirmed); transformers · Released 2025-04 · https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct - **InternVL3 (1B–78B)** — Strong general VLM line with good spatial/OCR; popular robotics fine-tune base · GPU: ≤14B: L40S; 38B: 1×80GB; 78B: 2×80GB · Serving: vLLM (confirmed); LMDeploy (native); transformers · Released 2025-04 · https://huggingface.co/OpenGVLab/InternVL3-8B - **DeepSeek-VL2 (Tiny / Small / Base)** — Efficient MoE VLM with strong grounding/OCR at low active-param cost · GPU: Tiny: L4; Small: L40S; 27B: 1×80GB · Serving: vLLM (confirmed); transformers (custom code) · Released 2024-12 · https://huggingface.co/deepseek-ai/deepseek-vl2 - **Kimi-K2.5 (vision)** — Frontier-class open multimodal brain for cloud-side escalation/teleop copilot — not an on-robot model · GPU: 8×H200 FP8 (~1TB) or 16×H100 · Serving: vLLM + SGLang (K2-family supported) · Released 2026-01 · https://huggingface.co/moonshotai/Kimi-K2.5 - **Nemotron Nano 12B v2 VL** — Efficient video-understanding VLM (doc/video intelligence) tuned for NVIDIA serving stack · GPU: L40S (BF16 ~25GB; FP8/NVFP4 repos provided) · Serving: vLLM (supported); TensorRT-LLM; NIM · Released 2025-10 · https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 - **Nemotron-3-Nano-Omni-30B-A3B-Reasoning** — 2026 omni reasoning model (audio+vision input) — candidate single-model voice+vision robot head on NVIDIA stack · GPU: 1×80GB BF16; official NVFP4/FP8 builds fit L40S · Serving: TRT-LLM/NIM; vLLM (verify version at launch); BF16+NVFP4+FP8 repos shipped · Released 2026-04 · https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 - **Cosmos-Reason1-7B** — Physical-common-sense + embodied chain-of-thought reasoning over video; the 2025 default 'is this safe/what happened' model · GPU: L40S · Serving: vLLM (Qwen2.5-VL arch); NIM; transformers · Released 2025-04 · https://huggingface.co/nvidia/Cosmos-Reason1-7B - **Cosmos-Reason2 (2B / 8B / 32B)** — Physical-AI reasoning upgrade — spatial/temporal reasoning and planning for robotics + AV; text+video/image in, text out · GPU: 2B: L4; 8B: L40S (card min 32GB); 32B: 1×80GB · Serving: vLLM via Qwen3-VL arch; NIM; transformers (card lists H100/A100 tested) · Released 2025-12 (32B 2026-04) · https://huggingface.co/nvidia/Cosmos-Reason2-8B - **RoboPoint v1 (7B / 13B)** — Spatial-affordance pointing VLM — 'point where to place/grasp' from language · GPU: L40S · Serving: LLaVA-style official repo; loadable via vLLM llava path; no LeRobot · Released 2024-09 · https://huggingface.co/wentao-yuan/robopoint-v1-vicuna-v1.5-13b - **ECoT — Embodied Chain-of-Thought (ecot-openvla-7b-bridge / -oxe; steerable-policy 2026 variants)** — Reasons step-by-step in text (plan→subtask→gripper) before emitting robot actions — the canonical embodied-CoT checkpoints; new 2026 'steerable policy' variants accept human interventions · GPU: L40S · Serving: OpenVLA codebase / transformers; no vLLM; not LeRobot-native · Released 2024-07 (steerable 2026-02) · https://huggingface.co/Embodied-CoT/ecot-openvla-7b-bridge - **SpaceLLaVA (13B)** — SpatialVLM-style quantitative spatial QA (distances, sizes, left/right-of) for scene sanity checks · GPU: L40S · Serving: LLaVA repo / GGUF (lite); vLLM llava path · Released 2024-03 · https://huggingface.co/remyxai/SpaceLLaVA - **LFM2-VL (450M / 1.6B)** — Fastest edge VLMs for embedded robot compute · GPU: CPU/Jetson/L4 · Serving: transformers; llama.cpp; LEAP edge SDK · Released 2025-08 · https://huggingface.co/LiquidAI/LFM2-VL-1.6B
### ❌ Research-only — do not serve | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Alpamayo-R1-10B / Alpamayo-1.5-10B** | NVIDIA | 10B (Cosmos-Reason substrate + trajector | NVIDIA Open Model License (tag license:other) | CLAIM CORRECTED: this is not the NVIDIA Open Model License — weights are research/eval-only, so charging for hosted inference is prohibited without a separately negotiated NVIDIA commercial license. | **Excluded (closed / NC / no grant):** Qwen2.5-VL-3B-Instruct (+AWQ) · Qwen2.5-Omni-3B · Remyx SpaceThinker-Qwen2.5VL-3B / SpaceOm / SpaceQwen2.5-VL-3B · kyutai/moshika-rl-seamless + personaplex-rl-seamless (2026) · NVIDIA parakeet-unified-en-0.6b and other 2026 Parakeet releases · Gemini Robotics-ER / Gemini Robotics On-Device (Google DeepMind) · OpenAI GPT-4o/realtime, Anthropic Claude · Llama 4 Behemoth · Kimi-K2.6 / K2.7-Code (Apr/Jun 2026) · Apple FastVLM · xAI Grok-2.5 open weights · MolmoAct2, RoboBrain-X0, GR00T N1.5, pi0/pi0.5, OpenVLA, Alpamayo (action side) · Qwen-AgentWorld-35B-A3B, MolmoMotion · Sesame full-size CSM (8B) ## Vision & perception ### ✅ Clean — serve without strings | Model | Org | Params | License | Notes | |---|---|---|---|---| | **SAM 2.1 (Hiera Tiny–Large)** | Meta | 39M–224M | Apache-2.0 (HF tag and repo agree | HF page confirms apache-2.0 tag, no gating; commercial hosted serving permitted. | | **SAM 1 (ViT-B/L/H)** | Meta | 91M/308M/636M | Apache-2.0 | HF page confirms apache-2.0, no gating or extra terms. | | **SAM-HQ / HQ-SAM 2** | SysCV (ETH Zurich) | ~10M (Light) – 640M (ViT-H) | Apache-2.0 | Repo is Apache-2.0 with no separate weight terms in README; upstream SAM/SAM2 are also Apache. | | **MobileSAM / MobileSAMv2** | Kyung Hee University | 9.7M | Apache-2.0 | Apache-2.0 repo, no research-only or separate weight license stated. | | **EfficientSAM (Ti/S)** | Meta | ~10M/26M | Apache-2.0 | Apache-2.0 repo covering code and checkpoints; no restrictions found. | | **EfficientTAM** | Meta | ~34M (small ViT variants) | Apache-2.0 | README explicitly licenses checkpoints under Apache-2.0 — cleanest possible weight coverage. | | **EdgeTAM** | Meta | ~14M | Apache-2.0 (HF tag | HF page confirms apache-2.0, no gating. | | **DINOv2 (w/ registers)** | Meta | 22M–1.1B (ViT-S…ViT-g) | Apache-2.0 (relicensed from CC-BY-NC in Sept 2023 | Current HF repo is apache-2.0; relicensing claim consistent — just avoid stale NC-era forks. | | **Depth Anything V2 — Small ONLY** | HKU / TikTok | 24.8M | Apache-2.0 (Small only | Small variant confirmed apache-2.0; Base/Large/Giant remain CC-BY-NC-4.0 — serve Small only. | | **Depth Anything 3 — Apache subset (Small/Base/Metric-Large/Mono-Large)** | ByteDance Seed | 0.08B/0.12B/0.35B/0.35B | Apache-2.0 for these four variants | Repo per-variant table matches claim exactly; DA3-LARGE/GIANT/NESTED (incl. 1.1) are CC-BY-NC-4.0 — pin checkpoint names. | | **Metric3D v2** | HKUST / DJI et al. | ViT-S / ViT-L / ViT-giant2 | BSD-2-Clause (repo states '2-clause BSD') | BSD-2 confirmed with no research-only weight terms; weights not separately licensed, so a brief email to the authors (they invite it) removes residual ambiguity. | | **MoGe-2** | Microsoft | 35M/104M/331M | MIT (repo | Both code and MoGe-2 HF weights verified MIT (unlike MoGe v1's NC weights); commercial serving fine. | | **MapAnything (Apache checkpoint)** | Meta (FAIR) | 1B | Apache-2.0 (facebook/map-anything-apache | Apache variant confirmed on HF; use this repo, not the default CC-BY-NC facebook/map-anything checkpoint. | | **Grounding DINO (T/B)** | IDEA-Research | 172M/233M | Apache-2.0 | Apache-2.0 repo, no separate weight restrictions found. | | **MM-Grounding-DINO** | OpenMMLab | ~172M+ (Tiny–Large) | Apache-2.0 (mmdetection) | HF page confirms apache-2.0, no restrictions. | | **OWLv2 (base/large)** | Google | 0.2B/0.4B | Apache-2.0 | Apache-2.0 governs; card's 'intended as a research output' framing is non-binding guidance, not a license term. | | **LLMDet (Tiny/Base/Large)** | Sun Yat-sen iSEE Lab | up to 0.3B | Apache-2.0 | Apache-2.0 confirmed for code and checkpoints; deps all permissive. | | **RT-DETR / RT-DETRv2** | Baidu | 20M–76M | Apache-2.0 (original lyuwenyu repo | Original repo Apache-2.0 with no weight restrictions; do NOT use the Ultralytics implementation (AGPL-3.0 — copyleft on your serving stack). | | **D-FINE (N–X)** | USTC | 4M–62M | Apache-2.0 | Apache-2.0 confirmed, no weight restrictions. | | **RF-DETR (Nano–Large + Seg)** | Roboflow | ~29M–128M | Apache-2.0 for Nano/Small/Medium/Large + segmentation varian | Repo license section confirms the split exactly as claimed; stay off detection-XL/2XL and the rfdetr_plus package. | | **Florence-2 (base/large)** | Microsoft | 0.23B/0.77B | MIT (HF tag confirmed | MIT confirmed on HF, no gating or restrictions. | | **CLIP ViT-L/14** | OpenAI | 428M | MIT | MIT governs; note the HF card's non-binding 'any deployed use case is currently out of scope' guidance — legal permission is clear, OpenAI's stated intent is research. | | **SigLIP 2 (B/L/So400m/g)** | Google DeepMind | 86M–1.2B | Apache-2.0 (HF tag confirmed | Apache-2.0 confirmed; no Gemma-style prohibited-use policy attached. | | **Perception Encoder (PE-Core B/L/G)** | Meta | 0.3B–2B | Apache-2.0 (HF tag confirmed | Apache-2.0 confirmed on HF, no restrictions. |
Serving details - **SAM 2.1 (Hiera Tiny–Large)** — Promptable image+video segmentation and object tracking — the default segmentation workhorse in real robot stacks · GPU: T4 16GB; RTX 4090/A10 for real-time video streams · Serving: HF Transformers native; Roboflow Inference server; abundant ONNX/TensorRT ports; no vLLM/LeRobot · Released 2024-07 (2.1 refresh 2024-10) · https://huggingface.co/facebook/sam2.1-hiera-large - **SAM 1 (ViT-B/L/H)** — Point/box-promptable image segmentation; still ubiquitous in Grounded-SAM style robot pipelines · GPU: T4 8GB (ViT-B) – 16GB (ViT-H) · Serving: HF Transformers native; huge ONNX/TensorRT ecosystem; Roboflow Inference; no vLLM · Released 2023-04 · https://huggingface.co/facebook/sam-vit-huge - **SAM-HQ / HQ-SAM 2** — Drop-in SAM/SAM2 variant with much higher-fidelity masks (learnable HQ output token) · GPU: T4 8–16GB; Light variant runs 41 FPS · Serving: HF Transformers support since 2025-06; official repo checkpoints · Released 2023-06 (HQ-SAM 2: 2024-11) · https://github.com/SysCV/sam-hq - **MobileSAM / MobileSAMv2** — Distilled SAM for edge robots — segment-anything on CPU/Jetson · GPU: CPU or Jetson Orin Nano; any 4GB GPU · Serving: Official repo + ONNX exports; community integrations; no vLLM · Released 2023-06 (v2: 2023-12) · https://github.com/ChaoningZhang/MobileSAM - **EfficientSAM (Ti/S)** — SAM-quality image segmentation at edge cost (SAMI-pretrained lightweight ViT) · GPU: CPU/Jetson; any 4GB GPU · Serving: Official repo checkpoints; ONNX/TorchScript exports provided; no vLLM · Released 2023-12 · https://github.com/yformer/EfficientSAM - **EfficientTAM** — SAM 2-class video object tracking+segmentation at >10 FPS on-device · GPU: Jetson Orin / T4 8GB · Serving: Official repo + HF checkpoints; no vLLM/LeRobot · Released 2024-12 · https://github.com/yformer/EfficientTAM - **EdgeTAM** — On-device SAM 2 variant — 22x faster, 16 FPS on iPhone 15; ideal for on-robot tracking · GPU: CPU/Jetson; any small GPU · Serving: HF checkpoint + GitHub repo; CoreML/ONNX paths; no vLLM · Released 2025-01-13 · https://huggingface.co/facebook/EdgeTAM - **DINOv2 (w/ registers)** — Self-supervised dense visual features — THE standard robot-policy vision backbone (OpenVLA and most VLA vision towers) · GPU: T4 8GB (ViT-L); A10 for ViT-g · Serving: HF Transformers + timm native; TensorRT-friendly; no vLLM needed · Released 2023-04 · https://huggingface.co/facebook/dinov2-large - **Depth Anything V2 — Small ONLY** — Fast monocular relative depth — widely used for cheap nav/obstacle depth on robots · GPU: CPU or T4 4GB · Serving: HF Transformers pipeline native; ONNX/TensorRT ports; no vLLM · Released 2024-06 · https://huggingface.co/depth-anything/Depth-Anything-V2-Small - **Depth Anything 3 — Apache subset (Small/Base/Metric-Large/Mono-Large)** — Any-view geometry foundation model: mono+multi-view depth, metric depth, camera pose, 3D Gaussians from one or many images · GPU: T4 8GB (Small/Base); T4 16GB (0.35B variants) · Serving: Official PyTorch repo + HF checkpoints; no vLLM/LeRobot yet · Released 2025-11-14 (v1.1 refresh 2025-12) · https://github.com/ByteDance-Seed/Depth-Anything-3 - **Metric3D v2** — Zero-shot metric monocular depth + surface normals (real-scale, camera-aware) — used for metric nav depth · GPU: T4 8–16GB; A10 24GB for giant2 · Serving: Official repo; ONNX variants released 2024-06; HF mirrors; no vLLM · Released 2024-03 (v2) · https://github.com/YvanYin/Metric3D - **MoGe-2** — Metric-scale 3D point maps, depth, normals and FoV from a single image — clean permissive alternative to NC 3D models · GPU: T4 8GB · Serving: pip package + HF checkpoints; official inference scripts; no vLLM · Released 2025-06 · https://github.com/microsoft/MoGe - **MapAnything (Apache checkpoint)** — Universal end-to-end multi-view 3D reconstruction transformer — 12+ tasks (SfM, multi-view depth, pose, mono depth) · GPU: A10/L4 24GB (scales with number of views) · Serving: Official repo + HF; no vLLM/LeRobot · Released 2025-09 (Apache ckpt updated 2026-01-20) · https://huggingface.co/facebook/map-anything-apache - **Grounding DINO (T/B)** — Open-vocabulary text→box detection; the detection half of every Grounded-SAM robot pipeline · GPU: T4 8GB · Serving: HF Transformers native; Roboflow Inference; ONNX ports; no vLLM · Released 2023-03 · https://github.com/IDEA-Research/GroundingDINO - **MM-Grounding-DINO** — Fully open retraining of Grounding DINO with stronger zero-shot numbers — the servable upgrade path since GDINO 1.5+ went API-only · GPU: T4 8GB · Serving: HF Transformers native; mmdetection; no vLLM · Released 2024-01 · https://huggingface.co/openmmlab-community/mm_grounding_dino_large_all - **OWLv2 (base/large)** — Zero-shot text-conditioned AND image-conditioned (one-shot) detection — used in OK-Robot-style pick pipelines · GPU: T4 8–16GB · Serving: HF Transformers native; Roboflow Inference; no vLLM · Released 2023-06 · https://huggingface.co/google/owlv2-large-patch14-ensemble - **LLMDet (Tiny/Base/Large)** — Open-vocab detector co-trained with an LLM captioner; beats Grounding DINO zero-shot (CVPR 2025 highlight) · GPU: T4 16GB · Serving: HF Transformers support; official repo; no vLLM · Released 2025-01-31 · https://github.com/iSEE-Laboratory/LLMDet - **RT-DETR / RT-DETRv2** — Real-time closed-set DETR detection — deterministic, anchor-free, Jetson-friendly for safety-critical detection · GPU: Jetson Orin / T4 4GB · Serving: HF Transformers native; TensorRT official path; no vLLM · Released 2023-04 (v2: 2024-07) · https://github.com/lyuwenyu/RT-DETR - **D-FINE (N–X)** — Real-time detection with fine-grained distribution refinement — SOTA COCO among real-time detectors at release · GPU: Jetson Orin / T4 4GB · Serving: HF Transformers support (2025); TensorRT/ONNX; no vLLM · Released 2024-10-18 · https://github.com/Peterande/D-FINE - **RF-DETR (Nano–Large + Seg)** — NAS-optimized real-time DETR (detection/segmentation/keypoints) — current real-time SOTA line, actively shipped through Jun 2026 · GPU: Jetson Orin / T4 4–8GB (TensorRT FP16 benchmarked) · Serving: Roboflow Inference server (official, first-class); pip rfdetr; TensorRT; no vLLM · Released 2025-03 (v1.8.3: 2026-06-29) · https://github.com/roboflow/rf-detr - **Florence-2 (base/large)** — Unified promptable vision model: caption, detect, ground, segment, OCR in one small seq2seq — cheap generalist perception endpoint · GPU: T4 8GB · Serving: HF Transformers (trust_remote_code); ONNX ports; vLLM had experimental encoder-decoder support only — treat as no · Released 2024-06 · https://huggingface.co/microsoft/Florence-2-large - **CLIP ViT-L/14** — Image-text embeddings for open-vocab labeling, retrieval, and reward/goal encoding in robot stacks · GPU: CPU or T4 4GB · Serving: HF Transformers/open_clip; vLLM pooling-model support; TEI; everywhere · Released 2021-01 · https://huggingface.co/openai/clip-vit-large-patch14 - **SigLIP 2 (B/L/So400m/g)** — SOTA image-text encoder with dense/localization skills — the robot VLA vision tower lineage (PaliGemma/pi0) · GPU: T4 8–16GB · Serving: HF Transformers native; vLLM pooling support; TEI · Released 2025-02-20 · https://huggingface.co/google/siglip2-so400m-patch16-384 - **Perception Encoder (PE-Core B/L/G)** — Meta's SOTA CLIP-style vision encoder family (beats SigLIP on hard OOD benchmarks) — strong new backbone option · GPU: T4 (L); A10 24GB (G) · Serving: HF Transformers + timm support; no vLLM · Released 2025-04-17 · https://huggingface.co/facebook/PE-Core-L14-336
### ⚠️ Servable with conditions | Model | Org | Params | License | Notes | |---|---|---|---|---| | **SAM 3** | Meta | 848M (shared vision encoder + detector + | SAM License (custom Meta license | SAM License text (GitHub) grants royalty-free commercial use but bans ITAR/military/weapons uses and reverse engineering, terminates on IP litigation vs Meta, and requires indemnification + gated access. | | **DINOv3** | Meta | 21M–7B (ViT-S…ViT-7B, ConvNeXt distills, | DINOv3 License (custom | License text at ai.meta.com permits commercial use but requires 'Built with DINOv3' attribution + license passthrough, bans military/nuclear/espionage/ITAR uses and reverse engineering; gated access. | | **VGGT-1B-Commercial** | Meta AI / Oxford VGG | 1B | vggt-aup-license (custom | Card states checkpoint is 'licensed for commercial use, with the exception of military applications'; gated access + AUP terms — commercial hosted serving OK once accepted. | | **Fast-FoundationStereo (C-FAST)** | NVIDIA | 14.6M | NVIDIA Open Model Agreement (HF tag | Agreement text permits commercial use/paid serving with attribution + litigation-termination clauses and no card deference — BUT the card's Description says 'for research and evaluation purposes only'; get written NVIDIA confirmation before revenue-bearing deployment. | | **C-RADIOv3 (B/L/H/g)** | NVIDIA | 90M–1.2B | NVIDIA Open Model License (HF tag 'nvidia-open-model-license | Commercial serving permitted, but NVOML terminates rights if you bypass guardrails without a substitute, requires 'Licensed by NVIDIA...' attribution on distribution, and has a litigation-termination clause. | | **SAM 3D Objects** | Meta | not stated (~1B-class) | SAM License (custom | Same SAM License as SAM 3 (commercial OK; no military/ITAR, no reverse engineering, litigation termination, indemnification) plus the heaviest KYC gate of the set — factor the identity disclosure into procurement. |
Serving details - **SAM 3** — Open-vocabulary concept segmentation: text/exemplar-prompted detection, segmentation and video tracking (SAM 2 successor) · GPU: L4/A10 24GB for video; T4 16GB for image-only · Serving: HF Transformers support; official PyTorch repo + notebooks; ONNX community ports; no vLLM/LeRobot · Released 2025-11-19 (SAM 3.1 update 2026-03-27) · https://huggingface.co/facebook/sam3 - **DINOv3** — Successor dense-feature backbone; SOTA frozen features for detection/segmentation/depth heads on robots · GPU: T4 8GB (≤ViT-L); A100/H100 80GB for ViT-7B · Serving: HF Transformers native; timm; no vLLM · Released 2025-08-13 · https://huggingface.co/facebook/dinov3-vitb16-pretrain-lvd1689m - **VGGT-1B-Commercial** — Feed-forward multi-image 3D: cameras, depth maps, point maps, tracks in one pass (CVPR 2025 best paper line) · GPU: A10/RTX 4090 24GB (16GB for few frames) · Serving: Official PyTorch repo; HF checkpoint; no vLLM · Released 2025-07-29 · https://huggingface.co/facebook/VGGT-1B-Commercial - **Fast-FoundationStereo (C-FAST)** — Real-time zero-shot stereo disparity/depth, distilled from FoundationStereo (10x faster, near-equal accuracy) · GPU: Jetson Orin / T4 8GB · Serving: TAO/TensorRT deployment path; HF checkpoint; no vLLM · Released 2026-02-01 · https://huggingface.co/nvidia/c-fast-foundationstereo - **C-RADIOv3 (B/L/H/g)** — Agglomerative backbone distilled from CLIP+DINOv2+SAM teachers — one encoder replacing three; used across NVIDIA robotics pipelines · GPU: T4 16GB (H); A10 24GB (g) · Serving: TensorRT/TAO official; HF trust_remote_code; no vLLM · Released 2025-03-26 · https://huggingface.co/nvidia/C-RADIOv3-H - **SAM 3D Objects** — Single image → full 3D object shape/texture/layout under occlusion — feeds grasp planning from one camera · GPU: A10 24GB · Serving: Official repo + HF checkpoints + demo; no vLLM/LeRobot · Released 2025-11-19 · https://huggingface.co/facebook/sam-3d-objects
**Excluded (closed / NC / no grant):** Grounding DINO 1.5 / 1.6 / DINO-X (incl. Edge) · AnyGrasp · FoundationPose · FoundationStereo (original checkpoints) · GraspGen · GraspNet-baseline · MASt3R / DUSt3R · UniDepth / UniDepthV2 · Depth Anything V2 Base/Large/Giant · Depth Anything 3 Giant / Nested-Giant / Large (any-view) · Apple Depth Pro · VGGT-1B (original) and VGGT-Omega (May 2026) · MapAnything default checkpoint (facebook/map-anything) · CoTracker3 · YOLO-World · YOLOE · Ultralytics YOLOv8/YOLO11 (+ their RT-DETR/FastSAM wrappers) · FastSAM · AM-RADIO original research checkpoints ## Driving / AV ### ✅ Clean — serve without strings | Model | Org | Params | License | Notes | |---|---|---|---|---| | **openpilot supercombo (driving_supercombo.onnx + big_driving_supercombo.onnx)** | comma.ai | undisclosed (~10^7–10^8 class ONNX; visi | MIT | Claim verified: weights ship inside the MIT repo with no separate weights license, so commercial hosted serving is permitted with MIT attribution. | | **Vista** | OpenDriveLab (Shanghai AI Lab) | ~2.5B (Stable Video Diffusion-based) | apache-2.0 (HF tag, matches repo). Not gated. | Clean Apache-2.0 on the weights repo; commercial hosted serving permitted with attribution/NOTICE compliance. | | **SimLingo** | University of Tübingen / Wayve-affiliated authors (RenzKa) | ~1B (InternVL2-1B base per paper) | Apache-2.0 (HF repo tag + GitHub). Not gated. | Apache-2.0 tag verified and ungated; commercial hosted serving permitted, though the empty HF card means the grant rests on the tag + GitHub license alone. | | **TransFuser++ (CARLA Garage, incl. Bench2Drive + Leaderboard 2.0 model sets)** | University of Tübingen (autonomousvision) | ~50–170M class (configurable ResNet back | Code MIT | CC BY 4.0 weights = commercial-OK with attribution. Sim-trained (CARLA), 3-seed model sets released for reproducible eval; 2nd place CVPR 2024 CARLA challenge. | | **commaVQ GPT2M (commavq-gpt2m)** | comma.ai | ~300M (GPT2-medium over VQ driving-video | MIT (HF tag) | Clean MIT tag on an ungated comma.ai repo; commercial hosted serving permitted with attribution. |
Serving details - **openpilot supercombo (driving_supercombo.onnx + big_driving_supercombo.onnx)** — Production L2 end-to-end driving policy (lane + longitudinal control) from camera video; drives ~10k real cars daily · GPU: None — runs on Snapdragon 845-class; CPU or any small GPU via ONNX Runtime/tinygrad · Serving: No vLLM/LeRobot; ONNX Runtime or tinygrad; no official server (on-device by design) · Released Rolling — current weights on master July 2026 (openpilot 0.10.x line); supercombo lineage since ~2019 · https://github.com/commaai/openpilot - **Vista** — Generalizable driving world model — action-conditioned future video prediction (steering/speed/trajectory/command inputs) + reward signal without ground-truth actions · GPU: 1× 40GB A100 recommended for full-res rollouts; 24GB workable at reduced res (est.) · Serving: Custom PyTorch (SVD-style) inference code; no vLLM/LeRobot/official server — research-grade serving effort · Released 2024-06-07 (NeurIPS 2024) · https://huggingface.co/OpenDriveLab/Vista - **SimLingo** — Driving VLA + language — camera-only closed-loop CARLA driving with VQA, commentary, and instruction following in one model · GPU: 1× 8–12GB (est.) · Serving: Custom PyTorch/Hydra research code; no vLLM/LeRobot/official server · Released Checkpoints 2025-06-25; code 2025-05-08 · https://huggingface.co/RenzKa/simlingo - **TransFuser++ (CARLA Garage, incl. Bench2Drive + Leaderboard 2.0 model sets)** — Camera+LiDAR end-to-end CARLA driving policy via imitation learning — the reference open CARLA baseline family · GPU: 1× 8–12GB (est.) · Serving: PyTorch + CARLA leaderboard harness; no vLLM/LeRobot · Released 2023; refreshed model sets through 2024-2025 · https://github.com/autonomousvision/carla_garage - **commaVQ GPT2M (commavq-gpt2m)** — Driving video world model — autoregressive next-token prediction of compressed driving video · GPU: 1× 8GB · Serving: Plain Transformers/GPT-2 stack — trivially servable; no vLLM recipe needed at this size · Released 2023-06-29 · https://huggingface.co/commaai/commavq-gpt2m
### ⚠️ Servable with conditions | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Cosmos-Reason2 (2B / 8B / 32B)** | NVIDIA | 2B / 8B / 32B (Qwen3-VL-Instruct base + | HF tag: 'other' = NVIDIA Open Model License. Card text: 'Mod | Commercial serving allowed, but rights auto-terminate if you bypass/weaken safety guardrails without a substantially similar replacement; gated access, litigation-termination and indemnity clauses apply. | | **Cosmos-Reason1-7B** | NVIDIA | 8.3B actual (676M ViT + 7.07B LLM) | NVIDIA Open Model License (card states 'Models are commercia | Commercial serving allowed under NVIDIA OML, subject to the guardrail-bypass auto-termination clause; ungated, derivatives permitted. | | **Cosmos-Transfer1-7B-Sample-AV (+ Sample-AV-Single2MultiView; sibling Cosmos-Predict1-7B Text2World/Video2World Sample-AV-Multiview)** | NVIDIA | 7B diffusion transformer (each variant) | HF tag 'other' = NVIDIA Open Model License | Commercial-OK; strings: HF gating, attribution, automatic termination if safety guardrails are bypassed. Five AV-specialized checkpoints verified live (all created 2025-03-06 to 2025-04-22). | | **UniAD (+ UniAD 2.0 R101 checkpoint)** | OpenDriveLab | ~150M class (R101/R34 encoder + task hea | Apache-2.0 (repo states all assets and code | License itself permits commercial serving, but the released checkpoints are trained on nuScenes (Motional's non-commercial dataset terms) — retrain on clean data or obtain a Motional commercial license for a defensible position. | | **VAD (Tiny / Base)** | HUST Vision Lab (hustvl) | ~60M class (vectorized scene encoder + p | Apache-2.0 (repo) | Code license wording covers code only and the Drive-hosted weights carry no express grant; weights are nuScenes-trained — get maintainer confirmation or retrain before charging for inference. | | **BEVFormer (tiny/small/base + V2)** | Fundamental Vision (OpenDriveLab-affiliated) | ~30–120M class depending on variant | Apache-2.0 | Repo Apache-2.0 nominally covers the zoo, but weights lack an explicit grant and are nuScenes-trained (non-commercial dataset terms) — same taint caveat as UniAD/VAD. | | **DiffusionDrive / DiffusionDriveV2** | HUST Vision Lab (hustvl) | 60M (ResNet-34 backbone, NAVSIM config); | V1: Apache-2.0 (HF hustvl/DiffusionDrive, 2024-11-30) | V2 (MIT) is clean for serving; V1's own model card declares 'Non-commercial license' despite the apache-2.0 tag — treat V1 as non-commercial until hustvl resolves the contradiction (likely LLaVA-template debris, but unproven). | | **LMDrive (llava-v1.5-7b / vicuna-v1.5-7b / llama-7b variants)** | OpenDILab / CUHK | 7B (each variant) + vision encoder | Apache-2.0 (repo) | The apache tag cannot relicense the bases: llava/vicuna-v1.5 variants inherit the Llama-2 Community License (commercial OK under its terms), while the LLaMA-7B variant inherits Llama-1 research-only weights — that variant is a hard no for paid serving. | | **InterFuser** | OpenDILab / Shanghai Jiao Tong | undisclosed (~100M-class multi-sensor tr | Apache-2.0 ('All code within this repository') | Downgraded from claimed 'yes': the license sentence covers code only and the externally hosted example weights carry no explicit grant — confirm with OpenDILab or retrain (CARLA data, so retraining is clean). | | **OpenDriveVLA-0.5B** | TU Munich (DriveVLA) | 0.5B (Qwen2.5-0.5B-based); 3B/7B paper v | Apache-2.0 (repo + HF checkpoint) | Apache tag verified but access requires accepting a gate the empty card doesn't document, and the planner is nuScenes-trained — clear the gate terms and dataset taint before paid serving. | | **ORION** | Xiaomi (xiaomi-mlab) / HUST | ~7B class (QT-Former + Vicuna-class LLM | Apache-2.0 (repo) | Tags are permissive, but the checkpoint embeds a Vicuna/Llama-2-lineage LLM (Meta license terms apply) and training includes nuScenes-derived data; checkpoint also sits in a personal HF account, weakening the provenance of the grant. | | **Mini-InternVL2-DA-DriveLM (1B / 2B / 4B)** | OpenGVLab (Shanghai AI Lab) | 1B / 2B / 4B | MIT (HF tag + card, verified on 4B). Not gated. | MIT tag verified on the 4B (Phi-3 base, MIT-compatible), but the 2B's InternLM2 base requires NVIDIA-style commercial registration with Shanghai AI Lab and the DriveLM fine-tune data derives from nuScenes — verify per-size before serving. |
Serving details - **Cosmos-Reason2 (2B / 8B / 32B)** — Physical-AI reasoning VLM — video+image reasoning for AV decision-making, robot planning, video analytics · GPU: 2B: 1× 8GB; 8B: 1× 24GB; 32B: 1× 80GB (est. BF16) · Serving: Official runtime is Transformers; Qwen3-VL architecture is vLLM-supported in practice; community GGUF quants exist. No LeRobot. · Released 2B/8B: 2025-12-19 (updated 2026-03-10); 32B: 2026-04-29. 400k+ downloads on 2B. · https://huggingface.co/nvidia/Cosmos-Reason2-8B - **Cosmos-Reason1-7B** — Physical-AI/embodied reasoning VLM — AV decision reasoning, robot planning, driving-data curation and captioning · GPU: 1× 24GB · Serving: Official vLLM inference examples (fps=4, max_tokens=4096) + Transformers. No LeRobot. · Released 2025-05-17 · https://huggingface.co/nvidia/Cosmos-Reason1-7B - **Cosmos-Transfer1-7B-Sample-AV (+ Sample-AV-Single2MultiView; sibling Cosmos-Predict1-7B Text2World/Video2World Sample-AV-Multiview)** — AV world simulation — HDMap/LiDAR-conditioned photoreal driving video generation (121 frames @ 704×1280), single-view→multiview expansion; synthetic data + scenario variation for AV eval · GPU: 1× 48–80GB (Ampere/Hopper/Blackwell, BF16 only; 80GB recommended) · Serving: Official NVIDIA Cosmos inference repo; no vLLM (video diffusion); no LeRobot · Released 2025-03-17 (Transfer1 AV); Single2MultiView 2025-04-22 · https://huggingface.co/nvidia/Cosmos-Transfer1-7B-Sample-AV - **UniAD (+ UniAD 2.0 R101 checkpoint)** — Planning-oriented end-to-end AV stack — joint detection/tracking/mapping/motion/occupancy/planning (CVPR 2023 best paper); UniAD 2.0 adds nuPlan/NAVSIM integration · GPU: 1× 16–24GB (est.; BEVFormer-base backbone) · Serving: mmdet3d-style research code; no vLLM/LeRobot/official server · Released 2023 (v1); UniAD2.0_R101_nuScenes checkpoint on HF 2025-10-24 · https://github.com/OpenDriveLab/UniAD - **VAD (Tiny / Base)** — Vectorized end-to-end driving — fully vectorized scene representation for efficient planning; UniAD's faster rival baseline · GPU: 1× 12–16GB (est.) · Serving: Research code (mmdet3d lineage); no vLLM/LeRobot; Google Drive weight hosting is a serving-pipeline smell · Released 2023 (ICCV 2023) · https://github.com/hustvl/VAD - **BEVFormer (tiny/small/base + V2)** — Camera-only BEV 3D perception — multi-view temporal transformer producing bird's-eye-view features for detection/segmentation · GPU: 1× 8–24GB depending on variant (tiny ~6.5GB, base ~28GB training-side; inference lighter) · Serving: mmdet3d research code; no vLLM/LeRobot/official server · Released 2022 (ECCV 2022); BEVFormerV2 2023 · https://github.com/fundamentalvision/BEVFormer - **DiffusionDrive / DiffusionDriveV2** — Truncated-diffusion end-to-end planner — multi-modal trajectory generation at real-time rates; the strongest tiny open planner · GPU: 1× 8GB — any modern GPU · Serving: PyTorch research code; trivially self-hostable given size; no vLLM/LeRobot · Released V1 2024-12-16; V2 2025-12-09 · https://huggingface.co/hustvl/DiffusionDriveV2 - **LMDrive (llava-v1.5-7b / vicuna-v1.5-7b / llama-7b variants)** — Language-guided closed-loop driving in CARLA — camera+LiDAR+natural-language instructions to driving actions; first LLM closed-loop driving framework · GPU: 1× 24GB (+ CARLA server for closed-loop) · Serving: Custom Transformers pipeline; no vLLM (interleaved sensor tokens); no LeRobot · Released 2023-12 (CVPR 2024) · https://github.com/opendilab/LMDrive - **InterFuser** — Safety-enhanced multi-modal multi-view CARLA driving — interpretable intermediate scene representation constraining actions · GPU: 1× 12GB (est.) · Serving: PyTorch + CARLA harness; no vLLM/LeRobot · Released 2022 (CoRL 2022) · https://github.com/opendilab/InterFuser - **OpenDriveVLA-0.5B** — End-to-end driving VLA — 3D-grounded vision-language-action trajectory planning on nuScenes · GPU: 1× 8GB · Serving: Transformers-loadable (Qwen2.5 lineage helps); custom action decoding; no official vLLM recipe, no LeRobot · Released Checkpoint 2025-11-14 (paper 2025-04-01) · https://huggingface.co/OpenDriveVLA/OpenDriveVLA-0.5B - **ORION** — Holistic E2E driving — VLM semantic reasoning bridged to numeric trajectory output; strong closed-loop Bench2Drive scores (ICCV 2025) · GPU: 1× 24GB (est.) · Serving: Custom research pipeline; no vLLM/LeRobot · Released Inference + checkpoint 2025-04-10; training code 2025-08-13 · https://github.com/xiaomi-mlab/Orion - **Mini-InternVL2-DA-DriveLM (1B / 2B / 4B)** — Driving visual QA / scene understanding — Mini-InternVL adapted to the DriveLM Graph-VQA driving domain (CVPR 2024 challenge lineage) · GPU: 1× 8–16GB · Serving: vLLM supports the InternVL2 architecture; Transformers with trust_remote_code; no LeRobot · Released Late 2024 (papers Oct–Dec 2024) · https://huggingface.co/OpenGVLab/Mini-InternVL2-4B-DA-DriveLM
### ❌ Research-only — do not serve | Model | Org | Params | License | Notes | |---|---|---|---|---| | **Alpamayo-R1-10B (the 'Alpamayo-1' release — same artifact, no separate model)** | NVIDIA | 10.8B (8.2B Cosmos-Reason1 VLM backbone | HF tag: 'other'. Model card text: weights 'released under a | NC confirmed from the live model card. Community requants (dwko/Alpamayo-R1-10B-4bit is mislabeled apache-2.0, z-lab DFlash/PARO variants) inherit the NC terms — do not treat the 4-bit tag as a loophole. | | **Alpamayo-1.5-10B (this IS the 'Alpamayo 2' slot — no model named Alpamayo-2 exists as of Jul 2026)** | NVIDIA | 11B (8.2B Cosmos-Reason2/Qwen3-VL-derive | HF tag: 'other'. Model card: non-commercial license, commerc | NC confirmed live. 58k downloads — the most-pulled open AV VLA. Its Cosmos-Reason2 backbone IS commercially licensed (see separate entry) — the NC applies to the driving-tuned composite. | | **Senna** | HUST Vision Lab (hustvl) | 7B (Vicuna-7B-v1.5 base, LLaVA codebase, | Code Apache-2.0 | The author's own card text declares the checkpoint non-commercial, overriding the permissive tag, and the LLaVA/Llama-2 lineage adds Meta license terms on top — do not serve commercially without written clarification from hustvl. | | **AutoVLA** | UCLA Mobility Lab | 3B (Qwen2.5-VL-3B-Instruct base, LoRA-me | Academic Software License (repo badge) | Downloadable but academically licensed — include for research-tier eval only. Dual fast/slow thinking modes; benchmarked on nuPlan/nuScenes/Waymo. | | **OmniDrive (OmniDrive-nuScenes)** | NVIDIA (NVlabs) | ~7B class LLM-agent (3D multimodal LLM) | NVIDIA License (custom): 'may be used... non-commercially' = | Verified NC from the LICENSE file text. Matters beyond itself: ORION initializes from these weights (taint propagation). | **Excluded (closed / NC / no grant):** Wayve GAIA-1 / GAIA-2 · Wayve LINGO-1 / LINGO-2 · Wayve Lingo-Judge · DriveVLM / DriveVLM-Dual (Tsinghua + Li Auto) · DriveGPT4 · Tesla (FSD / any layer) · Waymo EMMA · Senna-2 · TCP (OpenDriveLab) · Poutine (Mila; Waymo 2025 E2E challenge winner) · OpenEMMA / LightEMMA · XPeng world foundation model · DriveLM-Agent (official baseline) · z-lab / dwko Alpamayo requants (DFlash, PARO, 4-bit) · NVIDIA Hydra-MDP and other NAVSIM/challenge winners --- *Generated from the verified catalog sweep (wf_891f436e-42b). Re-verify license tags before adding any model to the serving plane — licenses change (GR00T N1.5→N1.7 flipped NC→commercial in one release).*