Skip to content
➜NextOS

On-device (offline)
in NextOS.

A small model that runs entirely in this browser tab via WebGPU - free, private, no key, and the one provider that still answers with no network at all. Download a model once from Settings > Assistant > Offline model.

Use On-device (offline)Get an API key

01Supported models

Every model
this plugin lists.

  • Llama-3.2-1B-Instruct-q4f16_1-MLC

    Llama 3.2 1B (fastest, ~0.9 GB)

    tools

    Smallest bundled model - quick drafts and short summaries on modest hardware. No reliable tool calling.

  • Qwen2.5-1.5B-Instruct-q4f16_1-MLC

    Qwen 2.5 1.5B (~1.1 GB)

    tools

    A step up in quality over the 1B model for about the same download. No reliable tool calling.

  • Llama-3.2-3B-Instruct-q4f16_1-MLC

    Llama 3.2 3B (balanced, ~2.2 GB)

    tools

    Noticeably better drafts and summaries; needs more VRAM. No reliable tool calling.

  • Hermes-2-Pro-Llama-3-8B-q4f16_1-MLC

    Hermes 2 Pro 8B (tools, ~5 GB, needs a capable GPU)

    tools

    The one bundled model with reliable tool calling - can use the offline file/notes/database tools, not just draft text.

02Bring your own key, or hosted

How you
pay for it.

On-device (offline) runs bring-your-own-key only: add your key in Settings > API keys, or send it per-request over the gateway.

  • Full tool calling - agents can use every app and MCP tool through this provider
  • No provider-hosted web search
  • No provider-hosted URL fetch

03Governance

The same controls
as every provider.

  • Restrict which providers and models a team may use at all - an allowed list on the org policy, enforced on every gateway call
  • Set a default model the whole workspace inherits, with per-agent pins for the job that needs a different one
  • Per-org and per-key budgets and rate limits, checked before a call reaches the provider
  • Guardrails (PII redaction, prompt-injection screening) applied to every request and response, regardless of provider
  • A full audit trail of governance and budget events, in the team's activity log

04Over the gateway

One endpoint,
OpenAI-compatible.

gateway — next.os
curl https://jonkum.in/api/v1/chat/completions \
  -H "Authorization: Bearer ngk_..." \
  -H "X-Provider-Key: your-local-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"local/Llama-3.2-1B-Instruct-q4f16_1-MLC","messages":[{"role":"user","content":"..."}]}'

Omit X-Provider-Key to run on the account's hosted allowance instead, where the plan supports it. See the gateway page for the full API.

05Every other provider

one plane · every provider · your key or ours

Run agents on On-device (offline).
Governed exactly like every other provider.

Set your defaultSee pricing