Models

Models define which LLM providers and model identifiers are available to agents, workflows, and runtime policies. SOIT is model-neutral: provider-specific details are handled behind the LLM port and provider adapters, so agents bind to model references (e.g. model:openai:gpt-4.1) rather than SDKs.

Provider support matrix

How models connect

• Agents and workflows bind to model references resolved through ModelHub.
• Model calls go through the LLM port and provider adapters — never directly to provider SDKs.
• Runtime traces record model usage, latency, token counts, and cost per call in the runtime ledger.
• Governance policy can constrain which agent versions can use which model references.

Configuring a provider

From the ModelHub UI (/model in the web workspace):

1. Create or update a provider with its credentials (stored as secret references, never plaintext).
2. Run the connectivity test — catalog, chat, and embedding diagnostics are available per provider.
3. Activate the models you want to expose; active models become selectable in Agent, Workflow, and Chat configuration.
4. Invalid credentials surface as a visible, understandable failure state rather than silent errors.

Gemini native adapter

New in v1.4.0. A Gemini provider can use a native adapter (adapter_backend: native on the provider) that calls the Gemini API directly. LiteLLM stays the default. The native adapter serves:

• Chat and streaming.
• Tool calls declared with their JSON Schema, returned with their thought signatures.
• Structured output.
• Images in messages, taken only as inline data: URLs.
• Embeddings, counted at four characters a token, since Gemini reports no embedding tokens.

With it, a Gemini provider no longer depends on LiteLLM for chat, tools, or retrieval. Image generation stays on LiteLLM: a provider on the native adapter serves no images, so an image call routed to it is refused as a missing capability (MODEL_CAPABILITY_UNAVAILABLE, reason native_adapter) and a virtual model moves on to its next target.

The native adapter has unit and mock coverage; it has not yet been checked against the live Gemini API with a real credential.

Model servers on your own network

New in v1.4.0. A model server on a private or loopback address — a local Ollama, an on-premises vLLM — can be used once two things are in place:

1. Its network is listed in EGRESS_PRIVATE_NETWORKS, for example ["127.0.0.1/32"] for an Ollama on the same host.
2. The server is on the workspace or tenant egress allowlist.

The list is empty by default, which refuses every non-public address. Before v1.4.0 every non-public address was refused whatever the configuration, so a local Ollama or an on-premises model server could not be used. See Private networks.

Virtual models

A virtual model (since v1.2.0) is a workspace name, called as vmodel:{slug}, for an ordered list of up to eight model: refs. The targets are tried in turn when a provider is down, rate-limited, or missing a capability the call needs. Manage them under Build › Models › Virtual models or with /api/v1/modelhub/virtual-models. The failover rules are described under Virtual models on the gateway page.

Trying a new model

Before switching agents to a new model, a model replay runs their regression sets on the candidate next to the model they use today and compares pass rate, latency, and cost. Start one under Build › Models › Regression replays or with soit eval.

Calling models from your own code

The OpenAI-compatible gateway serves the workspace's active models under /v1 to OpenAI SDKs and frameworks, by the same model refs, with every call governed and costed.

API and UI surfaces

• API prefix: /api/v1/modelhub
• Related runtime prefixes: /api/v1/runs, /api/v1/responses, /v1 (gateway)
• Web route: /model

Related capabilities

• Agents bind models as typed capabilities.
• Evaluation uses deterministic test models for repeatable scoring.
• Secrets and Security governs provider credentials.