Portal LLM Connections
Purpose
This functional area lets platform administrators configure the LLM provider connections used by semantic enrichment jobs, set default enrichment models per scope, and manage which tenants have access to each connection.
Its role is to give operators full control over which AI providers and models the platform uses, without requiring any restart or redeployment when the configuration changes.
Main Capabilities
- create, edit, enable or disable, and delete LLM connections for OpenAI-compatible or Azure OpenAI providers
- add, edit, enable or disable, and delete model profiles within each connection to define the exact model, deployment, temperature, and timeout
- manage tenant assignments for global connections so tenant administrators and enrichment jobs can use shared connections
- set the global default enrichment model profile so enrichment jobs fall back to it when no explicit model is chosen
- for global administrators: view and set a per-tenant default model override for every tenant through the Per-Tenant Defaults panel
- filter the connection list by tenant, name, status, and provider
Connection Scopes
LLM connections have two scopes:
- Global — owned by the platform, not tied to any tenant. Global connections are shared resources. To make a global connection visible to a tenant's administrators and enrichment jobs, the connection must be explicitly assigned to that tenant.
- Tenant — owned by one specific tenant. Only that tenant's administrators and enrichment jobs can see or use it.
Tenant administrators see only connections they own and global connections that have been assigned to their tenant. Global super-admins see all connections across all tenants.
Model Profiles
Each connection can have one or more model profiles. A profile defines a specific model identity within the connection, such as a specific GPT model variant, Azure deployment name, temperature, and timeout.
Having multiple profiles per connection lets the platform maintain, for example, a fast low-temperature profile for structured naming tasks and a separate profile for more creative enrichment scenarios, all sharing the same provider credentials.
Default Enrichment Selection
The platform resolves a default model for each enrichment run using a two-level hierarchy:
- Tenant default — if the mapping's tenant has a specific default model configured, that model is used.
- Global default — if no tenant-level override exists, the global default is used.
This means the same platform can serve different tenants using different LLM providers or models without any per-job configuration.
When an enrichment job is submitted with "Use default" selected, the job worker resolves the effective default for the mapping's own tenant at job execution time. The model shown as "default" in the enrichment modal reflects the tenant's effective default at the moment the modal is opened.
Per-Tenant Default Management
Global super-admins can view and change the default model for every individual tenant from a single panel without navigating tenant-by-tenant.
The Per-Tenant Defaults panel shows:
- each tenant name
- the currently effective default model profile (connection name · model name · model ID) or a label indicating it uses the global default
- a Set default action per row
Clicking Set default opens a modal listing all enabled model profiles accessible to that tenant. Choosing a profile writes a tenant-level override. Choosing "No default (clear)" removes the override and returns the tenant to the global default.
Provider Support
The platform supports:
- OpenAI / compatible — works with the standard OpenAI API and any compatible provider endpoint such as Mistral, Ollama, or locally hosted model servers
- Azure OpenAI — targets Azure-hosted deployments; requires a deployment name on each model profile in addition to the model ID
Credential Storage
LLM connections store their API key through one of two strategies:
- Inline secret — the API key value is encrypted and stored in the platform database. The raw value is only visible at creation time.
- Existing secret reference — the connection stores a reference name pointing to a secret already registered in the Secret Store. This is preferable when the key is managed centrally or in Azure Key Vault.
The platform resolves the credential at enrichment time through the secret store layer, so the enrichment job never handles the raw key directly.
Operational Notes
- A connection with zero model profiles cannot be used for enrichment even when enabled.
- Disabling a connection immediately excludes all its profiles from enrichment selection. Enrichment jobs that were already submitted but not yet started will fail if their selected profile becomes unavailable.
- Global connections must be explicitly assigned to tenants before those tenants can see or use them.
- Deleting a connection removes all its model profiles. If a default selection references one of those profiles, the default becomes invalid and enrichment will stop working for that scope until a new default is configured.
- The inline API key value is never retrievable after saving. Store it securely before closing the creation modal.