Portal Semantic Enrichment

Purpose

This functional area lets administrators request AI-assisted refinement of mappings.

Its role is to accelerate mapping authoring by generating better names, descriptions, and field suggestions from onboarded API content.

Main Capabilities

• request semantic enrichment for a mapping, with an optional choice of which LLM model to use for that specific run
• target specific enrichment scopes
• select which endpoints to include in a given enrichment run
• watch live progress for mappings currently in the Enriching state
• review enrichment proposals and selectively accept or discard suggestions
• review enrichment outcomes through dedicated governance views
• monitor enrichment runs and per-chunk LLM activity

What Can Be Enriched

The platform can assist with:

• mapping name suggestions
• mapping description suggestions
• endpoint naming suggestions
• endpoint annotation hints (read-only, destructive, idempotent, open-world)
• input field name and description suggestions
• output field name and description suggestions

This helps make the resulting tools more understandable and more usable for AI callers.

LLM Model Selection

When requesting enrichment, administrators can optionally select a specific LLM model profile to use for that run. The drop-down lists all enabled model profiles accessible to the mapping's own tenant — not the current viewer's scope. This matters when a global super-admin manages mappings for a specific tenant, because the effective model list and default label reflect the tenant's actual LLM configuration.

Leaving the model set to "Use default" submits the job without a specific model selection. The background job worker then resolves the effective default for the mapping's tenant at job execution time:

1. If the tenant has a per-tenant default configured, that model is used.
2. Otherwise, the global platform default is used.

This means the default shown in the enrichment modal is accurate for the mapping's tenant, and the resolved model at execution time will always match what was shown.

Reference Attachments

When requesting enrichment, administrators can attach one or more reference documents to provide the LLM with additional context beyond the imported API definition. Attachments can be supplied in two places: on the Create mapping proposal dialog when "run enrichment immediately" is enabled, and on the Request semantic enrichment dialog for an existing mapping. Both flows go through the same attachment pipeline and the same limits below.

Supported file types and limits

• Accepted formats: .txt, .md, and .pdf.
• Maximum total size across all attached files: 1 MB. A client-side guard enforces this limit before submission and shows an inline error if exceeded.
• Maximum file count: 5 files per enrichment request.

How files are processed

Attachment processing happens synchronously in the Portal API before the enrichment job is enqueued:

1. Each file's content is extracted — plain text and Markdown via StreamReader, PDF pages via PdfPig.
2. The extracted text is hashed with SHA-256 to produce a content fingerprint.
3. A prompt injection scan checks the extracted text against a set of compiled regex patterns that detect adversarial content (instruction overrides, role hijacking, system prompt reveal attempts, jailbreak phrases, data exfiltration markers, null bytes, and control characters). Content that matches the threshold is rejected with a 422 response before any job is enqueued.
4. Validated attachment snapshots are bundled into the enrichment job request and forwarded to the Jobs Worker.

In the Jobs Worker, the extracted text from all attachments is assembled into a --- ATTACHED REFERENCE DOCUMENTS --- block and forwarded to the enrichment pipeline as dedicated reference context (separate from custom instructions). By default the pipeline distills this document once into a compact glossary via a one-time reference-digest pass, and each operation chunk then injects only the glossary entries relevant to its own fields, parameters and endpoints — rather than re-embedding the full document in every chunk. Each glossary entry records the endpoints its meaning is valid for, so an opaque token that means different things on different endpoints (for example field001 as FirstName on one operation and customerCode on another) produces a separate scoped entry per meaning, and an optional note captures taxonomy values, constraints, or endpoint-specific logic drawn from the document. This preserves the document's authoritative meaning while keeping each chunk within the prompt budget. Distillation can be disabled with EnableReferenceDistillation = false, in which case the block is injected verbatim into every operation chunk as before.

UX: incremental file picking and individual removal

• Files can be added across multiple picks. Each pick merges the new selection with the existing list, deduplicating by filename.
• Individual files can be removed from the list before submitting.
• The total-size guard is evaluated client-side after each merge; the enrichment request action is blocked until the total size is within the 1 MB limit.

Traceability

Each attachment produces an EnrichmentAttachmentSnapshot stored as a JSON collection in the attachments_json column on the mapping_enrichment_runs table. The snapshot includes the filename, content type, size in bytes, SHA-256 content hash, extracted text (which may be truncated for very large documents), a truncation flag, and the processing timestamp.

The enrichment run detail modal in the governance dashboard shows a Reference attachments section listing each file's name, size, full SHA-256 hash (on a dedicated line), and a Truncated badge when the extracted text was cut off.

Security

The prompt injection scan uses 8 source-generated compiled regex patterns. If any pattern matches and the combined match score meets the configured threshold, the entire enrichment request is rejected with a 422 Unprocessable Entity response. Scanning can be disabled per environment via EnrichmentAttachmentsOptions, though this is not recommended for production.

Endpoint Selection

When requesting enrichment, administrators can choose which of the mapping's current endpoints to send to the LLM. The endpoint list defaults to all endpoints selected. Unchecking specific endpoints removes them from the LLM prompt for that run. Endpoints that are not selected are preserved in the resulting proposal exactly as they exist in the current draft — the LLM does not see them and does not generate suggestions for them.

This is useful when a mapping has many endpoints but only some need to be renamed or re-described, or when a previous enrichment pass already produced acceptable results for part of the mapping.

The selection is applied before the chunk planner runs. Only the selected operations are grouped into prompt chunks and sent to the LLM. The final proposal still includes all mapping endpoints, with non-selected ones carrying forward their current names and descriptions unchanged.

At least one endpoint must be selected before the start enrichment action becomes available.

How It Fits Into The Workflow

Semantic enrichment is not the source of truth by itself. Functionally, it acts as an accelerator inside the broader mapping lifecycle:

1. an API is onboarded
2. a mapping is created
3. enrichment is requested with selected targets and optionally a subset of endpoints
4. a background enrichment run is created and the job is enqueued
5. background processing generates suggestions through a chunked LLM pipeline
6. the completed proposal is stored as a side-car alongside the user's draft — the draft is never overwritten automatically
7. an administrator reviews the proposal, selects the suggestions they want to keep, and saves or discards

This keeps human administrators in control while still using AI to reduce manual effort.

Enrichment Runs

Each enrichment request creates a tracked enrichment run. The run captures which provider and model were used, how many operations were processed, how many chunks succeeded and failed, and the full timeline from request to completion.

The enrichment pipeline divides the imported API into operation chunks and executes each chunk as a separate LLM call. An optional overview pass can also enrich mapping-level metadata. Results are preserved progressively so completed work remains available for review.

Canceling An Enrichment Run

Operators can stop an in-progress enrichment run at any time from the mapping list.

Clicking the cancel action on an enriching mapping shows a confirmation dialog before proceeding. This prevents accidental interruption of long-running enrichment jobs.

On confirmation, the Portal:

1. Resets the mapping out of the Enriching state.
2. Marks the active enrichment run as Canceled in the database.
3. Publishes an abort signal to the Jobs Worker via the message bus.

The Jobs Worker receives the signal promptly and stops the in-flight enrichment work before any additional processing continues.

Canceled runs appear in the Enrichment Dashboard with a Canceled status. The mapping returns to the state it was in before enrichment was requested, so it can be enriched again or edited normally.

Operator Experience During Processing

While enrichment is running, the mapping remains in the Enriching state and is treated as a background operation rather than an inline editing task.

Functionally, the Portal supports:

• live chunk progress directly in the mapping list
• automatic refresh of mappings while enrichment is still in progress
• transient real-time progress updates showing processed item counts and cumulative token usage
• final notification and dashboard refresh when the run completes, fails, or is canceled

Reviewing And Accepting The Proposal

When enrichment completes, the Portal shows a notification banner inside the mapping editor signaling that a proposal is ready for review.

The review flow:

1. The Portal opens an Enrichment Proposal Panel showing a diff between the current draft and the suggestions.
2. Each endpoint that has at least one change is shown in a collapsible section. Changes include endpoint name, endpoint description, annotation hints, exposed field names, and field descriptions.
3. Each change is shown as a before/after comparison with strikethrough for the removed value and bold for the proposed value.
4. Every change has its own checkbox. All changes start as selected. Administrators deselect what they do not want to apply.
5. Acceptance is granular — administrators can accept all changes, accept changes per endpoint, or accept individual field changes.
6. Clicking Save applies the selected changes to the current draft and removes the accepted items from the pending proposal.
7. If any items remain un-accepted after a partial save, the proposal persists and can be reviewed again.
8. Clicking Discard proposal removes the entire proposal without altering the draft.

This means the administrator always chooses what enters the draft. The LLM output is a suggestion, not an automatic rewrite.

Enrichment Warnings

Separately from the proposal itself, an enrichment run can surface warning messages describing issues encountered while generating suggestions (for example, a chunk that could not be fully processed). These warnings are shown as their own alert in the mapping editor, next to the "Last enrichment" summary — they are not mixed into the change-summary text, so administrators can tell at a glance what changed versus what the run flagged as needing attention. Warnings are reset whenever the version is next edited or re-enriched.

Governance And Visibility

Enrichment is observable through dedicated governance views.

Operators can review:

• a paginated list of enrichment runs with status, chunk progress, provider, model, suggestion counts, and cumulative token totals
• detailed run records including provider information, timing, and chunk-level activity
• for each pass within a run: pass kind, prompt content, raw LLM output, suggestion counts, warnings, retries, token counts, and outcome. The one-time reference-digest pass is labelled Reference Documentation Digest rather than as a numbered chunk.

This is important because AI-assisted content generation needs traceability and operator oversight.

Operational Importance

This area improves documentation quality and tool usability while reducing the time required to prepare publishable mappings.