Prerequisites
- A LiteLLM proxy that already routes model traffic.
- A Silmaril guardrail endpoint and API key from deployment access.
- If you self-host, use the guardrail endpoint from the same deployment as your classify endpoint.
SILMARIL_GUARDRAIL_URL and SILMARIL_API_KEY in the proxy environment.
Configure the guardrail
Start in warn mode while validating proxy coverage. This configuration omitsreview to inherit your deployment’s default review setting.
additional_provider_specific_params.on_error: warn controls how Silmaril
represents a valid would-block decision. It does not control LiteLLM’s handling
of guardrail endpoint failures. Keep LiteLLM’s fail_on_error default; setting
fail_on_error: false broadly bypasses guardrail errors.
Choose where the guardrail runs
default_on: true attaches this guardrail globally. To scope it instead, set
default_on: false and attach its guardrail_name in the model configuration
or in one request’s guardrails list. Do not define duplicate guardrails to
cover the same stage. Post-call attachment and same-content behavior are
version-bound; use the qualification table below. All selected lanes recheck
response content changed by a native hook.
The repository qualification contract
contains reproducible commands for three separately hash-locked published CLI
profiles on Python 3.12:
These are exact qualification profiles, not a blanket package recommendation.
The patched LTS and stable permission-response profiles require the complete
hash-locked dependency graph, including explicit
prisma==0.15.0. Bare
litellm[proxy] installs of either version are not qualified for this behavior:
without Prisma they return a generic HTTP 500 while formatting the
authorization error. This qualification prerequisite does not prescribe
database setup or broader production dependency requirements.
With the complete profiles, both patched lanes reject low-privilege Azure
Entra overrides for tenant_id, client_id, client_secret,
azure_username, azure_password, and azure_scope as bounded HTTP 401
JSON authorization errors before the provider or Silmaril classifier runs.
The response contains neither the supplied value nor the qualification
environment path.
For a Gemini-native function_declarations tool without type, 1.108.9
fails closed before provider execution; that is not tool support. 1.109.1
accepts and preserves the exact type-less shape in the Generic Guardrail API
request.
Verify the installed proxy version and complete profile against the contract.
Read the evidence as four distinct states: source is repository behavior;
published is one exact PyPI package; tested is a completed version-selected
lane; deployed is not established by qualification. This guide does not claim
which version is currently installed or deployed.
Expected result
LiteLLM calls Silmaril before and after the model call. In this starter configuration, a valid would-block decision is advisory and the proxy call continues.unreachable_fallback: fail_open is narrower than fail_on_error: false. It
allows unreachable transport failures and HTTP 502, 503, and 504
responses to continue. Malformed or non-object JSON and HTTP 500 remain
guardrail errors under LiteLLM’s default fail_on_error: true. A valid
Silmaril BLOCKED response is still a block when native enforcement is
explicitly configured.
Qualified endpoint and modality scope
The matrix below is bounded to the exact, separately hash-locked historical1.108.3, patched LTS 1.108.9, and patched stable 1.109.1
packages on Python 3.12. Except for the version boundaries above and in the
table, the endpoint checks remain shared across all three lanes. Qualification
exercised the published CLI and deterministic provider and Firewall services
over loopback only. It used no model, cloud service, paid inference endpoint,
or customer account. This establishes tested compatibility, not deployment or
customer activation.
Pass-through endpoints are not qualified by this matrix. Their visible fields
depend on endpoint/provider-specific JSONPath configuration, and the selected
LiteLLM packages check extracted pass-through text without applying response
rewrites.
Responses WebSocket follow-up content enforcement is version-bound. On stable
1.109.1, two response.create frames on the same authenticated connection
receive pre-call checks: the clean frame reaches the provider, while the
malicious follow-up is blocked before provider execution and appears in
neither provider nor client output. The authenticated user ID and key hash are
preserved. Per-frame call and trace IDs are null because the published
upstream logging object is not propagated, so full sequence/request
correlation remains unqualified. LTS 1.108.9 runs no per-frame guardrail;
unsafe content reaches the provider and client, and this route is unsupported.
Historical 1.108.3 remains outside the retained receipt. Complete WebSocket
protection, post-call token blocking, and deployment are not qualified.
The endpoint rows are exercised directly rather than inferred from chat
coverage. Responses assertions preserve instructions in both the exact
guardrail payload and provider request. Messages assertions preserve top-level
system, tool_result text, and client input tool_use through the actual
selected package’s OpenAI Responses bridge. The provider returns the current
Responses function_call fields (id, call_id, name, arguments, and status);
qualification checks the exact guardrail tool-call payload and native Responses
or Anthropic tool_use delivery for clean calls, and verifies malicious
arguments are not delivered.
For buffered blocked streams, provider auditing records every emitted text
fragment and final source consumption. Endpoint-native SSE assertions prove
that no provider text fragment, including the safe prefix emitted before the
malicious fragment, reaches the client. Blocked paths also reject Responses
response.output_text.delta and Messages content_block_delta delivery.
The qualification also exercises LiteLLM’s endpoint-specific
structured_messages write-back for Responses and Messages. That case uses a
clearly marked synthetic GUARDRAIL_INTERVENED response only at the upstream
compatibility boundary. Production Silmaril responses remain NONE or
BLOCKED; this does not claim a Silmaril rewrite feature.
Native blocking and streaming
Native blocking is a separate, approved profile. Only the exact provider modemode: enforce selects it; do not add it to the starter configuration without
completing the agreed qualification. In that profile, a blocked pre-call can
stop provider execution, and a blocked non-streaming post-call can withhold
the response.
Streaming requires an additional delivery decision. Tokens already delivered
before a post-call verdict cannot be retracted, and
streaming_end_of_stream_only: true does not withhold them. If blocking must
prevent output exposure, qualify the complete proxy profile with
streaming_buffer_until_moderated: true and keep output buffered until the
verdict.
Optional review selection
Keepreview omitted to inherit your deployment’s default as shown above. Select standard or deep with additional_provider_specific_params.review.
mode: shadow explicitly selects Shadow response mode and takes precedence over the on_error: warn response-mode fallback. Review selection and response mode are independent.
standard selects standard review. deep requests a second opinion on eligible potentially harmful events and requires a deployment with deep review available. Otherwise Silmaril returns HTTP 400 before classification. An explicit deep selection overrides a standard-review default. Explicit null, wrong types, whitespace or case variants, and other strings are invalid.
See review depth for availability and response handling.
A successful explicit selection is acknowledged by the same exact string in top-level silmaril_review. This confirms the applied setting. An event may complete without additional review. Omitted review needs no acknowledgement and preserves the legacy response shape.