Inspect Both Sides of the LLM Conversation

In Do AI Firewall Sidecars Make Sense with Vendor-Hosted LLMs?, I worked through where an AI inspection control can sit: a centralized gateway, a Kubernetes sidecar, or an application library. Placement is only the first decision. The control still needs a policy that can inspect a request before inference and inspect the response before anything trusts it.

Those two passes solve different problems.

The input pass looks for prompt injection, jailbreak attempts, prohibited data, malformed requests, and requests that exceed the application's intended scope. I discussed this in my last post. The output pass looks for unsafe content, sensitive-data disclosure, prompt leakage, executable content, and model-generated instructions that should not reach a downstream tool.

I do not treat either pass as a magic prompt wrapped around another prompt. A useful implementation combines protocol validation, deterministic checks, classification, data-loss prevention, application context, and an explicit policy decision. It also assumes that some attacks will evade detection.

OWASP describes prompt injection as input that alters model behavior or output in unintended ways. It also states that foolproof prevention is unclear and recommends layered measures that include constrained model behavior, expected output formats, input and output filtering, least privilege, human approval for high-risk actions, and adversarial testing.[1] That is the right starting point. The inspection service is a control layer, not proof that a request is safe.

Chapter 13, "AI Security Architecture," in Cybersecurity Architect's Handbook, Second Edition provides the architectural basis for this design. Pages 410 through 412 explain why prompt injection requires layered defense, place input guardrails before inference, and place output guardrails before delivery to a user or downstream system. The chapter also identifies the tradeoff between buffering a complete response for full-context inspection and screening chunks during streaming. Pages 413 through 414 introduce the AI firewall and describe three deployment patterns: a centralized API gateway interceptor, a Kubernetes sidecar proxy, and an SDK or in-process library.

This post takes those input and output guardrail concepts and turns them into an implementation-oriented content-screening policy.

Define the two enforcement points

A complete inference path has at least two policy decisions:

caller
  -> authenticate and authorize caller
  -> validate request shape and size
  -> normalize and classify input
  -> decide: allow, transform, review, or deny
  -> call approved model and provider
  -> validate response shape
  -> classify content and inspect for disclosure
  -> decide: deliver, redact, replace, review, or deny
  -> encode for the destination context
  -> user, application, or tool broker

The model call sits between the decisions. That sounds obvious, but many systems screen the prompt and then stream the provider's response directly to the browser. Once a token has reached the user, a later detector cannot recall it. The same issue applies when model output is fed into a shell, SQL client, template renderer, browser, or agent tool.

OWASP's guidance on improper output handling says to treat model output as untrusted input, validate it before backend functions use it, and encode it for the destination context. A content-safety score does not replace HTML encoding, parameterized database operations, schema validation, or authorization. Those controls answer different questions.

Build an input detector as a stack

Prompt injection and jailbreak detection should use several signals. No single detector has enough context or reliability to make every decision.

Validate the request before reading its meaning

Start with controls that do not require a model:

• authenticate the caller and bind the request to an application, tenant, route, and use case
• allow only approved provider operations, model identifiers, parameters, media types, and tool schemas
• cap request bytes, message count, attachment count, image dimensions, decoded size, and total retrieved context
• reject duplicate or conflicting fields
• require well-formed UTF-8 and a defined policy for invalid byte sequences
• reject unsupported compressed, archived, encrypted, or nested content rather than forwarding what the scanner cannot inspect
• separate system instructions, user input, retrieved documents, tool output, and conversation history into typed fields

This first layer catches protocol abuse and closes inspection gaps. It also gives later detectors the route and source context they need.

Preserve both original and normalized forms

Attack text may use Unicode confusables, zero-width characters, mixed scripts, unusual whitespace, HTML entities, base64, URL encoding, or text split across fields. Normalize a copy for detection while preserving the exact original for hashing, forensic handling, and any permitted model call.

A reasonable text pipeline can:

1. enforce UTF-8
2. apply Unicode NFKC to the inspection copy
3. remove or flag zero-width and bidirectional control characters
4. collapse unusual whitespace for rule matching
5. decode one permitted layer of URL or HTML encoding where the field's declared format allows it
6. identify long encoded spans for separate analysis
7. join adjacent streaming or multipart fragments within a bounded rolling window

Do not recursively decode arbitrary content until something looks malicious. Recursive decoding is easy to abuse for resource exhaustion, and it can turn benign text into a different byte sequence. Record every normalization step and cap the expansion ratio.

Classify the source before classifying the words

A direct user prompt and a paragraph retrieved from the web are different security events. Both are untrusted, but they enter through different paths and may justify different actions.

I tag each segment with fields such as:

segment:
  id: seg-0187
  source_type: retrieved_web
  trust: untrusted
  intended_use: summarization
  may_supply_instructions: false
  may_select_tools: false
  may_select_destinations: false
  content_digest: sha256:<digest>

The inspection service should receive these labels from trusted application code, not from the model or user. A retrieved page that says "ignore previous instructions" should be treated as suspicious content from a data source. It should not become a higher-priority instruction because it appears late in the assembled prompt.

Use deterministic rules for high-signal conditions

Rules are useful for known indicators and policy violations:

• attempts to override system or developer instructions
• requests to reveal hidden prompts, credentials, private context, or chain-of-thought
• role or authority impersonation
• instructions embedded in retrieved content that ask the agent to call tools, change recipients, or contact a new destination
• known jailbreak markers maintained from tested attack cases
• encoded or invisible instruction-like content
• repeated attempts that vary wording after a refusal
• canary values that should never appear in user-controlled input

Keep these rules narrow. A cybersecurity article can legitimately discuss prompt injection and contain phrases used in attacks. A rule that blocks every occurrence of "ignore previous instructions" will spend its life blocking documentation, incident reports, and test cases.

Rules should produce named signals and evidence ranges, not a single unexplained verdict. Store the rule version and the location of the match. Do not put the complete sensitive payload in ordinary logs.

Add a classifier for semantic attacks

A classifier can catch paraphrases that rules miss. It can be a dedicated local model, a commercial guardrail service, or a second vendor-hosted model with a constrained classification schema. The choice changes latency, cost, privacy, and correlated-failure risk.

The classifier should return structured fields such as:

{
  "direct_injection": 0.07,
  "indirect_injection": 0.91,
  "jailbreak": 0.16,
  "secret_request": 0.84,
  "tool_manipulation": 0.88,
  "evidence": [
    {"segment_id": "seg-0187", "start": 412, "end": 538}
  ]
}

Treat the scores as signals. Thresholds must come from evaluation against the application's traffic, languages, user roles, and consequences. A threshold copied from a product example has no established meaning in another system.

Anthropic's published guidance recommends a layered approach that includes input validation, a lightweight screening model with constrained output, prompt hardening, repeated-offender handling, monitoring, and additional safeguards around untrusted tool content. OpenAI likewise describes moderation results as policy signals that can support filtering, review routing, or account intervention rather than as an automatic universal blocking decision.

A classifier call also creates another data boundary. If the inspection service sends the full prompt to a second hosted provider, the organization now has two processors, two retention configurations, and two incident paths. Redact what the classifier does not need, select an approved region, disable training or retention where the service permits it, and document the outage behavior.

Make a route-specific decision

Prompt injection risk depends on what the application can do. I would not use the same action thresholds for a public writing assistant and an agent that can modify cloud policy.

A useful decision model includes:

• caller identity and abuse history
• application and route
• data classification
• segment provenance
• detector signals and versions
• tool and network authority available to the run
• whether a human will review the result
• whether the request is read-only or can cause an external effect

A high score on a read-only public summarizer may result in isolation of the suspicious segment and a warning. The same score on an administrative agent should deny inference or remove all action authority. A clean score should never grant a tool permission the caller did not already possess.

Distinguish jailbreaks from injection

The terms are related, but I keep separate labels.

A jailbreak attempts to bypass the model's safety behavior. A prompt injection attempts to alter the application's intended behavior, directly through the user or indirectly through content the application retrieves. One request can do both.

That distinction improves policy. A jailbreak against a public chatbot may call for refusal, abuse throttling, or account review. An indirect injection inside a retrieved support ticket may call for quarantining that segment, preventing tool use, and notifying the application owner even when the user did nothing wrong.

The distinction also improves metrics. If every suspicious event is called a jailbreak, the security team cannot tell whether it has an abusive user problem, a poisoned data-source problem, or an application that cannot preserve instruction provenance.

Screen the response before release

The output pass should assume that model output is untrusted, even when the provider reports that its own safety controls ran.

There are three separate questions:

1. Does the response contain content the application should not deliver?
2. Does it disclose data the recipient is not allowed to receive?
3. Is it safe for the next software component to parse or execute?

One classifier rarely answers all three.

Moderate against an application policy

Define content categories according to the application's audience and purpose. Common categories include violence, threats, self-harm, sexual content, hate, harassment, fraud, malware assistance, regulated advice, and prohibited goods or services. The policy should say what happens at each severity and confidence level.

The response options are broader than allow or block:

• deliver unchanged
• deliver with a warning or age gate
• replace with a fixed safe response
• redact a bounded span
• route to trained human review
• block the response and record a reason code
• disable tools or external actions while still returning a safe explanation
• terminate or rate-limit an abusive session

Do not silently rewrite high-consequence answers and pretend the model produced the edited text. Preserve attribution in the internal record and tell the user when policy changed the response, unless doing so would expose detection details or sensitive data.

OpenAI's moderation documentation supports screening both inputs and generated outputs. It notes that category scores arrive only after the full generated output is available for streamed responses, and that a refusal can still be flagged because it discusses harmful content. That is a useful warning against treating one boolean as the entire policy.

Detect sensitive-data disclosure with more than regex

Sensitive output can come from the user prompt, retrieved documents, tool results, conversation memory, system instructions, training data, or a model inference that happens to be correct. OWASP lists personal data, financial and health records, credentials, confidential business information, and legal material among the affected classes, then recommends sanitization, strict access control, and restricting data sources.

I combine several methods:

• exact-data matching for values supplied during the current run
• canary strings placed in protected prompts, documents, or test records
• structured detectors for credentials, private keys, account numbers, government identifiers, and other stable formats
• entropy and prefix checks for secret-like values
• DLP or named-entity classification for personal, health, financial, legal, and proprietary data
• tenant and recipient policy that determines whether a detected class is allowed at this destination
• comparison with retrieval permissions and tool-response fields used to build the answer
• semantic checks for paraphrased confidential content that exact matching will miss

Exact-data matching is especially useful when the gateway can build a protected-value set from the material sent to the model. Store keyed digests or another protected representation where exact plaintext retention is unnecessary. The match service itself becomes sensitive because it can act as an oracle, so restrict access and rate-limit queries.

Pattern matches need validation. A 32-character hexadecimal string could be a harmless content hash or a credential. Context, prefix, entropy, known issuer format, and destination policy should determine the action. Do not call a live provider endpoint to verify a suspected secret unless the verification path is authorized, isolated from ordinary logs, and guaranteed not to mutate state.

Detect prompt and policy leakage

A system prompt is not a safe place for credentials, authorization rules, private keys, or hidden access-control data. If revealing the prompt compromises the system, the design has already placed too much trust in secrecy.

Still, output screening can detect leakage of internal instructions, canary phrases, policy identifiers, hidden context, and private tool schemas. Exact canaries work well here. Similarity checks can catch partial paraphrase, but they need careful tuning because generic instructions often resemble ordinary product documentation.

If a leak detector fires, block or replace the response, retain protected evidence under incident controls, and review the full context assembly path. A leak is often evidence of a larger provenance or authorization weakness rather than a reason to add another sentence to the system prompt.

Validate output for its destination

A response intended for a human-readable text box has different hazards from a response used as code or tool input.

For a browser, encode output for the exact HTML, attribute, URL, CSS, or JavaScript context. Do not rely on a generic HTML sanitizer for every sink. Use a strict Content Security Policy as another layer.

For tool calls, require a schema, reject unknown fields, resolve canonical resource identifiers, authorize the action outside the model, and bind any approval to the exact proposed effect. For SQL, use parameterized operations and an application-owned query interface. For files, constrain roots, canonicalize paths, reject traversal, and separate content from filenames. For shell execution, prefer typed operations over generated command strings.

Content moderation cannot make generated code safe. Secret scanning cannot prove a tool call is authorized. Destination validation remains required even when every AI-specific detector reports a clean result.

A vendor-neutral policy example

The following YAML is a reference policy shape, not configuration for a named product. Its purpose is to make the decisions explicit enough to implement in a gateway, sidecar, or SDK.

policy_version: 2026-09-08.1

routes:
  support_assistant:
    request:
      max_body_bytes: 1048576
      allowed_models: [approved-chat-model]
      allowed_content_types: [application/json]
      invalid_utf8: deny
      unsupported_archive: deny
      normalization:
        unicode: NFKC
        flag_zero_width: true
        max_decode_layers: 1
        max_expansion_ratio: 4
      detectors:
        - id: injection-rules
          version: 18
        - id: injection-classifier
          version: 2026-08-31
          timeout_ms: 350
        - id: outbound-dlp
          version: 12
      decision:
        direct_injection:
          review_at: 0.65
          deny_at: 0.88
        indirect_injection:
          isolate_segment_at: 0.55
          disable_tools_at: 0.70
          deny_at: 0.90
        unknown_detector_result: deny_if_tools_enabled

    response:
      mode: buffered
      max_body_bytes: 2097152
      detectors:
        - id: content-safety
          version: 2026-09-01
        - id: sensitive-data
          version: 12
        - id: prompt-leak-canaries
          version: 4
      decision:
        credential: block
        private_key: block
        cross_tenant_data: block_and_alert
        internal_prompt_canary: block_and_alert
        personal_data:
          allow_only_if_recipient_authorized: true
        high_severity_unsafe_content: replace_and_review
        detector_timeout: block

    downstream:
      browser_output:
        encode_for: html_text
        content_security_policy: required
      tool_calls:
        schema_validation: required
        authorization: required
        human_approval_for: [external_send, destructive_change]

failure_policy:
  read_only_public_route:
    input_classifier_unavailable: allow_with_tools_disabled
    output_classifier_unavailable: block
  privileged_route:
    any_required_detector_unavailable: block

logging:
  store_raw_prompts: false
  store_raw_responses: false
  fields:
    - request_id
    - caller_id
    - route
    - provider
    - model
    - detector_versions
    - scores
    - decision
    - reason_codes
    - content_digest
    - latency_ms
  evidence_store:
    enabled_for: [block_and_alert, human_review]
    encrypted: true
    access_role: ai-security-investigator
    retention_days: 30

The values are examples, not recommended universal thresholds. The useful properties are the separation of request and response policy, explicit detector failure behavior, route-specific consequences, named versions, bounded payloads, and limited evidence retention.

Buffering, streaming, and latency

Streaming forces a hard architecture choice.

A fully buffered response can be inspected before any content reaches the caller. This is the safer option for sensitive or high-consequence applications. It adds time to first byte, increases memory use, and requires a maximum response size.

Chunk scanning reduces latency but creates blind spots. An unsafe phrase or secret may span chunks. A rolling window helps, but semantic classifiers often need the complete answer. More importantly, a chunk that has already been released cannot be withdrawn.

I use three patterns:

1. Buffer the full response for privileged tools, regulated data, cross-tenant systems, and any route where disclosure would be material.
2. Stream only after a short holdback window for lower-risk conversational routes, then scan overlapping windows and terminate on a high-confidence match. Accept and document that some prefix may already have been disclosed.
3. Stream only metadata internally while withholding content from the end user until full-response checks pass.

If the provider supplies moderation signals only after generation completes, those signals cannot protect already released deltas. The gateway needs its own inline control or must buffer.

Envoy's external processing filter can send request and response headers, bodies, and trailers to a gRPC processor, and it can accept a locally generated response from that processor. A core filter block for buffered inspection looks like this:

name: envoy.filters.http.ext_proc
typed_config:
  "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
  grpc_service:
    envoy_grpc:
      cluster_name: ai_guardrail_processor
  failure_mode_allow: false
  message_timeout: 0.5s
  processing_mode:
    request_header_mode: SEND
    request_body_mode: BUFFERED
    response_header_mode: SEND
    response_body_mode: BUFFERED

That fragment is not a complete Envoy deployment. The listener, route, upstream provider cluster, external processor cluster, TLS validation, buffer limits, authentication headers, retries, and health checks still need configuration. failure_mode_allow: false is also not a complete outage policy. Separate routes or gateway cells may need different failure behavior based on data and action risk.

Test the actual proxy version before deployment. Body-processing mode, buffer limits, timeout behavior, header mutation, and response replacement are part of the security contract. A configuration that inspects headers but silently skips an oversized body is not equivalent to content inspection.

Configure failure behavior deliberately

Detector outages are inevitable. Decide the result before one occurs.

For an input detector failure, options include:

• deny the request
• permit inference but remove tool and network authority
• route to a safer model or a fixed retrieval-only response
• queue for review
• allow a low-risk route with an explicit degraded-state event

For an output detector failure, fail-open behavior is harder to justify because the uninspected content is about to leave the control point. I normally block or replace output on routes that can expose sensitive data, affect a protected user population, or feed another program.

Timeouts should not be reported as clean classifications. Use a distinct state such as unknown, attach a reason code, and apply the route's failure policy. Alert on sustained degraded operation and on sudden changes in block rate, review rate, classifier errors, response size, or latency.

Avoid correlated failure. If the main model and guardrail classifier use the same provider, region, identity service, or quota, one outage can disable both. A local deterministic layer and cached policy can preserve basic controls, but stale detection models and rules need an expiration policy.

Keep logs useful without building another leak

Raw prompt and response logging is tempting during tuning. It is also an efficient way to copy secrets, personal data, legal material, and proprietary documents into a system with broad analyst access.

The normal event should contain metadata:

• request and trace identifiers
• authenticated caller, application, tenant, route, provider, and model
• segment source types and trust labels
• detector and policy versions
• scores, matched rule IDs, decisions, and reason codes
• body sizes, content digests, timing, and degraded-state fields
• whether tools were available, removed, requested, authorized, and executed

Store raw evidence only when the use case requires it. Encrypt it separately, restrict it to an investigation role, apply a short retention period, and record every access. Redact authorization headers and provider credentials before any error body or trace is created.

NIST AI 600-1 frames generative AI risk management across the lifecycle and emphasizes governance, content provenance, pre-deployment testing, and incident disclosure. It also identifies data privacy, dangerous content, and information-security risks that can arise from both inputs and outputs. The gateway event model should support those activities without becoming an uncontrolled content archive.

Test the policy with adversarial and ordinary traffic

A detector that catches a few public jailbreak strings is not ready for production. The test set needs attacks, benign lookalikes, application-specific data, and failure cases.

I include at least:

• direct instruction override attempts
• indirect instructions inside web pages, tickets, documents, images, and tool output
• Unicode confusables, zero-width characters, mixed languages, encoded spans, and split tokens
• multi-turn attacks that become suspicious only when conversation history is included
• requests to reveal system prompts, retrieved private data, credentials, or another tenant's content
• unsafe output categories at each policy severity
• exact and paraphrased disclosure of protected test values
• secrets split across output chunks
• generated HTML, Markdown, URLs, SQL, paths, and tool arguments aimed at the real destination validators
• detector timeout, malformed detector response, stale policy, oversized body, provider retry, and partial stream failure
• legitimate security documentation, incident response, medical, legal, and educational content that contains sensitive vocabulary
• authorized sensitive-data use where the correct result is allow

Every case needs an expected policy outcome and an external-effect assertion. For a blocked output, verify that zero response bytes reached the caller. For a denied tool call, verify that the target received zero requests. For redaction, verify that the protected value is absent from the delivered body and ordinary logs. For failover, verify which controls remained active and which route entered a degraded state.

Track precision and recall by route and language, but do not stop there. Measure the effect of false positives on real work, the share of unknown results, review-queue age, detector latency, policy-version drift, and whether callers bypass the gateway after a denial.

Policy changes should move through version control, review, automated tests, canary release, and rollback. Keep the model or ruleset version with every decision so a later incident can be replayed against the policy that actually ran.

Native provider controls and your control point

Vendor safety controls are useful. Use them when they fit the application, but keep the application's decision at the boundary you own.

Provider moderation may have access to model-specific signals and may reduce harmful generation before it reaches the response. Your gateway knows the authenticated caller, tenant, business purpose, data classification, approved recipients, tool authority, and destination context. The provider cannot infer all of that from prompt text.

The two layers should complement each other:

• the provider enforces its platform safety policy and supplies available safety signals
• the application or gateway enforces organizational policy, authorization, data handling, and destination validation
• the tool broker enforces what actions can occur
• monitoring and testing verify the combined path

Do not assume a provider refusal means the input was harmless to log, that a provider-generated answer is authorized for the recipient, or that a moderation pass detected a credential. Record which layer made each decision.

The policy matters more than the proxy

A gateway or sidecar can see both directions only when traffic is routed through it in an inspectable form. Once that is true, the hard work is policy: what to normalize, which sources can provide instructions, which detectors run, what each score means for this route, what happens during an outage, when streaming is allowed, which data may reach which recipient, and how output is validated for its next use.

My baseline is simple:

1. Treat prompt-injection and jailbreak detectors as sensors, not authorization systems.
2. Keep provenance with every untrusted context segment.
3. Screen requests before inference and responses before release.
4. Use separate controls for unsafe content, sensitive-data disclosure, and destination safety.
5. Buffer output when disclosure cannot be tolerated.
6. Fail closed for privileged actions and protected data.
7. Log decisions and versions by default, raw content only under restricted evidence handling.
8. Test external effects, detector failures, and benign lookalikes before production.

The architecture in the earlier sidecar article creates the inspection point. This policy makes that point useful. The companion post, Prompt Injection Is an Authorization Failure with Words Attached, will carry the design one step further by showing why clean classification still cannot replace capability boundaries and tool authorization.