Build a useful synthetic reproducer for an AI support issue

Define the question before collecting a transcript

A support packet should help another engineer answer one question. “The assistant gave a bad answer” is too broad. A more useful question is whether a particular policy revision, message order and cache state explain a specific difference between expected and observed behavior.

Start with invented content where possible. RFC 6973, an Informational privacy-design document, describes minimizing collection, use, disclosure and retention to what a task needs. That principle supports a small diagnostic packet; it does not certify a packet as anonymous or establish that changing data preserves model behavior.

This guide uses no real prompt, customer record, personal name, credential or private path. Nothing was sent to a support service. A tiny offline teaching stub was run; actual AI, provider and production reproduction remains unrun.

Separate the packet from its dependency manifest

Give the case a stable synthetic ID and a short expected-versus-observed statement. Include the application or fixture revision, relevant policy identifier, ordered message roles, retrieval fixture, cache preparation and the exact operation under investigation. Use placeholders for systems that are not exercised. Do not copy a real system prompt merely to make the packet look complete.

Keep three statuses distinct: recorded application observation, reproduced locally, and proposed explanation. A screenshot of an answer is an observation, not necessarily a reproducible case. A suspected cache dependency is a hypothesis until an appropriate controlled comparison supports it. Unknown model or deployment versions should remain unknown rather than be filled with today's configuration.

A manifest should also name excluded dependencies and why they were excluded. If the packet is synthetic, describe how it differs from the original issue without retaining the original sensitive content inside the same packet.

Use a declared toy to practice reduction

For an inspectable example, the following fictional packet supplies four dependencies to an offline predicate. These role labels are local fixture labels, not a provider API's message schema.

{
  "schemaVersion": 1,
  "policyRevision": "P7",
  "roleOrder": ["policy", "document", "user"],
  "cacheState": "warm_prior_revision",
  "documentMarker": "DEMO-MARKER"
}

The toy was deliberately defined to emit legacy_demo_path only when all four values match; otherwise it emits current_demo_path. It does not discover or simulate a real model defect. Its original input also had an irrelevant displayTheme and note. The reduction procedure removed those two fields one at a time while checking that the label stayed the same.

Keep a useful packet and an over-reduced packet apart

The offline Node-only run produced the following observations. The reduced packet retained its label; each one-dependency variation lost it. These are six configurations of a canned predicate, not six AI security tests or an estimated success rate.

The missing-trigger packet is smaller but no longer illustrates the declared behavior. That is the practical difference between useful minimization and deleting the relevant condition. This procedure establishes a reduction under the chosen removals, not a globally minimal case. Real model behavior can vary with wording, sampling, provider changes and hidden state; equivalence requires separate evidence.

Inspect the packet before any authorized transfer

OWASP logging guidance identifies credentials, sensitive information and some identifying metadata as data that should not ordinarily be recorded directly. Apply the same caution when selecting diagnostic material. Review nested fields, attachments, screenshots, filenames and metadata, not just visible message text.

Use an allowlist of packet fields with a reason for each. An unexpected field should trigger local review rather than automatic inclusion. A content hash can identify fixture bytes but cannot prove that the material is safe to disclose. Replacing a name with a stable identifier may still preserve identifying relationships, so do not label the result anonymous merely because obvious names disappeared.

Prefer a self-contained synthetic document and recorder over a link to a live private resource. If a necessary dependency cannot be safely represented, state that limit and use the applicable authorized handling process. A prepared packet does not by itself determine a destination, permission to send or the recipient's retention behavior.

Report the evidence at its actual level

A useful handoff includes the fixture and source revisions, preparation steps, expected result, observed result, reduction history, runtime and unresolved dependencies. For this guide, the recorded runtime was Node v25.9.0. The observed evidence is restricted to the offline predicate and its six configurations; no model, provider, support ticket or customer outcome was reproduced.

Boundary's planned request router does not supply this diagnostic reduction workflow or guarantee safe support exports. Its separate eight mock policy checks concern other synthetic decisions. Keep those evidence sets separate, and describe a real AI issue as unresolved until the relevant application has actually been checked.