The Hidden Security Risk Inside OpenTelemetry OTLP Trace Exports

When debugging microservice tail latency, database deadlocks, or LLM agent chains, platform engineers regularly export OpenTelemetry (OTLP/JSON) trace files containing resourceSpans and scopeSpans. However, a single exported trace file routinely embeds production secrets and regulated customer PII across span attributes:

  • http.request.header.authorization: Bearer JWTs, session cookies, and internal API keys attached to outbound RPC spans.
  • db.statement & db.query.text: Unparameterized SQL queries containing literal customer email addresses, billing tokens, or tax identifiers.
  • gen_ai.prompt & gen_ai.completion: Raw user prompts and proprietary RAG context captured by OpenTelemetry GenAI semantic conventions.
  • net.peer.ip & service.instance.id: Internal Kubernetes pod CIDR addresses and private cluster endpoints.

Pasting an OTLP JSON trace dump into a cloud formatter or attaching it unredacted to a support ticket exposes production authentication tokens and customer PII outside your security boundary.

Architectural Solution: 100% Client-Side RAM OTLP Span Scrubbing

To eliminate third-party telemetry exposure during incident response, we built OpenTelemetry Trace Redactor — a zero-upload browser studio and standalone offline HTML5 workstation that sanitizes OpenTelemetry trace attributes entirely in local RAM.

1. Multi-Tier Span Attribute Redaction Without Breaking Trace Waterfalls

Instead of corrupting the JSON structure or breaking parent-child spanId links, the local redaction engine targets sensitive attribute values while preserving trace hierarchy:

// Client-side OTLP span attribute scrubbing pattern const otlpSecretRegex = /otel_bearer_[A-Za-z0-9._-]+|sql_bind_[A-Za-z0-9._-]+|genai_prompt_[A-Za-z0-9._-]+|eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}(?:\.[A-Za-z0-9_-]+)?|[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}|sk_live_[a-zA-Z0-9_]+|AKIA[0-9A-Z]{16}|\b\d{1,3}(?:\.\d{1,3}){3}\b/g; 

Engineers can toggle between three redaction strictness levels in real time:

  • 35% Partial Trace ID Mask: Retains attribute prefixes for correlation while masking token suffixes and final IPv4 octets.
  • 70% Standard Span Attribute Mask: Replaces bearer JWTs, SQL bind literals, GenAI prompt PII, and internal IPs with deterministic type-tagged placeholders ([OTLP_BEARER_MASKED], [SQL_BIND_MASKED]).
  • 95% SOC2 & GDPR Safe Harbor Scrub: Performs full irreversible token stripping across all sensitive span attributes and events.

2. Interactive 12-Column Free Workbench & 3-in-1 Offline PRO Appliance

  • Instant Browser Workbench: Load the built-in OTLP trace sample or paste your own trace table, adjust the redaction slider, and immediately copy or download the first 70% of sanitized span records for free.
  • Standalone Offline PRO Workstation: For air-gapped incident response teams, the PRO workstation compiles into a self-contained single-file HTML5 application featuring 4 dedicated OpenTelemetry batch modules (OTLP Span Attribute & HTTP Header Scrubber, SQL Bind Parameter Redactor, GenAI Trace Prompt PII Stripper, and TraceID & Internal Endpoint Anonymizer).

Try the Live Zero-Upload Sandbox

👉 Open OpenTelemetry Trace Redactor Live Sandbox