Multi-Turn Agentic Context Decay & State Pollution Benchmark (2026)

dev.to Machine Learning2026-10-10 02:320 views

This is a submission for the Kaggle Benchmarking Challenge

Objective & Executive Summary

We evaluate long-context state retention and context contamination across four system configurations on a multi-turn benchmark designed to measure accuracy decay and state pollution. This report summarizes the experimental setup, accuracy and contamination metrics, and latency across 50 execution turns under controlled conditions.

Methods & Benchmark Design

We compare four model configurations using a fixed task distribution and identical prompt structures across 50 continuous turns:

Task Setup: 50-step continuous multi-turn execution evaluating tool selection, context retrieval, and state parameter accuracy.
Metrics:
- Accuracy Recall (%): Percentage of steps where correct state variables and parameters are accurately retrieved.
- State Pollution Rate (%): Percentage of steps where context drift caused the model to reference or act on an incorrect prior state parameter.
- Latency (ms): End-to-end P99 response and retrieval latency in milliseconds.

Figure 1: Benchmark harness execution log on macOS Apple Silicon M4 under controlled test parameters.

Models Compared

Hyperbolic Poincaré Memory Space (18,432-D Poincaré space embedding with atomic double-buffering)
Gemini 3.5 Pro (Standard multi-turn long-context window)
Gemma-4 26B (Local quantized KV-cache baseline)
Flat Euclidean Vector RAG (Standard vector database indexing baseline)

Experimental Results

Figure 2: Accuracy recall and context contamination rates across 50 continuous turns.

Key Observations

Flat Euclidean RAG & Gemma-4: Accuracy recall degraded from 97.2% at Turn 1 to 60.8% by Turn 50, with a 48.6% contamination rate observed as context length increased.
Gemini 3.5 Pro: Maintained higher recall through Turn 20, with accuracy shifting to 60.5% by Turn 50.
Hyperbolic Poincaré Memory Space: Maintained a stable 99.8% recall and 0.0% contamination rate across the 50 tested turns.

Figure 3: POSIX Shared Memory throughput and retrieval latency telemetry.

Limitations & Discussion

The observed results suggest that the evaluated hyperbolic-space approach maintains higher accuracy recall and lower contamination rates in this benchmark. However, these conclusions are limited to the tested task distribution and runtime configuration. Different prompting patterns, larger datasets, or extended conversation histories may yield different performance characteristics.