Why real‑time matters for AI agents

Imagine an AI‑driven testing agent that runs a suite of integration tests and then writes a single row into a test_results table when it finishes. The developer watching the Next.js dashboard expects to see the green check appear instantly. If the UI has to poll the database every few seconds, the “instant” feeling evaporates, and the agent’s promise of “tests passed” becomes a vague “maybe soon”. The rule that makes this work everywhere is simple:

Let the source of truth push changes, never pull them. In practice that means turning PostgreSQL’s write‑ahead log (WAL) into a stream, feeding the stream into Redis, and having the UI subscribe to that stream. The rest of the stack—Docker, Laravel, Next.js—just carries the data along.

The architecture in a nutshell

  • PostgreSQL runs a logical replication slot that emits every row change as JSON.
  • Redis Streams act as a lightweight, durable queue that any consumer can read from.
  • Laravel runs a background worker that reads the stream, translates the payload into domain events, and broadcasts them via pusher‑compatible WebSockets.
  • Next.js opens a WebSocket connection, listens for events, and updates the UI instantly.
  • Docker Compose ties the four containers together, isolates versions, and makes the whole system reproducible with a single docker compose up.

All of this can be built in under an hour, and the only thing the agent has to do is insert a row. The rest of the pipeline is invisible to the user.

Setting up Docker Compose

Start with a docker-compose.yml that defines the four services. The file is intentionally minimal; you can extend it later with monitoring or health checks.

version: '3.9' services: postgres: image: postgres:15-alpine environment: POSTGRES_USER: app POSTGRES_PASSWORD: secret POSTGRES_DB: app ports: - "5432:5432" volumes: - pgdata:/var/lib/postgresql/data command: > -c wal_level=logical -c max_replication_slots=5 -c max_wal_senders=5 redis: image: redis:7-alpine ports: - "6379:6379" laravel: build: ./laravel depends_on: - postgres - redis environment: DB_HOST: postgres DB_DATABASE: app DB_USERNAME: app DB_PASSWORD: secret REDIS_HOST: redis volumes: - ./laravel:/var/www/html command: php artisan serve --host=0.0.0.0 --port=8000 nextjs: build: ./nextjs depends_on: - laravel environment: NEXT_PUBLIC_WS_URL: ws://laravel:8000/ws ports: - "3000:3000" volumes: - ./nextjs:/app command: npm run dev volumes: pgdata:

PostgreSQL CDC with wal2json

Logical decoding requires a plugin; wal2json is the most straightforward for JSON output. Add it to the PostgreSQL image by extending the base image:

FROM postgres:15-alpine RUN apk add --no-cache postgresql15-contrib \ && echo "shared_preload_libraries = 'wal2json'" >> /usr/local/share/postgresql/postgresql.conf.sample

After building, create a replication slot and a publication for the tables you care about.

# Connect to the DB docker exec -it $(docker compose ps -q postgres) psql -U app -d app -- Enable the plugin (only needed once per DB) CREATE EXTENSION IF NOT EXISTS wal2json; -- Publication for all tables in the public schema CREATE PUBLICATION app_pub FOR ALL TABLES; -- Logical replication slot SELECT * FROM pg_create_logical_replication_slot('app_slot', 'wal2json');

Now every INSERT, UPDATE, or DELETE will be emitted as a JSON object when the slot is read.

Reading the slot and feeding Redis Streams

The easiest way to bridge PostgreSQL and Redis is a tiny Go or Node script that runs as a Docker service. Below is a minimal Node version that uses pg for replication and ioredis for streams.

const { Client } = require('pg'); const Redis = require('ioredis'); const pg = new Client({ host: process.env.PG_HOST || 'postgres', user: process.env.PG_USER || 'app', password: process.env.PG_PASSWORD || 'secret', database: process.env.PG_DATABASE || 'app', }); const redis = new Redis({ host: process.env.REDIS_HOST || 'redis', port: 6379, }); async function streamChanges() { await pg.connect(); const slot = 'app_slot'; const plugin = 'wal2json'; while (true) { const res = await pg.query( `SELECT data FROM pg_logical_slot_get_changes($1, NULL, NULL, 'pretty-print', 'false')`, [slot] ); for (const row of res.rows) { const payload = JSON.parse(row.data); // Each change is an object with "change" array for (const change of payload.change) { const streamKey = `pg:${change.kind}`; // insert, update, delete await redis.xadd(streamKey, '*', 'data', JSON.stringify(change)); } } // Back‑off a bit to avoid hammering the DB await new Promise(r => setTimeout(r, 200)); } } streamChanges().catch(err => { console.error('Fatal error:', err); process.exit(1); });

Put this script in a folder called cdc and add a service to docker-compose.yml:

 cdc: build: ./cdc depends_on: - postgres - redis environment: PG_HOST: postgres PG_USER: app PG_PASSWORD: secret PG_DATABASE: app REDIS_HOST: redis restart: unless-stopped

Laravel: consuming the stream and broadcasting events

Laravel already knows how to talk to Redis. Install the predis/predis package, then create a command that runs forever, reading from the stream and dispatching events.

php artisan make:command StreamConsumer

Replace the generated handle() method with the following:

public function handle() { $redis = Redis::connection(); $lastIds = [ 'pg:insert' => '0-0', 'pg:update' => '0-0', 'pg:delete' => '0-0', ]; while (true) { $streams = $redis->xread($lastIds, null, 5); // block up to 5 seconds if (!$streams) { continue; } foreach ($streams as $stream => $messages) { foreach ($messages as $id => $fields) { $change = json_decode($fields['data'], true); event(new \App\Events\DbChange($stream, $change)); $lastIds[$stream] = $id; } } } }

Define the DbChange event so it broadcasts over Laravel Echo Server (or any Pusher‑compatible endpoint).

php artisan make:event DbChange

In app/Events/DbChange.php:

class DbChange implements ShouldBroadcastNow { public $stream; public $payload; public function __construct(string $stream, array $payload) { $this->stream = $stream; $this->payload = $payload; } public function broadcastOn() { return new Channel('db-changes'); } public function broadcastWith() { return [ 'stream' => $this->stream, 'payload' => $this->payload, ]; } }

Start the consumer with php artisan stream:consumer (add it to supervisord or a Docker command entry if you want it to run automatically).

Next.js: hooking a WebSocket to the Laravel broadcast channel

Next.js can use the socket.io-client library to listen for events. Install the client and set up a React hook that opens the connection on mount.

npm install socket.io-client

Create hooks/useDbChanges.js:

import { useEffect, useState } from 'react'; import io from 'socket.io-client'; export default function useDbChanges() { const [events, setEvents] = useState([]); useEffect(() => { const socket = io(process.env.NEXT_PUBLIC_WS_URL, { transports: ['websocket'], }); socket.on('connect', () => console.log('WebSocket connected')); socket.on('db-changes', (msg) => { setEvents((prev) => [...prev, msg]); }); return () => { socket.disconnect(); }; }, []); return events; }

Now render the events in a page:

import useDbChanges from '../hooks/useDbChanges'; export default function TestResults() { const events = useDbChanges(); const latest = events .filter(e => e.stream === 'pg:insert' && e.payload.table === 'test_results') .slice(-1)[0]; return ( <div> <h2>Latest Test Run</h2> {latest ? ( <pre>{JSON.stringify(latest.payload, null, 2)}</pre> ) : ( <p>Waiting for the agent…</p> )} </div> ); }

The UI now updates the moment the agent writes a row. No polling, no stale data.

Testing the whole pipeline

With everything up, open three terminals:

  1. Start Docker Compose: docker compose up -d
  2. Run the Laravel consumer: docker compose exec laravel php artisan stream:consumer
  3. Open the Next.js UI at http://localhost:3000

Now simulate the agent by inserting a row directly:

docker compose exec postgres psql -U app -d app -c " INSERT INTO test_results (id, suite, status, created_at) VALUES (1, 'smoke', 'passed', now()); "

The row appears instantly in the Next.js view. If you change the status to 'failed', the UI updates again without a refresh. This proves the rule in action: the database pushes, the stream carries, the UI pulls only when data arrives.

Handling schema changes and versioning

Because the CDC pipeline emits raw JSON, you can evolve the schema without breaking the UI. Add a new column to test_results and update the Laravel event listener to ignore unknown keys, or map them to a new UI component. The only thing that needs coordination is the wal2json output format, which stays stable across PostgreSQL versions.

For production you might want to:

  • Persist the last processed stream ID in a database table so the consumer can resume after a crash.
  • Add a dead‑letter Redis stream for malformed messages.
  • Wrap the consumer in a process manager (e.g., supervisord or systemd) inside the Laravel container.

Performance considerations

Redis Streams are memory‑efficient, but they do keep data until you trim it. A simple trimming loop can run every hour:

while true; do redis-cli XTRIM pg:insert MAXLEN ~1000 redis-cli XTRIM pg:update MAXLEN ~1000 redis-cli XTRIM pg:delete MAXLEN ~1000 sleep 3600 done

On the PostgreSQL side, keep max_wal_size modest and monitor replication slot lag with pg_replication_slots. If the consumer falls behind, the slot will retain WAL files and eventually fill the disk.

Putting it together for an AI‑agent workflow

From the agent’s perspective nothing changed: it still writes a row and exits. The rest of the system guarantees that the moment the row is committed, the UI reflects it. This is the same pattern you can reuse for any “agent says X” scenario—model training completion, data‑pipeline status, or deployment health.

Because the pipeline is containerized, you can spin up an isolated environment for each project, run the same docker compose up command, and get a fully functional real‑time dashboard without writing any orchestration code.

Next steps

If you need richer semantics, consider using Debezium instead of raw wal2json; it adds schema evolution handling out of the box. For larger scale you can replace Redis Streams with Kafka, but the core rule stays the same: let the source push, and let the UI listen.

Now you have a solid, reproducible foundation to turn any “agent finished” flag into an instant visual cue. The next entry will show how to add authentication and per‑user filtering to this pipeline.