Phase 2: Stateless Cross-Instance HITL & GCS Artifact Architecture

Recap

In the previous two articles of this series:

Entering Phase 2 (Issue #3 & Issue #4), we face a classic and common pitfall in distributed systems when pushing to production (Cloud Run / Agent Runtime auto-scaling clusters): When containers can Scale to Zero or be load-balanced to different instances at any time, where should multimodal business card photos (Binary Images) be stored? And how can the HITL pause state, waiting for a user to click a button, be recovered across instances?

In the third installment of this series, we will deep dive into how we used ADK Go v2.6.0's artifact/gcsartifact (GCS Artifact Service) and session/vertexai (Agent Platform Sessions) in PR #11 to completely cure the "HITL amnesia" of stateless containers!

1. Why did the original HITL suffer from "Amnesia" and "State Bloat" in cloud multi-instance environments?

During local development or with a single container instance, stuffing everything into memory is fast. However, once deployed to Google Cloud Run or Vertex AI Agent Runtime, which support Horizontal Auto-scaling or Scale-to-Zero, the first version's design immediately hits two major walls:

Pain Point 1: The "State Bloat" anti-pattern of stuffing multi-MB business card []byte into Session.State

In the first version, when a LINE user sent a high-resolution business card photo, we directly put the downloaded imageBytes []byte into ADK's Session.State["image_bytes"]:

// ❌ Version 1 approach: Directly stuffing multi-MB raw image []byte into Session State InitialState: map[string]any{ StateKeyInputType: RouteIngest, StateKeyImageBytes: imageBytes, // A 3MB JPEG inflates to a ~4MB Base64 string after JSON serialization! }

When using the local session.InMemoryService(), this is just a Go slice pointer. But when switching to cloud-persisted Agent Platform Sessions (session/vertexai) or Cloud SQL (session/database):

  1. Serialization and network transfer for every node state change (StateDelta): As the ADK Graph Workflow moves from RouterNode to VisionOCRNode, fans out to three parallel nodes, and finally pauses at HITLConfirmNode, keeping multi-MB image binary data in Session.State significantly slows down every Cloud Session API read/write and might even trigger Cloud Session payload size limits!
  2. Difficulty in auditing and versioning history: If a user sends multiple business cards across several turns, mixing raw binary files with structured JSON state makes the conversation state extremely bloated.

Pain Point 2: Container instance switching leads to loss of HITL pendingSession memory mapping

Let's look at a real-world user scenario:

  1. Turn 1 (hits Container Instance A): A user takes a photo of a business card at a conference and sends it to the LINE Bot. Instance A completes Vision OCR and parallel background checks, then pauses at HITLConfirmNode via workflow.NewRequestInputEvent, pushing a Flex Message preview card to the user.
  2. 10-minute break: The user goes to exchange thoughts with the contact and grabs a coffee. During this time, Instance A scales to 0 due to inactivity, or Container Instance B is scaled out due to other traffic.
  3. Turn 2 (hits Container Instance B): The user clicks "✏️ Edit Fields" on LINE and enters "Change title to CTO", or clicks "✅ Confirm Save".

In the first version, our Engine used an in-memory Go map[string]pendingHITL to track which userID was waiting for which SessionID and InterruptID:

// ❌ Version 1 approach: Relying solely on a local in-memory map to determine if there's a pending HITL type Engine struct { mu sync.RWMutex pendingSession map[string]pendingHITL // userID -> {SessionID, InterruptID} }

As a result, when the Turn 2 Webhook request is routed by the load balancer to the brand new Instance B, Instance B's pendingSession map is empty! HasPendingHITL(userID) returns false, and the system treats "Change title to CTO" as a brand new "Business Card Search" request, instantly breaking the HITL loop!

2. The Standard ADK 2.0 Solution: Separation of Concerns between Session State and Artifact Storage

To solve this, Google ADK 2.0's architectural design explicitly splits Agent runtime data into two independent services:

Comparison Dimension Session Service (session.Service) Artifact Service (artifact.Service)
Core Purpose Stores conversation event logs (Events), workflow breakpoints (Interrupts), and lightweight structured state (JSON State) Stores large binary files (images, PDFs, audio, CSV reports) and provides automatic version control (Versioning)
Cloud Implementation Package google.golang.org/adk/v2/session/vertexai (Agent Platform Sessions) or session/database google.golang.org/adk/v2/artifact/gcsartifact (underlying storage in Google Cloud Storage Bucket)
Local Development Implementation session.InMemoryService() artifact.InMemoryService()
Role in Business Card King Stores draft_card JSON, active_interrupt_id, and lightweight pointers (filename + version number) to the image Stores raw business card photo business_card.jpg (automatically increments Version 1, 2, 3... per upload)

With this dual-track design, the entire cross-instance multimodal business card recognition and HITL recovery process becomes crystal clear:

sequenceDiagram autonumber actor User as LINE / WebUI User participant InstA as Container Instance A (Turn 1: Upload Card) participant GCS as GCS Artifact Service<br/>(artifact/gcsartifact) participant Sess as Agent Platform Sessions<br/>(session/vertexai) participant InstB as Container Instance B (Turn 2: Edit Fields) participant InstC as Container Instance C (Turn 3: Confirm Save) User->>InstA: 1. Upload business card photo (JPEG []byte) InstA->>GCS: 2. Save("business_card.jpg", inlineData)<br/>Returns Version = 1 InstA->>Sess: 3. Create Session, State only records:<br/>card_image_artifact="business_card.jpg"<br/>card_image_version=1 InstA->>GCS: 4. VisionOCRNode calls LoadVersion("business_card.jpg", 1) Note over InstA,Sess: Fan-Out Parallel duplicate check / Company search / Interactive retrieval -> Generate preview card InstA->>Sess: 5. HITLConfirmNode triggers NewRequestInputEvent<br/>Persists active_interrupt_id and draft_card InstA-->>User: 6. Return LINE Flex preview card (⏸️ Workflow Paused) Note over InstA,InstB: ☁️ Container scales to 0 or load balancer switches to a brand new stateless Instance B User->>InstB: 7. Input: "Change title to CTO" InstB->>Sess: 8. resolvePendingHITL(userID):<br/>Query latest Session's active_interrupt_id Sess-->>InstB: 9. Retrieve {SessionID, InterruptID} and draft_card InstB->>Sess: 10. ResumeHITL -> ApplyEditNode updates title -> Re-pauses and updates active_interrupt_id InstB-->>User: 11. Return updated Flex preview card (Title: CTO) Note over InstB,InstC: ☁️ Switch again to brand new stateless Instance C User->>InstC: 12. Click "✅ Confirm Save" (action=confirm_save) InstC->>Sess: 13. resolvePendingHITL(userID) retrieves latest InterruptID InstC->>Sess: 14. ResumeHITL -> SaveCardNode writes to card library + clears active_interrupt_id InstC-->>User: 15. Return save success and AI icebreaker draft!

3. Code Implementation Breakdown (Phase 2 — PR #11)

Now let's look at the code to see how we completed this upgrade with less than a few hundred lines of clean Go code.

1. Creating a Multi-backend Artifact Factory: sessions/artifacts.go

We added sessions/artifacts.go under the sessions package, using ADK Go v2.6.0's built-in google.golang.org/adk/v2/artifact/gcsartifact package to automatically switch between Cloud GCS Bucket and local memory based on environment variables:

package sessions import ( "context" "log/slog" "os" "strings" "google.golang.org/adk/v2/artifact" "google.golang.org/adk/v2/artifact/gcsartifact" ) // NewArtifactService returns an ADK artifact.Service backed by Google Cloud Storage // when ARTIFACT_BUCKET_NAME or LOGS_BUCKET_NAME is configured, and falls back to // artifact.InMemoryService() for local development and unit tests. func NewArtifactService(ctx context.Context) (artifact.Service, error) { bucketName := resolveArtifactBucketName() if bucketName == "" { slog.Info("Using in-memory artifact service (set ARTIFACT_BUCKET_NAME or LOGS_BUCKET_NAME for GCS)") return artifact.InMemoryService(), nil } svc, err := gcsartifact.NewService(ctx, bucketName) if err != nil { return nil, err } slog.Info("Using GCS artifact service", "bucket", bucketName) return svc, nil }

A thoughtful detail: In the agents-cli Agent Runtime deployment environment, the system usually configures LOGS_BUCKET_NAME (or our custom ARTIFACT_BUCKET_NAME). Some might accidentally fill it in with a format like gs://my-bucket/prefix. Our resolveArtifactBucketName() automatically strips the gs:// prefix and path to ensure gcsartifact.NewService connects reliably, falling back to artifact.InMemoryService() if no bucket is set.

Simultaneously, we inject ArtifactService and SessionService into launcher.Config in main.go and cmd/server/main.go:

// main.go — Injecting both SessionService and ArtifactService into ADK Universal Launcher config := &launcher.Config{ SessionService: sessionService, ArtifactService: artifactService, AgentLoader: NewSingleAgentLoader(rootAgent), A2AOptions: a2aOpts, }

2. Storing Business Card Images in Artifacts, Keeping only "Lightweight Pointers" in Session State

Next, we refactor internal/workflow/graph.go and internal/workflow/nodes_ingest.go.

First, we define two new lightweight State Keys and a default filename:

const ( StateKeyImageBytes = "image_bytes" // Kept only for backward compatibility fallback StateKeyCardImageArtifact = "card_image_artifact" // Artifact filename StateKeyCardImageVersion = "card_image_version" // Artifact version number (int) DefaultCardImageArtifactName = "business_card.jpg" )

(1) Entering from LINE Webhook (Engine.StartIngest)

When a LINE user sends an image, StartIngest first calls e.artifactService.Save to write the image binary to the Artifact service and only records the filename and version number in InitialState. StateKeyImageBytes is no longer written to Session.State:

initialState := map[string]any{ StateKeyInputType: RouteIngest, StateKeyRecentEvents: recentEvents, } if e.artifactService != nil && len(imageBytes) > 0 { part := genai.NewPartFromBytes(imageBytes, detectImageMIMEType(imageBytes)) saveResp, saveErr := e.artifactService.Save(ctx, &artifact.SaveRequest{ AppName: appName, UserID: userID, SessionID: sess.ID(), FileName: DefaultCardImageArtifactName, Part: part, }) if saveErr == nil && saveResp != nil { initialState[StateKeyCardImageArtifact] = DefaultCardImageArtifactName initialState[StateKeyCardImageVersion] = saveResp.Version } }

(2) Uploading images directly from ADK WebUI / Agent Engine Playground / A2A (RouterNode)

What if the user isn't using LINE but uploads a business card photo directly via ADK WebUI (make playground) or Agent Engine Console? In RouterNode, we use ADK 2.0's ctx.Artifacts().Save(...) to transfer InlineData from ctx.UserContent() into the Artifact Service:

// internal/workflow/nodes_ingest.go — RouterNode if getStringState(ctx, StateKeyCardImageArtifact) == "" && len(getBytesState(ctx, StateKeyImageBytes)) == 0 && ctx.UserContent() != nil { for _, part := range ctx.UserContent().Parts { if part != nil && part.InlineData != nil && len(part.InlineData.Data) > 0 { if ctx.Artifacts() != nil { mimeType := part.InlineData.MIMEType if mimeType == "" { mimeType = detectImageMIMEType(part.InlineData.Data) } artifactPart := genai.NewPartFromBytes(part.InlineData.Data, mimeType) if saveResp, saveErr := ctx.Artifacts().Save(ctx, DefaultCardImageArtifactName, artifactPart); saveErr == nil { setState(ctx, ev, StateKeyCardImageArtifact, DefaultCardImageArtifactName) setState(ctx, ev, StateKeyCardImageVersion, saveResp.Version) } else { setState(ctx, ev, StateKeyImageBytes, part.InlineData.Data) } } if getStringState(ctx, StateKeyInputType) == "" { setState(ctx, ev, StateKeyInputType, RouteIngest) } break } } }

(3) VisionOCRNode Reads specific versions of business card images on demand (loadCardImageBytes)

When the workflow enters VisionOCRNode to call Gemini Vision for multimodal recognition, it uses ctx.Artifacts().LoadVersion (or Load) to load the image into memory for the model. It's discarded after use, keeping Session.State clean:

func loadCardImageBytes(ctx agent.Context) []byte { if artifactName := getStringState(ctx, StateKeyCardImageArtifact); artifactName != "" && ctx.Artifacts() != nil { version := getIntState(ctx, StateKeyCardImageVersion) var ( resp *artifact.LoadResponse err error ) if version > 0 { resp, err = ctx.Artifacts().LoadVersion(ctx, artifactName, version) } else { resp, err = ctx.Artifacts().Load(ctx, artifactName) } if err == nil && resp != nil && resp.Part != nil && resp.Part.InlineData != nil { return resp.Part.InlineData.Data } } return getBytesState(ctx, StateKeyImageBytes) // Fallback }

3. Implementing Stateless Cross-Instance HITL State Discovery: resolvePendingHITL

With image storage solved, here's the main event: When Container Instance B receives a user's "Change title to CTO" or "✅ Confirm Save", how does it automatically retrieve the HITL workflow paused by Instance A when its local memory is empty?

The secret is: when HITLConfirmNode calls workflow.NewRequestInputEvent to pause, we've already written the current InterruptID into Session.State["active_interrupt_id"] (and SaveCardNode clears it to "" once saving is complete).

Therefore, we implemented resolvePendingHITL(ctx, userID) in internal/workflow/graph.go:

// resolvePendingHITL checks the local cache first and falls back to querying // the shared session.Service for any session belonging to userID with a non-empty // StateKeyActiveInterruptID. This enables stateless cross-instance HITL resume // on Cloud Run / Vertex AI Agent Runtime. func (e *Engine) resolvePendingHITL(ctx context.Context, userID string) (pendingHITL, bool) { // 1. Fast path: If a record exists in the same instance's memory, verify with sessionService if it's still paused e.mu.RLock() cached, ok := e.pendingSession[userID] e.mu.RUnlock() if ok && cached.SessionID != "" && cached.InterruptID != "" { if getResp, err := e.sessionService.Get(ctx, &session.GetRequest{ AppName: appName, UserID: userID, SessionID: cached.SessionID, }); err == nil && getResp != nil && getResp.Session != nil { if activeID, err := getResp.Session.State().Get(StateKeyActiveInterruptID); err == nil { if s, ok := activeID.(string); ok && strings.TrimSpace(s) != "" { cached.InterruptID = strings.TrimSpace(s) return cached, true } } } } // 2. Stateless Slow Path: When hitting a brand new container instance, query the shared sessionService directly! listResp, err := e.sessionService.List(ctx, &session.ListRequest{ AppName: appName, UserID: userID, }) if err != nil || listResp == nil { return pendingHITL{}, false } var ( matched pendingHITL latestTime time.Time foundActive bool ) for _, sess := range listResp.Sessions { if sess == nil || sess.State() == ni...
Source：https://dev.to/gde/agent-platform-in-action-3-solving-stateless-container-hitl-amnesia-cross-instance-business-3c4h
My favorites