Workflow vs Agent: Deterministic Orchestration & Dynamic Tool Selection

Quick Diagnosis

Decision Flow

flowchart TD A["Is the full execution path known ahead of time?"] B["Can every required tool be enumerated?"] C["Use a deterministic workflow"] D["Is the request likely to need unknown sub‑tasks?"] E["Add an agent layer for dynamic tool selection"] F["Can live values be fetched via an API?"] G["Expose API call from the workflow"] H["Leave decision to the agent at runtime"] A -->|Yes| B B -->|Yes| C B -->|No| D D -->|Yes| E D -->|No| F F -->|Yes| G F -->|No| H 

Architectural Responsibilities

Python Decision Helper (Teaching‑Only)

# decision_helper.py – pure stdlib, no external packages import json from pathlib import Path from typing import Literal, TypedDict class DecisionInput(TypedDict): """Minimal data required to pick workflow vs. agent.""" steps_known: bool # True if the entire sequence can be enumerated  tools_enumerated: bool # True if every tool can be listed up‑front  live_value_needed: bool # True if an authoritative API must be called  dynamic_subtasks: bool # True if request may spawn unknown sub‑tasks  def choose_strategy(data: DecisionInput) -> Literal["workflow", "agent", "hybrid"]: """ Return the recommended execution style. * `workflow` – fully deterministic pipeline * `agent` – pure agentic control * `hybrid` – deterministic skeleton with optional runtime API or agent """ # Basic validation – fail fast if the contract is broken  assert isinstance(data["steps_known"], bool), "steps_known must be bool" assert isinstance(data["tools_enumerated"], bool), "tools_enumerated must be bool" assert isinstance(data["live_value_needed"], bool), "live_value_needed must be bool" assert isinstance(data["dynamic_subtasks"], bool), "dynamic_subtasks must be bool" if data["steps_known"] and data["tools_enumerated"]: # All actions are known → deterministic flow  return "workflow" if data["dynamic_subtasks"]: # Content can generate new branches → need agentic reasoning  return "agent" # Mixed case – known skeleton but may need live data or optional tool selection  return "hybrid" # ------------------------------------------------- # Example usage (for teaching only – not part of production) if __name__ == "__main__": example = { "steps_known": True, "tools_enumerated": True, "live_value_needed": False, "dynamic_subtasks": False, } print(json.dumps({"strategy": choose_strategy(example)})) 

Teaching‑Only Note – Production systems should encode this decision in a CI‑validated configuration file rather than an ad‑hoc script.

Architectural Decision Records (ADRs)

ADR 1 – Introducing an Agent for Policy Queries

Context

A fintech onboarding pipeline executes three fixed steps (identity → credit → welcome email). Occasionally, a regulator submits a policy_query that does not fit the fixed path.

Decision

Add an optional agent that is triggered only when the incoming payload contains a policy_query field. The agent will invoke a policy‑search tool, retrieve the relevant text, and attach it to the final output.

Consequences

• The core majority of requests remain in a deterministic workflow, preserving auditability.
• New logs capture the agent’s chosen tool, LLM prompt, and tool response, enabling traceability.
• Latency impact is limited to the subset of requests that require the agent.

Acceptance Criteria

1. Agent‑enabled paths generate a log entry that records the selected tool, the exact prompt sent to the LLM, and the tool’s raw response.
2. All agent‑generated outputs are stored with a cryptographic hash for integrity verification.
3. CI pipeline includes unit tests for both “with query” and “without query” scenarios, and validates that the workflow remains deterministic when the agent is not invoked.

ADR 2 – Handling Live Credit‑Limit Values

Context

The onboarding flow must embed the applicant’s current credit limit, which changes frequently. A stale document store could return an outdated figure.

Decision

Expose a tool in the workflow that calls the credit‑bureau’s authoritative REST endpoint. The workflow pauses, fetches the live limit, and continues deterministically. No caching of the numeric value is permitted.

Consequences

• Guarantees freshness of a critical compliance datum.
• Introduces a single external API call per onboarding, which is observable in cost‑monitoring dashboards.
• Enables deterministic testing by mocking the API response within a test harness.

Acceptance Criteria

1. Integration tests with a mock endpoint produce the same structural response shape as the real service.
2. Production logs record request‑ID, endpoint URL, response timestamp, and the returned numeric value.
3. Any failure to reach the API aborts the workflow with a defined error code and escalates to a human reviewer.

Evaluation & Maintenance Checklist

• Functional correctness

  • ✅ Deterministic steps produce identical output for identical inputs (regression suite).
  • ✅ Agent‑enabled paths have mocked‑LLM tests covering tool selection logic.

• Governance

  • ✅ Prompt templates are version‑controlled, signed, and reviewed.
  • ✅ Business rules (e.g., “identity verification before credit check”) live in code, not in LLM prompts.

• Cost visibility

  • ✅ Token usage per step logged to a cost‑center dashboard.
  • ✅ External‑API call count tracked; alerts trigger on abnormal spikes.

• Latency observability

  • ✅ End‑to‑end latency recorded; thresholds defined per SLA.
  • ✅ Separate latency metric for agent loops to bound their impact.

• Knowledge freshness

  • ✅ Retrieval index (Azure Cognitive Search) refreshed on a schedule aligned with policy updates.
  • ✅ Live‑value endpoints re‑validated after any vendor version change.

• Model updates

  • ✅ Prompting is the first line of defence; LoRA fine‑tuning is applied only after a documented gap analysis shows prompting insufficient.
  • ✅ When LoRA is applied, a full regression suite on pre‑existing deterministic tests is run; behavior preservation is not assumed.

• Security & Auditing

  • ✅ Secrets stored in a vault; never hard‑coded.
  • ✅ Execution logs are immutable, encrypted, and retained per compliance window.

References

• Microsoft Retrieval‑Augmented Generation concepts – https://learn.microsoft.com/en-us/azure/foundry/concepts/retrieval-augmented-generation
• “LoRA: Low‑Rank Adaptation of Large Language Models” – https://arxiv.org/abs/2106.09685