Agentic AI Workflow

One of the most important realizations in modern AI engineering is this:

Better workflows often outperform better models.

This sounds counterintuitive at first.

Most of the AI industry has been conditioned to think progress comes primarily from:

• Larger models
• More parameters
• Larger context windows
• More training data

But empirical evidence increasingly shows that workflow architecture can matter just as much, and sometimes more.

For example:

Model	Workflow	Accuracy
GPT-3.5	Direct Generation	~40%
GPT-4	Direct Generation	~67%

The jump from GPT-3.5 to GPT-4 is enormous.

But something even more interesting happens when we introduce an agentic workflow around the weaker model.

Why Agentic Workflows Beat Bigger Models

"A weaker model with iteration can often outperform a stronger model without iteration."

This is one of the most important ideas in Agentic AI.

A non-agentic workflow looks like this:

flowchart LR
    Problem --> LLM --> Code 

Instead of generating code once

Agentic workflow becomes:

graph TD
    A[Draft Output] --> B[Critique]
    B --> C[Identify Weaknesses]
    C --> D[Revise]
    D --> A

Now the model is no longer performing:

• single-pass generation

It is performing:

• iterative problem solving.

This changes everything.

The Power of Reflection Loops

A reflection loop enables the model to critique and improve its own output.

Conceptually:

$$Solution_{t+1} = Improve(Solution_t, Feedback_t)$$

This resembles iterative optimization systems found throughout computer science:

• gradient descent
• evolutionary search
• Monte Carlo refinement
• compiler optimization passes

reasoning quality compounds across iterations.

---

Agentic Reasoning Strategies

Reasoning strategies determine how an AI agent thinks, plans, and decides actions to achieve a goal.

mindmap
  root((🧠 Agentic Reasoning))
    Conditional Logic 🔀
    Heuristics 🎯
    ReAct 🔄
    ReWOO 📜
    Self-Reflection 🪞
   Multi-Agent Reasoning  🤝 

When to use what

Scenario	Best Strategy
Fixed approval workflow	Conditional Logic
Fast approximate decision	Heuristics
Tool-using assistant	ReAct
Pre-planned execution pipeline	ReWOO
Improve answer quality	Self-Reflection
Complex collaborative task	Multi-Agent Reasoning

Cost vs Complexity

Strategy	Adaptive	Tool Usage	Cost	Complexity
Conditional Logic	❌	Limited	Low	Low
Heuristics	⚠️ Partial	Limited	Low	Low
ReAct	✅	High	Medium	Medium
ReWOO	⚠️ Limited	High	Low-Medium	Medium
Self-Reflection	✅	Optional	High	Medium
Multi-Agent	✅	High	High	High

Strategies Overview

1. Conditional Logic 🔀 : Flow based

Rule-based decision making using predefined conditions.

Designing AI gents like CI/CD Pipelines

Flow

graph TD
    A[Input] --> B{Condition?}

    B -->|Yes| C[Action A]
    B -->|No| D[Action B]

Advantages

• Fast
• Predictable
• Explainable
• Easy to test

Disadvantages

• Rigid
• Poor adaptability
• Doesn't generalize well

Best For

• Business workflows
• Approval systems
• Compliance checks

Example

IF payment_failed
THEN notify_support

IF order_completed
THEN send_confirmation

---

2. Heuristics 🎯 : Goal Based

Uses rules of thumb or experience-based shortcuts.

Like goal-based agents, utility-based agents search for action sequences that achieve a goal

But they factor in utility as well. They employ a utility function to determine the most optimal outcome.

Flow

graph TD
    A[Problem]
    B[Apply Heuristic 🎯]
    C[Likely Good Solution ⚖️]

    A --> B --> C

Advantages

• Fast decisions
• Low computational cost
• Works well in uncertain environments

Disadvantages

• Not always optimal
• Can introduce bias

Best For

• Scheduling
• Resource allocation
• Recommendation systems

Example

Drive me to Home

Search through different routes and recommend the fastest 1.

Drive with fastest route

---

3. ReAct (Reason + Act) 🔄 : Dynamic Tool use

Alternates between reasoning and tool usage.

Combine Chain of Thoughts CoT prompt with tooling

• Thought — free-form reasoning in natural language about what to do next
• Action — a structured call to an external tool (search, calculator, code executor, API)
• Observation — the tool's return value, fed back into the context

Flow

graph TD
    A["Question ❓"]
    B["Thought 🤔"]
    C["Action :: Tool 🧰"]
    D["Observation 👀"]

    A --> B
    B --> C
    C --> D
    D --> B

Key Failure

Thought loops

the model can get stuck reasoning without committing to an action, especially if the prompt doesn't enforce the Thought/Action format strictly.

Observation hallucination

If the tool returns a noisy or ambiguous result, the next thought may misinterpret it rather than re-querying.

Context length

In long tasks, early observations get pushed out of the window, causing the model to lose track of earlier sub-goals.

Action space design

The quality of ReAct is heavily dependent on what tools are available and how well their interfaces are described in the prompt.

Ending Tool Call Loop

1. Max retries

Set maximum number of loop iterations to limit latency, costs and token usage, and avoid the possibility of an endless loop.

2. End Condition

When some specific condition is met, such as when the model has identified a potential final answer that exceeds a certain confidence threshold.

Advantages

• Dynamic
• Handles unknown situations
• Works well with tools

Disadvantages

• More LLM calls
• Higher latency
• Higher cost

Best For

• Tool-using agents
• Research agents
• Customer assistants

Example

Question:
What's the weather in Munich?

Thought:
Need weather data.

Action:
Call Weather API.

Observation:
24°C.

Thought:
Generate answer.

---

4. ReWOO 📜 (Reasoning Without Observation) : Fixed Plan based

The planner decomposes tasks, while the executor handles execution and tool calls.

• Plan does not change mid-execution so failure in a step can lead to complete failure

Planner-Executor Pattern

graph TD
    A[Goal] --> B[Planner]
    B --> C[Task Queue]
    C --> D[Executor]
    D --> E[Tool Calls]
    E --> F[Results]
    

Advantages

• Fewer LLM calls
• Lower cost
• More efficient

Disadvantages

• Cannot adapt mid-execution
• Plan may become invalid

Best For

• Predictable workflows
• Data collection pipelines
• Structured tasks

Example

Goal:
Compare Tesla and BMW stock performance.

Plan:
1. Fetch Tesla stock
2. Fetch BMW stock
3. Compare returns
4. Summarize findings

Execute all steps.

5. 🪞 Self-Reflection

Agent evaluates and improves its own output.

Flow

graph TD
    A[Generate Answer]
    B[Self Review]
    C{Good Enough?}

    A --> B
    B -->C
    D --> B

    C -->|No| D[Revise]
    C -->|Yes| E[Final Answer]

Why Iterative Systems Work Better

Single-shot prompting forces the model to solve the entire problem within one reasoning trajectory.

That creates several limitations:

• early mistakes propagate
• hallucinations remain uncorrected
• missing information cannot be recovered
• no verification occurs

Agentic systems introduce feedback cycles.

Instead of:

$$Output = f(Prompt)$$

we now have:

$$State_{t+1} = f(State_t, Memory_t, Feedback_t, Tools_t)$$

The system continuously improves its internal state.

Advantages

• Higher accuracy
• Better quality
• Reduces hallucinations

Disadvantages

• Slower
• More token usage
• Higher cost

Best For

• Code generation
• Writing assistants
• High-stakes decisions

Example

Answer Generated

Review:
Did I answer the question?
Are facts correct?
Is information missing?

Improve Answer

---

6. 🤝 Multi-Agent Reasoning

Multiple specialized agents collaborate to solve problems.

Supervisor Agent

Coordinate work across multiple sub agents

Worker Agents

Different agents specialize in different domains.

• Research agent
• Coding agent
• Reviewer agent

Flow

graph LR
    A[Supervisor Agent]

    A --> B[Research Agent]
    A --> C[Coding Agent]
    A --> D[Validation Agent]

    B --> E[Shared Knowledge]
    C --> E
    D --> E

    E --> A

Advantages

• Specialization
• Better scalability
• Handles complex tasks

Disadvantages

• Coordination overhead
• Higher infrastructure cost
• More complex debugging

Best For

• Enterprise AI systems
• Software development agents
• Autonomous workflows

Example

Research Agent
→ Finds information

Coding Agent
→ Builds solution

Validation Agent
→ Checks correctness

Supervisor
→ Produces final answer

---

👤 Human-in-the-Loop Systems (HITL)

Manual oversight remains crucial for high-stakes applications.

It is an oversight and governance pattern that can be inserted into almost any agent architecture.

Many production systems insert approval checkpoints for:

• compliance
• legal review
• financial decisions
• healthcare workflows

This hybrid architecture is usually more practical and safer.

graph TD
    A[User Request]
    B[Agent Reasoning]
    C[Proposed Decision]

    D{High Risk?}

    E[Human Approval 👤]
    F[Execute Action]

    A --> B
    B --> C
    C --> D

    D -->|Yes| E
    E --> F

    D -->|No| F

Example:

• Agent Create PR
• Developer Review and Approve PR

Using lang graph to add HITL

from typing import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver
from IPython.display import Image, display


class State(TypedDict):
    input: str


def step_1(state):
    print("---Step 1---")
    pass


def step_2(state):
    print("---Step 2---")
    pass


def step_3(state):
    print("---Step 3---")
    pass


builder = StateGraph(State)
builder.add_node("step_1", step_1)
builder.add_node("step_2", step_2)
builder.add_node("step_3", step_3)
builder.add_edge(START, "step_1")
builder.add_edge("step_1", "step_2")
builder.add_edge("step_2", "step_3")
builder.add_edge("step_3", END)

# Set up memory
memory = MemorySaver()

# Add
graph = builder.compile(checkpointer=memory, interrupt_before=["step_3"])

with step 3 we now need Human approval

user_approval = input("Do you want to go to Step 3? (yes/no): ")

flowchart TD

    User["User  👤"] --> AI[AI System]

    AI -->|Provides Output| User

    User -->|Feedback| AI

    AI -->|Improves Model| UpdatedAI[Updated AI System]

---

Final Thoughts

Agentic AI Is a Systems Engineering Problem

The future of AI engineering is increasingly shifting from:

"How do I write a better prompt?"

to:

"How do I design a better reasoning workflow?"

That is a profound transition.

The competitive advantage is no longer just:

• larger models
• larger context windows
• better prompting

It increasingly comes from:

• orchestration quality
• retrieval strategy
• evaluation loops
• workflow architecture
• system reliability

The most powerful AI applications over the next few years will likely not be single models.

They will be coordinated cognitive systems.

And that changes the role of software engineering entirely.

The Real Engineering Challenge

Most people think Agentic AI is primarily about prompting.

In practice, the hardest problems are:

• orchestration
• state management
• memory handling
• tool reliability
• retry logic
• evaluation
• cost optimization
• latency control
• permission boundaries

Prompt engineering becomes only one layer of a much larger system.