State based vs Prompt Engineering

State based vs Prompt Engineering

There are two ways to control how your AI Agent behaves on a call: 1. Write one long system prompt that tries to cover every situation or 2. Build a flow with states that change the agent's focus as the conversation moves forward. This article explains why flows almost always win for anything beyond simple Q&A.

The Problem with One Big Prompt

A single prompt has to hold every rule the agent might ever need. "Greet the caller. Answer questions. If they want sales, collect their info. If they want support, route to the right person. If they ask about hours, look it up. If they ask to book, use Calendly. But only for existing customers. Except on weekends..." Every rule you add dilutes the earlier ones. At some point the model stops following them reliably.

In practice we see routing accuracy hover around 50% on prompt-only agents that have to branch to more than a handful of destinations. The agent gets it right sometimes and gets it wrong other times, and there is no clean lever to fix it.


What a Flow Changes

A flow splits the conversation into states. Each state is its own mini-prompt with its own set of tools. The agent only sees the instructions and tools relevant to where it is in the call.

  • Shorter context means the agent pays attention to fewer rules at once.
  • Fewer tools means fewer wrong choices. If the only tool in the "Collect Info" state is the info collector, the agent will not randomly decide to transfer.
  • Explicit transitions replace hoping the prompt covers every branch. You write the trigger once, and the agent follows it.

Real-world result on the same routing problem: moving from prompt-only to state-based took accuracy from ~50% to near 100%.

You are not replacing the prompt. Each state still has a prompt. You are replacing one giant prompt with several focused ones that only apply while the conversation is in that state. 

When a Single Prompt Is Fine

Keep it simple when the agent's job is simple. A front-desk agent that only answers questions from company info and transfers to one receptionist does not need multiple states. Put everything in the initial Greeting / Assisting state and move on.


When to Reach for a Flow

  • You have more than two or three places a caller might end up.
    • Example: A caller might need to go to Billing for billing assistance or Support for technical assistance.
  • You need to collect information before doing something (lead capture, booking, order intake).
    • Example: You need to get Callers name, Phone Number, and Email before booking an appointment.
  • You need to verify the caller before a sensitive action (reschedule, cancel, account lookup).
    • Example: You need to verify the caller's information to ensure appropriately rescheduling them. 
  • You have multiple integrations (calendar + POS + KB) and want the agent to use them at different points in the call.
    • Example: You utilize both Calendly and Service titan integrations to use depending on what the caller is inquiring about.
  • You are seeing the agent "freelance" — picking the wrong tool or skipping steps.
    • Example: The agent did not confirm information before transferring you as needed.

Rule of Thumb

If you are adding a sentence to the prompt that starts with "if the caller..." or "only do this when...", that is a candidate for a new state or transition.

The prompt handles how the agent talks. The flow handles what the agent is allowed to do and when.

Still have questions? Submit a ticket and our team will help you pick the right shape for your use case.

    • Related Articles

    • State Type: Error Handling

      The Error Handling state is the recovery branch of your flow — where the agent goes when something did not go as expected. A caller spoke a language the agent does not understand, a tool returned an error, a required field was not captured, an ...

    • State Type: Assisting

      The Assisting state is the agent's general-purpose "stay and help" state. It is the default conversational state — broad in scope, liberal with tools, and the place most calls spend the majority of their time. What It Does Assisting is what it sounds ...

    • State Type: Closing

      The Closing state is where the conversation ends. The agent says goodbye, performs any final wrap-up, and hangs up. It is the flow's exit — when the agent reaches a Closing state with no outbound transitions, the call is over. What It Does A Closing ...

    • Chats vs. Channels in OneCloud Connect | OneCloud Support Portal

      In OneCloud Connect, effective communication is key to productivity and collaboration. Two primary ways to communicate within the platform are through Chats and Channels. This article will help you understand the differences between these two ...

    • State Calling Hours | OneCloud Support Portal

      Many countries regulate the hours during which businesses are allowed to contact consumers by phone. In some countries, such calling hours may also be regulated at the state level. To be compliant with such regulations during your outbound campaigns, ...