In order to process user requests and provide responses in natural language, every agent, including Ed, leverage large language models (LLMs). Every action in an agent flow triggers an individual prompt sent to the LLM, and results in a response from the LLM.
When you create agents, skills. and tools, you provide context for the LLM to accurately generate the necessary information required to complete the user's request in the platform. For example:
- An agent's persona provides the LLM with the context necessary to determine which of its tools to execute.
- An agent's skills provide the LLM with instructions that guide how it approaches and executes complex tasks, and define how results should be formatted and returned. Skills are incorporated into the agent's prompt before any tools are executed, meaning they directly influence the order and approach of the agent flow.
- The description of an input field on a logic block tool provides the LLM with the context necessary to extract the correct input value from the user's request.
Because skills are incorporated into the agent's system prompt at runtime, they also make agent flows more predictable and consistent. Without skills, the agent relies solely on its own reasoning to determine how to fulfill a request, which can vary depending on the underlying AI model. Skills ensure the agent handles requests and formats responses the same way every time, regardless of any underlying changes to the AI model.
The agent flow is determined in part by the Thought Reasoning that each agent is equipped with. There are different types of Thought Reasoning allows agents to perform multiple actions in a logical order. Learn more in Agent thought reasoning.
You can examine agent flows in detail with agent traces, which allow you to view each action in an agent flow, as well as the complete requests to and responses from the LLM. Learn more in Agent traces.
Agent traces
Examine an agent flow in detail with agent traces. Each trace is sequential, and includes all the agents accessed and tools executed as part of the agent flow. Additionally, traces include the complete requests made to the LLM, and its responses.
During an agent flow, each agent that's called upon and each tool that's executed results in an individual request to an LLM. For example, when Ed calls upon an agent on behalf of a user, the prompt that's sent to the LLM that contains the user's request, the agent's persona, descriptions of related agents, and descriptions of the agent's tools. Using this information, the LLM responds with instructions to either call a related agent or execute a tool, and the agent flow continues.
When you view these LLM inputs and outputs in an agent trace, you can better understand how your agents and tools perform in context of one another. This allows you to troubleshoot any unexpected behavior in the agent flow and make modifications to improve the performance of your agents.
For example, you might create an agent with the capability to display the contact information of your suppliers. The agent has a logic block tool that uses a semantic search configuration to determine the correct supplier, and a data retrieval tool to provide their phone number and email address.
However, when you test your agent, it's repeatedly unable to successfully retrieve the supplier's contact information. When you examine the agent trace, you determine that the logic block tool isn't being executed, which means that the data retrieval tool cannot work properly. With this information, you can modify the Agent Persona and the Agent Description Prompt of the agent's tools to indicate that the logic block tool should be executed first.
Learn more about accessing agent traces in the Agent Conversations application topic.
Agent Conversation Inquiry application
Use the Agent Conversation Inquiry application to review your conversations with Ed and other agents, including scheduled agent conversations, and to debug them and view their agent traces.
Every conversation you've had in the chat interface is stored. You can use the Agent Conversation Inquiry application to:
- View conversations with agents.
- View scheduled agent conversations.
- View a conversation similar to how you would see it in Ed.
- Debug a conversation if there are issues.
- View the agent traces for a conversation.
- Continue a conversation in Ed.
Browse conversations
Each conversation in the list shows its status, name, agent, type, user, and date. The agent list shows how many conversations each agent has, with a count for any that need input, are running, or have errors. Select an agent to see only its conversations. Select All Agents to see everything.
Use the status buttons above the list, Needs input, Running, Idle, and Error, to narrow the list by status. Each button shows how many conversations have that status. Two of the statuses mean:
Select Filters to filter by conversation, agent, type, user, and date range.
Run actions
Select the Run Action menu at the end of a conversation to open record options. Depending on the state of the conversation and the type of agent, you have different options available, such as:
- Run Now—Run a scheduled agent immediately.
- Open in Ed—Open Ed where you can debug the conversation and more.
- Resume—Resume running an agent.
View and continue a conversation
Open a conversation record to view additional information. To move the conversation from the panel into Ed and pick up where it left off, select the Continue in Ed button.
Select the form action menu next to Continue in Ed to access additional options, such as:
- Debug in Ed—Investigate a problem with the conversation.
- View Traces—View the agent traces for the conversation.
Agent traces and debugging
To see the agent traces for a conversation, select View Traces from the form action menu. A trace shows each step the agent took, the skills and tools it used, and any errors along the way. Application developers who are designing their own intelligent agents can use agent traces to examine agent flows in detail. Learn more in Agent traces.
There may be multiple agent traces in a single conversation, because each agent trace includes the details of a single agent flow. An agent flow starts when Ed, or another agent, receives a request, and ends when that request is satisfied. During a conversation with Ed, a user can make a series of requests, resulting in multiple agent flows. If an agent needs additional information from the user to satisfy a request, the additional message is still part of the original agent flow and is captured in the same agent trace.
Learn more about interpreting agent traces in the Agent Traces application topic.
Agent Traces application
Application developers use the Agent Traces application to access the details of individual agent flows.
Each record in the agent trace represents a single action performed throughout the agent flow. By default, each action is in order.
You can get a sense of the agent flow by examining the Trace Type of each action. In complex flows, you'll likely see many actions of type Agent and Tool, allowing you to determine if the agents called and tools executed were what you expected based on the the request.
Additionally, you will likely see multiple actions of type Language Model. Each time an agent is called or a tool is executed, an individual prompt is sent to an LLM. Examine the details of these Language Model actions to view the complete JSON data of the request, and of the model's response. This allows you to examine your related agents and tools in context of one another. Learn more in Agent traces.
Agent Conversations application
Use the Agent Conversations application to access the conversations you've had with Ed and other agents in the chat interface, and to access in-depth agent traces.
Every conversation you've had in the chat interface is stored, and the Conversation Name matches what you see in the chat history dropdown. Open a record in the detail form to access the complete conversation. By default, you can only access your own conversations.
Conversations you have with Ed have a Agent Conversation Type of Conversational. However, some agents run in the background. These agents may require additional input from you in the chat interface. These agents have a Agent Conversation Type of Automation.
Access agent traces
Application developers who are designing their own intelligent agents can access agent traces to examine agent flows in detail. Learn more in Agent traces.
In the detail form of a conversation, use the View Agent Traces row action on a message to view the details of each agent flow. This action only appears on the first message of an agent flow with a Message Type of User.
There may be multiple agent traces in a single conversation, as each agent trace includes the details of a single agent flow. An agent flow starts when Ed, or another agent, receives a request, and ends when that request is satisfied. During a conversation with Ed, a user can make a series of requests, resulting in multiple agent flows.
However, an agent may need additional information from the user in order to satisfy the request. Even though this results in an additional User message, it is still part of the original agent flow and is captured in the same agent trace.
Learn more about interpreting agent traces in the Agent Traces application topic.
Agent thought reasoning
All agents are equipped with Thought Reasoning, which allows them to execute multiple tools and call upon their related agents in a logical order to complete multiple tasks based on a single user prompt.
Each time a user sends a request to Ed, and each time his agents are accessed, a unique prompt is sent to the LLM. The external providers of the LLMs do not use any of these prompts to train their models. In order for Thought Reasoning to work, the responses returned by the LLM must be stored in the platform for upcoming agents and tools to access.
To provide this context for thought reasoning, the logic block associated with each of the agent's logic block type tools should typically produce an output in natural language that indicates the result of the logic block's execution, as well as the values produced by the logic block. The agent that executed the tool uses this output to determine whether to execute another tool, call on another agent, or respond to the user.
For example, the AppBuilderAgent can create a standard application. It's equipped with tools to create data items, include them in a table, and create an application over that table. However, these tools must be executed in a specific sequence in order to create the application, since these different metadata objects build upon one another.
When the AppBuilderAgent is tasked with creating a Movie Collection application, it first uses its data item tool to create the fields necessary for the application, such as MovieTitle and ReleaseDate. When this tool is executed, the associated logic block produces an output in natural language which indicates that these data items were created. When the AppBuilderAgent is called again to determine which tool to execute next, the prompt that is sent to LLM automatically includes the output from the first tool. This context enables the LLM to know that it should tell the AppBuilderAgent to execute its table creation tool to using those data items next.
There are multiple types of Thought Reasoning, including:
Chain of Thought—Allows an agent to perform an action, either by executing a tool or calling another agent, and then evaluate the results of that action to determine whether another is necessary to satisfy the user prompt. After the agent determines the user's prompt is satisfied, a response is sent and the chain of thought concludes. This is the default form of thought reasoning.Single Selection—Allows an agent to either execute a tool or call on another agent. Only one action occurs each time the agent is called.Directed—Allows an agent to execute its tools in the order you define. Learn more in Agent Toolset Builder application and Agent Builder application.