Coding Agents: Understanding the Difference Between Conversations and Tasks

The development of coding agents has led to the realization that simply remembering conversations is not enough. These agents need to understand the task at hand to provide effective assistance. The initial version of a coding agent had a major bug: it could recall conversations but not the actual work being done. This limitation became apparent when the agent was required to perform real tasks, such as continuing a task from a mobile channel or approving a command.

The Importance of Task Identity

The issue arose from the fact that most agent systems are designed with a simple structure in mind: conversation, runtime session, and messages. This design works well for demos where the user performs one task at a time. However, it breaks down when the user engages in normal behavior, such as switching between different channels or asking about previous tasks. The system needs to be able to distinguish between a chat session and actual work.

Redesigning the Agent's Memory Model

To address this issue, a new design model was implemented: Person, Project, Task, Execution, and RuntimeSession. The key aspect of this model is the boundary between these components. A Task represents the user's objective, such as fixing routing or reviewing a code change. An Execution is a specific attempt to complete the task, and a RuntimeSession is the current process or provider session underlying the execution.

Benefits of the New Model

The new model allows the agent to understand the task's context, even if the runtime process changes. This enables the agent to provide more accurate and relevant responses to user queries. For example, when a user asks about the status of a task, the agent can provide an answer based on the task's state, rather than just the chat history. The new model also enables multiple agents to collaborate on the same task without conflicts.

Lessons Learned

The experience of developing the coding agent highlighted the importance of understanding the difference between conversations and tasks. The agent's ability to remember sessions is not enough; it needs to have a deeper understanding of the task at hand. By redesigning the agent's memory model, it is possible to create a more effective and efficient coding assistant. The takeaway is that agent memory should be focused on summarizing the task, rather than just the conversation.