Michael Flanagan

Posted on Dec 11

From Scrolls to Scripts: Orchestrating Agents with Chronicles

#aws #ai #typescript #learning

🏰 Embarking on the Next Chapter

In the previous installments of my AI adventure, we laid down the foundational stones of our grand project. We began by exploring the Documents—the core entities like Actors and Locations that populate our creative worlds. Then, we ventured into the realm of Agents, capable helpers designed to perform specific tasks and bring our documents to life.

However, these Agents, while powerful individually, lack a unifying narrative to coordinate their actions. This brings us to the next exciting phase: introducing Chronicles. Just as a storyteller weaves characters and events into a captivating narrative, Chronicles orchestrate our Agents, guiding them through complex workflows to achieve larger goals.

Quick Stop: Time for reality

As I said before, I come bearing gifts this is my repository for this project:
Codex of Agents

For some help to specifically get this initial version off the ground feel free to give this first step readme a read:
First Steps

Why Chronicles?

In AI and automation, managing individual tasks is just the beginning. To tackle more intricate challenges, we need a way to coordinate multiple Agents seamlessly. Chronicles serve as scripts that outline each Agent's role, the sequence of actions, and how data flows between them.

By orchestrating our Agents with Chronicles, we transform individual performances into a harmonious ensemble, where each member knows their part and contributes to the harmony of the whole.

Setting the Stage

In this post, we'll dive into the art of orchestrating Agents using Chronicles. We'll focus on a few illustrative examples to keep things clear and manageable. Through these scenarios, we'll see how Agents can be coordinated to perform complex tasks that would be challenging for them to handle individually.

Looking ahead, there's much more on the horizon. In the next set of paired posts, we'll explore:

🛠️ Building a More Resilient Serverless Architecture: We'll delve into designing an infrastructure that can gracefully handle the demands of our growing system, ensuring scalability and robustness.
📜 Expanding the Agent Codex: We'll work on building out our list of Agents, enriching our system's capabilities and paving the way for more sophisticated workflows.

But for now, let's focus on mastering the basics of orchestration with Chronicles. After all, every epic tale begins with a single chapter.

The Power of Names in Software Development

🌟 What's in a Name?
When building anything—be it a world for a tabletop game or a piece of software—the words we choose matter. Precise terminology isn't just about semantics; it's about clear communication and shared understanding.

Clarity and Communication

In coding, names are more than labels; they're guides that help us navigate complex systems. A well-chosen name illuminates purpose and intent, reducing confusion for anyone reading the code or documentation. Yet, finding the right names is surprisingly tough. It's a balancing act between technical accuracy and engaging descriptiveness.

Our Thematic Shift

✨ Embracing Storytelling Elements

To address these challenges, we've embraced storytelling elements in our project. Given my love for narratives, I've decided to infuse our work with a bit of fantasy flair. By adopting terms from storytelling, we're not just writing code—we're crafting an experience.

Introducing Our Cast

📜 Scribe: The master planner who crafts Chronicles—the blueprints that guide our Agents.
- 📖 Chronicles: Detailed plans outlining the steps our Agents will take to accomplish complex tasks.
  - 🔖 Chapters: Individual steps within a Chronicle, each representing a specific action.
🧭 Navigator: An Agent that adjusts Chronicles in response to changes.
🤖 Agents: Our trusty helpers who carry out the tasks detailed in the Chronicles.
🛠️ Implements: The tools our Agents use to perform their duties.

By framing our technical components within a storytelling context, we make the system more engaging and relatable. This consistent theme enhances understanding and fosters a stronger connection to the project.

(Note: The "Navigator" will be introduced in future posts when its role becomes relevant.)

To prevent confusion, I'll provide clear definitions whenever introducing a new concept going forward!

Understanding Orchestration in AI Systems

🔗 The Art of Coordination
In the realm of artificial intelligence, individual components can achieve impressive feats on their own. However, as tasks grow in complexity, these components need to work together harmoniously. This is where orchestration comes into play.

What is Orchestration?

Orchestration is the coordination of multiple Agents to achieve complex objectives that would be challenging for a single entity. It's about managing workflows, handling dependencies, and ensuring that each part of the system knows its role within the bigger picture.

Think of it as conducting an orchestra: each musician plays their instrument, but it's the conductor who ensures they all perform in harmony. Similarly, in AI systems, orchestration involves planning and overseeing the interactions between various Agents to execute a multifaceted task seamlessly.

Why Orchestration Matters

While single agents can perform specific tasks efficiently, they often hit limitations when confronted with more intricate challenges. Coordinating multiple agents allows us to:

Divide and Conquer: Break down complex goals into manageable, discrete tasks.
Leverage Specialization: Utilize the unique capabilities of different agents, each designed for specific functions.
Enhance Flexibility: Adapt to changing requirements by rearranging or swapping agents without disrupting the entire system.
Improve Reliability: Design systems that can handle failures gracefully by redistributing tasks among agents. In essence, orchestration is essential for building AI systems that are robust, scalable, and capable of tackling real-world problems with agility.

Designing Our Orchestration Strategy

🛠️ Crafting the Plan
With orchestration and our storytelling theme in place, let's delve into how we're implementing this strategy in our project.

Our Approach

At the heart of our strategy is the separation of planning and execution:

📜 Scribe: The master planner who composes Chronicles based on our goals.
🤖 Agents: Specialists who execute tasks specified in the Chronicles, using their Implements.

This division maintains a clean architecture where each component has a distinct responsibility, enhancing scalability and maintainability. The Scribe focuses on crafting adaptable high-level plans, while Agents concentrate on efficiently performing their designated tasks, guided by the detailed instructions within each Chapter.

Key Features of Chronicles

We're concentrating on the basics to keep things straightforward:

Chapters: Individual steps within a Chronicle, each representing a specific action.
Dependencies: Relationships between Chapters that determine execution order.

By defining clear dependencies, we control the flow of execution, ensuring that Agents perform their tasks in the correct sequence—crucial when certain actions rely on the results of previous steps.

Keeping It Simple

Our initial focus is on these core elements to establish a solid foundation. By mastering the fundamentals of orchestrating Agents with Chronicles, we set the stage for introducing more complex features down the line.

Looking Ahead

While we're starting with the essentials, there's plenty of room for growth. In future posts, we'll explore advanced capabilities such as:

Parallelism: Allowing multiple Chapters to be executed simultaneously when there are no dependencies between them.
Retries and Error Handling: Implementing mechanisms to handle failures gracefully and ensure the robustness of our system.
Dynamic Adjustments: Introducing the Navigator to adjust Chronicles in response to changes or new information.

By building upon the basics, we'll gradually enhance our system's capabilities without overwhelming complexity.

Architecture of Chronicles in Our System

🧩 Building the Blueprint
Let's delve into the architecture that brings our system to life, focusing on how Chronicles are structured and how they interact with our Agents and Implements.

Defining the Chronicle Structure

At the core of our system lies the Chronicle, the orchestrated plan that guides our Agents.

Components of a Chronicle

🔖 Chapters: Goal-driven actions assigned to specific Agents. Think of them as individual quests in our grand adventure.
🧩 Dependencies: Define the order and prerequisites for Chapter execution, maintaining logical flow.
⏳ Status Tracking: Both Chronicles and Chapters have statuses to indicate their progress through the workflow.

Data Structures

Here's how we represent Chronicles and Chapters in code:

type ChronicleStatus = 'created' | 'ready' | 'inProgress' | 'complete' | 'failed';
type ChapterStatus = 'created' | 'ready' | 'inProgress' | 'complete' | 'failed';

interface Chronicle {
  id: string;             // Unique identifier
  title: string;          // Human-readable title
  goal: string;           // Overarching objective
  status: ChronicleStatus; // Current state
  chapters: Chapter[];    // List of Chapters
}

interface Chapter {
  id: string;             // Unique identifier
  targetAgent: string;    // Responsible Agent
  goal: string;           // Specific objective
  status: ChapterStatus;  // Current state
  inputContext?: Record<string, any>;  // Required data
  outputContext?: Record<string, any>; // Resulting data
  dependencies?: string[];             // Prerequisites
}

Integration with Agents and Implements

📜 Scribe: Generates the Chronicle using LLM requests (e.g., AWS Bedrock's Converse API), transforming user goals into detailed plans.
🤖 Agents: Execute assigned Chapters, utilizing Implements to perform actions. They update the status and output context accordingly.
🛠️ Implements: Tools that empower Agents, such as database connectors or API clients, allowing Agents to focus on their tasks without worrying about underlying mechanics.

Integrating with Agents and Implements

Now that we have our Chronicles defined, let's see how they integrate with our Agents and Implements.

📜 Scribe

The Scribe plays a pivotal role in our architecture. Using Large Language Model (LLM) requests—specifically AWS Bedrock's Converse API calls—the Scribe generates the orchestrating Chronicles. It takes the user's goals and transforms them into a detailed plan that our Agents can execute.

Here's how the Scribe operates:

Input Processing: Receives a natural language prompt from the user or other input sources, which may include multiple tasks.
Chronicle Generation: Crafts a Chronicle with structured Chapters, assigning each to the appropriate Agent and defining goals, contexts, and dependencies.
Status Initialization: Sets the initial status of the Chronicle and its Chapters to created.
Preparation for Execution: Updates the status to ready once the Chronicle is fully defined and dependencies are mapped.

🤖 Agents

Agents are the executors of our Chronicles. Each Agent is responsible for one or more Chapters within a Chronicle. When an Agent receives a Chapter:

Reads the Goal: Understands the specific objective of the Chapter.
Checks Dependencies: Ensures all dependencies have been satisfied before proceeding.
Uses Implements: Leverages its tools (Implements) to perform the necessary actions.
Updates Status:
- Sets the Chapter status to inProgress at the start of execution.
- Upon completion, updates the status to complete and records any output in the outputContext.
- If an error occurs, updates the status to failed.
Output Sharing: The outputContext can be accessed by dependent Chapters, facilitating data flow between Agents.

🛠️ Implements

Implements are the tools that empower our Agents to perform their duties. They provide the capabilities required for executing each Agent's Chapters. Implements can include:

Database Connectors: For storing or retrieving data (e.g., DynamoDB operations).
API Clients: To interact with external services or APIs.
Utility Libraries: For tasks like data transformation, validation, or formatting.

By abstracting functionalities into Implements, we enable Agents to focus on their specific tasks without worrying about the underlying mechanics. This modularity enhances reusability and simplifies maintenance.

Example Workflow: Simple Chronicle Execution

🎭 Bringing It All Together
Now that we've laid out the architecture and roles within our system, it's time to see how everything works in practice. Let's walk through a simple yet illustrative example that showcases how the Scribe, Agents, and Implements collaborate to achieve a common goal.

Scenario Overview

Goal (from the user's prompt):

I need your help to create some new characters and a location for our game world. First, there's Elthien Pettlebloom, an elf ranger who loves the forest. She's an expert archer and is seeking revenge against the orcs who destroyed her village. Next, Thorgar Ironfist, a dwarf warrior. He's a master blacksmith, enjoys hearty ale, and is on a quest to find the legendary Hammer of Durin. Finally, we need the Misty Mountains, a treacherous mountain range filled with ancient mines, hidden dangers, and rumored to house a sleeping dragon.

Agents Involved:

📜 DocumentRequestHandlingScribe: Our Scribe specialized in handling document-related requests and ensuring data integrity.
🤖 ActorSubmissionAgent: Responsible for processing and submitting Actor documents.
🤖 LocationSubmissionAgent: Responsible for processing and submitting Location documents.
🤖 DocumentRelationshipMappingAgent: Handles mapping relationships within our relationship datastore. This Agent prefers to depend on the resolution of submission-related Agents to reduce executions.

Note: For those curious, I use the term Ancestry when referring to the types and origins of our characters—yes, I'm a big fan of Pathfinder 2nd Edition!

Our Scenario’s Chronicle

Upon receiving the user's prompt, the DocumentRequestHandlingScribe processes the input and generates the following Chronicle:

<chronicle>
  <chapter id="chapter-001">
    <targetAgent>ActorSubmissionAgent</targetAgent>
    <dependencies/>
    <goal>Process and submit the document for the character Elthien Pettlebloom.</goal>
    <context>
      <name>Elthien Pettlebloom</name>
      <ancestry>Elf</ancestry>
      <class>Ranger</class>
      <traits>
        <trait>Forest lover</trait>
        <trait>Expert archer</trait>
        <trait>Seeking revenge against orcs who destroyed her village</trait>
      </traits>
    </context>
  </chapter>
  <chapter id="chapter-002">
    <targetAgent>ActorSubmissionAgent</targetAgent>
    <dependencies/>
    <goal>Process and submit the document for the character Thorgar Ironfist.</goal>
    <context>
      <name>Thorgar Ironfist</name>
      <ancestry>Dwarf</ancestry>
      <class>Warrior</class>
      <traits>
        <trait>Master blacksmith</trait>
        <trait>Enjoys hearty ale</trait>
        <trait>On a quest to find the legendary Hammer of Durin</trait>
      </traits>
    </context>
  </chapter>
  <chapter id="chapter-003">
    <targetAgent>LocationSubmissionAgent</targetAgent>
    <dependencies/>
    <goal>Process and submit the document for the location Misty Mountains.</goal>
    <context>
      <name>Misty Mountains</name>
      <description>A treacherous mountain range filled with ancient mines, hidden dangers, and rumored to house a sleeping dragon.</description>
    </context>
  </chapter>
  <chapter id="chapter-004">
    <targetAgent>DocumentRelationshipMappingAgent</targetAgent>
    <dependencies>
      <dependency>chapter-001</dependency>
      <dependency>chapter-002</dependency>
      <dependency>chapter-003</dependency>
    </dependencies>
    <goal>Map relationships among the submitted characters and location in the relationship datastore.</goal>
    <context/>
  </chapter>
</chronicle>

Execution Flow

Let's step through how this Chronicle is executed within our system.

Scribe Generates the Chronicle
- The DocumentRequestHandlingScribe receives the user's prompt.
- Parses the prompt to identify entities (characters and location).
- Constructs the Chronicle with four Chapters, each assigned to the appropriate Agent.
- Sets the status of the Chronicle and all Chapters to "created".
Scribe Prepares the Chronicle for Execution
- Updates the status of the Chronicle to "ready".
- Sets the status of Chapters 1 to 3 to "ready" since they have no dependencies.
- Chapter 4 remains at "created" because it depends on the completion of the first three Chapters.
Agents Execute Chapters 1 to 3
- For each of Chapters 1 to 3:
- The assigned Agent retrieves the Chapter when its status is "ready".
- Updates the Chapter status to "inProgress".
- Agent Execution Pseudocode:

async function executeChapter(chapter: Chapter) {
  chapter.status = 'inProgress';
  try {
    const inputData = chapter.inputContext;

    // Augment the document using the style guide
    const augmentedData = applyStyleGuide(inputData);

    // Submit the document to DynamoDB (simulated)
    const documentId = await submitToDynamoDB(augmentedData);

    // Update the Chapter's output context with the document ID
    chapter.outputContext = { documentId };

    chapter.status = 'complete';
  } catch (error) {
    chapter.status = 'failed';
    // Handle error (e.g., log it, retry logic)
  }
}

For Chapters 1 and 2 (ActorSubmissionAgent):
- Processes the character data.
- Augments the data according to the style guide (e.g., ensuring required fields are present).
- Stores the Actor document in DynamoDB.
- Records the document ID in the outputContext.
For Chapter 3 (LocationSubmissionAgent):
- Processes the location data.
- Augments the data as per the style guide.
- Stores the Location document in DynamoDB.
- Records the document ID in the outputContext.
Scribe Updates Chapter 4 to Ready
- Once Chapters 1 to 3 are "complete", the Scribe (or a simple status-checking mechanism within the Scribe Lambda) updates Chapter 4's status to "ready".
DocumentRelationshipMappingAgent Executes Chapter 4
- The DocumentRelationshipMappingAgent retrieves Chapter 4.
- Updates the Chapter status to "inProgress".
- Agent Execution Pseudocode:

async function executeRelationshipMapping(chapter: Chapter, chronicle: Chronicle) {
  chapter.status = 'inProgress';
  try {
    // Gather outputs from dependencies
    const dependencyOutputs = chapter.dependencies?.map(depId => {
      const depChapter = chronicle.chapters.find(ch => ch.id === depId);
      return depChapter?.outputContext;
    });

    // Process relationships
    const relationships = mapRelationships(dependencyOutputs);

    // Optionally store relationships
    await storeRelationships(relationships);

    chapter.status = 'complete';
  } catch (error) {
    chapter.status = 'failed';
    // Handle error
  }
}

Functionality:
- Collects the document IDs from Chapters 1 to 3.
- Identifies potential relationships between the characters and the location.
- For example, associating Thorgar Ironfist with the Hammer of Durin if it exists in the datastore.
- Updates the relationship datastore accordingly.
- Marks the Chapter as "complete" upon success.
Completion of the Chronicle
- Once all Chapters are "complete", the Chronicle's status is updated to "complete".
- The workflow is successfully executed, and the new characters and location are now part of the game world, with relationships mapped.
- Code Snippets
- Here's a simplified version of how this might look within our Scribe Lambda Function:

export const handler = async (event: any) => {
  // Assume event contains the user's prompt  const userPrompt = event.userPrompt;
  // Step 1: Generate the Chronicle  const chronicle = generateChronicle(userPrompt);
  // Step 2: Prepare Chapters 1-3  chronicle.chapters.forEach(chapter => {
    if (!chapter.dependencies || chapter.dependencies.length === 0) {
      chapter.status = 'ready';
    }
  });
  // Step 3: Execute Chapters 1-3  for (const chapter of chronicle.chapters) {
    if (chapter.status === 'ready') {
      await executeChapter(chapter);
    }
  }
  // Step 4: Update Chapter 4 to Ready  const chapter4 = chronicle.chapters.find(ch => ch.id === 'chapter-004');
  if (chapter4 && chapter4.dependencies?.every(depId => {
    const depChapter = chronicle.chapters.find(ch => ch.id === depId);
    return depChapter?.status === 'complete';
  })) {
    chapter4.status = 'ready';
    await executeRelationshipMapping(chapter4, chronicle);
  }
  // Step 5: Update Chronicle Status  if (chronicle.chapters.every(ch => ch.status === 'complete')) {
    chronicle.status = 'complete';
  } else {
    chronicle.status = 'failed';
  }
  // Return the final status  return {
    statusCode: 200,
    body: JSON.stringify({
      message: 'Chronicle execution complete',
      chronicleStatus: chronicle.status,
    }),
  };
};

Notes on Execution

All execution occurs within the Scribe Lambda function, simplifying our architecture for this example.
Status Management: We use the status fields to control the flow of execution and handle dependencies.
Error Handling: If any Chapter fails, its status is set to "failed", and appropriate error handling can be implemented (e.g., retries, logging).
Data Flow: The outputContext of Chapters allows us to pass data between Agents, crucial for dependent tasks like relationship mapping.

Implements in Action

applyStyleGuide: A utility function (Implement) used by Agents to ensure that the data adheres to our predefined style guidelines.
submitToDynamoDB: An Implement that abstracts the process of storing documents in DynamoDB.
mapRelationships: Used by the DocumentRelationshipMappingAgent to establish connections between entities.
storeRelationships: An Implement for persisting relationship data.

Outcome

By the end of this workflow:

Elthien Pettlebloom and Thorgar Ironfist are added to our game world's database as new characters.
The Misty Mountains becomes part of our location data.
Relationships among the characters and location are established, enriching the game's narrative fabric.

This example demonstrates how our system orchestrates multiple Agents to achieve a complex goal, all within a single execution flow. It showcases the power of the Chronicle structure, the coordination of Agents, and the utility of Implements.

In the next section, we'll reflect on how this workflow highlights the strengths of our orchestration strategy and discuss potential enhancements for future iterations. The journey doesn't end here—there are many more adventures to embark upon as we continue to build and refine our system.

Highlighting the Potential of Chronicles

🌟 Unleashing the Power of Coordination
By now, we've seen how Chronicles serve as the orchestrators of our Agents, guiding them through complex tasks with ease. But what makes Chronicles truly shine in our system? Let's delve into the benefits that Chronicles bring to the table and how they set the stage for future enhancements.

Benefits of Using Chronicles

Scalability

One of the standout advantages of utilizing Chronicles is the inherent scalability they offer.

Effortless Expansion: Need to add a new feature or incorporate another Agent? With Chronicles, it's as simple as writing a new Chapter and assigning it to the appropriate Agent.
Parallel Development: Multiple Chronicles can be crafted and executed simultaneously, allowing different teams or Agents to work on various aspects of the project without stepping on each other's toes.

Maintainability

A well-structured Chronicle makes maintaining and updating our system a breeze.

Clear Structure: Each Chapter within a Chronicle is a self-contained unit with a specific goal, making it easier to identify where changes need to be made.
Simplified Debugging: If something goes awry, the status tracking within Chronicles and Chapters helps pinpoint exactly where the issue occurred, reducing downtime and frustration.

Flexibility

Perhaps one of the most exciting aspects is the flexibility that Chronicles introduce.

Adaptive Agents: Agents can interpret and execute Chapters based on the context provided, allowing for dynamic responses to varying inputs or situations.
Easy Reconfiguration: Want to change the flow of tasks? Adjusting dependencies or reordering Chapters in a Chronicle allows us to reshape the workflow without extensive rewrites.

Setting the Stage for Future Enhancements

Our journey doesn't stop here. Chronicles open up a world of possibilities for more sophisticated workflows and integrations.

Complex Workflows

Intricate Dependencies: Future Chronicles can incorporate more complex dependency structures, enabling the modeling of intricate processes that mirror real-world scenarios.
Conditional Logic: Introducing conditions within Chapters can allow Agents to make decisions on the fly, further enhancing the system's responsiveness.

Integration with the Agent Codex

Expanding the Roster: As we build out our Agent Codex, we'll introduce new Agents with specialized skills, broadening the system's capabilities.
Enhanced Functionality: With a richer set of Agents and Implements, Chronicles can orchestrate more diverse and powerful workflows, from data analytics to real-time decision-making.

Next Steps

🚀 Charting the Path Forward
With the foundational pieces in place, it's time to look ahead at what's on the horizon for our project. The journey of building this system is akin to an epic tale, and the next chapters promise to be just as thrilling.

Exploring Resilient Serverless Architecture

In the upcoming posts, we'll dive into designing a more robust serverless architecture.

Scalability at Scale: We'll explore how to ensure our system can handle increased load without breaking a sweat.
State Management: Managing state in a distributed, serverless environment presents unique challenges. We'll tackle these head-on, implementing solutions that keep our system responsive and reliable.

Building Out the Agent Codex

Our Agents are the heart of the system, and it's time to expand their ranks.

New Agents and Implements: We'll introduce a host of new Agents, each bringing unique capabilities to the table. From data processors to AI-driven decision-makers, the possibilities are vast.
Enhanced Flexibility: With more Agents at our disposal, Chronicles can orchestrate increasingly complex tasks, opening doors to functionalities we haven't yet imagined.

By focusing on these areas, we're setting ourselves up for a future where our system isn't just functional—it's exceptional.

Conclusion

📝 Wrapping Up Our Journey
As we close this chapter, let's reflect on the ground we've covered and look forward to the adventures yet to come.

Recap of Key Takeaways

The Power of Chronicles: We've seen how Chronicles serve as the orchestrators, uniting our Agents to achieve complex goals efficiently.
Thoughtful Naming Matters: By embracing storytelling elements, we've made our system more engaging and easier to understand, demonstrating that clarity and creativity can go hand in hand.
Simplicity in Action: Even a simple Chronicle can coordinate multiple Agents to perform tasks that would be challenging to handle individually.

Invitation for Engagement

I encourage you to explore the repository and experiment with the code. Whether you're a seasoned developer or just starting out, there's a place for you in this journey.

Try It Out: Clone the repo, run the examples, and see how Chronicles and Agents work together firsthand.
Share Your Thoughts: Your feedback is invaluable. Feel free to open issues, suggest enhancements, or share your experiences.

Looking Forward

The story doesn't end here—it's just beginning. Stay tuned for our next installment, where we'll dive into advanced orchestration mechanisms and expand our Agent Codex, setting the stage for even more ambitious projects.

References (The Big One)

GitHub Repository: Codex of Agents