AI Orchestration and Automation (feat. Andrew Brooks)

In this conversation, Krish Palaniappan interviews Andrew Brooks, co-founder of Contextual, an AI orchestration platform. The conversation covers Andrew's background in technology and entrepreneurship, the evolution of Contextual, and its unique offerings in AI workflow automation. The episode includes a live demonstration of the platform, showcasing its features, user interface, and capabilities for integrating AI solutions into business processes. Key topics include the importance of AI orchestration, the role of developers in utilizing the platform, and best practices for user management and security. In this conversation, Andrew Brooks and Krish Palaniappan discuss the functionalities and features of Contextual, focusing on managing permissions, API access, usage-based pricing, and the role of services. They explore the mapping of services to tenants, promoting services across environments, and the importance of atomic services. The discussion also covers navigating the Contextual interface, getting started with the Hello AI World demo, and understanding documentation and instructions. They highlight the significance of reusing flows, interacting with agents, and visualizing real-time data flow. The conversation concludes with insights on uptime, reliability, and the flow editor's capabilities. In this conversation, Andrew Brooks and Krish Palaniappan delve into the intricacies of data flow in AI applications, focusing on the differences between event and HTTP flows, the importance of designing efficient flows for scalability, and the seamless integration of third-party APIs. They also discuss debugging techniques, the mission of Contextual, and the value proposition of visual orchestration in making AI accessible to developers with basic skills.

Takeaways

Andrew Brooks has a rich background in technology and entrepreneurship.
Contextual focuses on AI orchestration and workflow automation.
The platform is designed to be user-friendly for developers.
AI solutions can significantly enhance business processes.
Contextual allows for custom solutions tailored to specific organizational needs.
The platform supports integration with various third-party services.
User management and security are critical components of the platform.
Developers can create and manipulate object types and flows easily.
The visual editor simplifies the development process for AI solutions.
Contextual's API allows for seamless interaction with external systems. Managing permissions is crucial to prevent unauthorized access.
Contextual offers a usage-based pricing model for flexibility.
Services in Contextual package components for deployment.
Understanding tenant mapping is essential for service management.
Promoting services can be done selectively across environments.
Atomic services allow for independent promotion of components.
Navigating the Contextual interface is user-friendly and intuitive.
The Hello AI World demo is a great starting point for new users.
Documentation is vital, but many developers prefer hands-on exploration.
Exporting and importing flows as JSON enhances collaboration. Data flows through nodes in AI applications.
Understanding the difference between event and HTTP flows is crucial.
Designing flows for scalability can enhance efficiency.
Debugging is essential for maintaining flow integrity.
Integrating third-party APIs can simplify complex workflows.
Visual orchestration makes AI tools accessible to developers.
Usage-based pricing aligns costs with value delivered.
Multiple flows can be organized into tabs for clarity.
Splits and joins within flows allow for parallel processing.
Contextual aims to simplify AI integration for users.

Chapters

00:00 Introduction to Andrew Brooks and Contextual
08:44 Exploring Contextual's AI Orchestration Platform
15:30 Understanding AI Workflows and Use Cases
24:18 Live Demo: Navigating Contextual's Interface
38:00 Creating Object Types and API Integration
58:43 User Management and Security Features
01:07:44 Managing Permissions and API Access
01:16:39 The Importance of Atomic Services
01:22:19 Understanding Instructions and Documentation
01:30:05 Real-Time Data Flow Visualization
01:38:02 Uptime and Reliability of Services
01:51:59 Customizing the Flow Editor
02:05:07 Understanding Data Flow in AI Applications
02:12:13 Navigating Event and HTTP Flows
02:19:20 Designing Efficient Flows for Scalability
02:28:24 Integrating Third-Party APIs Seamlessly
02:37:29 Debugging and Monitoring Flows
02:49:53 Contextual's Mission and Value Proposition

Podcast

Check out on Spotify.

Summary

1. Decision on Tool/Template Selection

Krish and Andrew discuss which template or tool to focus on for their project. While Krish considers a "data warehouse" option, Andrew suggests diving into one of the AI tools, specifically the OpenAI chatbot with functions, because of its applicability across many solutions.

2. Overview of OpenAI Assistant and its Functions

Andrew explains how the OpenAI Assistant operates, highlighting key features like custom instructions, data vectorization for retrieval-augmented responses, and the use of function calls. He elaborates on how these assistants differ from traditional LLMs, offering developers pre-coded solutions and flexibility for enterprise deployment.

3. Clarification on RAG (Retrieval-Augmented Generation)

Krish restates his understanding of RAG, emphasizing its relevance for his company's AI solutions. He seeks confirmation from Andrew, who validates his explanation and adds more detail on how RAG enables LLMs to work with private, custom data. Andrew provides additional context on how businesses can leverage RAG to integrate their own data into AI workflows.

4. Example of RAG Application in Business Context

Andrew provides a real-world use case where a client uses RAG to match musical artists with brands based on regional popularity. This illustrates how RAG can combine external AI training data with a company's proprietary data to produce customized responses.

5. Lighthearted Discussion on Brand Promotion and Taylor Swift

Krish humorously suggests that Taylor Swift is the default choice for brand promotion, and Andrew jokingly agrees but notes that many artists can serve similar roles, albeit with different budgets.

6. Setting Up a Tenant in Contextual

Krish describes setting up a tenant in Contextual’s system. He checks the availability of his chosen tenant name and confirms how naming conventions work across the platform. Andrew provides clarification on platform-wide naming uniqueness and gives Krish feedback on the successful setup.

7. Exploring the Contextual Interface

Krish navigates through the Contextual platform, exploring different options such as services, catalog, and components. He asks Andrew about the ideal role for managing these tools in a startup context, and Andrew clarifies that developers typically handle these tasks, but it's part of a broader vision crafted by business analysts or architects.

8. Clarifying Terminology: Plugins vs. Templates

Krish asks about the proper terminology for what they’re using. Andrew explains that "template" is the correct term rather than "plugin." These templates are created by Contextual's team to help developers understand how to assemble the various components for their projects. While the templates are currently created in-house, there’s potential for external contributions in the future.

9. Understanding Tenants and Environments

Krish speculates on how tenants and environments (such as dev, prod, etc.) are managed within the Contextual platform. He envisions how he might organize them across multiple projects and clients. Andrew confirms that the multi-tenant structure allows for this and explains how tenants are listed in a dropdown menu for easy management.

10. Exploring Components and Their Connection to Catalog Items

Krish continues navigating through the platform and speculates that once templates from the catalog are selected, they will populate various component sections like object types, flows, agents, and connections. Andrew confirms this understanding, agreeing that the catalog-driven approach will lead to filling out those areas as the user works with the templates.