Building an Embed SDK for AI-Powered Workflows

Originally published at orquesta.live/blog/building-embed-sdk-for-ai-powered-workflows

Integrating complex AI workflows directly into web applications has become essential for SaaS products looking to bring AI-powered operations closer to their users. At Orquesta, we developed an Embed SDK that allows any web app to incorporate our AI-driven workflow engine with nothing more than a single script tag. Here's how we built it, the architectural choices we made, and the features that enable real-time AI operations.

Designing the Embed SDK

Creating an SDK that integrates seamlessly into any existing web app requires careful consideration of multiple factors, especially when dealing with AI operations that need real-time updates and secure execution.

Simplifying Integration

Our primary goal was to minimize the friction involved in embedding Orquesta. We aimed for a single line of code: a script tag. This would ensure that developers can quickly add our platform to their applications without overhauling existing systems.

<script src="https://cdn.orquesta.live/embed-sdk.js"></script>

The script tag loads our SDK, which is designed to automatically initialize and hook into the host application. Behind this simplicity lies a robust framework that handles user authentication, UI component rendering, and real-time interaction with our AI agents.

Architecture Decisions

Modular Design

We opted for a modular architecture, allowing developers to selectively include only the components they need. This approach reduces the footprint of our SDK and ensures that unnecessary functionalities do not bloat the host application.

Authentication and Security

Security is crucial when embedding AI operations. We implemented an authentication flow based on OAuth 2.0, integrating seamlessly with existing user authentication systems. The SDK includes a lightweight client that handles token management and session renewal, ensuring secure interactions with the Orquesta platform.

Real-Time AI Operations

Live Terminal Streaming

One of the standout features of Orquesta is the ability to execute AI-powered workflows in real-time, with live streaming of terminal output. This required us to build a robust WebSocket infrastructure within the SDK to handle bi-directional communication efficiently.

const socket = new WebSocket('wss://api.orquesta.live/stream');
socket.onmessage = (event) => {
  console.log('Real-time data:', event.data);
};

This setup allows developers to watch AI operations unfold in their web applications, providing transparency and immediate feedback.

Agent Grid Monitoring

Another key feature is the Agent Grid, which provides a live view of multiple AI agents. We built this using a combination of WebSockets for real-time updates and React for component rendering, ensuring that the interface is both responsive and informative.

White-Label Capabilities

For SaaS providers, offering AI-powered features as a native part of their product is a significant advantage. The Orquesta Embed SDK is fully white-label, allowing companies to rebrand and style the components according to their application's look and feel.

Customizable UI Components

The SDK provides a set of customizable UI components for integrating AI operations. These components are built using standard web technologies (HTML, CSS, JavaScript), making them easy to style with custom themes.

API-First Approach

We've built the SDK on an API-first principle, ensuring that every feature and component can be accessed and controlled via our comprehensive APIs. This gives developers the flexibility to build bespoke interfaces and workflows tailored to their specific needs.

Challenges and Solutions

Handling Concurrency

A major challenge in building the SDK was ensuring it could handle concurrent AI operations without performance degradation. We implemented a task queue system within the SDK to manage simultaneous requests effectively, allowing operations to be queued and processed efficiently.

Ensuring Robustness

Building an embed solution that works flawlessly across various browsers and environments required thorough testing and optimization. We employed a CI/CD pipeline that runs automated tests across multiple scenarios, ensuring consistent performance and reliability.

Conclusion

The Orquesta Embed SDK represents a significant step forward in making AI-powered workflows accessible and manageable within any web application. By focusing on simplicity, security, and real-time capabilities, we've created a tool that empowers SaaS products to offer sophisticated AI operations seamlessly. As we continue to evolve, our goal remains to provide the most efficient and developer-friendly solution for integrating AI into existing infrastructures.