DEV Community: Ankur Tyagi

5 must know open-source repositories to build cool AI apps

Ankur Tyagi — Wed, 29 Oct 2025 07:45:50 +0000

Everywhere I look, teams are racing to ship AI-powered features, from solo founders building chatbots to enterprise teams automating workflows. The momentum is massive, and the big players (OpenAI, Google, and Meta) are pouring billions into new models.

But here’s the truth: you don’t need their budgets to build something impressive. What you do need are the right open-source tools and frameworks that give you full control, transparency, and freedom to experiment.

After experimenting with tons of AI integrations, I’ve found a handful of open-source repositories that make building real-time, multimodal apps actually doable.

These tools let you move from idea to prototype fast, no black boxes, no vendor lock-in.

1. Stream Vision Agents: Build Real-Time Video + Audio Intelligence

One of the cooler projects I’ve seen lately is Stream Vision Agents, an open source framework for building real-time, multimodal AI that can see, hear, and respond in milliseconds.

It’s built for developers who want to bring true intelligence to live video without being locked into a single model or transport provider.

Open Source: Fork it, read it, improve it.
Open Platform: Works with Stream Video or any WebRTC-based SDK.
Flexible Providers: Plug in OpenAI Realtime, Gemini Live, or your favorite STT/TTS and vision models.
It’s a bit like LiveKit Agents, but with a bigger focus on real-time vision and multimodal intelligence.

Let’s take a look at this example:

Sports Coach:

You can spin up a golf coaching AI with YOLO and OpenAI Realtime as the brain. YOLO handles pose detection, while the Realtime API reacts to movements as they happen. No lag, no buffering.

The cool part is, it’s not just for golf. The same setup works for stuff like drone-based fire detection, sports or gaming analytics, physical therapy assistance, workout form correction, and interactive dance or movement-based games. Basically, anything that needs a live “eyes and ears” AI.

agent = Agent(
    edge=getstream.Edge(),
    agent_user=agent_user,
    instructions="Read @golf_coach.md",
    llm=openai.Realtime(fps=10),
    #llm=gemini.Realtime(fps=1), # Careful with FPS can get expensive
    processors=[ultralytics.YOLOPoseProcessor(model_path="yolo11n-pose.pt")],
)

For more about Vision Agents, visit their documentation.

Star the Vision Agents repository ⭐

2. Open-Sora: High-Fidelity Text-to-Video Generation

Open-Sora is a super interesting open-source take on OpenAI’s Sora. It lets you convert text or images into short, high-quality videos that actually look stable (smooth motion, consistent frames, the whole thing). You can fine-tune it on your own datasets if you want to generate domain-specific stuff like marketing clips, story scenes, or quick simulations. It’s still early, but there’s a lot of room to experiment.

Why you’ll like it:

Supports text-to-video and image-to-video generation
Built for efficiency with diffusion-based architecture
Ideal for short clips (up to 15 seconds)
Actively maintained and open for contributions.

Star the Open Sora repository ⭐

3. OpenVoice v2: Instant Voice Cloning and Speech Synthesis

OpenVoice v2, built by the BentoML team, is one of the most impressive open-source voice cloning projects out there right now.

It can replicate a speaker’s tone and accent from just a few seconds of reference audio. It’s great for anything voice-driven. Think interactive AI agents, dubbing, or voice-enabled interfaces.

Why you’ll like it:

Multilingual and emotion-aware voice synthesis
Works well with real-time frameworks such as Stream Vision Agents
Simple API for inference and fine-tuning

Star the Open Voice repository ⭐

4. SpeechBrain: All-in-One Toolkit for Speech and Audio Intelligence

SpeechBrain is a PyTorch-based open-source toolkit that covers pretty much everything audio: ASR, TTS, speaker recognition, even speech enhancement.

It’s modular, easy to experiment with, and surprisingly production-ready. There are tons of prebuilt recipes if you just want to prototype fast or plug audio intelligence into something bigger you’re already building. .

Why you’ll like it:

Unified library for speech recognition and generation
Integrates easily with LLMs and real-time frameworks
Supports distributed and on-device inference

Star the Speech Brain repository ⭐

5. LiveKit Agents – Build Real-Time Voice and Video AI Applications

LiveKit Agents makes it easy to build real-time voice and video AI apps that actually feel live. Low latency, no awkward lag. You can run it locally or in the cloud, and hook it up to models like OpenAI Realtime, Gemini, or Whisper to handle the heavy lifting. It’s great for stuff like virtual meeting assistants, customer-support bots, or live translation apps.

Why you’ll like it:

Real-time streaming via WebRTC
Scales to thousands of concurrent sessions
Works seamlessly with custom or hosted LLMs

Star the agents repository ⭐

Thanks for reading the article.

In the comments below, let me know if other cool AI tools or frameworks have helped you build your application.

P.S. Feel free to follow me on X; I share valuable stuff - promise!

How to Implement Context-Aware AI Responses in Your Chat App

Ankur Tyagi — Wed, 07 May 2025 06:50:10 +0000

If you are used to using the ChatGPT or Claude chat interfaces, you might think that these models have incredible memories, able to maintain knowledge of the conversation across multiple interactions.

You're being tricked. The reality is that LLMs like Claude's have no inherent memory between calls. Each API request is stateless by design. When you use the official Claude web interface, it maintains your entire conversation history in the background and sends it with each new request. They are faking memory.

This "memory facade" pattern is fundamental to building any conversational AI application that maintains context over time, whether you're using Claude, GPT, or any other LLM. When building your own AI chatbot application, you'll need to implement this same pattern:

Store each message exchange (both user and assistant messages)
Bundle the entire conversation history with each new API call
Pass this history in the messages array parameter

If you have built on Stream’s AI integration examples, you will have already used this pattern. Here, we’ll walk through doing that using the Anthropic API, showing how to use Stream to easily build a chatbot with memory, context, and history.

Constructing Our Message Memory

Let's walk through how our AnthropicAgent class works to maintain conversation context in a Stream Chat application.

We’re going to end up with this:

This seems simple–it looks like any chat app. And with Stream and Chat AI, it is simple, but we need to set up our implementation correctly for this to work.

First, our agent needs to connect to both the Stream chat service and the Anthropic API:

export class AnthropicAgent implements AIAgent {
  private anthropic?: Anthropic;
  private handlers: AnthropicResponseHandler[] = [];
  private lastInteractionTs = Date.now();

  constructor(
    readonly chatClient: StreamChat,
    readonly channel: Channel,
  ) {}

  init = async () => {
    const apiKey = process.env.ANTHROPIC_API_KEY as string | undefined;
    if (!apiKey) {
      throw new Error('Anthropic API key is required');
    }
    this.anthropic = new Anthropic({ apiKey });
    this.chatClient.on('message.new', this.handleMessage);
  };

The constructor takes two parameters:

chatClient: A StreamChat instance for connecting to Stream's service
channel: The specific Stream channel where conversations happen

During initialization, we configure the Anthropic client with your API key, then set up an event listener for new messages in the Stream channel. This line is important:

this.chatClient.on('message.new', this.handleMessage);

This sets up our event listener to respond whenever a new message appears in the Stream channel. When a user sends a message, Stream triggers the message.new event, which our agent handles:

private handleMessage = async (e: Event<DefaultGenerics>) => {
  if (!this.anthropic) {
    console.error('Anthropic SDK is not initialized');
    return;
  }
  if (!e.message || e.message.ai_generated) {
    console.log('Skip handling ai generated message');
    return;
  }
  const message = e.message.text;
  if (!message) return;
  this.lastInteractionTs = Date.now();

  // Continue with handling the message...
}

Note that we skip processing if:

The Anthropic client isn't initialized
The message is AI-generated (to prevent infinite loops)
The message has no text content

Here's the crucial part where we construct context from the conversation history:

const messages = this.channel.state.messages
  .slice(-5)                                     // Take last 5 messages
  .filter((msg) => msg.text && msg.text.trim() !== '')
  .map<MessageParam>((message) => ({
    role: message.user?.id.startsWith('ai-bot') ? 'assistant' : 'user',
    content: message.text || '',
  }));

This is the "memory trick" in action. We:

Access the pre-maintained message history from channel.state.messages
Take the last five messages
Filter out empty messages
Transform them into Anthropic's message format with proper roles

This approach creates the illusion of memory for our AI, allowing it to maintain context throughout the conversation even though each API call is stateless.

Continuing the AI Conversation

Next, we want to send our message array to our LLM. We can leverage threaded conversations for more contextual AI responses:

if (e.message.parent_id !== undefined) {
  messages.push({
    role: 'user',
    content: message,
  });
}

The parent_id property is provided by Stream's messaging system, allowing us to identify thread replies. This threading capability is built into Stream, making it easy to create more organized conversations with humans and AI.

Now we send the constructed conversation history to Anthropic:

const anthropicStream = await this.anthropic.messages.create({
  max_tokens: 1024,
  messages,
  model: 'claude-3-5-sonnet-20241022',
  stream: true,
});

Notice how we're passing our messages array containing the conversation history. This gives Claude the context it needs to generate a relevant response. We then create a new message in the Stream channel and use Stream's built-in events system to show the AI's response in real-time:

const { message: channelMessage } = await this.channel.sendMessage({
  text: '',
  ai_generated: true,
});

try {
  await this.channel.sendEvent({
    type: 'ai_indicator.update',
    ai_state: 'AI_STATE_THINKING',
    message_id: channelMessage.id,
  });
} catch (error) {
  console.error('Failed to send ai indicator update', error);
}

Stream provides several features we're leveraging here:

channel.sendMessage() for creating new messages
Custom fields like ai_generated for adding metadata to messages
channel.sendEvent() for triggering real-time UI updates
Message IDs for tracking and updating specific messages

These capabilities make creating interactive, real-time AI experiences easy without building custom pub/sub systems or socket connections. The AnthropicResponseHandler uses partial message updates to create a smooth typing effect:

await this.chatClient.partialUpdateMessage(this.message.id, {
  set: { text: this.message_text, generating: true },
});

Stream's partialUpdateMessage function is particularly valuable for AI applications. It allows us to update message content incrementally without sending full message objects each time. This creates a fluid typing experience that shows the AI response being generated in real time without the overhead of creating new messages for each text chunk.

The run method processes the Anthropic stream events:

run = async () => {
  try {
    for await (const messageStreamEvent of this.anthropicStream) {
      await this.handle(messageStreamEvent);
    }
  } catch (error) {
    console.error('Error handling message stream event', error);
    await this.channel.sendEvent({
      type: 'ai_indicator.update',
      ai_state: 'AI_STATE_ERROR',
      message_id: this.message.id,
    });
  }
};

We then see our messages in the client:

If we were to console.log this for the messages above, we’d see:

messages [
  { role: 'user', content: 'Are you AI?' },
  {
    role: 'assistant',
    content: "Yes, I'm Claude, an AI assistant created by Anthropic. I aim to be direct and honest about what I am."
  },
  { role: 'user', content: 'What did I just ask you?' },
  { role: 'assistant', content: 'You asked me "Are you AI?"' }
]

And so on for the last 5 messages. By changing .slice(-5), we can increase or decrease the memory of the AI. You might want to adjust this based on your specific needs:

For simple Q&A bots, 2-3 messages may be enough
For complex discussions, you might need 10+ messages
For specialized applications, you might need to include specific earlier messages

Stream’s memory is much larger. The Channel object provides a convenient state management system through channel.state.messages, automatically maintaining our entire message history. This is a significant advantage of using Stream–you don't need to build your own message storage and retrieval system, and can then just choose the messages to send to AI.

This creates a conversation history that Claude can use to maintain context. We properly format the conversation back and forth by determining whether messages come from a user or an AI bot (based on user ID).

We can test whether the AI is maintaining its own memory of the conversation by taking advantage of the ability to delete messages in Stream:

If we then continue in the same message thread, the AI has lost all context:

If the LLM were maintaining its history, it would still have access to the previous messages. But as we’ve deleted them in Stream, we can see the API is truly stateless, and only our message “memory” code can allow us to give the AI the context it needs.

Tuning Our Message History Context Window

You can hardcode a message history, such as “5,” to give the LLM the most recent messages. But when you use the chat interface of Claude or ChatGPT, it remembers everything within the current chat. The interface sends all previous messages up to the LLM's maximum context window size.

The context window for an LLM is the maximum amount of text (tokens) that the model can process in a single API call, including both the input (prompt/history) and the generated output.

Instead of hardcoding our message history, we can dynamically decide how much of our history to send by computing whether all the messages will fit within the context window. The context window for Claude 3.5 (the model we’re using here) is 200k tokens.

function constructMessageHistory() {
  const MAX_TOKENS = 200000;  // Claude 3.5 Sonnet's limit
  const RESERVE_FOR_RESPONSE = 2000;  // Space for AI's reply
  const AVAILABLE_TOKENS = MAX_TOKENS - RESERVE_FOR_RESPONSE;

  // Simple token estimation (4 chars ≈ 1 token for English)
  function estimateTokens(text) {
    return Math.ceil(text.length / 4);
  }

  // Get all messages in chronological order
  const allMessages = this.channel.state.messages
    .filter(msg => msg.text && msg.text.trim() !== '')
    .map(message => ({
      role: message.user?.id.startsWith('ai-bot') ? 'assistant' : 'user',
      content: message.text || '',
      tokens: estimateTokens(message.text || '')
    }));

  // Work backwards from most recent, adding messages until we approach the limit
  let tokenCount = 0;
  const messages = [];

  for (let i = allMessages.length - 1; i >= 0; i--) {
    const msg = allMessages[i];

    if (tokenCount + msg.tokens <= AVAILABLE_TOKENS) {
      messages.unshift(msg);  // Add to beginning to maintain order
      tokenCount += msg.tokens;
    } else {
      break;  // Stop once we'd exceed the limit
    }
  }

  console.log(`Using ${tokenCount} tokens out of ${AVAILABLE_TOKENS} available`);

  return messages.map(({ role, content }) => ({ role, content }));
}

This approach:

Uses a simple character-to-token ratio (sufficient for getting started)
Prioritizes recent messages (most relevant to current context)
Automatically adjusts to different message lengths
Stops adding messages when approaching the token limit
Logs usage so you can monitor and adjust as needed

For better accuracy in production, you can replace the estimateTokens function with a proper tokenizer library. Still, this simple approach will get you started with dynamic context management without much complexity.

Fake Your LLM Memories

Implementing the “memory facade” with Stream and Anthropic is relatively straightforward. By leveraging Stream's built-in message history capabilities, we can easily maintain conversation context for our AI agent without building custom storage solutions. This approach eliminates the need to create your own database schema or message retrieval logic, allowing you to focus on the AI experience rather than infrastructure concerns.

Following this pattern, you can create AI chatbot experiences that maintain context over time, making interactions feel natural and coherent to your users. The simplicity of this implementation belies its power–it's the same technique used by major AI platforms to create the illusion of continuous conversation. Yet, it's accessible enough for developers to implement in their applications with minimal overhead.

Stream's built-in message history and event system make this straightforward. Instead of building custom databases and complex state management, developers can focus on unique features rather than infrastructure concerns. As AI continues evolving, this foundation of conversation context remains essential, and with Stream and Claude working together, you can build sophisticated, memory-aware chat applications that feel intelligent and responsive without reinventing the wheel.

Originally published on Stream’s blog: Implementing Context-Aware AI Responses in Your Chat App

Ready to build with Stream?

Follow a hands‑on video app guide: How I built a custom video conferencing app with Stream and Next.js
Sign up for the free Maker plan: Stream Maker plan (free)
Explore the Chat SDK: Stream Chat SDK
Explore the Video SDK: Stream Video SDK

Happy building.

How to build a job app with real-time chat & video using Stream, Next.js & Firebase. Perfect for devs building SaaS or interview platforms. Great tutorial by @DavidAsaolu 👏 #Nextjs #Stream #Firebase #webdev

Ankur Tyagi — Wed, 23 Apr 2025 11:57:12 +0000

David Asaolu for Stream

Apr 23 '25

Build a Job Application and Interview App with NextJS, Stream & Firebase

#javascript #tutorial #webdev #react

Comments 7

28 min read

Agent Cloud vs Qdrant

Ankur Tyagi — Mon, 20 May 2024 13:55:37 +0000

Over the past year, generative AI tools have exploded on the internet, captivating us with their ability to craft human-quality text, translate languages with remarkable accuracy, and even generate breathtakingly realistic images.

This rapid advancement in artificial intelligence is fueled by a crucial element that often goes unnoticed: data. Just as a powerful engine requires high-grade fuel, generative AI tools rely on vast repositories of high-quality data to deliver reliable and informative responses.

These Large Language Models (LLMs) draw their inferences almost exclusively from this pre-trained data. However, imagine if these tools could not only leverage data they have been pre-trained on but also be able to tap into your data for enhanced context. This is the revolutionary concept behind the Retrieval-Augmented Generation (RAG). LLMs can now reach beyond their internal training data thanks to RAG, which acts as a bridge to external information sources. This access allows them to generate more comprehensive, accurate, and up-to-date results.

AgentCloud is an open-source generative AI platform offering a built-in RAG service. The RAG, as a service offering from AgentCloud, includes a built-in pipeline that allows you to talk to your data by abstracting away all the complexities of setting up the underlying infrastructure yourself.

With AgentCloud, you can ingest data from over 300 sources and build a private LLM chat application with an interface similar to ChatGPT. Some common enterprise platforms where you can integrate your data include Google BigQuery, Salesforce, Atlassian Confluence, Zendesk, Airbyte, PostgreSQL, MongoDB, and OneDrive.

Now, let's shift our focus to Qdrant, AgentCloud's perfect partner. At its core, Qdrant shines as a high-performance vector database and vector similarity search engine that allows you to store and retrieve dense vector representations of data. Qdrant excels at facilitating similarity search operations, allowing LLMs to quickly retrieve the most relevant data points based on their queries.

This search capability makes Qdrant perfect for handling large volumes of data embeddings, which are multidimensional representations of data commonly used in AI generative applications such as AgentCloud and ChatGPT.

Before we get started

If you are an open Source lover and then please consider supporting Agent Cloud, by giving us a Star on GitHub ⭐️

Just click on the cat 🙏

Thanks. You're cool.

Understanding AgentCloud - Securely talk to your data.

AgentCloud is an open-source AI application platform whose main focus is enabling companies to securely talk to their data through an internal GPT builder that can use any LLM and access hundreds of data sources.

AgentCloud offers a seamless end-to-end solution for building powerful RAG applications using a powerful, scalable open stack of tools under the hood.

The stack includes a powerful, built-in ELT pipeline built on AirByte that can seamlessly ingest data from over 300 sources, including popular options like databases, documents, collaboration platforms, and cloud storage. Other supported data formats include pdf, txt, xlsx, and CSV files.

AgentCloud also incorporates a robust message bus (RabbitMQ), ensuring smooth communication between components within your RAG application, and an open-source vector database(Qdrant) for efficient data storage and retrieval.

Using its end-to-end pipeline, AgentCloud also offers features for tasks like splitting, chunking, and embedding data, as well as all crucial steps for efficient information retrieval.

For complex data sources like BigQuery, AgentCloud offers advanced chunking strategies like semantic chunking. This ensures efficient processing of large datasets without overwhelming the system or losing information. Looking ahead, AgentCloud will empower users with even greater control over their data. You'll soon be able to selectively choose which fields to embed for retrieval by your AI agents and which fields to store as metadata for additional context.

AgentCloud stores the prepared data securely in a vector database(Qdrant) and keeps it fresh through manual, scheduled, or automated updates. You can now create apps using a no-code builder interface built for anyone and share them with Users, Teams, or the entire Organisation.

Away from building chatbots for data interaction, AgentCloud also offers process automation capabilities with multi-agent workflows powered by Autogen.

Autogen is a platform that provides a multi-agent conversation framework as a high-level abstraction, allowing you to build LLM workflows conveniently.

Using this feature, you can build powerful teams of AI agents, each powered by your choice of Large Language Models (LLMs) from providers like OpenAI or Hugging Face. These agents can work together, share information, and leverage various data sources to automate complex processes.

One of the most exciting aspects of AgentCloud is its flexibility regarding Large Language Models (LLMs). Unlike some platforms that lock you into their ecosystem, AgentCloud empowers you to choose the LLM that best suits your needs.

Do you have a specific open-source LLM in mind, like LLAMA2 or a model from Hugging Face?

No problem! AgentCloud allows you to connect your LLM.

Do you need the processing muscle of a cloud-based LLM?

AgentCloud seamlessly integrates with popular providers like OpenAI, Cohere, and Anthropic Claude.

For organizations prioritizing maximum security, AgentCloud lets you connect the platform to your private LLM endpoints. This ensures your data remains completely isolated, with no access to the internet and no risk of exposure to external LLM providers.

Agent Cloud caters to self-hosting and cloud-based deployment options. For companies seeking complete control and data isolation, self-hosting allows them to deploy the platform on their infrastructure using Kubernetes Helm files. This approach requires managing the infrastructure yourself.

Alternatively, the cloud option is ideal for businesses prioritizing a quicker setup and avoiding infrastructure management. This cloud-based deployment allows you to get up and running with AgentCloud functionality without the burden of maintaining infrastructure.

AgentCloud prioritizes your data security. You can self-host everything or leverage self-hosted AI models to prevent unauthorized access. Additionally, you can control and limit what your AI agents can access and do.

AgentCloud Usecases

AgentCloud product offerings cater to the needs of businesses in different ways. Let’s highlight some use cases for Agent Cloud:

Customer support: AgentCloud empowers businesses to deploy chatbots that can handle customer inquiries, troubleshoot issues, and even resolve support tickets. This frees up human agents for more complex tasks, improving overall customer service efficiency.
Internal knowledge management: Go beyond simple FAQs. AgentCloud allows you to create process apps that automate internal workflows. For instance, you can create an onboarding chatbot that guides new employees through company policies or a document approval chatbot that streamlines internal processes.
Data-Driven decision-making: Break down data silos! AgentCloud facilitates the building of conversational interfaces for data analysis, facilitating data-driven decision-making.
Security and Control: AgentCloud offers a unique advantage for organizations with strict data privacy regulations - complete on-premise or private cloud deployment. This ensures maximum control over sensitive data and compliance with data sovereignty requirements.
Financial services: Build chatbots for banking inquiries, fraud detection powered by AgentCloud, and even personalized financial advisory services delivered through chat interfaces.
Healthcare solutions: AgentCloud can be the foundation for virtual healthcare assistants, medical chatbots for appointment scheduling or symptom evaluation, and even remote patient monitoring systems.

Understanding Qdrant

Qdrant is a high-performance vector database designed for efficiently storing, searching, and managing vector embeddings. Unlike traditional OLTP (Online Transaction Processing) and OLAP (Online Analytical Processing) databases that rely on tables and keywords, Qdrant excels at handling high-dimensional data represented as dense vector embeddings.

This capability makes Qdrant particularly useful for applications dealing with complex data types like text, images, audio, and more, which are often used in generative AI platforms like AgentCloud. Vector embeddings allow for a compressed representation of these complex data types that is optimized for machine learning algorithms.

Before we highlight the key features of Qdrant, consider these vectors as a unique, multi-dimensional representation of data where each dimension represents a characteristic of your data.

For example, text data might be transformed into an embedding that captures its semantic meaning, while image data might be converted into an embedding that reflects its color composition, shapes, and textures.

Ditch the keyword struggle.

Qdrant leverages semantic embeddings to understand the true meaning of your text data, even for short texts.

This allows you to build and deploy powerful semantic neural search functionalities on your data in minutes.

Traditional search engines tend to struggle with data containing thousands of dimensions. Qdrant tackles this challenge head-on with advanced indexing techniques. It utilizes a graph-based indexing algorithm called Hierarchical Navigable Small World (HNSW) graphs to create a network of interconnected data points.

Qdrant allows you to measure the quality of your search using the built-in exact search mode, which can measure the quality of the search results. In this mode, Qdrant performs a full kNN search for each query without approximation.

This vector database integrates effortlessly with your existing projects, regardless of programming language.

It offers a user-friendly RESTful API as the primary method for interaction, including official client libraries for popular languages to simplify the process.

If your programming language isn't on the list, you can still interact with Qdrant directly using the REST API or generate a custom client using OpenAPI.

Generative AI tools are constantly working with ever-growing data needs. Quadrant is built to efficiently store and retrieve vast amounts of data embeddings, making it ideal for Generative AI applications working with ever-growing data repositories.

Qdrant Usecases

Qdrant vector search engine transcends traditional keyword search, unlocking a wide range of applications across various industries.

Here's a glimpse into how what you can do with Qdrant vector database:

Matching engines for semantically complex objects: In HR processes and Job search platforms, Qdrant's vector search engine can match candidates and jobs based on skills and experience described in natural language, even if they don't use the exact same keywords. This eliminates the need for rigid categorization and allows for a better understanding of qualifications.
Similar image search: This can apply to the fashion industry. Qdrant's visual search functionality empowers shoppers to search for clothing based on appearance, removing the limitations of keyword searches. Large companies like Zalando are already using in this technology.
E-Commerce search: Qdrant's AI-powered search goes beyond traditional keyword-based search, allowing users to identify relevant products even when users don't use the exact terminology.
Recommendations engines: As seen in the previous examples, Qdrant can recommend food options or media content (music, movies, games) based on visual or user preference similarity.
Customer support and sales optimization: Chatbots powered by AgentCloud and Qdrant can automate answering frequently asked questions (FAQs), freeing up human customer service representatives for more complex issues.

What to Choose- AgentCloud or Qdrant?

In essence, AgentCloud and Qdrant are not competing tools; they complement each other.

By understanding their distinct functionalities, you can choose the best option or leverage them together for a powerful combination.

AgentCloud is an open-source platform that enables companies to build LLM-powered conversational chat apps that allow them to talk with their data. You can also conveniently build a group of agents to solve more complex tasks by providing these agents access to functions and data sources.

AgentCloud utilizes various open-source tools and services, and one of its core components is a vector database, Qdrant itself.

AgentCloud provides a comprehensive suite of data ingestion, transformation, model training, and application deployment functionalities. It also leverages the power of Qdrant for vector search within its workflows.

If your primary focus is efficient vector search and retrieval, Qdrant is the ideal choice.

On the other hand, if you need a comprehensive platform for building and deploying intelligent applications that leverage vector search, then AgentCloud is the way to go.

Conclusion

You won't be facing a situation where you must choose between Agent Cloud and Qdrant. However, by using the two platforms, you can create powerful generative AI applications that allow you to talk to your data securely through a simple chat interface.

AgentCloud abstracts away the complexities of setting up Qdrant yourself, allowing you to focus on building conversational and process automation apps.

AgentCloud offers a compelling solution for technical and non-technical users when building generative AI applications. With its built-in RAG pipeline, anyone within your company can find the data they need, when needed, without the hassle of searching through disparate systems or requesting access from colleagues through endless email exchanges.

🔍 Want to learn more about AgentCloud?

How to Build a Chat App with Your Postgres Data using Agent Cloud

Ankur Tyagi — Mon, 13 May 2024 15:02:38 +0000

Introduction

Enterprises are turning to RAG (Retrieval-Augmented Generation) because it fundamentally changes how AI can be used for better decision-making.

RAG combines the power of information retrieval with AI generation, allowing businesses to:

Leverage vast amounts of data: RAG unlocks the value of internal and external data sources, providing a more comprehensive picture for informed choices.
Enhance accuracy and confidence: By grounding AI generation in reliable information, RAG reduces the risk of bias or errors in decision-making.
Streamline the process: RAG automates parts of the information-gathering and analysis process, leading to faster and more efficient decision-making cycles.
RAG is a framework designed to improve language models by pulling in relevant, up-to-date data directly related to a user’s query and RAG chat apps allow you to chat with your data.

In this blog, we will learn to build a RAG chatbot in minutes using AgentCloud.

AgentCloud is an open-source platform enabling companies to build and deploy private LLM chat apps, empowering teams to securely interact with their data.

AgentCloud uses Airbyte to build data pipelines, which allow us to split, chunk, and embed data from over 300 data sources, including Postgres.
AgentCloud uses Qdrant as the vector store to efficiently store and manage large sets of vector embeddings. For a given user query the RAG application fetches relevant documents from vector store by analyzing how similar their vector representation is compared to the query vector.

So, let’s get started building a RAG chat application using Agent Cloud to talk privately and securely with your Postgres data.

Before we start building the RAG app

If you are an open Source lover and then please consider supporting Agent Cloud, by giving us a Star on GitHub ⭐️

Just click on the cat 🙏

Thanks. You're cool.

Setting up Agent Cloud via Docker

To run AgentCloud locally, ensure Docker is installed. Follow these steps to get AgentCloud up and running.

Clone the repo:

git clone https://github.com/rnadigital/agentcloud.git

Navigate to the agentcloud directory: cd agentcloud
Run this command:

chmod +x install.sh && ./install.sh
Running the install command will download necessary Docker images and launch the containers.

Once the install script is executed successfully we can view the containers running in the docker app:

To access Agent Cloud in the browser, we can hit the below URL: http://localhost:3000/register

Next, we need to sign to the platform

Post sign-up, and log in to the App, to get to this landing screen.

Congrats, setup is complete now.

Now let's move towards building our RAG app.

Adding New Model

Agent Cloud allows us to use models like FastEmbed and OpenAI in app.

To add a new model let’s go to the Models screen and click the Add Model option.

In the configuration screen, select a Model—such as the fast-bge-small-en for embedding text. Then, save your settings to finish the model setup.

Post successfully adding the model we will be able to view the model in the Models list.

Creating New Datasource

The data source for our app will be a Postgres table which contains information about Indian dishes.

Here is the snapshot of the table data.

Our table features 181 Indian dishes representing various Indian states.

Here is a description of the table columns.

name : Name of the dish
ingredients : Main ingredients used
diet : Type of diet - either vegetarian or non-vegetarian
prep_time : Preparation time
cook_time : Cooking time
flavor_profile : Flavor profile includes whether the dish is spicy, sweet, bitter, etc
course : Course of meal - starter, main course, dessert, etc
state : State where the dish is famous or is originated
region : Region where the state belongs
meta_info: This column contains the dish information in an unstructured format. The column data will be embedded for vector search.

To integrate table data into our app, add the data source by navigating to the Data Source page, clicking the New Connection button, and selecting a source from the modal.

Let's choose Postgres as our data source and then configure the connection.

Here's what's needed to set up the Postgres data source.

Schedule type
Host name
Port
Password
Postgres username
Postgres password
Database name

Let's disabled the SSL modes as we're running Postgres on our local machine.

Next, we need to select the replication method; I have selected Xmin.

Xmin replication is the new cursor-less replication method for Postgres.

Cursorless syncs enable syncing new or updated rows without explicitly choosing a cursor field. The Xmin system column is used to track inserts and updates to your source data.

Now, we need to click on the Test and Save button, which will first test the DB connection and, if successful, save the data connector.

We'll select the table and all its columns.

The column meta_info is selected for embedding, which means the vector search will be performed on the contents of this column.

Then we press the Continue button to create the Data Source.

It will take some time to embed the data, and post that you will be able to see this data in a new collection in the Qdrant collection dashboard.

After setting up the data source, Agent Cloud uses Airbyte in the background to create a data pipeline between Postgres and Qdrant.

Once the sync is complete, the new collection will appear in the Qdrant DB.

Example of how the data is stored inside a Qdrant point.

Next, we will be setting up a tool for using our data source.

Setting up Tools

Tools are essential for enabling the AI agent to interact effectively with its environment, process information, and take appropriate actions to achieve its goals.

The tools used by an AI agent can include functions, APIs, data sources, and other resources that help the agent perform specific tasks autonomously and efficiently.

The tool we will be setting up will be responsible for querying the data source and fetching relevant documents. Agent Cloud creates a tool for us when we add a new data source by default.

The screenshot below shows that the Postgres Food data tool has been automatically created by the platform.

I have updated the tool configuration to change the description.

This description is very important as it will be used by the Agent to decide which tool to use for a given task.

Click on the Save button to update the tool.

Creating Agent

An Agent is an intelligent system that can efficiently accomplish complex tasks.

To create an Agent, let’s go to the Agents screen and click on the New Agent button.

In the Agent configuration, we need to provide certain information one of the information is the LLM Model. Agents use LLMs to enhance their capabilities and improve communication.

We are using gpt-4-0125-preview OpenAI model for our agent.

Here are the remaining details needed for configuration:

Agent Name
Agent role
Agent Goal
Agent Backstory
Agent Tools

Now click on the Save button to complete the Agent configuration.

Creating Task

Tasks are specific assignments assigned to an agent for completion.

To create a new task let’s click the Add Task button in the Task screen.

When configuring the task, provide the following information:

Task name
Task description
Task tools
Task Agent

Post that we can click on the Save button to save the configuration.

Creating App

So till now, we have created a data source for the App, a tool to retrieve to information from the data source, an LLM Agent that will be working on complex tasks using the tool, and a Task that will be assigned to the Agent based on the user’s instruction.

Now we will be creating a conversational App that will allow us to chat with our data.

To create the App, let’s go to the App screen and click on the New App button.

In the configuration page, let’s enter the app name and select the app type, which will be a conversation chat app in our case.

Select the Task and Agent that we created. In the process, we will select sequential, and finally, in the Chat manager model, we will select the OpenAI GPT 4 model.

Now we can click on the Save button to complete the app setup.

After completing the app setup, the newly created app will appear in the dashboard.

Now let's see the app in action by clicking the Play button.

A conversion window is opened for us where we can chat with our data.

Here is the app's response to our question.

Yay, we did it 😎

Conclusion

In this blog, we learned to build a RAG chat app with Agent Cloud and Postgres.

We covered creating a data source, embedding it, and storing it in Qdrant DB. Additionally, we learned how to build tools for Agents and create an app where users can interact with their private data.

🔍 Want to learn more about Agent Cloud?

How I Built a Blog with Next.js and Firebase

Ankur Tyagi — Wed, 08 May 2024 09:31:27 +0000

Introduction

Building full-stack web applications has never been this easy.

You don't need to write backend and frontend code to create fully functional web applications that authenticate users, store data, and upload files to cloud storage. Firebase allows you to build these applications easily within a few minutes.

In this tutorial, you'll learn how to build a blog that allows anyone to read posts and create an account as post authors to write posts, comment, and interact with other users.
The application uses Firebase Google authentication methods to sign users into the application and Firebase Firestore to perform CRUD operations when users create, delete, and comment on posts.

PS: A basic knowledge of React/Next.js is required to fully understand this article.

What is Firebase?

Firebase is a Backend-as-a-Service (Baas) owned by Google that enables us to build full-stack web applications in a few minutes. Services like Firebase make it very easy for front-end developers to build full-stack web applications with little or no backend programming skills.

Firebase provides various authentication methods, a NoSQL database, a real-time database, file storage, cloud functions, hosting services, and many more.

Why you should use Firebase?

Firebase is an exceptional platform that empowers developers to build full-stack applications effortlessly. Let's explore some unique features that set Firebase apart from other Backend as a Service (BaaS) platforms.

Complete Authentication System:

Firebase offers a complete authentication system that allows you to easily manage user authentication in your application. With Firebase Authentication, you can authenticate users using email/password, phone numbers, Google, Facebook, and more.

Cloud Firestore

Firebase Firestore is a flexible, scalable database for mobile, web, and server development. It allows you to store and sync data between your users in real-time, making it ideal for applications that require CRUD operations and complex data queries.

File Storage
Firebase provides an efficient Cloud Storage that enables you to upload and download files easily within your application. It seamlessly integrates with Firebase Authentication and Firebase Firestore, making it easy to build complex applications that requires serving and retrieving various file formats.

Hosting
Firebase provides hosting services that allow you to deploy web apps and static content with a single command. It provides SSL encryption, global CDN delivery, and continuous deployment, ensuring fast and reliable performance for your users.

Building the application interface with Next.js

Here, I'll walk you through creating the various components and pages within the blogging application.

The application is divided into four pages:

Home page
Login page
Create Post page
Post Slug page that displays the posts' content

Before we proceed, create a types.ts file at the root of your Next.js project to define the attributes of the posts and comments within the application.

export interface Comment {
    id: string;
    content: string;
    author_name: string;
    pub_date: string;
}
export interface Post {
    post_id: string;
    author_id: string;
    title: string;
    content: string;
    author_name: string;
    image_url: string;
    pub_date: string;
    slug: string;
    comments: Comment[];
}

Next, create a utils.ts file within the Next.js app folder and copy the code snippet below into the file:

// Within the utils.ts file
export const getCurrentDate = (): string => {
    const currentDate = new Date();
    const day = currentDate.getDate();
    const monthIndex = currentDate.getMonth();
    const year = currentDate.getFullYear();
    const suffixes = ["th", "st", "nd", "rd"];
    const suffix = suffixes[(day - 1) % 10 > 3 ? 0 : (day - 1) % 10];
    const months = [
        "Jan.",
        "Feb.",
        "Mar.",
        "Apr.",
        "May",
        "Jun.",
        "Jul.",
        "Aug.",
        "Sept.",
        "Oct.",
        "Nov.",
        "Dec.",
    ];
    const month = months[monthIndex];
    return `${day}${suffix} ${month} ${year}`;
};
//👇🏻 Create a unique slug from posts' titles
export const slugifySentences = (sentence: string): string => {
    // Remove special characters and replace spaces with hyphens
    const slug = sentence
        .toLowerCase()
        .replace(/[^a-z0-9\s-]/g, "")
        .replace(/\s+/g, "-");
    // Generate 5 random letters
    const randomLetters = Array.from({ length: 5 }, () =>
        String.fromCharCode(97 + Math.floor(Math.random() * 26))
    ).join("");
    return `${slug}-${randomLetters}`;
};

The getCurrentDate function accepts the current date as a parameter and returns it in a more readable format, suitable for use within blog posts. The slugifySentences function creates a slug from each post's title.

The Home Page

The application home page displays all the available posts and allow users to read their preferred posts.

Copy the code snippet below into the app/page.tsx file.

"use client";
import Image from "next/image";
import Nav from "./components/Nav";
import Link from "next/link";
import { useEffect, useState } from "react";
export default function Home() {
    const [posts, setPosts] = useState<Post[]>();
    const shortenText = (text: string): string => {
        return text.length <= 55 ? text : text.slice(0, 55) + "...";
    };
    useEffect(() => {
        fetchAllPosts();
    }, []);
    const fetchAllPosts = async () => {
        // 👉🏻 fetch all posts
    };
    return (
        <div>
            <Nav />
            <main className='p-8 w-full flex lg:flex-row flex-col items-center gap-5 flex-wrap justify-center'>
                {posts?.map((post) => (
                    <Link
                        href={`/posts/${post.slug}`}
                        className='cursor-pointer lg:w-1/3 rounded-lg w-full border-2 h-[400px] bg-white'
                        key={post.post_id}
                    >
                        <Image
                            src={post.image_url}
                            alt='Image'
                            width={300}
                            height={300}
                            className='w-full h-2/3 rounded-t-lg'
                        />
                        <section className='h-1/3 w-full p-4 flex items-center'>
                            <p className='font-semibold text-xl text-blue-500'>
                                {shortenText(post.title)}
                            </p>
                        </section>
                    </Link>
                ))}
            </main>
        </div>
    );
}

The code snippet above fetches all the existing blog posts and displays their titles and cover images.

There is also a Nav component that displays the blog title and the Sign In button.

Create a components folder containing the Nav component within the Next.js app folder and update it as shown below.

"use client";
import Link from "next/link";
import { useEffect, useState } from "react";
import { handleSignOut } from "../utils";
import { usePathname } from "next/navigation";

export default function Nav() {
    const [displayName, setDisplayName] = useState<string>("");
    const pathname = usePathname();

    useEffect(() => {
        if (localStorage.getItem("user")) {
            const user = JSON.parse(localStorage.getItem("user") || "");
            setDisplayName(user.displayName);
        }
    }, []);

    return (
        <nav className='w-full py-4 border-b-2 px-8 text-center flex items-center justify-between sticky top-0 bg-[#f5f7ff]'>
            <Link href='/' className='text-2xl font-bold'>
                Blog
            </Link>
            {displayName ? (
                <div className='flex items-center gap-5'>
                    <p className='text-sm font-semibold'>{displayName}</p>
                    {pathname !== "/posts/create" && (
                        <Link href='/posts/create' className='underline text-blue-500'>
                            Create Post
                        </Link>
                    )}
                    <button
                        className='bg-red-500 text-white px-6 py-3 rounded-md'
                        onClick={handleSignOut}
                    >
                        Log Out
                    </button>
                </div>
            ) : (
                <Link
                    href='/login'
                    className='bg-blue-500 text-white px-8 py-3 rounded-md'
                >
                    Sign in
                </Link>
            )}
        </nav>
    );
}

The Nav component checks if the user is authenticated to determine which buttons to display. If the user is not authenticated, it renders the Sign-in button; otherwise, it displays the user's name, a Create Post button, and a Log Out button.

The Login Page

The Login page simply displays a Google sign-in button that enables users to log in using their Gmail account.

To create the Login page, first, create a login folder within the Next.js app folder. Inside the login folder, create two files: layout.tsx and page.tsx.

cd app
mkdir login && cd login
touch layout.tsx page.tsx

Copy the code snippet below into the page.tsx file. It renders a button that enables users to sign in to the application using their Gmail account.

"use client";
import { useState } from "react";

export default function Nav() {
    const [loading, setLoading] = useState<boolean>(false);

    const handleGoogleSignIn = () => {
        setLoading(true);
        //👉🏻 handle Firebase Google authentication
    };

    return (
        <div className='w-full min-h-screen flex flex-col items-center justify-center'>
            <h2 className='font-bold text-2xl mb-6'>Sign into Blog</h2>
            <button
                className='w-1/3 border-2 border-gray-600 mb-6 rounded-md hover:bg-black hover:text-white px-8 py-4'
                disabled={loading}
                onClick={handleGoogleSignIn}
            >
                {loading ? "Signing in..." : "Sign in with Google"}
            </button>
        </div>
    );
}

Update the layout.tsx file as shown below. It changes the page's title.

import type { Metadata } from "next";
import { Sora } from "next/font/google";
const inter = Sora({ subsets: ["latin"] });

export const metadata: Metadata = {
    title: "Log in | Blogging App",
    description: "Generated by create next app",
};

export default function RootLayout({
    children,
}: Readonly<{
    children: React.ReactNode;
}>) {
    return (
        <html lang='en'>
            <body className={inter.className}>{children}</body>
        </html>
    );
}

The Posts pages

First, create a posts folder containing a create folder and a [..slug] folder within the Next.js app folder.

The create folder adds a /posts/create client route to the application, and the [..slug] folder creates a dynamic route for each blog post.

Next, create a page.tsx file within the /posts/create folder and add a form that enables users to enter the details of a blog post.

"use client";
import { useEffect, useState } from "react";
import Nav from "@/app/components/Nav";
import { getCurrentDate, slugifySentences } from "@/app/utils";

export default function CreatePost() {
    const [coverPhoto, setCoverPhoto] = useState<string>("");
    const [content, setContent] = useState<string>("");
    const [title, setTitle] = useState<string>("");
    const [uploading, setUploading] = useState<boolean>(false);

    const handleFileReader = (e: React.ChangeEvent<HTMLInputElement>) => {
        const reader = new FileReader();
        if (e.target.files && e.target.files[0]) {
            reader.readAsDataURL(e.target.files[0]);
        }
        reader.onload = (readerEvent) => {
            if (readerEvent.target && readerEvent.target.result) {
                setCoverPhoto(readerEvent.target.result as string);
            }
        };
    };

    const handleCreatePost = async (e: React.FormEvent<HTMLFormElement>) => {
        e.preventDefault();
        setUploading(true);
        console.log({
            title,
            content,
            author_name: userData.displayName,
            pub_date: getCurrentDate(),
            slug: slugifySentences(title),
            comments: [],
        });
        //👉🏻 save blog post to the database
    };
    return (
        <div>
            <Nav />
            <main className='md:px-8 py-8 px-4 w-full'>
                <form className='flex flex-col w-full' onSubmit={handleCreatePost}>
                    <label htmlFor='title' className='text-sm text-blue-600'>
                        Title
                    </label>
                    <input
                        type='text'
                        name='title'
                        id='title'
                        value={title}
                        required
                        onChange={(e) => setTitle(e.target.value)}
                        className=' px-4 py-3 border-2 rounded-md text-lg mb-4'
                    />
                    <label htmlFor='content' className='text-sm text-blue-600'>
                        Content
                    </label>
                    <textarea
                        name='content'
                        rows={15}
                        value={content}
                        required
                        onChange={(e) => setContent(e.target.value)}
                        id='content'
                        className=' px-4 py-3 border-2 rounded-md mb-4'
                    ></textarea>
                    <label htmlFor='upload' className='text-sm text-blue-600'>
                        Upload Cover Photo
                    </label>
                    <input
                        type='file'
                        name='upload'
                        id='upload'
                        required
                        onChange={handleFileReader}
                        className=' px-4 py-3 border-2 rounded-md mb-4'
                        accept='image/jpeg, image/png'
                    />
                    <button
                        type='submit'
                        className='bg-blue-600 mt-4 text-white py-4 rounded-md'
                        disabled={uploading}
                    >
                        {uploading ? "Creating post..." : "Create Post"}
                    </button>
                </form>
            </main>
        </div>
    );
}

The [...slug] page displays the contents of a blog post. Add a page.tsx file within the folder and copy the code snippet below into the file.

"use client";
import Comments from "@/app/components/Comments";
import Nav from "@/app/components/Nav";
import { extractSlugFromURL } from "@/app/utils";
import { usePathname, useRouter } from "next/navigation";
import { useEffect, useState } from "react";

export default function Post() {
    const params = usePathname();
    const router = useRouter();
    const slug = extractSlugFromURL(params);
    const [loading, setLoading] = useState<boolean>(true);
    const [post, setPost] = useState<Post | null>(null);
    return (
        <div>
            <Nav />
            <main className='w-full md:p-8 px-4'>
                <header className='mb-6 py-4'>
                    <h2 className='text-3xl text-blue-700 font-bold mb-2'>
                        {post?.title}
                    </h2>
                    <div className='flex'>
                        <p className='text-red-500 mr-8 text-sm'>
                            Author: <span className='text-gray-700'>{post?.author_name}</span>
                        </p>
                        <p className='text-red-500 mr-6 text-sm'>
                            Posted on: <span className='text-gray-700'>{post?.pub_date}</span>
                        </p>
                    </div>
                </header>
                <div>
                    <p className=' text-gray-700 mb-3'>{post?.content}</p>
                </div>
            </main>
            <Comments comments={post?.comments} post_id={post?.post_id} />
        </div>
    );
}

The code snippet above displays the post's title, author, published date, and content, and passes the post's comments into a Comments component.

Finally, create a Comments component that accepts a post's comments as props and renders the available comments, including an input field for adding new comments.

"use client";
import Link from "next/link";
import React, { useEffect, useState } from "react";
import { getCurrentDate } from "../utils";
export const generateCommentID = () =>
    Math.random().toString(36).substring(2, 10);

export default function Comments({
    comments,
    post_id,
}: {
    comments: Comment[] | undefined;
    post_id: string | undefined;
}) {
    const [user, setUser] = useState({ displayName: "", u_id: "" });
    const [newComment, setNewComment] = useState<string>("");
    const [postingComment, setPostingComment] = useState<boolean>(false);
    useEffect(() => {
        if (localStorage.getItem("user")) {
            const user = JSON.parse(localStorage.getItem("user") || "");
            setUser(user);
        }
    }, []);
    const handleSubmit = (e: React.FormEvent<HTMLFormElement>) => {
        e.preventDefault();
        // 👉🏻 postComment();
    };
    if (!user.displayName)
        return (
            <main className='p-8 mt-8'>
                <h3 className=' font-semibold'>
                    You need to{" "}
                    <Link href='/login' className='text-blue-500 underline'>
                        sign in
                    </Link>{" "}
                    before you can comment on this post
                </h3>
            </main>
        );
    return (
        <main className='p-8 mt-8'>
            <h3 className='font-semibold text-xl mb-4 text-blue-500'>Comments</h3>
            <form onSubmit={handleSubmit} className='mb-8'>
                <textarea
                    className='w-full p-4 border border-gray-300 rounded-md mb-2'
                    placeholder='Leave a comment'
                    value={newComment}
                    required
                    onChange={(e) => setNewComment(e.target.value)}
                />
                <button
                    className='bg-blue-500 text-white px-4 py-2 rounded-md'
                    disabled={postingComment}
                >
                    {postingComment ? "Posting..." : "Post Comment"}
                </button>
            </form>
            {comments && comments.length > 0 && (
                <h3 className='font-semibold text-xl mb-4 text-blue-500'>
                    Recent comments
                </h3>
            )}
            <div className='w-full flex items-start md:flex-row flex-col justify-center gap-4 flex-wrap'>
                {comments?.map((comment) => (
                    <div
                        className='bg-white p-4 rounded-lg md:w-1/4 w-full shadow-lg'
                        key={comment.id}
                    >
                        <p className='mb-4'>{comment.content}</p>
                        <div>
                            <h4 className='font-semibold text-sm'>{comment.author_name}</h4>
                            <p className='text-gray-500 text-sm'>{comment.pub_date}</p>
                        </div>
                    </div>
                ))}
            </div>
        </main>
    );
}

The code snippet above accepts the post's comments as props and renders them within the component. However, it ensures that only authenticated users can create and view existing comments. Unauthenticated users can only read a blog post.
Congratulations! You've completed the user interface for the application.

How to add Firebase to a Next.js app

In this section, you'll learn how to set up Firebase project and add it to a Next.js application. A Firebase project has all the features provided by Firebase and enables you to use them within your software applications.

First, you need to visit the Firebase console and sign in with a Gmail account.

Create a Firebase project and select the </> icon to create a new Firebase web app.

Provide your app name and register it.

Install the Firebase SDK in your Next.js project by running the code snippet below.

npm install firebase
Next, Create a firebase.ts file at the root of your Next.js project and copy the Firebase configuration code for your app into the file.

import { initializeApp, getApps } from "firebase/app";
import { getFirestore } from "firebase/firestore";
import { getAuth, GoogleAuthProvider } from "firebase/auth";
import { getStorage } from "firebase/storage";
// Your web app's Firebase configuration
const firebaseConfig = {
    apiKey: "***********",
    authDomain: "***********",
    projectId: "**************",
    storageBucket: "**********",
    messagingSenderId: "************",
    appId: "********************",
};
//👇🏻 Initialize Firebase
const app =
    getApps().length === 0 ? initializeApp(firebaseConfig) : getApps()[0];
//👇🏻 required functions
const db = getFirestore(app);
const auth = getAuth(app);
const storage = getStorage(app);
const googleProvider = new GoogleAuthProvider();
//👇🏻 required exports
export { db, auth, googleProvider, storage };

The code snippet above sets up Firebase in your Next.js application. It initializes Firebase Firestore, authentication, and storage instances, allowing you to access these features within the application.
Before you can start using these features, you'll need to set them up.

Configuring the Firebase features

For this application, you'll need to set up Firebase Storage for saving cover images for posts, Firebase Firestore for storing post contents, and Firebase Authentication for user authentication.

To get started, select Build from the left-hand panel, then click on Firestore Database to create a database.

Create the database in test mode, and pick your closest region.

Next, select Storage from the sidebar menu to set up the cloud storage for the files within the application.

Finally, select Authentication and pick Google from the list of authentication providers.

Select the support email for the Firebase project and click Save to enable Google authentication.

Congratulations. You can start saving files to the Firebase Cloud Storage, interacting with the Firebase Firestore for CRUD operations, and authenticating users via Firebase Google Authentication.

Handling user authentication within the application

Here, you'll learn how to sign users in and out of the application and how to retrieve the current user details from Firebase.

Within the login/page.tsx file, update the handleGoogleSignIn function as done below:

"use client";
import { useState } from "react";
import { handleSignIn } from "../utils";
import { googleProvider } from "../../../firebase";
import { GoogleAuthProvider } from "firebase/auth";

export default function Nav() {
    const [loading, setLoading] = useState<boolean>(false);
    const handleGoogleSignIn = () => {
        setLoading(true);
        handleSignIn(googleProvider, GoogleAuthProvider);
    };
    return (
        <div className='w-full min-h-screen flex flex-col items-center justify-center'>
            <h2 className='font-bold text-2xl mb-6'>Sign into Blog</h2>
            <button
                className='w-1/3 border-2 border-gray-600 mb-6 rounded-md hover:bg-black hover:text-white px-8 py-4'
                disabled={loading}
                onClick={handleGoogleSignIn}
            >
                {loading ? "Signing in..." : "Sign in with Google"}
            </button>
        </div>
    );
}

The code snippet above triggers the handleSignIn function when a user clicks the Sign in button.

Add the handleSignIn function to the utils.ts file. It redirects the user to the Google authentication page to enable the application to access the user's profile details.

import { signInWithPopup } from "firebase/auth";
import { auth } from "../../firebase";
export const handleSignIn = (provider: any, authProvider: any) => {
    signInWithPopup(auth, provider)
        .then((result) => {
            const credential = authProvider.credentialFromResult(result);
            const token = credential?.accessToken;
            //👉🏻 sign in successful
            if (token) {
                const user = result.user;
                //👉🏻 log user's details to the console
                console.log({ displayName: user.displayName, u_id: user.uid });
                window.location.replace("/");
            }
        })
        .catch((error) => {
            console.log({ error: error.message });
            alert(
                `Refresh page! An error occurred while signing out - ${error.message}`
            );
        });
};

To sign users out of the application, create a handleSignOut function within the utils.ts file and copy the code snippet below into the file. It logs the current user out of the application.

import { signOut } from "firebase/auth";
import { auth } from "../../firebase";
export const handleSignOut = () => {
    signOut(auth)
        .then(() => {
            window.location.replace("/");
        })
        .catch((error) => {
            alert(
                `Refresh page! An error occurred while signing out - ${error.message}`
            );
        });
};

Protecting pages from unauthenticated users

Firebase allows us to determine whether the current user is signed in or not. This can be helpful when you need to protect certain pages from unauthenticated users, such as the Post creation page.

Add the code snippet below to the posts/create/page.tsx file. It listens to the authentication state of the signed-in user and updates the current view of the application based on the authentication status.

import { onAuthStateChanged } from "firebase/auth";
const [userData, setUserData] = useState<any>({});
export default function PostCreate() {
    useEffect(() => {
        onAuthStateChanged(auth, (user) => {
            user ? setUserData(user) : router.back();
        });
    }, [router]);
    return <div>{/** -- UI elements --*/}</div>;
}

How to save and retrieve posts from Firebase Firestore

Here, you'll learn how to save and retrieve from Firebase, and also how to update each posts when a user adds a comment on a post.

Creating new posts

Update the handleCreatePost function within the posts/create/page.tsx file as done below:

import { addDoc, collection, updateDoc, doc } from "firebase/firestore";
import { auth, db, storage } from "../../../../firebase";
import { getCurrentDate, slugifySentences } from "@/app/utils";
import { ref, uploadString, getDownloadURL } from "firebase/storage";

const handleCreatePost = async (e: React.FormEvent<HTMLFormElement>) => {
    e.preventDefault();
    setUploading(true);
    //👇🏻 add post to Firebase Firestore
    const docRef = await addDoc(collection(db, "posts"), {
        author_id: userData.uid,
        title,
        content,
        author_name: userData.displayName,
        pub_date: getCurrentDate(),
        slug: slugifySentences(title),
        comments: [],
    });
    //👇🏻 creates a storage reference using the docRef id
    const imageRef = ref(storage, `posts/${docRef.id}/image`);

    if (coverPhoto) {
        await uploadString(imageRef, coverPhoto, "data_url").then(async () => {
            //👇🏻 Gets the image URL
            const downloadURL = await getDownloadURL(imageRef);
            //👇🏻 Updates the docRef, by adding the logo URL to the document
            await updateDoc(doc(db, "posts", docRef.id), {
                image_url: downloadURL,
            });
        });

        setUploading(false);
        alert("Post created successfully!");
        router.push("/");
    }
};

Now, let me explain the function:

The code snippet below adds the author's ID and name, the post's title, content, published date, and slug to a posts collection within the database.

const docRef = await addDoc(collection(db, "posts"), {
    author_id: userData.uid,
    title,
    content,
    author_name: userData.displayName,
    pub_date: getCurrentDate(),
    slug: slugifySentences(title),
    comments: [],
});

After saving the post's details to Firebase Firestore, the function uploads the post's cover image to Firebase Cloud Storage and updates the posts collection with a new attribute (image_url) containing the URL of the uploaded cover image.

// creates a storage reference using the docRef id
const imageRef = ref(storage, `posts/${docRef.id}/image`);
if (coverPhoto) {
    await uploadString(imageRef, coverPhoto, "data_url").then(async () => {
        //👇🏻 Gets the uploaded image URL
        const downloadURL = await getDownloadURL(imageRef);
        //👇🏻 Updates the docRef, by adding the cover image URL to the document
        await updateDoc(doc(db, "posts", docRef.id), {
            image_url: downloadURL,
        });
    });
}

Fetching existing posts

To fetch all the saved posts from Firebase, add the code snippet below within the app/page.tsx file.

import { collection, getDocs } from "firebase/firestore";
import { useEffect, useState } from "react";
import { db } from "../../firebase";
export default function Home() {
    const [posts, setPosts] = useState<Post[]>();
    useEffect(() => {
        fetchAllPosts();
    }, []);
    const fetchAllPosts = async () => {
        const querySnapshot = await getDocs(collection(db, "posts"));
        const posts: any = [];
        querySnapshot.forEach((doc) => {
            posts.push({ ...doc.data(), post_id: doc.id });
        });
        setPosts(posts);
    };
    return <div>{/** -- UI elements --*/}</div>;
}

The code snippet above retrieves the saved posts from Firebase and displays them within the application.

Adding comments to posts

To add comments to a post, update the postComment function with the Comments.tsx file to fetch the particular post and add the newly created comment to the comments array attribute.

import { getCurrentDate } from "../utils";
import { doc, updateDoc, arrayUnion } from "firebase/firestore";
import { db } from "../../../firebase";
//👇🏻 generate comment ID
export const generateCommentID = () => {
    return Math.random().toString(36).substring(2, 10);
};
const postComment = async (post_id: string) => {
    try {
        const postRef = doc(db, "posts", post_id);
        await updateDoc(postRef, {
            comments: arrayUnion({
                id: generateCommentID(),
                content: newComment,
                author_name: user.displayName,
                pub_date: getCurrentDate(),
            }),
        });
        alert("Comment posted successfully");
    } catch (err) {
        console.log(err);
        alert("An error occurred while posting your comment. Please try again.");
    }
};

Congratulations. You've completed the tutorial.

Conclusion

So far! You’ve learned the following:

what is Firebase
how to add Firebase to a Next.js app
how to work with Firebase Auth, Storage, and Database
how to build a blog app with Next.js and Firebase

The source for this tutorial is available here.

Thank you for reading and If you enjoyed this blog and want to learn more, check out my other articles.

How to Build a RAG Chatbot with Agent Cloud and MongoDB

Ankur Tyagi — Mon, 06 May 2024 12:33:30 +0000

Introduction

Enterprises are constantly seeking ways to improve efficiency, gain a competitive edge, and deliver exceptional customer service. Retrieval-Augmented Generation (RAG) technology is emerging as a powerful tool that addresses these needs by combining information retrieval with AI generation. This innovative approach unlocks a range of benefits that can significantly transform how enterprises operate.

One of the most impactful applications of RAG lies in enhanced customer support. By retrieving information from a company's knowledge base or past customer interactions, RAG can empower chatbots and virtual assistants to provide more accurate and contextually relevant responses. This translates to faster resolution times, improved customer satisfaction, and a reduction in the burden on human support teams.

Another key advantage of RAG is its ability to streamline knowledge management. Enterprises often struggle with vast amounts of unstructured data stored in documents, emails, and reports. RAG tackles this challenge by enabling users to quickly retrieve the information they need. This empowers employees to find answers to internal queries, access relevant documents for decision-making, and conduct research more efficiently, ultimately boosting overall productivity.

RAG goes beyond simply retrieving and generating information. It can also play a crucial role in data analysis. By identifying relevant data points and insights from large datasets, RAG can automate parts of the data analysis pipeline, leading to faster extraction of actionable insights. This empowers enterprises to make data-driven decisions with greater speed and accuracy.

In this blog, we will learn to build a RAG chatbot in minutes using AgentCloud and MongoDB.

AgentCloud is an open-source platform enabling companies to build and deploy private LLM chat apps, empowering teams to securely interact with their data. AgentCloud internally uses Airbyte to build data pipelines allowing us to split, chunk, and embed data from over 300 data sources, including NoSQL databases like MongoDB. It simplifies the process of ingesting data into the vector store for the initial setup and subsequent scheduled updates, ensuring that the vector store information is always updated. AgentCloud uses Qdrant as the vector store to efficiently store and manage large sets of vector embeddings. For a given user query the RAG application fetches relevant documents from vector store by analyzing how similar their vector representation is compared to the query vector.

Setting up Agent Cloud via Docker

To run AgentCloud in Local you must have Docker installed in your system. Then you can execute the below steps to run AgentCloud.

clone the repo:

git clone https://github.com/rnadigital/agentcloud.git

go to the agentcloud directory:

cd agentcloud

run locally using this command:

chmod +x install.sh && ./install.sh

Here is a Quickstart Guide in Docs.

On running the install command it will download the required docker images and start the containers in Docker.

Once the install script is executed successfully we can view the containers running in the docker app:

To access Agent Cloud in browser we can hit the below url:
http://localhost:3000/register

Next, we need to sign to the platform.

Post sign-up, log in to the App, to get to this landing screen.

Congrats. Our setup is complete now.

Next, we will move towards building our RAG application.

Adding New Model

Agent Cloud allows us to use models like FastEmbed and OpenAI in our app.

To add a new model let's go to the Models screen and click the Add Model option.

In the configure screen you can select Model, I have selected fast-bge-small-en model to embed the text content.

Then click on the Save button to complete the model setup.

FastEmbed, a lightweight library with minimal dependencies, is ideal for serverless environments like AWS Lambda. The core model, fast-bge-small-en, efficiently captures text meaning for tasks like classification and retrieval due to its compact size. This combination offers developers a powerful solution for real-time text analysis in serverless deployments.

Post successfully adding the model we will be able to view the model in the Models list.

Creating DataSource

We will be using MongoDB as our data source.

MongoDB is a NoSQL database, that offers a flexible alternative to traditional relational databases. Unlike relational databases with rigid schemas, MongoDB stores data in JSON-like documents, allowing for easy adaptation to ever-changing data structures.

In our MongoDB, we have a database called course_db which contains a collection called course_catalog.

Inside this collection, we have stored different course information.

There are multiple fields in each document but the fields which we are interested in are:

title
description
level
duration
skills_covered
url
meta_data

To access and utilize MongoDB data within the RAG, we'll create a MongoDB data source.

First we need to go to the Data Sources page and click on the New Connection button.

We will select MongoDB as the Datasource.

We will select the Datasource Name as course_db_mongo which is derived from the database name. We will add a short description of the new data source. We have kept the Schedule Type as Manual which means the MongoDB data will get synced to the vectorstore manually.

I am running MongoDB on my local machine with Docker. For connecting Airbyte to MongoDB we need to provide the MongoDB connection string and the Mongo database name. In the cluster type, I have selected Self-Managed Replica Set since MongoDB is running in my local. Rest we can keep the default value as it is.

Next, we need to select the collection that we want to sync, which is course_catalog, we will be syncing all the fields to the vector store.

Post that we need to select the field to be embedded and click continue. The meta_data field in the Mongo DB has all the relevant information required so we will select this field for embedding.

The data source is created now. In the first run, it embeds the Mongo data and stores it in the Qdrant vector store.

We can check the Qdrant DB running in our local to verify the data sync.

The Qdrant DB is running in port 6333 and can be accessed from the below link.
http://localhost:6333/dashboard#/collections

On the collection page we can see a new collection is created.

As the data syncs, this collection gets populated with the documents.

Setting up tools

Tools are an essential component for enabling the AI agent to interact with its environment effectively, process information, and take appropriate actions to achieve its goals. The tools used by an AI agent can include functions, APIs, data sources, and other resources that help the agent perform specific tasks autonomously and efficiently.

The tool we will be setting up will be responsible for querying the data source and fetching relevant documents. Agent Cloud by default creates a tool for us when a new data source is added. In the below screenshot, we can see that the course_db_mongo tool has already been created for us by the platform.

The AI agents read the tool's description and make a judgment about using the tool for the task, so we need to make sure that the description of the tool covers all the information that the agent would require.

Creating Agent

An AI Agent is a sophisticated system that utilizes LLM technology to reason through problems, create plans to solve these problems and execute these plans with the assistance of various tools. These agents are characterized by their complex reasoning capabilities, memory functions, and the ability to execute tasks autonomously.

For creating the agent we will first go to the Agents page and then click on the New Agent button.

This brings us to the agent configuration page where we define the Name, Role, Goal, and Backstory of an Agent. We have selected both the Model and Function Calling Model as Open AI GPT 4.

In the Tools section, we will select the course_db_mongo tool.

If you don’t have the Open AI GPT 4 model configured then you can click on the Model option and add a new model. A modal for configuring new model opens and there you can add the name of the model, the model type, the Credentials which will be the OpenAI API key, and finally the LLM model. On clicking the save button the Open AI GPT 4 model we be configured.

Now click on the Save button on the agent configuration page and a new agent will be created for us.

Creating Task

Tasks are specific assignments assigned to an agent for completion. For creating a new task we need to click the Add Task button in the Task screen.

In the Task configuration page, we need to define the Name and Task Description.

We will be selecting the Tools and Preferred Agent as course_db_mongo and Course Information Agent.

On clicking the Save button a new task will be created for us.

Creating App

We will now look into the App creation part.

In our app, we bind the Agent and Task together to create a conversational RAG. This RAG will help users in answering questions related to courses.

In the app configuration we will select the App Type as Conversation Chat App.

The Task will be the Course Information Taskwhich we created before and the Agent will be the Course Information Agent. We will want the App to process tasks sequentially so we will select the Process as sequential. And finally, we will select the LLM model as OpenAI GPT 4. Then we can click on the save button to save our configuration.

Now lets test our App, on click the play button it will open a chat window for us where we can have conversation.

let's check if there are any Python courses on the list.

The Agent uses the course_db_mongo tool to retrieve the Python courses.

let's see another example where we can inquiring for any beginner course on Google Workspace and agent was able to retrieve the course with the difficulty level as beginner.

Ok, Ok last example in which let's try where we will trying to fetch a web development course with a duration of 2 weeks.

That's it for today.

🌐 Also, don't forget to check out our open source GitHub repository: AgentCloud GitHub Repo.

If you like what you see, give us a ⭐. Just click on the cat.

Thanks. You're cool.

Conclusion

In this blog, we learned to build a RAG chat app with Agent Cloud and MongoDB.

We covered:

How to create a data source
How to do embedding
How to store it in Qdrant DB
How to build tools for Agents
How to create an app where users can interact with their private data using Agent Cloud

🔍 Want to learn more about Agent Cloud? read other blogs.

Agent Cloud vs Google Cloud Agents

Ankur Tyagi — Mon, 29 Apr 2024 09:46:37 +0000

Introduction

Data has become the new gold mine. Businesses of all sizes are scrambling to extract valuable insights and unlock the potential hidden within their ever-growing data warehouses. Generative AI tools powered by Large Language Models (LLMs) are emerging as powerful tools for such tasks.

Based on large pre-trained data and the context given in the input, these AI marvels can generate responses in the form of text, image, code, and even videos that you can leverage for different needs.

But what if LLMs could also leverage external data sources to provide even more insightful and accurate results instead of only relying on pre-trained data? This is where Retrieval-Augmented Generation (RAG) comes in.

RAG takes LLM capabilities to the next level, allowing them to access and integrate information from external databases and repositories.

Imagine an LLM that can write a compelling product description and dynamically incorporate real-time customer feedback to enhance its accuracy. This is the power of RAG.

AgentCloud is an open-source generative AI platform that offers a built-in RAG pipeline to help you securely talk to your data using your preferred LLM. AgentCloud's built-in RAG pipeline simplifies data integration from over 300 sources, including Google BigQuery, Salesforce, Atlassian Confluence, Zendesk, Airbyte, Drive, SharePoint, and OneDrive.

Oh, By The Way

AgentCloud is a community-powered open-source project so If you find value in our work and wish to fuel our journey so the best way to show your support is by starring us on GitHub ⭐

Here is our public roadmap.

Now, let's discuss Google’s most recent addition, Vertex AI Agent Builder. This suite of tools allows developers of all experience levels to build and deploy generative AI applications. It offers a no-code interface, an open-source library(LangChain) for advanced users, and tools to ensure the AI responses are based on real-world data. With features like document processing and security controls, Vertex AI Agent Builder is an all-in-one solution for creating production-ready generative AI experiences.

This article will compare AgentCloud and Vertex AI agent builders, exploring their functionalities, strengths, and weaknesses to help you choose the most suitable solution for your needs.

Understanding AgentCloud - Talk to Your Data.

AgentCloud is an open-source AI application platform designed to empower you and your businesses to build and use conversational AI tools, like ChatGPT, privately within your organization

AgentCloud facilitates through an advanced built-in RAG as a service that allows you to split, chunk, embed, and retrieve data from over 300 sources. You can also upload your own files in the formats of PDF, docx, txt, xlsx, and CSV.

For databases, you can choose specific tables and columns for data ingestion. Once you have your data, you can define how it's processed. This includes splitting and chunking files and choosing an embedding model. Finally, AgentCloud stores the prepared data securely in a vector database and keeps it fresh through manual, scheduled, or automated updates. With everything set up, you can create an AI agent and interact with your data through a chat-like interface.

Suppose you're in the marketing department, scrambling to meet a deadline for a new marketing campaign. You need some data on customer acquisition costs from last quarter, but you can't remember exactly where it's stored. You fire off a quick message to Sarah in Sales.

A few minutes tick by, and you have yet to hear back. Sarah may be engaged in another task or digging through spreadsheets from last quarter, further delaying you.

With AgentCloud, instead of relying on colleagues' availability, you open the AgentCloud interface. It's like having a chat window directly connected to your company's data. You type your question.

"What were our customer acquisition costs in Q3?"

AgentCloud instantly retrieves the relevant data from your CRM, marketing automation tool, or any other designated source. You see a clear breakdown of the costs, presented in a user-friendly format.

This is the power of AgentCloud.

It eliminates the need for endless email chains and searches. You can chat with your data directly, just like you would with a colleague, unlocking valuable insights and accelerating your decision-making process.

AgentCloud is designed to be LLM-agnostic, meaning you can choose the LLM that best suits your needs.

Here's a breakdown of your options:

Open-source LLMs: Since AgentCloud is open-source, you can integrate various open-source LLMs, such as LLAMA2, Mistral 7B, Hugging Face, or BLOOM.
Cloud-based LLMs: AgentCloud allows you to connect to cloud-based LLMs offered by providers like Open AI, Cohere, and Anthropic Claude. This gives you access to powerful pre-trained models like OpenAI's GPT-3 series, known for their capabilities in text generation and code completion. This is perfect for companies that want to get up and running fast or don't want to manage infrastructure.

While AgentCloud excels at facilitating conversations with your data through a chat interface, its capabilities extend far beyond this. AgentCloud's Multi-agent assistant, powered by Autogen, empowers you to build robust, collaborative teams of AI agents. AutoGen provides a multi-agent conversation framework as a high-level abstraction.

With this framework, you can conveniently build a group of agents and provide them access to functions and data sources. Once provided with a prompt, these agents can securely leverage those data sets and converse with each other to solve tasks.

The power of AgentCloud continues beyond multi-agent collaboration. You can further extend your AI agents' capabilities by building custom tools and functions. These custom Python functions allow your agents to interact seamlessly with any third-party API service. Imagine needing sales data from your CRM system or marketing performance metrics from an external platform.

With AgentCloud, you can build custom functions that securely access these APIs and integrate the retrieved data directly into your agent workflows. This empowers you to create genuinely comprehensive AI solutions that bridge the gap between your internal data and external resources.

Finally, AgentCloud allows you to maintain complete control over your data and ensure chat applications don't share more information than needed. We recommend self-hosting on your computer if you’d like to operate in an ultra-secure environment.

You can deploy the entire platform on your infrastructure and leverage private LLM endpoints. This ensures your data remains completely isolated and LLM providers won't have access to train on it. Furthermore, AgentCloud offers granular access controls at multiple levels. Teams can only access data designated for them, while individual AI agents have restricted access to specific data, functions, and LLM models.

Key AgentCloud Features at a Glance

Talk to data, no coding (RAG as a service) - Build chatbots that chat with your data! No coding is needed; use AgentCloud's built-in RAG (fancy term for "knowledge retrieval") feature. Plus, it uses multiple data sources to keep things accurate.
Conversation management - You can build chatbots that know what they're talking about! AgentCloud uses your connected data sources as a giant knowledge base, so your chatbots can answer questions and act like a supercharged internal search engine for your data.
Data on demand (Data ingestion) - AgentCloud syncs information from all your favorite sources (think Confluence, databases, even PDFs!) into a central hub with a built-in Qdrant vector database. Plus, you can choose how often to refresh the data, keeping it nice and up-to-date.
Teamwork makes the dream work (Multi-agent engine) - You can automate complex tasks with a team of AI agents working together. Think of them as your data dream team each with their specialty, but also able to collaborate to get the job done.
You're in control - You get to decide who gets access to what data with AgentCloud's team and user permissions. Keep things organized and secure.
Your data, your rules - Put your data privacy first. AgentCloud lets you deploy it on your cloud servers, keeping your information securely within your control.

Understanding Google Cloud agents - Vertex AI Agents builder

Vertex AI Agent Builder is one of Google's most recent AI platforms. It allows you to create and deploy generative AI experiences for businesses. Unlike traditional coding, Vertex AI Agent Builder allows you to build agents using natural language instructions. This no-code interface makes it accessible to a wider range of developers, even those without extensive programming experience. You can define your agent's goals, provide step-by-step instructions, and share conversational examples to guide its responses.

Vertex AI Agent Builder allows you to integrate your agents with your data for more accurate and relevant responses. It offers pre-built options like Vertex AI Search for out-of-the-box grounding, or for more control, you can build custom Retrieval-Augmented Generation (RAG) systems using its search component APIs. Additionally, vector search capabilities allow for building even more accurate and valuable embedding-based agents.

With Vertex AI data connectors you can also ingest your data from third-party applications like ServiceNow, Hadoop, Salesforce, and other commonly used enterprise systems.

Complex tasks can be broken down into smaller, more manageable components. Vertex AI Agent builder allows you to create multiple agents, with a "main" agent and supporting "subagents." These subagents can collaborate seamlessly, passing information and collaborating to achieve the overall goal.

Vertex AI Agent builder goes beyond prototyping. It provides tools to refine your initial concepts into production-grade agents. You can monitor agent performance in real time, identify areas for improvement, and use natural language training to enhance responses to specific queries. You can monitor key metrics like usage, latency, safety, and cost to identify potential issues and optimize performance over time.

Vertex AI Agent builder includes built-in security features to ensure your agents meet enterprise scaling needs, compliance, and security standards. It also includes features that allow you to easily manage access and ensure the responsible use of AI models and data.

Key Vertex AI Agent builder features at a Glance.

Ditch the complex coding - Vertex AI Agent builder features an intuitive no-code interface that allows you to design conversational AI agents using plain English. Define your agent's goals, provide step-by-step instructions for achieving them, and share sample conversations to guide its responses.
Flexibility - Vertex AI Agent builder caters to non-technical users through the no-code interface. For experienced coders, LangChain on Vertex AI offers a code-first approach for creating highly customized and powerful agents.
Modular agent architecture - For complex tasks, you can build subagents that work seamlessly together. Imagine a customer service scenario where a primary agent handles initial inquiries. At the same time, a subagent retrieves user purchase history directly from your CRM system, providing a more holistic and personalized experience.
Pre-built grounding systems - Out-of-the-box options like Vertex AI search seamlessly connect your agents to your company data, ensuring responses are anchored in reliable sources.
Customizable RAG — You can use Vertex AI's Search Component APIs to build bespoke RAG systems. These systems empower you to process documents, rank retrieved information based on relevance, and communicate with your own data.
Real-time Monitoring and Optimization - Continuously monitor agent performance, identify areas for improvement, and use natural language training to enhance responses to specific queries. This ensures your agents remain relevant and effective over time.
Enterprise-Grade Security and Compliance - Built-in security features ensure your AI agents operate within the boundaries of your organization's compliance standards.

Comparison between AgentCloud and Vertex AI Agent builder

Both AgentCloud and Vertex AI Agent Builder offer compelling features. But which one is the right fit for your needs? Let's delve into their key functionalities to help you decide.

Focus and flexibility: Agent Cloud specializes in data interaction and conversation building, making it ideal for building chatbots that users can interact with to access information. It also offers a built-in process automation functionality. You can leverage multiple AI agents working together "multi-agents" to automate complex and repetitive business processes.

Vertex AI Agent Builder offers both conversational AI agents and process automation functionalities. This allows you to build a broader range of virtual assistants. You can create customer service chatbots, data analysis tools, or even workflow automation tools using subagents and function calls. While it offers process automation, it might not be as robust as Agent Cloud's multi-agent approach designed for this purpose.

Data integration and grounding: AgentCloud boasts a built-in Retrieval-Augmented Generation (RAG) system as a service, facilitating the creation of data-driven chat applications. You can connect to over 300 data sources and embed information for retrieval through your chat app.

Vertex AI agent builder also provides multiple options for data grounding. You can leverage pre-built systems like Vertex AI Search or build custom RAG systems using Search Component APIs. This grants more significant control over how your agents access and utilize data.

Process automation: AgentCloud excels in automating processes by enabling the use of multiple AI agents working together. This is ideal for streamlining complex workflows and eliminating repetitive tasks.

Vertex AI Agent builder: While it supports process automation, its focus is less pronounced than Agent Cloud. However, it offers functionalities like subagents and function calls, allowing for some level of workflow automation.

Development approach and security: AgentCloud’s open-source nature allows for self-deployment on your preferred cloud infrastructure, offering greater control and potentially enhanced data privacy. However, you'll need to manage the underlying applications yourself.

Vertex AI Agent builder on the other hand operates within Google Cloud's infrastructure, providing a managed service with built-in security features for responsible use of AI models and data. This eliminates the need for in-depth infrastructure management but may not offer the same level of customization as a self-hosted solution.

Openness and scalability: Agent Cloud is LLM-agnostic, meaning you can connect your open-source models or use cloud models from providers like OpenAI or Azure. This offers excellent flexibility but may require additional configuration.

At the time of writing this article, Vertex AI Agent Builder supported Google's latest Gemini models. It integrates seamlessly with other Google Cloud services, facilitating scalability within the Google Cloud ecosystem.

What to Consider When Choosing Between Agent Cloud and Vertex Agent Builder.

If your primary focus is building chatbots for users to interact with and gain insights from your data, AgentCloud excels in this area. Its built-in RAG system and data embedding functionalities streamline data retrieval and conversation management within your chat application.

For complex workflows requiring automation with multiple AI agents working together, Agent Cloud's multi-agent approach is a strong choice. AgentCloud also offers a broader range of data source connections (over 300) and the ability to use your open-source models. This provides greater control over integrating and utilizing data within your AI agents.

If you prioritize open-source solutions and want complete control over deployment on your preferred cloud infrastructure, AgentCloud allows for self-deployment. This can be ideal if your company has strict data privacy requirements.

When your project demands a broader range of virtual assistant capabilities, Vertex AI Agent Builder offers both conversational AI agents and process automation functionalities. This allows you to build a broader spectrum of AI-powered tools.

If you have developers new to AI development, Vertex AI Agent builder's no-code interface simplifies the agent creation process. This can significantly reduce development time and resources. AgentCloud has a similar interface.

Vertex AI Agent Builder operates within Google Cloud's infrastructure, offering built-in security features and seamless integration with other Google Cloud services. If you're comfortable using Google's cloud-based AI models Vertex AI Agent Builder is a strong choice.

Conclusion

AgentCloud and Vertex AI Agent builder (Google Cloud Agents) offer outstanding features that empower you to interact with your data through conversation and automate tasks. While Vertex AI Agent Builder excels in its versatility and ease of use, AgentCloud shines for its focus on data interaction and complex process automation with multi-agent workflows. Ultimately, the choice depends on your specific needs.

For the most control over data integration and the power of multi-agent automation, consider exploring AgentCloud. It offers an open-source approach and a wider range of data source connections, making it ideal for organizations with complex data interaction requirements.

That's it for today. I hope you found this article helpful.

Ready to jump in? our Quickstart guide is the perfect next step to get you going with AgentCloud.

Got thoughts or questions?
Have you tried building a RAG Chat App?

🤝 Let's learn from each other.

🔍 learn more about AgentCloud? read out other comparison blogs.📊

Explore how AgentCloud compares with other platforms:

and discover which option suits you the best ✨

How to Build a RAG Chat App With Agent Cloud and BigQuery

Ankur Tyagi — Wed, 17 Apr 2024 08:57:49 +0000

Introduction

Interacting with your data helps you and your team to gain relevant insights and make informed decisions for your business. Retrieval-augmented generation (RAG) chat applications allow you to chat with your data.

Retrieval-augmented generation enhances the accuracy of large language models by allowing them to consult an external data source before generating an output. The output of these LLMs is based on facts gathered from the embedded data sources.

Agent Cloud is an open-source generative AI platform with a built-in RAG as a Service that enables you to build and deploy LLM-powered conversation chat apps for talking with your data.

Agent Cloud’s RAG as a Service also has a built-in data pipeline that allows you to split, chunk, and embed data from over 300 data sources, including BigQuery.

This article will guide you on how to build a RAG chat application using Agent Cloud to privately and securely talk with your Google BigQuery data.

Prerequisite

This article is a comprehensive guide that takes you on a step-by-step journey, explaining each concept and configuration clearly. No prior experience in building RAG chat apps or interacting with Google BigQuery data is required to follow along.

Setting-Up BigQuery

Let’s set up our BigQuery data warehouse. You may skip this section if your data is on BigQuery already.

Create a Google Cloud Platform account.
Create a new project and navigate into the BigQuery pane.

Here, we gave our project the name test-agent-cloud.

You can give yours any name whatever you prefer.

Create dataset. Click on the kebab menu near your project name and select Create Dataset.

Configure the necessary information for your dataset e.g id and location.

Here are the details of our newly created dataset.

Create tables inside your dataset. To create tables, click on the kebab button and select Create table.

Add the table data. You can create an empty table, upload data, or add data from Google Cloud Storage, Drive, Bigtable, Amazon S3, and Amazon Blob Storage. In this demo, we will upload the World Economic Indicator dataset, publicly available data on Maven Analytics.

Here is our dataset schema.

Here is our dataset preview.

We are done setting up our BigQuery data warehouse. Next, we will setup Agent Cloud on our local machine.

Creating Your GCP Service Account Key

You must provide your GCP service account key to embed your BigQuery data into Agent Cloud. This section will show you how to create a service account. You may skip this section if you have your service account JSON already.

Follow the steps below to create and download your service account key:

Inside GCP, navigate to IAM & Admin > Service Accounts ** and select **CREATE SERVICE ACCOUNT.

Fill in the details for your service account.

Assign it a Storage Admin role and click Done.

Your new service account is now live, but no key yet.

Click on the service account email, go to the KEYS tab, select ADD KEY, and then Create new key.

Select JSON as key type then click on create. Your service account JSON or key JSON will automatically download.

Keep it safe. You will need it later - Agent Cloud will request it (as Credential JSON) when embedding your BigQuery data.

Running Agent Cloud Locally

We need to set up Agent Cloud on our local machine to build our RAG chat app. Soon, you will have the option to use Agent Cloud’s managed cloud platform.

Use the following steps to set up Agent Cloud on your local machine:

Clone the repository
Navigate into the repository clone and run Docker Desktop.
Start Services. To start services, run the following command for Mac and Linux users. For Windows users, we recommend using WSL.

chmod +x install.sh && ./install.sh

If everything goes well with the installation, there will be seven containers running.

❗️Heads up: Installing might take a while depending on your computer. It's a good idea to check out our docs for tips on setting up your system. You can find them here.

Once all the Docker containers are running.
Go to http://localhost:3000 to signin/signup on Agent Cloud and start building your application.

This is what Agent Cloud looks like after logging in.

Add Your Model

Go into the Models page and add two models.

the fast embed model
an LLM

The fast embed model is a lightweight model that will run locally on your machine to split, chunk, and embed data.

For your LLM, you can use either OpenAI, Azure OpenAI, or LMStudio and add your credentials.

Connect Datasource

Agent Cloud allows you to split, chunk, and embed data from over 300 sources. In this case, our data source is BigQuery. Let’s connect to BigQuery.

Navigate to the Data Sources screen and click New Connection

Select BigQuery as your datasource

Give your datasource a name and select the sync schedule for your data (for now, you may set it to Manual).

Under the hood, Agent Cloud runs Airbyte on localhost:8000 to process your data. Other application running under the hood are Qdrant (http://localhost:6333/dashboard#/collections) and RabbitMQ (http://localhost:15672)

Add your BigQuery Dataset ID so that Airbyte won’t have to search all datasets which may take too long if you have many datasets. Enter your Google Cloud Project ID and your credentials JSON. Your credential JSON is the service account JSON you created in a previous section of this article.

Note: Ensure to convert your credential JSON to a single line code to avoid getting an error from Agent Cloud.

Select tables and fields to sync. Then select your embedding model.

Create a Tool

Go into the Tools screen and create a new tool.

Create a detailed description of your tool. It helps the LLM decide what tool to use.

Create an Agent

Go into the Agents screen and create a new agent.

Give the Agent a Role, Goal, and Backstory.

Also, select the LLM and tool you want to use. Select the RAG tool that we created earlier. Note that you can use multiple tools if you have multiple data sources.

You can use the following information for the role, goal, and backstory:

Role: You are an AI Assistant.
Goal: Have a back-and-forth conversation with the user.
Backstory: You are an Agent Cloud Assistant, a helpful assistant designed to answer questions provided by a user.

Create a Task

Navigate into the Tasks screen and set up a task.

Add the following description for your task description:

Have a back-and-forth conversation with the user.
Be clear in your answers always.
If you don't know the answer, say "I do not know."

Create an App

Let’s create a conversation interface for chatting with your data.

Go into the Apps and click on New App.

Select Conversation Chat App as the App Type.
Select the Conversation Agent you created earlier.
Also, select the Task you created earlier.
Select Sequential as the Process type.

Save the app and run it.

Start chatting with your data

Yay, we did it 😎

🚀 Want more? Check out our video tutorial! 🎥

If you prefer a visual walkthrough, we've got you covered.

Watch the complete guide on "How to Build a RAG Chat App With Agent Cloud and BigQuery" on our YouTube channel: Video Tutorial

And 👋 hey, if you have any questions or need further assistance, don't hesitate to join our Discord community.

We're here to help and chat anytime.

Join us here.

Conclusion

This article showed a step-by-step guide for building a RAG chat app with Agent Cloud and BigQuery.

Agent Cloud is an open-source generative AI platform that enables organizations and teams to build and deploy a conversation chat app for interacting with their data. The built-in RAG as a Service has a data pipeline that allows you to embed data from over 300 sources. Agent Cloud can be useful for customer service, sales automation, streamlining data analysis, improved employee onboarding, etc.

🔍 Want to learn more? Check out our comparison blogs! 📊

Explore how Agent Cloud compares with other platforms:

Discover which option suits you best ✨

🔥 6 Open-source Libraries You Must Know in 2024 🪄🔥

Ankur Tyagi — Mon, 15 Apr 2024 07:54:47 +0000

Last week I have discovered some awesome libraries!
Check them, use them, help them with a star ⭐️

“I will never stop learning.
I won’t just work on things that are assigned to me.
I know there’s no such thing as a status quo.
I will build our business sustainably through passionate and loyal customers.
I will never pass up an opportunity to help out a colleague, and I’ll remember the days before I knew everything.
I am more motivated by impact than money, and I know that Open Source is one of the most powerful ideas of our generation.

I will communicate as much as possible, because it’s the oxygen of a distributed company. I am in a marathon, not a sprint, and no matter how far away the goal is, the only way to get there is by putting one foot in front of another every day. Given time, there is no problem that’s insurmountable.”

Matt Mullenweg CEO of Automattic

Coolify

Introducing Coolify, an open-source and self-hostable alternative to Heroku, Netlify, and Vercel

Developed by Andras Bacsai, this innovative platform empowers users to deploy and manage their web applications with ease, all while maintaining full control over their infrastructure.

Key Features:

Self-Hostable: With Coolify, you have the freedom to host your web applications on your own servers, giving you complete control over your deployment environment.
Easy Deployment: The platform simplifies the deployment process, allowing you to deploy your applications with just a few clicks or commands.
Scalability: Coolify is designed to scale effortlessly, accommodating growing traffic and workload demands without compromising performance.
Customization: You can customize and configure your deployment environment to suit your specific requirements, ensuring optimal performance and security.
Integration: Coolify seamlessly integrates with popular development tools and workflows, making it easy to incorporate into your existing projects.
Community Support: Benefit from a vibrant community of developers and contributors who provide support, share resources, and collaborate on improving the platform.

Check out Coolify on GitHub and give it a star: Coolify Repository

Glasskube

Glasskube is an open-source Kubernetes package manager that simplifies package management for Kubernetes deployments. It provides a user-friendly interface, reducing complexity and increasing transparency in managing your favorite Kubernetes packages.

Key Features:

User-Friendly UI: Glasskube offers an intuitive user interface, making Kubernetes package management more accessible and understandable for users.
Brew-Inspired CLI: For advanced users familiar with the command-line interface, Glasskube provides a brew-inspired CLI, allowing for efficient package management operations.
Dependency Awareness: Glasskube packages are dependency aware, ensuring smooth management and installation of Kubernetes packages with consideration of dependencies.
Cloud Native Application: Designed as a cloud-native application, Glasskube aligns with GitOps principles, facilitating seamless integration into Kubernetes environments and workflows.

Glasskube stands out as a valuable tool for simplifying Kubernetes package management. Whether you're a novice Kubernetes user or an experienced DevOps engineer, Glasskube offers a user-friendly solution for managing your Kubernetes packages with ease and efficiency.

Check out Glasskube on GitHub and give it a star

AgentCloud

AgentCloud is your go-to platform for setting up your very own AI App playground! Picture it like having your own personal GPT (Generative Pre-trained Transformer) builder platform, but with some awesome extra features.

With Agent Cloud, companies have the flexibility to create and deploy two distinct types of apps for their employees:

Conversational Chat Apps: These apps operate similarly to OpenAI's GPTs, allowing users to develop single-agent chat applications. However, they come with the added advantage of utilizing any Large Language Model (LLM), accessing a rich library of tools, and tapping into a vast repository of knowledge from numerous data sources.
Process Apps: Say hello to automation heaven! These apps help you streamline processes by setting goals and tasks for your agents to tackle. It's all about making work easier, faster, and more collaborative.

So, whether you're looking to boost productivity with nifty chatbots or automate tasks like a pro, Agent Cloud has got you covered. It's all about giving your team the tools they need to work smarter, not harder!

And the best part? It's open-source! If you liked using Agent Cloud, why not show some love and give it a star on GitHub? Your support means the world to open source community. 🌟

Check out Agent Cloud on GitHub and give it a star: Agent Cloud Repository

I also came across a cool demo that gives you a sneak peek into what Agent Cloud is all about. Check out the intro video here, along with some nifty insights on RAG Google Bigquery by the founder himself on YouTube.

CrewAI

CrewAI is a groundbreaking framework designed to orchestrate role-playing and autonomous AI agents. Developed by João Moura, this open-source project empowers agents to collaborate intelligently, enabling them to work together seamlessly on complex tasks.

Key Features:

Collaborative Intelligence: CrewAI fosters collaborative intelligence among AI agents, allowing them to pool their resources and expertise to accomplish tasks more efficiently.
Autonomous Agents: The framework supports the development of autonomous AI agents capable of making independent decisions and taking actions based on their environment and objectives.
Task Orchestration: CrewAI provides tools for orchestrating tasks and coordinating the actions of multiple agents, ensuring smooth collaboration and synchronization.
Scalability: With CrewAI, developers can scale their AI systems to accommodate a large number of agents, making it suitable for applications ranging from small-scale simulations to large-scale environments.
Flexibility: The framework offers flexibility in designing agent behaviors and interactions, allowing developers to tailor the system to suit specific use cases and scenarios.
Community Support: CrewAI benefits from a vibrant community of developers and researchers, fostering collaboration and innovation in the field of autonomous agents and artificial intelligence.

Check out CrewAI on GitHub and give it a star: CrewAI Repository

Learn about the difference between Agent Cloud and CrewAI.

SuperAGI

SuperAGI is a cutting-edge platform designed to push the boundaries of conversational AI, integrating principles of artificial general intelligence (AGI) to create highly intelligent and adaptable chatbots.

Key Features:

Artificial General Intelligence (AGI): SuperAGI leverages AGI principles to develop chatbots with human-like understanding and reasoning capabilities.
Deep Learning Models: The platform harnesses state-of-the-art deep learning models, such as transformers and neural networks, to process and generate natural language responses.
Knowledge Graphs: SuperAGI incorporates knowledge graphs to represent and organize information, enabling chatbots to access a vast repository of structured knowledge.
Meta-Learning: Employing meta-learning techniques, SuperAGI enables chatbots to continuously improve and adapt their behavior based on past interactions and feedback.
Multi-Turn Conversations: Supporting multi-turn conversations, SuperAGI allows chatbots to maintain context and engage in more natural and fluid interactions with users.
Extensibility: The platform is highly extensible, enabling developers to integrate custom modules and expand chatbot functionality as needed.

Explore SuperAGI on GitHub: SuperAGI Repository

Gitroom

Gitroom provides tools and helps your grow your open-source repository. Gain more visibility, stars, contributions and customers.

Key Features:

Unified Collaboration: Gitroom brings team collaboration directly to your Git repositories, enabling seamless communication and coordination.
Real-time Updates: Stay in the loop with real-time updates on repository activity, allowing teams to track changes and progress effortlessly.
Interactive Discussions: Engage in interactive discussions directly within your Git repositories, making it easy to share ideas and provide feedback.
Task Management: Keep track of tasks and issues with built-in task management features, streamlining project workflows and improving productivity.
Integration: Gitroom integrates smoothly with GitHub, providing a familiar and intuitive interface for users already accustomed to the platform.
Customization: Tailor Gitroom to fit your team's specific needs with customizable settings and preferences.

Explore Gitroom on GitHub: Gitroom Repository

In summary, these repositories - AgentCloud, Gitroom, Glasskube CrewAI, Coolify, and SuperAGI - are packed with amazing tools for building smarter applications. Whether you're into chatbots, AI agents, or web deployment, there's something for everyone.

Give them a star ⭐️ on GitHub if you like what you see, and why not use their tools to create something awesome?

See you soon in another blog post.

Thank you for reading and If you enjoyed this blog and want to learn more in depth about developer tools, React and more check out my other articles on my website.

Agent Cloud VS OpenAI

Ankur Tyagi — Thu, 11 Apr 2024 11:07:55 +0000

Introduction

Generative AI is a growing field of artificial intelligence where large language models (LLM) are pre-trained on large amounts of data to generate text, images, video, or voice data.

Sometimes, models may not need to rely solely on pre-training data to generate accurate results. It can leverage external data sources to generate accurate and factually consistent responses. This process is known as Retrieval-Augmented Generation (RAG).

RAG can help organizations talk to their data and generate meaningful insights from the conversation. RAG elevates the capabilities of large language models and helps them generate reliable and contextually relevant content.

Agent Cloud is an open-source generative AI platform that offers a built-in RAG pipeline to help you securely talk to your data using your preferred LLM.

The RAG as a Service offering of Agent Cloud allows you to ingest or sync data from over 300 sources including Google BigQuery, Salesforce, Atlassian Confluence, Zendesk, Airbyte, Drive, SharePoint and OneDrive. You can chat with your data from multiple sources from a single point - Agent Cloud. More than building private LLM chat apps, Agent Cloud also enables you to automate manual processes by assigning tasks to its LLM-powered agents to complete. These agents work in groups to achieve a given objective. With Agent Cloud’s process automation, you can automate complex business processes and abstract away manual redundancies.

⭐️ Support the open-source community, Star Agent Cloud ⭐️

OpenAI is an AI research organization and a technology company building large language models with different capabilities like text, image, and voice generation. It also focuses on the research and building of safe artificial general intelligence (AGI). OpenAI’s products include GPTs for generating text, DALL-E for generating realistic images, and Whisper a speech recognition system for identifying, transcribing, and translating multiple human languages.

This article will do a comparison between Agent Cloud and OpenAI using key indicators like core functionalities, key features, differences, and use cases. In the end, it will help you understand what these tools do and what to consider when choosing between them.

Core Functionalities of Agent Cloud and OpenAI

Agent Cloud and OpenAI, while focused on generative AI, have distinction in their core functionalities. Agent Cloud’s core focus is RAG as a Service for creating conversation chat apps using any LLM of your choice and process automation using multiple AI agents. The RAG pipeline of Agent Cloud enables you to split, chunk, and natively embed data from over 300 sources. This way, you can chat securely and privately with your data.

Under the hood, Agent Cloud uses open source stack like Airbyte for its ELT pipeline, RabbitMQ for message bus, and Qdrant for vector database.

Agent Cloud is completely abstracted so you do not need to manage any of these applications if you don’t want to. Because it is open source, you can deploy it securely to your own cloud service. Currently, it supports OpenAI and Azure OpenAI cloud models, but in the future, it will support all cloud providers, making it easy to use any cloud model you prefer.

Furthermore, with Agent Cloud process automation functionality, you can build process apps that leverage multi-agents to complete tasks. This way, complex and redundant business processes can be automated. The multi-agent engine of Agent Cloud is an abstracted Lanchain-based runtime called CrewAI. The engine enables AI agents to perform and complete their assigned objectives.

The following is a bullet breakdown of the core functionalities of Agent Cloud:

Focus on data interaction and conversation building.
Focus on process automation using multiple AI agents that work in groups to complete tasks.
Ability to connect to various data sources and extract information.
Construction of conversational interfaces (chatbots) for interacting with your data.
Abstraction. You do not need to know how the apps under the hood work.
LLM agnostic. You are not constrained to any LLM - You can connect your open-source model or use OpenAI.

OpenAI, on the other hand, is a proprietary (closed source) AI research and development company.

Although its research interest spans a broad range of AI disciplines, its primary focus is generative artificial intelligence using large language models. It has produced several notable AI systems and LLM, including GPTs, DALL-E, Whisper, Sora, a text-to-video LLM, and OpenAI Gym, a toolkit for reinforcement learning algorithms.

These are some of OpenAI’s core product offerings. Its models are pre-trained on very large data parameters making it capable of generating accurate and factually consistent text, voice, video, and image content.

OpenAI GPTs are advanced large language models pre-trained on massive amounts of data and hence are capable of generating text content like code, letters, articles, etc. GPT-4, OpenAI’s latest and most advanced text generation LLM, is a multimodal model with enhanced capabilities like improved natural language processing. OpenAI’s core functionality is generative AI, focusing on text, voice, video, and image data. Its API allows the integration of LLMs like GPTs, Whisper, and DALL-E into your different projects.

The following is a bullet breakdown of some core functionalities of Open AI:

Research and development of AI systems like GPTs, Whisper, Sora, and DALL-E
Focus on safe and beneficial artificial general intelligence (AGI)
API integration with its LLMs. OpenAI allows you to integrate its LLMs into your projects.

Key Features

Let's explore the main features of Agent Cloud and OpenAI.

Agent Cloud’s Key Features

Below are the key features of Agent Cloud:

Built-in RAG Pipeline: Agent Cloud has a built-in RAG as a Service that enables you to build and deploy conversational chat apps seamlessly. These chat apps allow you to talk to your data and gain relevant insights.
Data Embedding: Split, chunk, and embed data from over 300 data sources that you can chat with in your RAG chat app. Agent Cloud’s built-in vector database stores your data as vectors, making them easily retrievable. You can also specify how frequently you want your data to sync.
Process Automation: Automate business processes using Agent Cloud’s multiple AI agents. These AI agents can work in groups to complete an assigned task. Process automation eliminates manual work by letting capable AI agents handle tasks for you. You can also enable these AI agents to access third-party APIs.
Conversation Management: Manage conversation with your data securely from a single point - Agent Cloud. Agent Cloud, because it is capable of syncing data from multiple sources, can be a powerful search engine for your organization’s data.
Data Privacy and Security: Agent Cloud is open source, allowing you to deploy your application to your own cloud infrastructure. This is important for organizations that are concerned about data privacy.
Permission: Agent Cloud allows you to enable team and user permission for your app.

OpenAI’s Key Features

Below are the key features of OpenAI:

Generative AI Models: OpenAI offers broad generative AI models such as GPTs capable of generating text data, Whisper for voice data generation, Sora for text-to-video generation, and DALL-E for text-to-image generation.
Cutting-Edge AI Research: OpenAI also specializes in cutting-edge research in models capable of natural language processing (e.g text generation, translation, and code completion), computer vision, robotic and artificial general intelligence.
API Offering: Through OpenAI’s API, you can access LLMs like GPTs, Whisper, DALL-E, and Sora for easy integration into your existing projects.
Model Fine-Tuning: OpenAI allows customization of its LLMs for specific tasks. This is called model fine-tuning.

Key Differences Between Agent Cloud and OpenAI

In this section, we will go over some key differences between Agent Cloud and OpenAI.

Focus: Agent Cloud’s core focus is RAG as a Service and process automation. It emphasizes data interaction through building and deploying RAG conversational chat apps using LLMs as a component. OpenAI, on the other hand, prioritizes research and development of LLMs, offering them as API integration into various applications.
Tools and Functionality: Agent Cloud provides a RAG pipeline with native data embedding and a process automation tool with multi-agent task collaboration. OpenAI offers pre-trained LLMs with abilities to fine-tune them for your needs.
Deployment and Control: Agent Cloud gives more data privacy by allowing you to self-deploy to your own cloud. OpenAI offers only Cloud-based and API-level access to their LLMs.
LLM Agnosticism: Agent Cloud is LLM agnostic, allowing you to use any open-source LLM or OpenAI for your conversational chat apps. OpenAI only allows you to use their LLM offerings (GPTs, Sora, DALL-E, and Whisper).

Use Case

Agent Cloud product offerings cater to the needs of businesses in different ways. Let’s highlight some use cases for Agent Cloud:

Building data-driven virtual assistants and chatbots for customer service, sales, or employee interaction.
Interacting with your data to gain relevant insights.
Automating business processes. Automate repetitive tasks like scheduling appointments, business analysis, social media posts, product description, customer analysis, etc.
Enhancing customer experience. Because Agent Cloud can securely access user data, it can be helpful for personalized user engagements like marketing offers and product suggestions.

The following are some use cases of OpenAI:

Text generation for marketing copywriting, content marketing, or code generation.
Identifying, translating, and transcribing languages in both text and voice format.
Integration of LLMs into your existing products.
Using LLMs for research in various fields.
Computer vision. Models like DALL-E 2 help in creative image content creation and imaging analysis.

What to Consider When Choosing Between Agent Cloud and OpenAI

Agent Cloud and OpenAI are both great tools. However, the choice of which to choose depends on your needs.

Agent Cloud offers a RAG pipeline that can embed data from over 300 sources. You can easily create a chat interface for interaction with your data and gain relevant insights from it. Agent Cloud takes privacy seriously, allowing you to deploy your app to your own cloud infrastructure. Furthermore, you can automate business processes, including repetitive tasks using Agent Cloud’s process automation solution. This solution enables you to deploy AI agents that work collaboratively to complete tasks for you.

OpenAI offers extensive AI solutions, including text, voice, and media data generation. Its pre-trained large language models are integrable to your application and can be fine-tuned for specific purposes.

If your needs involve quick deployment of AI-powered chat apps, task automation, data embedding, and privacy, then Agent Cloud should be your go-to tool.

Conclusion

Agent Cloud and OpenAI stand out as great generative AI tools. Both tools have outstanding features. OpenAI shines with API offering, LLM capabilities, AGI research, and model fine-tuning. Plus, it has a large community around it. Agent Cloud shines in RAG chat apps, process automation, conversation management, and data privacy.

Finally, deciding what tools to use between Agent Cloud and OpenAI depends on your project’s needs. However, if you are looking to quickly build chatbots, virtual assistants, or other conversation AI applications, Agent Cloud is a good choice. Its built-in RAG pipeline simplifies data integration from multiple sources. It also prioritizes data privacy by allowing deployment to your own cloud infrastructure. If you need to build and deploy LLM-powered conversational chat apps quickly using any LLM of your choice or deploy process apps, consider using Agent Cloud.

Enjoyed this article? check out more blog on our website or Join our discord to learn/discuss more about Agent Cloud.

Agent Cloud vs CrewAI

Ankur Tyagi — Fri, 05 Apr 2024 16:26:48 +0000

Introduction

The demand for collaborative and generative artificial intelligence is increasing as more companies are interested in interacting with their data in a natural language, gaining valuable insights from the interaction, and automating complex business processes. AI collaboration or collaborative intelligence, refers to humans and AI working together to build powerful and efficient systems that automate tasks, make decisions, interpret data, and generate relevant outputs. Generative AI and collaborative intelligence tools like Agent Cloud and CrewAI are important for conversing with your data and automating processes.

Agent Cloud is an open-source platform that enables companies to build LLM-powered conversational chat apps to talk with their data. These conversational chat apps are private and secure. They can retrieve information from hundreds of data sources. Furthermore, Agent Cloud allows you to build process apps for automating tasks and complex business processes.
It uses multiple AI agents to ensure autonomous, collaborative, and scalable process applications that can access third-party APIs. The built-in data pipeline of Agent Cloud allows for the splitting, chunking, and embedding of data from more than 300 data sources including Postgres, Docs, Slack Google Drive, etc, making your RAG app seamless.

Oh, By The Way Before We Proceed- Do You Know

Agent Cloud is like having your own GPT builder with a bunch extra goodies.

The Top GUI features Are:

RAG pipeline which can natively embed 260+ datasources
Create Conversational apps (like GPTs)
Create Multi Agent process automation apps (crewai)
Tools
Teams+user permissions. Get started fast with Docker and our install.sh

We're open source so the easy way to support us is by giving a star on GitHub ⭐

CrewAI is a collaborative process automation framework that utilizes multiple AI agents working together to automate assigned tasks. It simplifies the processes of building multi-agent automation. You can deploy your multi-agent automation within seconds using the prebuilt templates. CrewAI also provides a flexible approach for building your automation locally.

In this article, we will compare Agent Cloud and CrewAI exhaustively using indicators like architectural paradigm, key features, use cases, similarities, and differences to help you understand how these tools work and how to select the more appropriate one for your organization's needs.

Architectural Philosophy

Next, let's go into the architectural philosophy of both tools - Agent Cloud and CrewAI - to understand how things work at their core.

Agent Cloud

Agent Cloud is an open-source generative AI platform that uses a modular architecture designed to scale with your organization. This means users can go fully private by deploying to their own cloud environment or connecting it to their locally hosted model.

With Agent Cloud being LLM agnostic, you are not constrained to any LLM. It also connects to any open-source model or defaults to OpenAI.
Agent Cloud has a built-in RAG as a Service that can ingest data from over 300 sources for knowledge retrieval for conversational chat apps. Alternatively, you can upload your own files. It supports pdf, docx, txt, xlsx, and csv files.

Cron expressions help you control the sync frequency of your data from their various sources. Agent Cloud allows you to select what tables and fields get synced. You can also specify how you want your data to be split and chunked.

The tool supports both character splitting and semantic chunking for files. For data sources like Bigquery and others, it automatically chunks the messages that come through the message bus. Soon, Agent Cloud users can select the fields to embed and fields to store as metadata from their data sources, and shortly, it will enable a vector upsert to give more flexibility for custom chunking.

The Agent Cloud multi-agent engine is an abstraction of CrewAI. The multi-agent engine powers the AI agents that help you automate manual processes.

Under the hood, Agent Cloud uses the following open-source stack:

Airtbyte for its ELT pipeline
RabbitMQ for message bus.
Qdrant for vector database.

Although you can access Airbyte and Qdrant (the vector database instance) locally, you don’t have to manage any of these apps. Agent Cloud is fully abstracted.

For installation, Agent Cloud currently has a docker install.sh script for Mac/Linux users. WSL is recommended for users on Windows.

CrewAI

CrewAI is an open-source framework for multi-agent collaboration built on Langchain. As a multi-agent runtime, Its entire architecture relies heavily on Langchain.

Building multi-agent automation with CrewAI involves the following:

Installation: Install CrewAI and other dependencies using pip.

pip install crewai

pip install 'crewai[tools]'

Agents Assembly: CrewAI allows you to define your agents and assign distinct roles and backstories to them. Doing so helps agents in the way they execute tasks and how they interact in the crew.
Task Definition: After assembling and defining your agents, next is to define tasks. Defining tasks helps set the objectives for your agents.
Crew Formation: A crew is a group of agents working collaboratively to complete an assigned task. Form your agents into a crew and set the workflow process then start the task execution process.

Functionalities like verbose mode, language model configuration, and memory capabilities can be incorporated into your crew to improve performance. You can choose to let your crew perform tasks based on either a sequential process or a hierarchical process. The former process executes tasks one after the other while the latter follows the hierarchical approach.

Although CrewAI uses OpenAI’s GPT-4 model by default for language processing, it allows connection with any LLM. Because it was built on top of Lanchain, CrewAI is compatible with all Langchain LLM components thus enabling diverse integrations. It also allows the flexibility of customizing agents and integrating human input into agent execution.

Key Features

Let’s do a breakdown of the key features of Agent Cloud and CrewAI

Key Features of Agent Cloud:

RAG as a Service: Agent Cloud has a built-in RAG as a Service feature for building conversational chat apps that interact with your data. The RAG as a Service feature can reference multiple data sources to mitigate hallucination.
Data Ingestion/Syncing: Agent Cloud allows you to sync data from multiple sources with a built-in Qdrant vector database. You can also specify the sync frequency for your data.
Conversation Management: Agent Cloud allows you to build chatbots that interact with users based on the information retrieved from your data. Because it uses connected data sources as knowledge points, it is a powerful internal search engine for your data.
Multi-Agent Engine: Automate complex business processes with multiple AI agents working together to complete tasks. The agents can specialize in different tasks and also collaborate to complete a task. Single-agent applications like chatbots can also be created with Agent Cloud.
Permissions: Agent Cloud allows team and user permissions.
Data Privacy: You can deploy Agent Cloud to your cloud servers, keeping your data within your control.

Key Features of CrewAI:

The following are the key features of CrewAI:

Multi-Agent Collaboration: Multi-agent collaboration is the core of CrewAI’s strength. It allows you to define agents, assign distinct roles, and define tasks. Agents can communicate and collaborate to achieve their shared objective.
Role-Based Design: Assign distinct roles to agents to promote efficiency and avoid redundant efforts. For example, you could have an “analyst” agent analyzing data and a “summary” agent summarizing the data.
Shared Goals: Agents in CrewAI can work together to complete an assigned task. They exchange information and share resources to achieve their objective.
Process Execution: CrewAI allows the execution of agents in both a sequential and a hierarchical process. You can seamlessly delegate tasks and validate results.
Privacy and Security: CrewAI runs each crew in standalone virtual private servers (VPSs) making it private and secure.

Similarities and Differences

Next, let's explore the similarities and differences between Agent Cloud and CrewAI

Similarities

They both allow you to connect to any LLM including local LLMs like Ollama.
Both Agent Cloud and CrewAI are open source.
They both take privacy and security seriously.
They both leverage multi-agent collaboration.

Differences

Core Focus: Agent Cloud is dual-focused. It offers RAG as a Service (single agent conversational chat app) for interacting with your data and also offers process automation using its multi-agent engine. CrewAI’s only focus is process automation.
Underlying Technology: Agent Cloud uses a modular architecture that is scalable with your organization. The open-source stack like Airbyte and Qdrant that it uses under the hood are well abstracted so you don’t need to manage any of them. CrewAI was built on top of Lanchain making it easy to access pre-trained LLMs thereby reducing your development workload.
Use Cases: Agent Cloud shines in information processing from multiple data sources and task automation. It therefore has a more diverse use case than CrewAI. The use cases for CrewAI are streamlined to process automation.
Community and Support: Agent Cloud currently has a small but growing community compared to CrewAI which benefits from the larger Langchain community and resources.

Choosing Between Agent Cloud and CrewAI: What to Consider

Choosing between Agent Cloud and CrewAI depends largely on your use cases and needs. Unlike CrewAI, the use cases for Agent Cloud are more robust. It combines the collaborative process automation in CrewAI with a built-in RAG as a Service for building private and secure conversational chat apps.

CrewAI shines in facilitating communication, task delegation, result validation, and resource sharing among agents to achieve an objective. Agent Cloud on the other hand shines for both process automation and RAG app use cases. It can sync to hundreds of data sources to retrieve information and present it through conversational interfaces like chatbots. The ability to reference data from multiple sources mitigates hallucination.

Ultimately, Agent Cloud is a more robust solution that covers your conversational chat apps and process automation needs.

Conclusion

Tools like Agent Cloud and CrewAI leverage generative artificial intelligence and collaborative intelligence to facilitate natural language interaction with data and automate complex and redundant business processes.

While both tools share similarities such as LLM compatibility, open-source nature, and emphasis on privacy and security, they differ in their core focus, underlying technology, use cases, and community support. Agent Cloud offers a diverse range of use cases, combining conversational chat apps and process automation. At the same time, CrewAI excels in facilitating communication and collaboration among agents for process automation tasks.

In conclusion, the choice between Agent Cloud and CrewAI depends on your organization’s specific use cases and needs. However, if you are looking for a robust solution for your RAG and process automation needs, you should consider trying out Agent Cloud.