DEV Community: Amos Gyamfi

Compare the 3 Best Agentic CLI Coding Tools

Amos Gyamfi — Mon, 09 Jun 2025 07:09:10 +0000

Discover how CLI-based AI coding tools let you work directly with LLMs and agents from your Terminal. Use them to improve code, debug and fix errors, and collaborate on a Git project without ever opening an IDE. Agentic AI coding tools vary in how they help you write, debug, and ship code.

Some, like Lovable and Bolt, allow developers to quickly build web and mobile apps using prompts. Others, like Cursor and Windsurf, provide developers with a fully AI-featured IDE to solve engineering problems.

Generally, AI coding platforms can be categorized into the following:

CLI-Based Agents: Interact with AI agents through the command line using Aider, Claude Code, and Codex CLI.
AI Code Editors: Interact with agents through GitHub Copilot, Cursor, and Windsurf.
Vibe Coding: Build web and mobile applications with prompts using Bolt, Lovable, v0, Replit, Firebase Studio, and more.
AI Teammate: A collaborative AI teammate for engineering teams. Examples include Devin and Genie by Cosine.

This article focuses on the first group: CLI-based coding platforms. You'll explore the top three tools, compare their capabilities, and see how to get started with each one.

What Is a CLI Coding Tool?

Think of a CLI-based AI coding tool as any LLM like Claude, OpenAI models, or Gemini in your Terminal. This category consists of closed and open-source tools that allow developers to work on engineering projects directly by accessing coding agents from model providers like Anthropic, OpenAI, xAI, and Google.

To understand how CLI tools differ, consider how IDE-based agents like Cursor work: You pick the agent you want to use in your project and add a prompt to begin interacting with it. Cursor then presents a UI to accept, reject, and review the agent's changes based on your prompt.

In contrast, CLI coding tools streamline that experience. You run commands directly through the Terminal at the root of your project. After the agent analyzes your code, it asks yes/no questions about the task without leaving the Terminal.

CLI-Based Coding Platforms vs. AI-First Code Editors

CLI-based coding platforms use a minimal interface to communicate with AI models. Since these tools have minimal UI, you get things done faster. There are no chunky interfaces for confirming changes to your source code when communicating through the command line.

By contrast, interacting with agents via AI code editors is sometimes difficult due to clunky interfaces. For example, accepting code sections and reviewing the entire code may be confusing in Cursor since it involves several clicks and UI changes.

Although developers can use both categories to work on large projects, command-line coding tools provide raw and low-level access to LLMs. They do not offer a graphical UI, so there’s a steeper learning curve and more control.

Many developers do not consider the pricing and cost per session aspects of using these agentic coding tools, whether working alone or in a team. It is recommended that the pricing for using the model on each platform be checked before starting a project.

Although Codex CLI and Aider are open-source and free to use, using Claude Code, for example, can be expensive.

If you do not regularly check the cost of running an agent, it can cost you about 6x Cursor's monthly cost of $20.

CLI Coding Tools Use Cases Across Teams

CLI-based agentic tools have many application areas, from engineering to research teams. They work well across both small and large code bases.

Use cases include:

Codebase reviews: Answer questions, fix bugs, commit code changes, and create pull requests.
Vibe coding: CLI coding tools excel at vibe coding like traditional AI code editors. They make it quicker to skim, review, or supervise generated code easily.
Repetitive engineering tasks: Automate the repetitive and boring development tasks that teams often procrastinate.
Productivity booster for ICs: Help individual contributors on your team increase their coding productivity by about 10x.
Finance: Use these tools for non-coding functions like editing and improving data workflows, or even parsing CSVs for finance or operations teams.
Data teams: Build production-ready dashboard apps for visualizing information.
Infrastructure teams: Perform effective security analysis, approvals, and reviews.
Marketing: Evaluate the performance of Ads and campaigns, and suggest improvements.
DevRel: Write, edit, review, and manage documentation and technical tutorials in a GitHub repo.
Product design team: Perform visual edits using prompts and manage the state of designs.

The 3 Leading Command-Line Coding Platforms

As the number of AI coding editors increases weekly, several agentic coding tools for the command line may emerge soon. Still, as of this writing, three platforms stand out for their adoption, capabilities, and developer experience.

The next sections break down what each one offers and how to get started.

What is Aider?

Think of Aider as an AI pair engineer in your command-line utility. Aider lets developers select the LLMs they prefer when creating a new project or working with an existing codebase. Unlike Claude Code in this category, it is open-source and has the most installs, GitHub stars, and more than 135 contributors.

Key Features of Aider

Being open-source, Aider has evolved to support many features that suit small and large code bases.

Get usage information: Aider displays information about the sent and received tokens and the cost of your messages after each session. This feature helps you unlock which models are cheaper for your projects.
Local and cloud LLMs support: Access state-of-the-art models like Claude 4 Opus and Sonnet, OpenAI o3, DeepSeek-R1-0528, and more.
Auto Git support: Aider automatically commits your changes to GitHub.
Access via an IDE: Use Aider through the integrated Terminal of IDEs like Cursor or GitHub Copilot.
Voice mode: Instead of writing instructions to perform tasks in Aider, use your voice as input.
Linting support: Automatically lint and test your code after changes.
Image and web contexts: Use screenshots and web content to provide context when working with Aider.
Instant code changes: Aider shows code changes directly in the Terminal and syncs them instantly in your IDE.

Get Started With Aider

Aider is available on Pypi and provides a simple installation. To start a new project or work with an existing one, use the following commands to install Aider and configure your preferred LLM. Check out its user guide to discover the various commands you can use for your projects.

python -m pip install aider-install
aider-install

# Change directory into your codebase
cd /to/your/project

# DeepSeek
aider --model deepseek --api-key deepseek=<key>

# Claude 3.7 Sonnet
aider --model sonnet --api-key anthropic=<key>

# o3-mini
aider --model o3-mini --api-key openai=<key>

The sample project below uses OpenAI’s o3-mini model to generate a SwiftUI code for an existing codebase.

What is Claude Code?

Claude Code is a Terminal tool from Anthropic designed for agentic coding to help developers work faster using Claude models. It works across several environments, codebases, and languages.

Unlike Aider and Codex CLI, Claude Code is closed-source. The agent can modify files, fix errors, execute test commands in your codebase, and integrate them with GitHub Actions for automated pull request management and code reviews. Since it understands any codebase, you do not need to provide files and directories as context for your project.

It also has powerful built-in tools like WebSearch, WebFetch, and MultiEdit for editing your codebase. To ensure enhanced security, using some of these tools requires configuring the permissions of your working environment.

Claude Code also has a command-line SDK for integration with applications. With the CLI-based SDK, you can build AI systems and assistants that use the capabilities of Claude models.

Key Features of Claude Code

Claude Code stands out with its privacy and security by default and its ability to seamlessly integrate with enterprise applications. It makes it easy to manage security to safeguard your workflow and working environment. You can also define and specify robust permissions for tool access to restrict who can access what.

Top features of Claude Code include:

Connect third-party APIs: Claude Code projects can securely integrate enterprises with applications and deploy them on Amazon Bedrock and Vertex AI. It also provides secure, built-in tools for user management for large teams.
Code privacy and security: Your prompts and private data stay with your Anthropic's API dashboard only and are inaccessible to third-party servers.
Prompt injection prevention: It provides ready-made tools to prevent any code base of enterprise teams from prompt injection, a technique used by attackers to manipulate an LLM’s instruction.
Tracing and usage monitoring: Claude Code supports OpenTelemetry, allowing you to observe and trace the performance of selected models and monitor their usage.
CLI commands: Work effectively with Claude Code using CLI flags and slash commands such as --model, /cost, and /review. To learn more, check out the Claude Code guide.
Vim mode: Easily switch to Vim mode (/vim) for Vim keybindings and command modes. You can also use the /configure slash command to set up your Vim mode.
Connect your favorite IDE: Claude Code provides developers seamless integration with popular IDEs such as VS Code, Cursor, Windsurf, PyCharm, WebStorm, and IntelliJ for improved workflows.
Instant launch: Press cmd + esc and ctrl + esc to quickly launch Claude Code from your Terminal.
Built-in memory: Claude Code can keep information about your sessions, commands, and style guides in its memory. This feature helps it remember your project preferences and actions for faster workflow.

Get Started With Claude Code

Like the other CLI coding tools, Claude Code makes it easy to start and create your first project. However, unlike Aider, a Python package, Claude Code's installation requires Node 18+.

To start, run this command to install and authenticate it.

npm install -g @anthropic-ai/claude-code

The following preview demonstrates Claude Code’s usage when writing a test.

Using Claude Code in a large team can be expensive if several people use it to contribute to a codebase. To monitor a team's usage closely, you can limit Claude Code Workflow spending. You can learn more about it in this Anthropic guide.

Use Claude Code With Non-Anthropic Models

Due to the incredible coding capabilities, developers love Claude models like 4 Opus and Sonnet, and even previous ones like 3.5 and 3.7 Sonnet. Engineering teams and indie devs normally complain about these models because of their high running costs. As highlighted in some sections of this article, running 4 Opus or Sonnet in Claude Code can cost you 6x the monthly cost of Cursor. For this reason, you and your team may want to use Claude Code with non-Anthropic models like Gemini 2.5 Pro, which is much cheaper but has coding performance on par with the SOTA models of Anthropic. Luckily, you can utilize Claude Bridge, allowing you to use Claude Code with Germini, OpenAI, xAI, and Ollama models.

The example above uses Claude Code with OpenAI's gpt-4.1 model.

What is Codex CLI?

Codex CLI is an experimental, lightweight, and fully open-source coding agent that lives and pairs programs with you in the Terminal.

You can collaborate with it to read and modify your source code and run tests using closed and open-source models. It uses the reasoning capabilities of OpenAI's o3 and o4-mini models and integrates them seamlessly into your local environment. However, you can bring your preferred models, such as Gemini 2.5 Pro, Claude 4 Opus, and Sonnet.

Key Features of Codex CLI

Easy setup: Once your OpenAI API key is ready, you can start using Codex CLI by running only two commands.
Approval modes: Decide how you want an agent to read and write files with approval modes such as Suggest, Auto Edit, and Full Auto. Like the other platforms, it can accept and approve changes safely and securely.
Multimodal input: Start a conversation and implement features by passing low-fidelity sketches, diagrams, and images to your model.
Configure codebase standards: When using Codex CLI, add an AGENTS.md file to the root of your project or in your repo. Like README.md, you can define in this file how an agent should navigate your codebase, testing commands, projects' style guide, standard practices, etc.
Security and permissions: Codex CLI safeguards your applications against prompt injection and refuses malicious requests. With its built-in security, you can define different levels of autonomy for an agent using the --approval-mode flag.
Low-latency code Q&A: It uses codex-1 as the default model, an optimized version of o4-mini that assists with faster code editing and workflows. You can also use the model via the OpenAI API ( codex-mini-latest).
Sign in with ChatGPT: Codex CLI supports logging in with ChatGPT. This feature lets you connect your developer account without manual API token generation and setup.

Get Started With Codex CLI

Similar to the other command-line AI coding platforms, creating your first project with Codex CLI requires running only two commands. Ensure you have Node.js 22 or newer (LTS recommended) installed. Globally install the tool and store your API key as an environment variable.

npm install -g @openai/codex
export OPENAI_API_KEY="your-api-key-here"

Now, you can use Codex CLI for your projects by running codex in your Terminal. The preview below uses Codex CLI to commit changes to a technical tutorial, create a new branch, and push it to GitHub.

This pull request shows that the tutorial changes have been successfully pushed to GitHub by Codex CLI.

What to Consider When Using CLI Coding Platforms

CLI coding assistants can help solve engineering tasks easily by giving developers raw access to an AI model without interacting with a UI. But they also come with tradeoffs.

Watch for Hidden Costs

Some tools, like Aider and Codex CLI, are open-source and free. Others, like Claude Code, can get expensive fast. Each prompt interaction consumes tokens, and daily use can easily exceed $6 per developer. In team settings, that adds up quickly. To track and monitor your costs, you should always check out your Anthropic dashboard’s usage history. Additionally, you could consider setting a limit on your workspace spending for your Claude Code workspace.

Learning Curve and UX Tradeoffs

If you are not a power CLI user, interacting with these agents via the Terminal may look dull and intimidating. Unlike AI IDEs like Cursor, CLI platforms offer no graphical interface for organizing files or visualizing agent suggestions. That can make managing extensive engineering projects, where context is crucial, more challenging.

If that’s a concern, consider Cursor for managing large projects instead. For example, Cursor easily tags files and folders for a precise response.

Set Your Project Up for Success

To get the best out of these Terminal coding assistants, always start your projects with AGENTS.md or Claude.md files when using Codex CLI or Claude Code.

You can use them to specify things like:

Test instructions
Core files
Code styling
Guidelines
Other important information you want an agent to remember in the files

Whenever you send a query, the coding assistant retrieves the information in the files to guide its operations and help it respond with accurate results.

The Future of CLI Coding Platforms

Command-line AI coding tools like Aider, Claude Code, and Codex CLI help developers work faster and improve workflows with task delegation and automation. They are direct alternatives to leading AI code editors such as VS Code and Cursor, and they open a new landscape for engineers to collaborate with AI agents. As engineering teams and indie developers gradually adopt these tools, we should prepare for a future where agentic coding is done primarily within your Terminal.

As these platforms evolve, expect stronger integration with third-party APIs, improved security features, and support for real-time interaction, possibly even voice collaboration. With better enterprise-grade controls and broader model support, CLI agents are poised to become an essential part of the modern dev stack.

Top 7 Frameworks To Integrate MCP With LLMs

Amos Gyamfi — Wed, 14 May 2025 10:36:09 +0000

Toolkits for AI agents expose developers to various APIs to equip AI solutions with tools to carry out tasks and ensure accurate results for user satisfaction. However, integrating these tools into AI apps and managing them can be messy. This article introduces you to an industry standard of providing context to LLMs and agents using Model Context Protocol (MCP).

LLM Context Provision Approaches and Specifications

By default, without giving LLMs and AI chatbots a proper context, they cannot fetch real-time information, execute code, call external tools and APIs, or even use a web browser on behalf of users. Developers can utilize the following approaches to fix this limitation of LLMs and agents.

Composio: Composio has specifications and a library of toolkits for integrating AI agents and LLMs. Aside from Composio's library of ready-made toolkits, they recently announced Composio MCP, allowing developers to connect with 100+ MCP servers for IDEs. Check out the Composio MCP tool categories from the above link to connect several applications to your projects in MCP-supported IDEs like Cursor, Claude, and Windsurf.

Agents.json: A specification built on top of OpenAI standards to ensure seamless and enhanced interactions between AI agents and their access to APIs and external tools. Although Agent.json is an excellent specification, it is not widely used and has not been adopted, unlike MCP. Refer to its GitHub repo to learn more and get started.
MCP: MCP gives developers the best way to provide contextual data to LLMs and AI assistants to solve problems. For instance, you can build an MCP documentation server to offer complete access to your documentation to IDEs and agentic frameworks, just like using llms.txt file.

What is MCP?

Think of MCP as a third evolution of LLMs. In the first evolution, we had LLMs capable of answering user prompts accurately if they found the queries in their training data. In this phase, they cannot meaningfully respond to prompts outside their training data since they don’t have access to external tools. In LLMs' second evolution, we give them access to additional context (tools) that are not intuitive to work with. However, they are capable of helping the LLMs to predict and answer user intents accurately. The third evolution still consists of LLMs and tools, but we implement a proper infrastructure to give them access to external applications and ensure they are easily maintainable.

When building an AI service, your data may live in the cloud for an AI assistant app that answers customer support tickets in an enterprise setting. MCP is an open-source protocol from Anthropic that you can use to connect your enterprise data to AI systems.

It offers a standard way to communicate and link content repositories (GitHub, Notion), development environments, web, and business tools to assistive AI technologies. A popular and continuously growing use case of MCP is AI-assisted coding. Hundreds of MCP integrations with development environments and tools like Cursor and Windsurf allow developers to connect and interact with external applications for development.

Note: This article aims to implement MCP with AI assistants and agentic systems developers built using Python and TypeScript, not IDE-based MCP integrations.

How MCP Works

In the context of LLMs and agents, MCP assists them in providing meaningful responses to user queries outside their built-in knowledge. For instance, ask ChatGPT to message a particular Slack channel, check availability on your calendar, and schedule a meeting with a teammate today. You will be disappointed by ChatGPT's response because it does not have access to these applications. Implementation of MCP helps these assistants output useful outcomes.

The first question developers typically ask is how MCP works. In MCP's basic operation, a user sends a query to an agent. The agent then decides which MCP server and tool to call to get relevant information for the operation. The agent then uses the data from a specific tool to respond to the user.

Why Adopt MCP for AI Agents and LLM-based Apps?

MCP is becoming a standard that helps developers build AI systems where these systems can effectively communicate with other external applications. Microsoft recently announced the integration of MCP in its Copilot Studio to simplify how AI apps and agent access tools. Also, OpenAI has announced the support of MCP across its products, such as the Agents SDK and the desktop app for ChatGPT. There is nothing wrong with equipping tools directly with AI assistants. However, it becomes cumbersome for an AI agent system consisting of several multi-agents performing multiple tasks, such as reading and answering emails, doing web scrapping, financial analysis, fetching real-time weather information, etc.

An AI Agent With A Tool Integration

In the diagram above, three external tools are connected to the LLM. Managing and securing them all will be frustrating if the number increases to, for example, 100+. An improved approach will be accessing the same tools or more than 100+ of them through an MCP registry, as demonstrated in the following section.

An AI Agent With MCP Integration

In this diagram, we combine the tools needed for the agentic system and access them through an MCP server to provide a more cohesive user experience. The MCP approach makes securing and managing these tools via a central location easier.

Advantages of Using MCP Over Traditional Toolkits Integration

MCP has several key benefits over using the traditional way of integrating tools with AI agents. For example, the reliability of tool integration without MCP is questionable because it can result in several errors when making multiple API calls to external applications due to incompatible AI infrastructures. Before MCP, every tool you would like to add to agents must be implemented with custom code, which takes several weeks to implement.

Architecture: Unlike the pre-built tool specifications for AI agents, MCP has a clean and flexible architecture for interacting with tools and APIs.
Improved external tools access and management: It provides tools access to AI models through a standardized interface to bridge the communication gap between LLMs and their interactions with third-party systems.
Addresses the limitations of a stand-alone tool implementation: MCP tools are suitable for single-user scenarios and teams.
Community-driven: MCP has many open-source servers and a developer ecosystem. It is also widely adopted in the developer community for many use cases.
Authentication: It has a robust built-in authentication and permissions system to control tool access. For example, when using Composio-provided MCP tools, you can authenticate users with Google Sheets or Gmail.
Tools search: Unlike the traditional method of installing, configuring, and integrating tools with AI chatbots, MCP makes searching for and finding external tools easier.
Scalability: MCP is easily scalable to many users and applications.
Industry standard: You can install and hard-code tools to provide context to AI applications. However, MCP provides an industry standard to give the required context to agents and LLMs.

Kinds of MCP Servers

The Anthropic's specification of MCP has two forms of servers for adding tools to agents and AI projects. These MCP server connection types include the following.

Server-Sent Events (SSE): Connects to a remote service via HTTP.
STDIO: Allows execution of local commands and communication via standard I/O.

The framework you select to build your AI applications provides the necessary classes to connect to these servers.

Access an Ecosystem of MCP Registries/Servers

There are several open-source libraries of hosted MCP tools to augment LLMs and agents to ensure the reliability of the responses they generate. These libraries of hosted MCP tools are called registries that provide curated collections of services. You can connect your AI applications to the following registries to utilize their tools. Also, you can use different server types, such as uvx, which consists of Python-based tools without installation requirements. There is also a Docker option for running MCP tools and an npx-based server that requires installing Node.js.

MCP servers on GitHub: A collection of community-built servers with additional MCP resources.
Glama Registry: Production-ready and open-source MCP servers for developers.
Smithery Registry: With Smithery, developers can access 2000+ MCP servers to augment the capabilities of AI agents and LLMs.
** OpenTools **: OpenTools provides generative APIs for MCP tool use. You can access hundreds of ready-made MCP tools to implement with your AI projects. Using the OpenTools API, developers can extend the capabilities of LLMs’ web search, fetch real-time location data, and web scrape. The API supports Curl, Python, and TypeScript. Visit the OpenTools quickstart guide to use the API.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.opentools.com",
    api_key="<OPENTOOLS_API_KEY>"
)

completion = client.chat.completions.create(
    model="anthropic/claude-3.7-sonnet",
    messages=[
        { "role": "user", "content": "Compare specs of top 5 EVs on caranddriver.com" }
    ],
    tools=[{ "type": "mcp", "ref": "firecrawl" }]
)

PulseMCP Registry: Using PulseMCP, you can browse hosted MCP tools and use cases for your AI projects. Check out PulseMCP News for recently trending MCP servers and apps.
mcp.run: This registry gives developers access to hundreds of MCP apps for their businesses.

Composio Registry: Composio's SSE-based MCP servers allow easy integration of tools with different AI frameworks to build applications.
guMCP: Gumloop's guMCP provides free, open-source, and fully hosted MCP servers for seamless integration with any AI app.

Top 7 Client Frameworks to Add MCP to LLMs and Agents

Although MCP has become a buzzword, and all developer communities have recently been discussing it, it isn't easy to know which MCP client frameworks to use for integrating with AI apps and agents. We researched and found the following leading MCP client platforms for Python and TypeScript-based agentic workflows and AI assistants.

Note: The following sections demonstrate the implementation of MCP in frameworks for building AI solutions and not MCP integration with AI code editors like Cursor or Windsurf.

1. Build a Git MCP Agent With OpenAI Agents SDK

When building agents with the OpenAI Agents SDK, you connect to these community-built MCP servers using the SDK's MCPServerStdio and MCPServerSse classes. The following MCP agent implementation accesses the root directory of your local Git repo and responds to user queries about the repo.

import asyncio
import shutil
import streamlit as st
from agents import Agent, Runner, trace
from agents.mcp import MCPServer, MCPServerStdio

async def query_git_repo(mcp_server: MCPServer, directory_path: str, query: str):
    agent = Agent(
        name="Assistant",
        instructions=f"Answer questions about the localgit repository at {directory_path}, use that for repo_path",
        mcp_servers=[mcp_server],
    )

    with st.spinner(f"Running query: {query}"):
        result = await Runner.run(starting_agent=agent, input=query)
        return result.final_output

async def run_streamlit_app():
    st.title("Local Git Repo Explorer")
    st.write("This app allows you to query information about a local git repository.")

    directory_path = st.text_input("Enter the path to the git repository:")

    if directory_path:
        # Common queries as buttons
        col1, col2 = st.columns(2)
        with col1:
            if st.button("Most frequent contributor"):
                query = "Who's the most frequent contributor?"
                run_query(directory_path, query)

        with col2:
            if st.button("Last change summary"):
                query = "Summarize the last change in the repository."
                run_query(directory_path, query)

        # Custom query
        custom_query = st.text_input("Or enter your own query:")
        if st.button("Run Custom Query") and custom_query:
            run_query(directory_path, custom_query)

def run_query(directory_path, query):
    if not shutil.which("uvx"):
        st.error("uvx is not installed. Please install it with `pip install uvx`.")
        return

    async def execute_query():
        async with MCPServerStdio(
            cache_tools_list=True,
            params={
                "command": "python", 
                "args": [
                    "-m", 
                    "mcp_server_git", 
                    "--repository", 
                    directory_path
                ]
            },
        ) as server:
            with trace(workflow_name="MCP Git Query"):
                result = await query_git_repo(server, directory_path, query)
                st.markdown("### Result")
                st.write(result)

    asyncio.run(execute_query())

if __name__ == "__main__":
    st.set_page_config(
        page_title="Local Git Repo Explorer",
        page_icon="📊",
        layout="centered"
    )
    # Change from async to synchronous implementation
    # Since Streamlit doesn't work well with asyncio in the main thread

    # Define a synchronous version of our app
    def main_streamlit_app():
        st.title("Local Git Repo Explorer")
        st.write("This app allows you to query information about a Git repository.")

        directory_path = st.text_input("Enter the path to the git repository:")

        if directory_path:
            # Common queries as buttons
            col1, col2 = st.columns(2)
            with col1:
                if st.button("Most frequent contributor"):
                    query = "Who's the most frequent contributor?"
                    run_query(directory_path, query)

            with col2:
                if st.button("Last change summary"):
                    query = "Summarize the last change in the repository."
                    run_query(directory_path, query)

            # Custom query
            custom_query = st.text_input("Or enter your own query:")
            if st.button("Run Custom Query") and custom_query:
                run_query(directory_path, custom_query)

    # Run the synchronous app
    main_streamlit_app()

The code above integrates Streamlit with the OpenAI MCP agent, allowing you to chat with your local Git repo using the Git MCP server. To run this example, you should install the following.

pip install streamlit openai-agents mcp-server-git.
Then, export your OpenAI API keys using export OPENAI_API_KEY=sk-.... When you run the Python file, you should see a result similar to this preview.

You can explore other OpenAI MCP examples from GitHub.

One advantage of using the Agents SDK's MCP integration is its built-in MCP agent monitoring system on OpenAI's dashboard. This feature automatically captures your agents' MCP operations, such as tool listing, POST responses, and getting data about function calls. The image below represents traces of the Git MCP example in this section after running the code above. You can access all the logged information from your OpenAI's dashboard.

2. Build MCP AI Agents With Praison AI

Praison AI is a Python-based AI framework for building a team of agents. It provides the simplest way to add the MCP server tools to agentic workflows with a single line of code, just like you equip agents with traditional tooling.

The following example integrates the Airbnb MCP server with a Praison AI agent using a Streamlit UI to help find apartments in specified locations. You should install these to create your first MCP agent with Praison AI.

pip install praisonaiagents mcp streamlit
Next, export your OpenAI API key export OPENAI_API_KEY='sk-proj-qZIGbi....

Create a Python file, for example, streamlit_praison_airbnb_mcp_agent.py, and fill out its content with this code.

import streamlit as st
from praisonaiagents import Agent, MCP

st.title("🏠 Airbnb Booking Assistant")

# Create the agent
@st.cache_resource
def get_agent():
    return Agent(
        instructions="""You help book apartments on Airbnb.""",
        llm="gpt-4o-mini",
        tools=MCP("npx -y @openbnb/mcp-server-airbnb --ignore-robots-txt")
    )

# Initialize chat history
if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat history
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

# User input form
with st.form("booking_form"):
    st.subheader("Enter your booking details")

    destination = st.text_input("Destination:", "Paris")

    col1, col2 = st.columns(2)
    with col1:
        check_in = st.date_input("Check-in date")
    with col2:
        check_out = st.date_input("Check-out date")

    adults = st.number_input("Number of adults:", min_value=1, max_value=10, value=2)

    submitted = st.form_submit_button("Search for accommodations")

    if submitted:
        search_agent = get_agent()

        # Format the query
        query = f"I want to book an apartment in {destination} from {check_in.strftime('%m/%d/%Y')} to {check_out.strftime('%m/%d/%Y')} for {adults} adults"

        # Add user message to chat history
        st.session_state.messages.append({"role": "user", "content": query})

        # Display user message
        with st.chat_message("user"):
            st.markdown(query)

        # Get response from the agent
        with st.chat_message("assistant"):
            with st.spinner("Searching for accommodations..."):
                response = search_agent.start(query)
                st.markdown(response)

        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": response})

# Allow for follow-up questions
if st.session_state.messages:
    prompt = st.chat_input("Ask a follow-up question about the accommodations")
    if prompt:
        search_agent = get_agent()

        # Add user message to chat history
        st.session_state.messages.append({"role": "user", "content": prompt})

        # Display user message
        with st.chat_message("user"):
            st.markdown(prompt)

        # Get response from the agent
        with st.chat_message("assistant"):
            with st.spinner("Thinking..."):
                response = search_agent.start(prompt)
                st.markdown(response)

        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": response})

Running the sample code will call the required Airbnb MCP tools to look for apartments for you in a specific location, as demonstrated in the image above.

You have noticed it adds MCP support to the agent with a single line of code tools=MCP("npx -y @openbnb/mcp-server-airbnb --ignore-robots-txt"), where npx represents the command to run to start the MCP server. -y is the command line argument to pass to the command. Refer MCP Servers in the OpenAI Agents SDK documentation to learn more.

3. Using MCP for LangChain AI Apps

LangChain has tool-calling support for MCP. This support allows you to set up Python functions to access different MCP servers and retrieve tools to perform tasks in AI projects. The sample code below connects to a secure MCP file system server, enabling an LLM to answer questions about any file you provide accurately.

# Copyright (C) 2024 Andrew Wason
# SPDX-License-Identifier: MIT

import asyncio
import pathlib
import sys
import typing as t

from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
from langchain_core.output_parsers import StrOutputParser
from langchain_core.tools import BaseTool
from langchain_groq import ChatGroq
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

from langchain_mcp import MCPToolkit


async def run(tools: list[BaseTool], prompt: str) -> str:
    model = ChatGroq(model_name="llama-3.1-8b-instant", stop_sequences=None)  # requires GROQ_API_KEY
    tools_map = {tool.name: tool for tool in tools}
    tools_model = model.bind_tools(tools)
    messages: list[BaseMessage] = [HumanMessage(prompt)]
    ai_message = t.cast(AIMessage, await tools_model.ainvoke(messages))
    messages.append(ai_message)
    for tool_call in ai_message.tool_calls:
        selected_tool = tools_map[tool_call["name"].lower()]
        tool_msg = await selected_tool.ainvoke(tool_call)
        messages.append(tool_msg)
    return await (tools_model | StrOutputParser()).ainvoke(messages)


async def main(prompt: str) -> None:
    server_params = StdioServerParameters(
        command="npx",
        args=["-y", "@modelcontextprotocol/server-filesystem", str(pathlib.Path(__file__).parent.parent)],
    )
    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read, write) as session:
            toolkit = MCPToolkit(session=session)
            await toolkit.initialize()
            response = await run(toolkit.get_tools(), prompt)
            print(response)


if __name__ == "__main__":
    prompt = sys.argv[1] if len(sys.argv) > 1 else "Read and summarize the file ./readme.md"
    asyncio.run(main(prompt))

Before running this Python script, you should install the required dependencies, langchain-core, langchain-groq, and langchain-mcp.

pip install langchain-core langchain-groq langchain-mcp

The MCP configuration above uses the npx server type. So, you should install the server-filesystem package.

pm install -g @modelcontextprotocol/server-filesystem
With all the required packages installed, if you add a file to your project and reference it in the Python script as shown in the sample code ./readme.md above, you should see an output similar to this image.

Note: This example was taken from LangChain’s GitHub repo.

4. Using MCP for Chainlit AI Apps

Chainlit is a platform for building AI applications in Python. It has built-in support for MCP servers, so you can configure your app to discover available MCP tools and integrate tool calls into your application’s flow for improved results. You can integrate Chainlit apps with server-sent events (SSE) and command-line (stdio) based services. In the following example, we will connect a Chainlit app to the Linear MCP server to allow the app to manage Linear issues, projects, and teams. You can use the Linear tools available in this example to create, update, search, and get user issues or add a comment to an issue.

Configure Your Chainlit App To Connect to an MCP Server

Connecting your Chainlit app to access tools from an MCP server involves two main steps.

Register the MCP connection: In this step, you should implement Chainlit's on_mcp_connect async function to create a successful connection. You can also implement the on_mcp_disconnect function to handle cleanup.

# pip install chainlit

import chainlit as cl
from mcp import ClientSession

@cl.on_mcp_connect
async def on_mcp_connect(connection, session: ClientSession):
    """Called when an MCP connection is established"""
    # Your connection initialization code here
    # This handler is required for MCP to work

@cl.on_mcp_disconnect
async def on_mcp_disconnect(name: str, session: ClientSession):
    """Called when an MCP connection is terminated"""
    # Optional handler: Cleanup your code here

Configure the MCP Client (Chainlit, LangChain, Mastra): For an MCP server to work with a Chainlit app, the client should provide connection details via the Chainlit's UI. This configuration involves the following.

A unique identifier representing the name of the connection.
Client type: You should specify whether you want to use sse or stdio. For sse, you should add a URL endpoint. When using stdio, a complete command (e.g., npx your-tool-package or uvx your-tool-package) is required. A complete command example is shown below.
npx -y linear-mcp-server --tools=all --api-key=lin_api_your_linear_API_Key

After establishing an MCP server connection, you can execute tools using an MCP session. Finally, you integrate the MCP tools seamlessly with the models/agents of your Chainlit app via a tool call. You can find the complete source code for this Linear MCP integration from Chainlit's example apps on GitHub.

When you get the above source code from Chainlit's GitHub repo, run it, and set npx -y linear-mcp-server --tools=all --api-key=lin_api_your_linear_API_Key through the Chainlit interface, you will then be able to create and update Linear issues/projects. However, performing these actions requires your Linear team's ID.

5. Integrate MCP for Agno AI Agents

Agno is a Python framework for building complex agentic workflows. It is popular for its simplicity, ease of use, and seamless integration with MCP servers. The example MCP implementation in this section integrates with a multi-agent team of four individual contributing agents, such as Airbnb, Google Maps, web search, and weather MCP agents. The Agno multi-agents collaborate to provide information about a trip in a specified location.

Prerequisites

To test the Agno MCP implementation example in this section,

Install Agno, DuckDuckGo, and Exa: pip install -U openai agno duckduckgo-search exa-py.
Get a GOOGLE_MAPS_API_KEY and add it to your project's .env file.
Get an APIFY_TOKEN and add it to your .env.
Validate the Google Address API.

Configure the Agno MCP Team of Agents

For this step, you should define your MCP server parameters and manage multiple context managers with AsyncExitStack. Then, create the agents and run them.

    # Define server parameters
    airbnb_server_params = StdioServerParameters(
        command="npx",
        args=["-y", "@openbnb/mcp-server-airbnb", "--ignore-robots-txt"],
        env=env,
    )

    maps_server_params = StdioServerParameters(
        command="npx", args=["-y", "@modelcontextprotocol/server-google-maps"], env=env
    )

    # Use contextlib.AsyncExitStack to manage multiple async context managers
    async with contextlib.AsyncExitStack() as stack:
        # Create stdio clients for each server
        airbnb_client, _ = await stack.enter_async_context(stdio_client(airbnb_server_params))
        maps_client, _ = await stack.enter_async_context(stdio_client(maps_server_params))

        # Create all agents
        airbnb_agent = Agent(
            name="Airbnb",
            role="Airbnb Agent",
            model=OpenAIChat("gpt-4o"),
            tools=[airbnb_client],
            instructions=dedent("""\
                You are an agent that can find Airbnb listings for a given location.\
            """),
            add_datetime_to_instructions=True,
        )

Get the complete source code from Agno’s GitHub repo.

6. Using MCP for Upsonic Agents

Upsonic is a Python framework for creating AI agents. Using Upsonic, you can build your agents, define tasks for the agents, and handle each task definition with an MCP tool, as demonstrated in the sample code below.

import os
from dotenv import load_dotenv
from upsonic import Task, Agent, Direct
from upsonic.client.tools import Search  # Adding Search as a fallback tool

# Load environment variables from .env file
load_dotenv()

# Get the OpenAI API key from environment variables
openai_api_key = os.getenv("OPENAI_API_KEY")
if not openai_api_key:
    raise ValueError("OPENAI_API_KEY not found in .env file")

# Set your OpenAI API key for the session
os.environ["OPENAI_API_KEY"] = openai_api_key

# Define the HackerNews MCP tool
# Using the correct MCP setup for HackerNews based on Upsonic documentation
class HackerNewsMCP:
    command = "uvx"
    args = ["mcp-hn"]
    # No environment variables are needed for this MCP

# Create a task to analyze the latest HackerNews stories
# Adding Search as a fallback in case HackerNews MCP fails
task = Task(
    "Analyze the top 5 HackerNews stories for today. Provide a brief summary of each story, "
    "identify any common themes or trends, and highlight which stories might be most relevant "
    "for someone interested in AI and software development.",
    tools=[HackerNewsMCP, Search]  # Include both HackerNews MCP and Search tools
)

# Create an agent specialized in tech news analysis
agent = Agent(
    "Tech News Analyst",
    company_url="https://news.ycombinator.com/",
    company_objective="To provide insightful analysis of tech industry news and trends"
)

# Execute the task with the agent and print the results
print("Analyzing HackerNews stories...")
agent.print_do(task)

# Alternatively, you can use a Direct LLM call if the task is straightforward
# print("Direct analysis of HackerNews stories...")
# Direct.print_do(task)

# If you want to access the response programmatically:
# agent.do(task)
# result = task.response
# print(result)

In the example above, we create an AI agent in Upsonic that retrieves the top five most recent stories from Hackernews. If you pip install upsonic and run the Python code above, you should see an output similar to this image.

7. Using MCP for Mastra Agents

Mastra is a TypeScript framework for building prototypes and production-ready AI agents. Like Chainlit, Mastra offers a standardized way to connect to MCP servers to access a wide range of tools with stdio or sse-based connections.

To connect your Mastra agent to an MCP server, you should use its MCPConfiguration class. This class handles multiple server connections, such as lifecycle, namespacing, and tools, in any Mastra agentic workflow. Creating a connection between a Master app and an MCP server involves the following.

Create an instance of the MCPConfiguration class and add server configurations.
Retrieve MCP tools using the getTools() or getToolsets() method.

The sample code below represents a basic usage of implementing an MCP server with a Mastra agent.

import { MCPConfiguration } from "@mastra/mcp";
import { Agent } from "@mastra/core/agent";
import { openai } from "@ai-sdk/openai";

const mcp = new MCPConfiguration({
  servers: {
    stockPrice: {
      command: "npx",
      args: ["tsx", "stock-price.ts"],
      env: {
        API_KEY: "your-api-key",
      },
    },
    weather: {
      url: new URL("http://localhost:8080/sse"),
    },
  },
});

// Create an agent with access to all tools
const agent = new Agent({
  name: "Multi-tool Agent",
  instructions: "You have access to multiple tool servers.",
  model: openai("gpt-4"),
  tools: await mcp.getTools(),
});

Refer to Mastra’s MCPConfiguration to learn more.

Challenges and What’s Next for MCP in LLM Apps and Agents

This tutorial introduces you to MCP and explains why it has become popular in the developer community. We highlight MCP integration with IDEs such as Cursor and Windsurf. In addition to the above, we implemented MCP in seven different Python and TypeScript frameworks for building LLM-based apps, AI assistants, and agents.

MCP's awesomeness also comes with the following challenges. When searching for MCP tools for your projects, you may find it challenging to examine or verify the quality and see the exact applications for your AI project. This is because its tool search and discovery are not yet standardized. Also, its configuration does not offer a consistent user experience due to the different schemas of the MCP server providers.

Currently, the MCP ecosystem is discussing standardizing various aspects of MCP. In the future, there might be a standard way to install MCP-based apps, just like how we pip install packages in Python. PulseMCP also attempts to make browsing and discovering MCP easier.

How To Run OpenAI Agents SDK Locally With 100+ LLMs and Custom Tracing

Amos Gyamfi — Wed, 07 May 2025 15:58:19 +0000

The OpenAI Agents SDK for Python provides developers with the building blocks to implement two agentic solutions for AI applications. You can create text-generation agents, allowing users to get responses from text prompts. Additionally, you can build voice agents using the SDK. To create your first agent with the OpenAI Agents SDK, get started here.

This tutorial will cover three main goals and aspects of building AI agents.

100% local: Use the OpenAI Agents SDK to build agentic workflows that run locally on your computer without compromising business data and private information.
Use 100+ models: The Agents SDK can now use open-source models and 100+ LLMs, eliminating the limitation of using only OpenAI models.
Own tracing: Implement a custom solution for logging and monitoring the performance of your agents.

If these three goals excite you, continue reading to unlock the best way to run the OpenAI Agents SDK entirely locally using the hundreds of AI models you prefer and monitoring their operations with trusted solutions.

Prerequisites

Before we create the demo projects for this article, ensure you have installed the following Python packages and libraries. This is our tech stack.

OpenAI Agents SDK: Build multi-agent and voice AI apps.
Ollama: Run large and small language models locally.
DeepSeek-R1-Distill-Llama-8B: Download a distilled version of DeepSeek R1 from Hugging Face.
Streamlit and Gradio: Interact with OpenAI agents via an AI chat UI.
AgentOps: Tracing and observing agents' performances.
LiteLLM: Gives developers access to 100+ LLMs via Ollama.

Why is Tracing Important in Agentic Workflows?

When building AI applications, tracing operations of the underlying LLMs powering the app helps developers build more reliable solutions. Tracing also assists you in auditing your app to gain insights into logs, errors, and prompt injection attacks.

Record the agent's events: When an agent runs, you can collect a comprehensive record of events like LLM calls and conversations. Evaluate prompts and LLM Switching costs: This will help you gain insights into model spending. Track latency: Easily monitor LLM output generations, tool calls, and handoffs and detect failures.
Debug and visualize agentic workflows: See interactive visualizations of logs and fine-tune your agents for better performance.

What Problems Are We Solving?

The OpenAI Agents SDK is an excellent choice for building voice and text generation agents. It has more features than even some of the leading agent-building frameworks for Python. However, depending on your use case and your business's needs, you may be interested in the following ways to use the SDK.

There are several reasons why developers may want to experiment with their agentic projects locally. Your data may be sensitive, and you want to keep it offline on your machine. You can use any compatible LLM for your agents instead of being locked to a specific vendor. Another reason may be the ability to try several open-source models and the free choice of closed ones.

Run locally

We want to use the Agents SDK but keep our data private and offline without sending it to OpenAI.

Not limited to only the OpenAI models

OpenAI recommends creating agents using its Responses API. However, since the responses API supports only OpenAI models, using it means you are vendor-locked into only OpenAI and its family of large language models. Therefore, we will find alternative ways to utilize the Agents SDK entirely but using non-OpenAI models.

Tracing

When building agentic workflows, tracing or agent monitoring is one of the key features to consider and implement in your project. The OpenAI Agents SDK is open-source, but its built-in monitoring system is not. This feature is turned on by default, but can be turned off if you do not want your agents to be traced. In this tutorial, we will look at how to implement an alternative open-source tracing option for the agents we build.

Build a File System MCP Agent

Let's create an OpenAI agent to read files and answer questions about them using the File System MCP server on GitHub. The agent will access the contents of your specified files and respond to queries about them. We will use the following text files on GitHub, but feel free to replace them with yours.

Create a Python file, for example, filesystem_mcp_streamlit_agent.py, and fill it out with the following sample code. Here, we interact with the file system agent via a Streamlit UI. Running the code successfully requires installing the OpenAI agents SDK and Streamlit.

# https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem

import streamlit as st
import asyncio
import os
import shutil
import tempfile

from agents import Agent, Runner, gen_trace_id, trace
from agents.mcp import MCPServer, MCPServerStdio


# Create a sample file for demonstration if needed
def ensure_sample_files():
    current_dir = os.path.dirname(os.path.abspath(__file__))
    samples_dir = os.path.join(current_dir, "sample_files")

    # Create the directory if it doesn't exist
    os.makedirs(samples_dir, exist_ok=True)

    # Create a sample WWDC predictions file
    predictions_file = os.path.join(samples_dir, "wwdc25_predictions.md")
    if not os.path.exists(predictions_file):
        with open(predictions_file, "w") as f:
            f.write("# WWDC25 Predictions\n\n")
            f.write("1. Apple Intelligence features for iPad\n")
            f.write("2. New Apple Watch with health sensors\n")
            f.write("3. Vision Pro 2 announcement\n")
            f.write("4. iOS 18 with advanced customization\n")
            f.write("5. macOS 15 with AI features\n")

    # Create a sample WWDC activities file
    activities_file = os.path.join(samples_dir, "wwdc_activities.txt")
    if not os.path.exists(activities_file):
        with open(activities_file, "w") as f:
            f.write("My favorite WWDC activities:\n\n")
            f.write("1. Attending sessions\n")
            f.write("2. Labs with Apple engineers\n")
            f.write("3. Networking events\n")
            f.write("4. Exploring new APIs\n")
            f.write("5. Hands-on demos\n")

    return samples_dir


# Using a separate event loop to run async code in Streamlit
class AsyncRunner:
    @staticmethod
    def run_async(func, *args, **kwargs):
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        try:
            return loop.run_until_complete(func(*args, **kwargs))
        finally:
            loop.close()


# Function to run a query with error handling
def run_agent_query(query):
    try:
        # Create a fresh server and agent for each query
        samples_dir = ensure_sample_files()

        async def run_query():
            server = None
            try:
                server = MCPServerStdio(
                    name="Filesystem Server, via npx",
                    params={
                        "command": "npx",
                        "args": ["-y", "@modelcontextprotocol/server-filesystem", samples_dir],
                    },
                )

                # Enter the server context
                mcp_server = await server.__aenter__()

                agent = Agent(
                    name="Assistant for Content in Files",
                    instructions="Use the tools to read the filesystem and answer questions based on those files.",
                    mcp_servers=[mcp_server],
                )

                trace_id = gen_trace_id()
                with trace(workflow_name="MCP Filesystem Query", trace_id=trace_id):
                    result = await Runner.run(starting_agent=agent, input=query)
                    return result.final_output, trace_id
            finally:
                # Make sure to properly exit the server context
                if server:
                    await server.__aexit__(None, None, None)

        return AsyncRunner.run_async(run_query)
    except Exception as e:
        st.error(f"Error processing query: {str(e)}")
        return f"Failed to process query: {str(e)}", None


def main():
    st.title("File Explorer Assistant")
    st.write("This app uses an AI agent to read files and answer questions about them.")

    # Ensure sample files exist
    ensure_sample_files()

    # Input area for user queries
    query = st.text_area("Ask me about the files:", height=100)

    if st.button("Submit"):
        if query:
            with st.spinner("Processing your request..."):
                result, trace_id = run_agent_query(query)

                if trace_id:
                    st.write("### Response:")
                    st.write(result)

                    trace_url = f"https://platform.openai.com/traces/trace?trace_id={trace_id}"
                    st.write(f"[View trace]({trace_url})")

    # Sample queries
    st.sidebar.header("Sample Queries")
    if st.sidebar.button("List all files"):
        with st.spinner("Processing..."):
            result, trace_id = run_agent_query("Read the files and list them.")
            if trace_id:
                st.write("### Files in the system:")
                st.write(result)

    if st.sidebar.button("WWDC Activities"):
        with st.spinner("Processing..."):
            result, trace_id = run_agent_query("What are my favorite WWDC activities?")
            if trace_id:
                st.write("### WWDC Activities:")
                st.write(result)

    if st.sidebar.button("WWDC25 Predictions"):
        with st.spinner("Processing..."):
            result, trace_id = run_agent_query("Look at my wwdc25 predictions. List the predictions that are most likely to be true.")
            if trace_id:
                st.write("### WWDC25 Predictions Analysis:")
                st.write(result)


if __name__ == "__main__":
    # Let's make sure the user has npx installed
    if not shutil.which("npx"):
        st.error("npx is not installed. Please install it with `npm install -g npx`.")
    else:
        main()

The output of the sample code will be similar to this image when you run it.

In the image preview above, clicking View trace will take you to your OpenAI dashboard, where you can view logs about the agent, such as the list of tools, responses, and more.

Configure Your Local Environment

To run any agent or even the previous file system MCP locally, we can use a tool like Ollama and supported AI models such as DeepSeek R1, Phi-4, etc. Check out the Prerequisites section at the beginning of the article to download and install Ollama and a distilled GGUF version of DeepSeek R1. To ensure Ollma and the DeepSeek R1 model will operate successfully, run this command in your Terminal.

ollama run deepseek-r1

You will be given an output in the Terminal similar to the following to initiate a new chat with the model.

>>> Send a message (/? for help)

What we did was just for testing purposes. Let's continue from the next section by converting our previous OpenAI file system MCP agent into a local one.

Ollama-Supported Models and Streamlit UI Example

In this section, we modify the previous section’s code to use the OpenAI SDK but run the agent locally using a non-OpenAI model. To run the file system MCP agent with an Ollama-supported model (DeepSeek R1), create a new Python file and copy the content of local_openai_filesystem_mcp_agent.py from GitHub to fill out yours.

The part of the sample code worth mentioning is implementing the OpenAI agent to run locally via Ollama. The code snippet below can initialize any open-source AI model Ollama supports. To serve the model through Ollama, you specify the name of the open-source model and the base URL base_url="http://localhost:11434/v1".

# Initialize the model and agent
local_model = OpenAIChatCompletionsModel(
    model="deepseek-r1:8b",
    openai_client=AsyncOpenAI(base_url="http://localhost:11434/v1")
)

After downloading and running the complete code from GitHub, you should see an output similar to this preview.

Our example local OpenAI agent works excellently with all the models Ollama provides. However, you may need more model support options for large-scale or enterprise AI applications than those available on Ollama. Since we use an open-source model, DeepSeek R1, the agent's operations won't be traced on your OpenAI dashboard for this example. In the next section, we will implement another tech stack to run the agent with any model provider.

Ollama + Gradio + LiteLLM: Access 100+ LLMs

This section will build an OpenAI agent in Python that can call 100+ APIs and LLMs. We will implement the example with the following tech stack to make it possible to work with any model provider.

OpenAI Agents SDK: Provides the underlying agent-building API.
Ollama: To support running any selected model locally.
LiteLLM: Enables the app to access 100+ models.
Gradio: To build a UI to chat with the OpenAI agent.

The main objective is to extend the demo agent we created to make it more accessible to any model. Let's start by installing the following.

pip install gradio litellm

Then, launch your favorite IDE, create a new Python file, and use the following sample code for its content.

Note: Why create the UI for this example using Gradio instead of Streamlit? Gradio has seamless support for LiteLLM and Ollama, making the integrations much more manageable.

# https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem

import asyncio
import os
import shutil
import gradio as gr
import litellm

from agents import Agent, Runner
from agents.mcp import MCPServer, MCPServerStdio

# Configure litellm to use Ollama
litellm.set_verbose = True

# List of available models - can be expanded
AVAILABLE_MODELS = [
    "ollama/deepseek-r1:8b",  # Default model
    "ollama/llama3:8b",
    "ollama/mistral:7b",
    "ollama/gemma:7b",
    "ollama/phi3:mini",
    # Add other providers if needed
    "openai/gpt-3.5-turbo",
    "anthropic/claude-3-haiku"
]

DEFAULT_MODEL = "ollama/deepseek-r1:8b"

async def process_query(query, mcp_server, model_name):
    # Configure LiteLLM to use the selected model
    litellm.model = model_name

    # Create the agent 
    agent = Agent(
        name="Assistant for Content in Files",
        instructions="Use the tools to read the filesystem and answer questions based on those files.",
        mcp_servers=[mcp_server],
    )

    # Run the agent
    result = await Runner.run(starting_agent=agent, input=query)
    return result.final_output


async def handle_query(query, model_name):
    current_dir = os.path.dirname(os.path.abspath(__file__))
    samples_dir = os.path.join(current_dir, "sample_files")

    async with MCPServerStdio(
        name="Filesystem Server, via npx",
        params={
            "command": "npx",
            "args": ["-y", "@modelcontextprotocol/server-filesystem", samples_dir],
        },
    ) as server:
        output = await process_query(query, server, model_name)
        return output


def gradio_query(query, model_name):
    if not shutil.which("npx"):
        return "Error: npx is not installed. Please install it with `npm install -g npx`."

    # Check if Ollama is selected but not running
    if model_name.startswith("ollama/"):
        try:
            # Simple check to see if Ollama is accessible
            asyncio.run(litellm.acompletion(
                model=model_name,
                messages=[{"role": "user", "content": "test"}],
                max_tokens=1
            ))
        except Exception as e:
            return f"Error: Ollama is not running or model {model_name} is not available. Please start Ollama first."

    try:
        result = asyncio.run(handle_query(query, model_name))
        return result
    except Exception as e:
        return f"Error: {str(e)}"


def create_demo():
    with gr.Blocks() as demo:
        gr.Markdown("# File Content Agent")
        gr.Markdown("Ask questions about files in your sample directory.")

        with gr.Row():
            with gr.Column():
                model_selector = gr.Dropdown(
                    choices=AVAILABLE_MODELS,
                    value=DEFAULT_MODEL,
                    label="Select LLM Model"
                )
                query_input = gr.Textbox(
                    label="Your Question",
                    placeholder="e.g., Read the files and list them.",
                    lines=3
                )
                submit_btn = gr.Button("Submit Query")

            with gr.Column():
                output = gr.Textbox(label="Response", lines=10)

        # Add example queries
        example_queries = [
            "What are my most likely predictions?",
            "What are my favorite WWDC activities?",
            "Look at my wwdc25 predictions. List the predictions that are unlikely."
        ]
        gr.Examples(
            examples=example_queries,
            inputs=query_input
        )

        submit_btn.click(
            fn=gradio_query,
            inputs=[query_input, model_selector],
            outputs=output
        )

    return demo


if __name__ == "__main__":
    # Create and launch the Gradio app
    demo = create_demo()
    demo.launch()

Running the app will display a UI similar to the image at the beginning of this section. One unique feature of this example is that we can run the agent with closed, open-source, large, and small language models from any provider. Thanks 🙏 to LiteLLM and Ollama, you can access and add your preferred models from OpenAI, Anthropic, Gemini, Azure, BedRock, DeepSeek, Meta AI, xAI, Mistral, etc.

Implement Your Agents’ Logging System With AgentOps

For an AI agent to work reliably, robust tracing and monitoring of the performance of the underlying LLM must be implemented. Several solutions are available to integrate and monitor your AI apps and agents. You can, for example, add Literal AI to your agents to evaluate and manage prompts with beautiful visual representations.

However, the OpenAI Agents SDK supports selected tracing solutions. You can use any monitoring services OpenAI recommends, but we will use AgentOps for this section. AgentsOps AI provides developers the tools to debug, trace, and deploy AI applications securely and reliably.

Let's use it to trace our local OpenAI agent.

First, install the AgentOps Python package with this command.

pip install agentops

Next, modify the code in the previous section to implement AgentOps for tracing the local OpenAI agent. We create the tracing implementation using this sample code.

# https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem

import asyncio
import os
import shutil
import gradio as gr
import litellm
import agentops

from agents import Agent, Runner
from agents.mcp import MCPServer, MCPServerStdio

# Initialize AgentOps - configure with your API key if needed
# agentops.configure(api_key="your_api_key_here")
agentops.init()

# Configure litellm logging using environment variable 
os.environ['LITELLM_LOG'] = 'DEBUG'

# List of available models - can be expanded
AVAILABLE_MODELS = [
    "ollama/deepseek-r1:8b",  # Default model
    "ollama/llama3:8b",
    "ollama/mistral:7b",
    "ollama/gemma:7b",
    "ollama/phi3:mini",
    # Add other providers if needed
    "openai/gpt-4o-mini",
    "anthropic/claude-3-haiku"
]

DEFAULT_MODEL = "ollama/deepseek-r1:8b"

async def process_query(query, mcp_server, model_name):
    # Configure LiteLLM to use the selected model
    litellm.model = model_name

    # Create the agent 
    agent = Agent(
        name="Assistant for Content in Files",
        instructions="Use the tools to read the filesystem and answer questions based on those files.",
        mcp_servers=[mcp_server],
    )

    # Start tracking a session with tags
    session = agentops.start_session(tags={
        "model": model_name,
        "query_type": "file_content",
    })

    try:
        # Run the agent
        result = await Runner.run(starting_agent=agent, input=query)

        # End the session successfully
        agentops.end_session(session)

        return result.final_output
    except Exception as e:
        # End the session with error
        agentops.end_session(session, error=str(e))
        raise


async def handle_query(query, model_name):
    current_dir = os.path.dirname(os.path.abspath(__file__))
    samples_dir = os.path.join(current_dir, "sample_files")

    try:
        async with MCPServerStdio(
            name="Filesystem Server, via npx",
            params={
                "command": "npx",
                "args": ["-y", "@modelcontextprotocol/server-filesystem", samples_dir],
            },
        ) as server:
            output = await process_query(query, server, model_name)
            return output
    except Exception as e:
        raise


def gradio_query(query, model_name):
    if not shutil.which("npx"):
        return "Error: npx is not installed. Please install it with `npm install -g npx`."

    # Check if Ollama is selected but not running
    if model_name.startswith("ollama/"):
        try:
            # Simple check to see if Ollama is accessible
            asyncio.run(litellm.acompletion(
                model=model_name,
                messages=[{"role": "user", "content": "test"}],
                max_tokens=1
            ))
        except Exception as e:
            return f"Error: Ollama is not running or model {model_name} is unavailable. Please start Ollama first."

    try:
        result = asyncio.run(handle_query(query, model_name))
        return result
    except Exception as e:
        return f"Error: {str(e)}"


def create_demo():
    with gr.Blocks() as demo:
        gr.Markdown("# File Content Agent")
        gr.Markdown("Ask questions about files in your sample directory.")

        with gr.Row():
            with gr.Column():
                model_selector = gr.Dropdown(
                    choices=AVAILABLE_MODELS,
                    value=DEFAULT_MODEL,
                    label="Select LLM Model"
                )
                query_input = gr.Textbox(
                    label="Your Question",
                    placeholder="e.g., Read the files and list them.",
                    lines=3
                )
                submit_btn = gr.Button("Submit Query")

            with gr.Column():
                output = gr.Textbox(label="Response", lines=10)

        # Add example queries
        example_queries = [
            "What are my most likely predictions?",
            "What are my favorite WWDC activities?",
            "Look at my wwdc25 predictions. List the predictions that are unlikely."
        ]
        gr.Examples(
            examples=example_queries,
            inputs=query_input
        )

        submit_btn.click(
            fn=gradio_query,
            inputs=[query_input, model_selector],
            outputs=output
        )

    return demo


if __name__ == "__main__":
    # Create and launch the Gradio app
    demo = create_demo()
    demo.launch()

    # End all sessions when the application exits
    agentops.end_all_sessions()

The code's functionality remains the same as in the previous section, except for the tracing feature. When you run the code, you will receive a Session Replay link to view the agent's traces in your AgentOps dashboard.

Clicking the link will display the agent's sessions and overall metrics on the dashboard.

Congratulations 👏! You can now create local voice and AI apps with the OpenAI Agents SDK by integrating them with other technologies, such as Ollama, LiteLLM, AgentOps, etc.

Where to Go From Here

In this article, we examined building and running OpenAI agents with local toolkits that support free choices of LLM providers and implement custom AI model tracing. We covered only the offline aspect of using the Agents SDK. However, several features can be added to your AI applications, such as voice agents, providing context to models, and more.

The 3 Best Python Frameworks To Build UIs for AI Apps

Amos Gyamfi — Fri, 21 Mar 2025 15:41:37 +0000

Python offers various packages and frameworks for building interactive, production-ready AI app interfaces, including chat UIs. This article details the top platforms.

AI Chat UIs: Overview

Chat UIs for AI applications provide the front-end interface to interact with AI assistants, local, and cloud-based large language models (LLMs), and agentic workflows. Like popular AI apps Grok for iOS, Le Chat, and ChatGPT for the Web and mobile, UIs are available for sending and generating text, images, audio, or video (multimodal input and output).

Generally, developers can utilize chat SDKs and ready-made AI UI components from Stream to assemble interactive chatbots and assistants. One advantage of this approach is that developers can implement these AI components for popular platforms such as iOS, Android, Flutter, React, React Native, and NodeJS with minimal effort using available code samples and taking inspiration from demo apps.

How To Build AI Apps in Python

Developers can use several model provider APIs when building AI apps. Popular AI model solutions include OpenAI, Anthropic, Mistral, Grok, and DeepSeek. Typically, developers build these AI solutions and agents using platforms like LlamaIndex, LangChain, PydanticAI, Embedchain, LangGraph, CrewAI, and more. Using the command line to test and debug these applications is excellent.

However, an intuitive interface is needed to interact with the underlying AI functionalities to perform specific tasks, test them with real users, or share them through the world for feedback. Additionally, an AI app you build in Python may need to connect to other external services to access knowledge bases, tools to execute tasks, monitor performance, etc.

Why Use Python Libraries For AI Chat UIs?

Creating UIs for an AI app involves many considerations, such as text generation style, response streaming, output structuring, and how all the user-AI interactions create a coherent experience. Depending on the platform you choose to build the front end of your AI app, there are numerous built-in functionalities and AI chat components you get out of the box.

Output streaming: An LLM's response streaming is complex if you need to implement it from scratch for your AI project. Although many AI models implement the feature by default, most UI-building platforms provide easy ways to enable output streaming unless custom streaming is preferred.
Chatbot animations: You can get free chatbot response animations, such as typing indication and thinking, by using some libraries to build UIs for your AI projects.
Feedback: They help to get feedback quickly for a project's improvement.
Monitor and visualize performance: Chainlit, for example, uses Literal AI (AI observability service) playground to observe the performance of underlying LLMs, manage prompts, and see analytics.
Easy to test: These libraries provide easy ways to test and demo AI apps to people.

AI Chat UI Builders and Use Cases

You can build the front end of AI apps for many use cases, depending on the app's target. Generally, the AI chat UI you may want to make will fall into some of the following categories.

Multimodal AI: AI apps in this category involve Audio-text-to-text, Image-text-to-text, Visual question answering, Document question answering, Video-text-to-text, Visual document retrieval, etc.
Computer vision: In this category, you can build AI apps for text and image classification in a chat messaging app, video classification in a conferencing app like Zoom, and Text-to-video generation.
Natural Language Processing: In the natural language processing category, developers can build AI app UIs for summarization in a video calling app, translation in a chat app, and an AI chatbot for question answering and text generation.
Audio: In this category, you can easily create front-end UIs for Text-to-speech, Text-to-audio, Automatic speech recognition for video conferencing, Audio classification for voice messaging, and Voice activity detection AI apps.

Tools For Building AI Chat UIs in Python

Depending on a project's objectives, several ways and UI styles exist to build AI apps. For example, a multi-modal AI app that allows users to send text, voice, and image inputs may differ from the one supporting only voice cloning. For Python-based AI projects, you can build the front ends using frameworks like Gradio, Streamlit, and Chainlit. There are other Python-based frameworks for building web and data apps. However, those are beyond the scope of this article.

Our primary focus is on creating front-end interfaces for AI apps. The tools mentioned here have their strength and limitations, so we discuss them in the following sections, demonstrating sample code and implementation examples to help you pick the suitable framework for your next AI app.

1. Gradio: Build UIs for Testing and Deploying AI Apps

Gradio is an open-source Python library from Hugging Face that allows developers to create UIs for LLMs, agents, and real-time AI voice and video applications. It provides a fast and easy way to test and share AI applications through a web interface. Gradio offers an easy-to-use and low-code platform for building UIs for unlimited AI use cases.

Easy to start: You can quickly begin with Gradio and set up a fully functioning interface for an AI app shortly with a few lines of Python code.
Showcase and share: Easily embed UIs in Jupyter Notebook, Google Colab or share them on Hugging Face using a public link.
Deploy to production: You can permanently host the UI of a Gradio-powered AI project on Hugging Face Spaces.

Key Features and Benefits of Gradio

Gradio is one of the most famously used Python libraries for building the front end for machine learning (ML) applications. Its popularity and widespread use result from feature-rich components and building blocks for constructing AI chat interfaces with minimal effort. The following are the key features of Gradio.

Custom components: Create custom-made components as UI libraries and integrate them into your AI apps powered by Gradio's UI. Custom components can be published as Python packages and distributed to the developer community.
Agent UI: Easily create a Gradio interface for text-to-image generation, etc., using Hugging Face's transformers.agents.
Support for leading AI building platforms: Gradio integrates well into leading Python frameworks for creating AI and agent applications such as LangChain, LlamaIndex, Embedchain, and more.
Native intermediate thoughts: Leverage the built-in support for chain of thought (CoT) to build interfaces and demos for projects implementing reasoning LLMs.
Support for handling different AI app use cases: One essential characteristic when looking for a UI-building framework for an AI app is the platform's ability to handle different app use cases. Luckily, Gradio's gradio.Interface class allows developers to create interfaces around four distinct use cases. The standard use case allows you to build UIs with input and output. This interface type is excellent for creating image classifiers or speech-to-text AI apps. You can create an interface that supports only an output use case, like image generation. It also supports an input-only interface, which can be used to build an LLM app that stores uploaded images in external databases. With the unified interface option, you can make UIs whereby input and output do not differ. In this option, the output overrides the input. An excellent example in this category is an AI UI that autocompletes text or code.
Custom interfaces: Blocks in Gradio allow you to implement bespoke UIs quickly for AI projects.
Chatbot UI: Use Gradio's high-level abstraction gr.ChatInterface to quickly build chatbot and assistant interfaces.

Gradio Installation and Quickstart

This section uses Gradio's building blocks to create the chatbot UI in the image above. First, in your favorite IDE, you should set up a Python virtual environment and install Gradio's Python package using the following commands.

python -m venv venv
source venv/bin/activate
pip install gradio

Next, create a Python file grok_ui.py and replace its content with this sample code.

import gradio as gr
import os
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

# Function to handle chat interactions
def chat_with_grok(message, history, system_message, model_name, temperature, max_tokens):
    # In a real implementation, this would call the Grok API
    # For now, we'll just echo the inputs to demonstrate the UI is working
    bot_message = f"You selected model: {model_name}\nSystem message: {system_message}\nTemperature: {temperature}\nMax tokens: {max_tokens}\n\nYour message: {message}"
    return bot_message

# Create the Gradio interface
with gr.Blocks(theme=gr.themes.Soft(primary_hue="blue")) as demo:
    gr.Markdown("# Grok AI Chat Interface")

    with gr.Row():
        with gr.Column(scale=3):
            # Main chat interface
            chatbot = gr.Chatbot(
                height=600,
                show_copy_button=True,
                avatar_images=("👤", "🤖"),
                bubble_full_width=False,
            )

            # Message input
            msg = gr.Textbox(
                placeholder="Send a message...",
                container=False,
                scale=7,
                show_label=False,
            )

            with gr.Row():
                submit_btn = gr.Button("Send", variant="primary", scale=1)
                clear_btn = gr.Button("Clear", variant="secondary", scale=1)

        with gr.Column(scale=1):
            # Model settings sidebar
            gr.Markdown("### Model Settings")

            model_dropdown = gr.Dropdown(
                choices=["grok-1", "grok-2", "grok-3-beta"],
                value="grok-3-beta",
                label="Model"
            )

            system_message = gr.Textbox(
                placeholder="You are a helpful AI assistant...",
                label="System Message",
                lines=4
            )

            with gr.Accordion("Advanced Settings", open=False):
                temperature = gr.Slider(
                    minimum=0.0,
                    maximum=1.0,
                    value=0.7,
                    step=0.01,
                    label="Temperature"
                )

                max_tokens = gr.Slider(
                    minimum=100,
                    maximum=4000,
                    value=1000,
                    step=100,
                    label="Max Tokens"
                )

    # Set up event handlers
    submit_btn.click(
        chat_with_grok,
        inputs=[msg, chatbot, system_message, model_dropdown, temperature, max_tokens],
        outputs=[chatbot],
    ).then(
        lambda: "",
        None,
        msg,
        queue=False
    )

    msg.submit(
        chat_with_grok,
        inputs=[msg, chatbot, system_message, model_dropdown, temperature, max_tokens],
        outputs=[chatbot],
    ).then(
        lambda: "",
        None,
        msg,
        queue=False
    )

    clear_btn.click(lambda: None, None, chatbot, queue=False)

# Launch the app
if __name__ == "__main__":
    demo.launch()

In this example, we use Gradio's gr.Blocks() component to create a chatbot UI for sending and generating text output from an AI. Using the built-in blocks, we can partition the UI into message input and output, model settings, and advanced settings sections.

Running the Python file grok_ui.py will display the above chatbot interface via your local URL: http://127.0.0.1:7860.

Bravo! 👏 You have created your first AI chat UI using Gradio. As discussed earlier, Gradio is not limited to only building text and image generation interfaces. You can easily integrate it with other AI app development platforms to create UIs for any use case. Let's make a real-time voice interface with Gradio, allowing you to have a live audio conversation with an AI in the next section.

Gradio: Build a Real-Time Voice UI

Gradio is an excellent choice for building conversational AI UIs. Let's use its built-in components to create a real-time voice app using Hugging Face's FastRTC. Think of FastRTC as a direct competitor and alternative to OpenAI's real-time API. It is a Python library that allows developers to build real-time communication apps for AI. The interface in this section is simple. We create a voice UI with Gradio, allowing you to input a speech and get the output back as an echo using FastRTC.

Like the previous example, we create a virtual environment and install Gradio, the FastRTC Python library, and optional dependencies using the following Terminal commands.

python -m venv venv
source venv/bin/activate
pip install gradio
pip install fastrtc

# Install the vad, stt, and tts extras
pip install "fastrtc[vad, stt, tts]"

Next, create a new Python file gradio_fastrtc_voice_ui.py and fill its content with the following.

from fastrtc import Stream, ReplyOnPause
import numpy as np

def echo(audio: tuple[int, np.ndarray]):
    # The function will be passed the audio until the user pauses
    # Implement any iterator that yields audio
    # See "LLM Voice Chat" for a more complete example
    yield audio

stream = Stream(
    handler=ReplyOnPause(echo),
    modality="audio", 
    mode="send-receive",
)
stream.ui.launch()

In the above sample code, the stream.ui.launch() method launches a built-in Gradio UI for quickly testing and sharing your real-time stream. After running the Python file, you should see this information in the Terminal Running on local URL: http://127.0.0.1:7860. Launch http://127.0.0.1:7860 in your browser to see a Gradio voice interface similar to the image above. To initiate real-time communication, you should grant access to your webcam. Enjoy 😊 your first real-time voice UI with Gradio.

Gradio: Integration With Popular APIs and LLM Libraries

When working with popular LLM APIs like OpenAI, Anthropic, xAI, and libraries such as Embedchain and LlamaIndex, you can easily use Gradio to construct the front end of your projects. Refer to the following guides and examples to get started.

2. Streamlit: Build AI Apps and Share in the Cloud

Aside from Gradio, another excellent open-source app framework for constructing UIs for AI projects is Streamlit. It is well known for being magically easy to use. Want to create UIs to run LLMs locally? Connect Streamlit with Ollama to access open-source reasoning models like QwQ-32.

Key Features and Benefits of Streamlit

Using Streamlit, you can begin with a fully working app with just five lines of Python code. It allows developers to extend their apps with pre-built widgets and supports instant deployment to the web. The following highlights the essential functionalities of Streamlit.

Local LLMs support: Streamlit integrates seamlessly with local and open-source models. That means you can quickly run any GGUF model from Hugging Face or run models using Llama.cpp, Llamafile, and more.
Streaming and model progress animations: Get a beautiful text streaming animation for output generation for free and loading/status animations while a model generates responses.
Components: Go beyond what is possible with Streamlit by extending and integrating your AI apps and LLMs with third-party tools.
Deployment: Quickly build shareable web apps that require no front-end experience and let users test with any browser. Streamlit provides a beautiful web UI for your AI apps.
Build Streamlit UIs with LLMs: Due to Streamlit's popularity and large community-shared code samples on GitHub the Web, you can use State-of-the-Art (SOTA) LLMs to generate Streamlit code to create multi-modal AI app UIs.
Chat interface: Build UIs for generative AI apps with interactive real-time visualizations.
LLM building tools agnostic: Streamlit is supported by all popular AI app-building applications, such as LangChain, Embedchain, LlamaIndex, and agentic frameworks.

Build a Streamlit UI To Run Models Locally

Local LLM tools like LMStudio or Ollama are excellent for offline running a model like DeepSeek R1 through an app interface and the command line. However, in most cases, you may prefer having a UI you built to interact with LLMs locally. In this circumstance, you can create a Streamlit UI and connect it with a GGUF or any Ollama-supported model.

This section adopts a privacy-focused, local-first approach to cloning the DeepSeek Chat UI and running the DeepSeek R1 model via a custom-made Streamlit offline interface powered by Ollama. Feel free to download and run other Ollama-supported models with the code sample in this section. In summary, we want to:

Build a local Streamlit interface to run and test open-source models, avoiding the fear of data sharing.
Benefit from Streamlit's streaming (typewriting animation) feature for answer generation.
Get an animated spinner for free while the chatbot is generating responses.
Enhance the user experience of waiting for answer generation with an animated count-up duration timer.

To begin, you should install the following required and optional dependencies.

# Set up a Python virtual environment you prefer 
python -m venv venv
source venv/bin/activate

# Install dependencies
pip install streamlit
pip install ollama

# Download your open-source model with Ollama 
ollama run deepseek-r1:8b

Here, we set up a Python virtual environment, installed Streamlit and Ollama, and pulled one of the distilled versions of DeepSeek R1.

Next, create a Python file and name it whatever you want, local_deepseek_r1_streamlit.py. Add the following code to the file you just created.

import streamlit as st
import ollama
import time

def stream_data(text, delay: float=0.02):
    for word in text.split():
        yield word + " "
        time.sleep(delay)

# Input for the prompt
prompt = st.chat_input("Ask DeepSeek R1")

# Initialize chat history in session state if it doesn't exist
if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat history
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

if prompt:
    # Add user message to chat history
    st.session_state.messages.append({"role": "user", "content": prompt})

    # Display input prompt from user
    with st.chat_message("user"):
        st.markdown(prompt)

    # Processing
    with st.chat_message("assistant"):
        message_placeholder = st.empty()
        full_response = ""

        # Stream the response with a spinner while waiting for the initial response
        with st.spinner("Thinking...", show_time=True):
            response = ollama.chat(
                model="deepseek-r1:8b", 
                messages=[{"role": m["role"], "content": m["content"]} for m in st.session_state.messages],
                stream=True  # Enable streaming if supported by your ollama version
            )

            # If streaming is supported
            if hasattr(response, '__iter__'):
                for chunk in response:
                    if chunk and "message" in chunk and "content" in chunk["message"]:
                        content = chunk["message"]["content"]
                        full_response += content
                        message_placeholder.markdown(full_response + "▌")
                message_placeholder.markdown(full_response)
            else:
                # Fallback for non-streaming response
                full_response = response["message"]["content"]
                # Simulate streaming for better UX
                for word in stream_data(full_response):
                    message_placeholder.markdown(full_response[:len(word)] + "▌")
                message_placeholder.markdown(full_response)

        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": full_response})

The above sample code creates a simple input field for capturing user prompts. Once a user sends a prompt, we stream the response generation, making a typewriting effect. The chatbot displays a spinner while generating a response. The Streamlit spinner also shows an animated count-up timer to simulate the duration of output generation.

Use the streamlit run command and specify the name of your Python file streamlit run local_deepseek_r1_streamlit.py to view your local AI chat app on http://localhost:8501/. You can now use any prompt to generate a response similar to the preview below.

Thanks to Streamlit 🙏, we have a beautiful streaming, spinning, and animated count-up duration timer for free. Check out Streamlit's website for more integration examples for building AI chat and LLM apps with other frameworks.

3. Chainlit: Build UIs for Conversational AI

Like Streamlit, you can use Chainlit to create aesthetically pleasing and pragmatic UIs to interact with AI systems and LLMs. Streamlit can be used to build general-purpose web applications for data manipulation and AI in pure Python. However, Chainlit has one primary focus, which is creating AI and LLM apps. If you want to make a universal web app in Python rather than AI-focused, Chainlit may not suit you.

Key Features and Benefits of Using Chainlit

Chainlit's broad integration with external third-party applications and deployment options make it an excellent choice for creating AI chat UIs. Work faster with your team by utilizing Chainlit's monitoring, observability, and authentication mechanism for building secure and extensible AI apps.

Built-in monitoring and analytics: Monitor your API calls to model providers like Anthropic and OpenAI on a beautiful dashboard and prompt playground using Literal AI.
Customization: Chainlit provides different customization options to help you customize the look and feel of the UIs you create. Perform basic modifications like swapping a logo and favicon. You can do advanced theming of Chainlit’s UI or use custom CSS to theme interface components.
Authentication: Authenticate users with industry-standard protocols like OAuth. The above means you can authenticate users with third-party authorization services like Google or GitHub. Another great feature of Chainlit is its ability to authenticate people with password/login.
Supported functionalities: Interact with your AI applications with messages, streaming, audio, chat history, chat profiles, feedback (upvote or downvote an LLM's responses, like in ChatGPT and Grok), and more.
Integrations: Build a Chainlit UI and integrate it with popular Python libraries and APIs like Mistral AI, Autogen, and Haystack.

Deployment: Create Chainlit UIs and use them as standalone AI web apps or in-app features for applications like Slack, Discord, and Microsoft Teams.

Build a Chainlit UI to Access 100+ LLMs

In this section, we create a local Chainlit UI powered by the LiteLLM platform. LiteLLM is a gateway that provides model access and spend tracking across hundreds of LLMs in an OpenAI-compatible format. This example uses one of the distilled models of DeepSeek R1 via Ollama for local access.

Begin by installing Ollama as discussed in one of the previous sections and the following.

pip install chainlit
pip install litellm

Next, create a new Python file in your favorite editor and use this code sample to fill its content.

import chainlit as cl
import litellm

@cl.on_message
async def on_message(message: cl.Message):
  msg = cl.Message(content="")
  await msg.send()

  system_message = {
    "role": "system", 
    "content": """You are an advanced AI assistant powered by the deepseek-r1:8b model.

Your strengths:
- Providing clear, accurate, and thoughtful responses
- Breaking down complex topics into understandable explanations
- Offering balanced perspectives on questions
- Being helpful while acknowledging your limitations

Guidelines:
- If you're uncertain about something, acknowledge it rather than making up information
- When appropriate, suggest related questions the user might want to ask
- Maintain a friendly, respectful tone
- Format your responses with markdown when it improves readability
"""
  }

  response = await litellm.acompletion(
    model="ollama/deepseek-r1:8b",
    messages = [
      system_message,
      {"role": "user", "content": message.content}
    ],
    api_base="http://localhost:11434",
    stream=True
  )

  async for chunk in response:
    if chunk:
      content = chunk.choices[0].delta.content
      if content:
        await msg.stream_token(content)

  await msg.update()

Running the Python code above will display a Chainlit AI chat interface similar to the preview below. You are now up and running with the DeepSeek R1 model powered by Ollama and LiteLLM. To test with different models for AI chat and content generation, you can, for example, pull the SOTA reasoning model, QwQ-32 with ollama run qwq in the Terminal.

Other Frameworks To Build AI App UIs in Python

As discussed earlier in this article, you can build the front end of your Python-based AI application using many different solutions. In addition to the three leading libraries mentioned above, you can use the following and other platforms like Flask to create web UIs for AI apps, although some have broader use cases than solely AI.

Messop: Similar to Chainlit and Streamlit, Messop allows developers to rapidly create AI apps in Python with tens of UI components. Although it is not as widely used and feature-rich as the UI building frameworks above, its GitHub repo indicates it is used by an engineering team at Google.
FastHTML allows developers to build modern web applications entirely in Python without touching JavaScript or React. As its name implies, it is quicker to begin with FastHTML. However, it does not have pre-built UI components and styling. Getting the best out of this framework requires the knowledge of HTMX and UI styling using CSS libraries like Tailwind and Bootstrap.
Reflex: Build your AI app's frontend and backend in Python. Check out the Reflex website’s implementation examples for AI image generation and chatbots.

Final Thoughts on Chat UIs for AI Apps

Choosing a framework for building chat UIs for AI applications requires many considerations, including features, scalability, integration with other AI technologies and platforms, and deployment options. This article explored real-world examples of building chat UIs for AI apps using industry-leading platforms like Gradio, Streamlit, and Chainlit. Hopefully, you enjoyed the different use cases and capabilities unlocked from each AI UI building framework. Stream offers an alternative in-app AI assistant integration approach using six platforms' pre-built AI chat UI components. To begin, check out how to create an AI assistant and start coding AI apps with Stream for free.

Using a Speech Language Model That Can Listen While Speaking

Amos Gyamfi — Fri, 27 Sep 2024 11:20:19 +0000

Traditional Speech Language Models (SLMs) and conversational AI apps can only have one-sided (turn-based) conversations. For example, you speak to Siri and wait for it to respond. SLMs are incapable of real time interruptions that make conversations interactive to mimic how humans communicate. However, a recent finding shows that a language model can Listen While Speaking, paving the way to more natural human-to-SLM interactions.

This article introduces the listening-while-speaking language model (LSLM), which makes speech conversations with AI feel more dynamic and natural.

Traditional Turn-Based Conversational Models

SLMs are generally voice-based conversational AI models trained for speech-to-speech interactions. Think of turn-based major conversational models and voice assistants like Siri, Amazon Alexa, Google Assistant, etc. Although these voice assistants can have turn-by-turn conversations, they cannot handle real-time interruptions when users are not okay with the answers they provide. Additionally, SLMs find it challenging to operate in noisy surroundings, like speaking to Siri with a background construction noise or other people talking close to you. The reasons above occur because SLMs are half duplex systems and cannot speak and listen to users simultaneously.

Continue reading the following sections to learn about a proposed advanced SLM designed to handle turn-taking and human interruptions like the GPT-4o voice assistant.

A Proposed SLM That Can Speak and Listen Simultaneously

The listening-while-speaking language model (LSLM) is designed to handle simultaneous speaking and listening capabilities using a communication mechanism known as Full-Duplex Modeling (FDM). An FDM communication allows voice assistants or conversational models and users to speak and listen at the same time, embracing real-time user interference. Implementing FDM communication in a voice assistant like Siri removes the need to always wait for turns during human-AI speech conversations.

The new GPT-4o model demonstrates an excellent integration of the LSLM technology in a real-world scenario. It can handle real-time user interruptions originating from vision and sound during conversations. For example, when you start speaking to the model and another person suddenly appears in the environment near you, the model can quickly distinguish you from the other person. The model can also be interrupted with sound while it is answering a prompt. For instance, while the GPT-4o model describes a scenario, you can interrupt it with your voice to stop and sing instead or do something else.

Key Features of LSLMs

The following are the main characteristics of LSLMs that differentiate them from traditional SLMs like Google Assistant.

Speak and listen at the same time: Unlike traditional turn-based voice assistants, LSLMs and their users can speak and listen simultaneously. This feature is feasible due to its underlying full duplex mechanism.
Interruption detection: Regardless of the modalities, such as speech, text, and vision, LSLMs can be obstructed at any point during an ongoing conversation. They can recognize and respond instantly to obstructions like task switching.
Adaptation and noise detection: During an active or ongoing conversation with these models, they can detect sudden and continuous noises that may occur in the speaker's environment. This ability to detect noise is helpful in other situations, such as differentiating the main speaker from the voices of other people who may be talking in close range.
Context awareness: They can track users' preferences in real time and immediately switch an ongoing task or conversation when necessary.
Unseen speaker detection: An LSLM can detect and respond to unseen speakers without interpreting their voices as noise.

LSLM: How It Works

The LSLM uses a full duplex model to allow a model and users to speak and listen at the same time rather than waiting for either party to finish speaking. Its token-based decoder-only text-to-speech (TTS) mechanism incorporates tokenized (breaking words into smaller units or chunks) data for speech generation. Using a text-to-speech token-based decoder improves the model's response time and output. It uses a system known as a Streaming Self-Supervised Learning (SSL) Encoder to process voice/speech input continuously. The SSL encoder transforms audio into a model-understandable data format for real time speech interactions.

LSLM: Fusion Mechanisms

The LSLM employs three fusion techniques, early fusion, middle fusion, and late fusion, to integrate its speaking and listening capabilities and ensure optimized data processing. It also uses a command-based FDM to handle and respond to voice instructions/prompts. The system mentioned here helps the model distinguish voice interruptions and background noises from speech input. Its companion voice-based FDM helps evaluate the model's interruption handling capability and how well it can manage different speech inputs from users.

The LSLM operates using two main channels for speaking and listening. The speaking channel handles speech generation using a TTS system. The listening channel is used for input speech processing and identifying environmental noise and interruptions to switch and adjust contexts.

LSLM Interruption Handling

While the LSLM is talking, and a user interferes with an adjustment or switch of context, the LSLM can stop speaking immediately, listen, and shift focus using an interruption token. The interruption token helps the model terminate an ongoing task quickly if turn-taking occurs, respond to interruptions and other unseen human voices, and dismiss environmental noises.

Full Duplex Speech Language Models

As stated in one of the earlier sections, LSLMs can speak, listen, and respond to sudden user interruptions due to their full duplex capabilities. A full-duplex communication is similar to human-human conversations. One talks and listens at the same time. The other person listening can also interrupt the speech at any time. On the other hand, half-duplex communication is similar to direct chat messaging. While one chat participant starts typing, the other needs to wait until the one typing sends the message.

GPT-4o is an excellent example of a commercial spoken dialogue system using full duplex speech technology. However, GPT-4o is a text-centric LLM. Another example of a speech language model that uses a full duplex speech system is Moshi Chat, an experimental conversational AI that can listen, think, and talk at the same time.

Advantages of LSLM Over SLM

The following highlights the benefits of LSLMs in voice conversational apps.

GPT-4o-like interactions: Similarly to GPT-4o, implementing LSLMs in consumer products can help directly observe tone, multiple speakers, background noises, laughter, singing, and user emotion.
Not limited to turn-based conversations: Although the model allows users to wait for turns, you can interfere with it using new prompts at any point in the conversation. The ability to obstruct the LSLM with a new instruction helps remove awkward pauses, like those in traditional SLM interactions.
Ability to distinguish human voices from noise: The LSLM can differentiate high and low frequency white noises and traffic noises. However, traditional SLMs cannot detect unseen speakers and high-frequency background noises such as phone ringtones and noise from nearby road construction.
Integration with other modalities: Like GPT-4o, LSLMs can integrate seamlessly with other modalities, like vision. In some situations, multi-modal input with voice and vision is necessary to improve the model's effectiveness and performance.

Limitations of LSLMs

Although LSLMs have several application areas in real-world scenarios and advantages over traditional SLMs, sometimes, these systems cannot interpret wrong user instructions. They can also lead to misjudgment of interruptions in high-frequency noisy conditions. The following are some other limitations associated with LSLMs.

Universal Accessibility: Currently, the model supports only English. Providing support for other languages with unique rhythms and structures will make it more accessible to varying cultures.
Performance: Based on the test conducted in the [listening-while-speaking] paper (https://arxiv.org/pdf/2408.02622), maintaining high performance under noisy conditions can be challenging when the model is used in voice-based applications. A high-frequency background noise, for example, can cause a drop in performance.
Interruption handling by the model in conditions whereby human voices mix with low and high-frequency noises can be challenging.
Cybersecurity risks: Similarly to any other application or service that processes real time voice inputs, an LSLM can face cybersecurity risks. Since the model continuously listens to capture human speech, it can unintentionally record sensitive personal data that hackers can intercept if not handled correctly.
Accuracy and reliability: Since the LSLM relies heavily on voice inputs, it can be challenging when verbal commands are less clear. Also, varying accents and misinterpretation of voice instructions can affect the model's reliability and accuracy of results.
Limited to few voice presets: While these AI models, like GPT-4o, excel in real-time voice communication, they are often restricted to a limited number of predefined male and female voice presets for output generation and user interactions. This limitation can make interacting with these models feel a bit unnatural and less personal in diverse user settings. Also, users cannot customize predefined voices that may be supplied by the model or add personalized ones.

LSLMs Use Cases

LSLMs can enhance speech communication in various ways, including instant translation and collaboration, customer service, virtual assistance, and more. The following highlights some of their potential application areas.

Healthcare assistance: LSLMs can act as assistants for patients and doctors to ensure improved healthcare. They could listen to patients, interpret symptoms, and help in early diagnosis of illnesses.
Real-time collaboration: In a collaborative environment setting, LSLM-based applications can facilitate seamless collaboration with multiple participants, allowing them to have real-time discussions during brainstorming sessions.
Real time tutoring: LSLMs can help provide personalized tutoring to various users. For example, they can assist kids in solving complex math problems by interacting with them in real time and step-by-step.
Language learning and translation: In a language learning setting, people can ask the models to pronounce words and read sentences from one language to another that they are not familiar with. For example, one can speak in English and ask the AI model to translate it into another language.
Meeting AI and companion: When implemented in video meeting apps, for example, meeting participants can ask the LSLM apps about specific things in a meeting, like summarization and note-taking. LSLMs can also help to retrieve action points from a previous meeting in real time.
Augmenting vision experiences: Combining these interactive speech language models with other modalities like vision, users can ask a model to describe what it observes in a particular surrounding and explain what is happening.
Customer service interactions: Integrating these voice models in a customer setting can help resolve order issues and troubleshooting.

LSLMs Conclusion

This article introduced you to the exciting research area of interactive speech dialogue systems and speech-based conversational AI. You have learned several reasons why traditional speech conversational systems like Alexa or Siri do not support real-time spoken scenarios and user interference during conversations. Unlike the above models, LSLMs support real time audio/voice communication, allowing both turn-based interactions and users to interfere and adjust voice commands at their preference.

We discussed the key features, benefits, and limitations of LSLMs and how they can be helpful in various scenarios and application areas. The accompanying research paper provides more information about LSLMs and tools for evaluating them. The related links in this article also provide helpful information on AI chat and LLMs.

The 6 Best LLM Tools To Run Models Locally

Amos Gyamfi — Thu, 29 Aug 2024 08:43:36 +0000

Running large language models (LLMs) like ChatGPT and Claude usually involves sending data to servers managed by OpenAI and other AI model providers. While these services are secure, some businesses prefer to keep their data entirely offline for greater privacy.

This article covers the top six tools developers can use to run and test LLMs locally, ensuring their data never leaves their devices, similar to how end-to-end encryption protects privacy.

Why Use Local LLMs?

A tool like LM Studio does not collect user data or track users' actions when they use it to run local LLMs. It lets all your chat data stay on your local machine without sharing with an AI/ML server.

Privacy: You can prompt local LLMs in a multi-turn manner without your prompt data leaving your localhost.
Customization Options: Local LLMs provide advanced configurations for CPU threads, temperature, context length, GPU settings, and more. This is similar to OpenAI’s playground.
Support and Security: They provide similar support and security as OpenAI or Claude.
Subscription and Cost: These tools are free to use and they do not require monthly subscription. For cloud services like OpenAI, each API request requires payment. Local LLMs help to save money since there are no monthly subscriptions.
Offline Support: You can load and connect with large language models while offline.
Connectivity: Sometimes, connecting to a cloud service like OpenAI may result in poor signal and connection.

Top Six and Free Local LLM Tools

Depending on your specific use case, there are several offline LLM applications you can choose. Some of these tools are completely free for personal and commercial use. Others may require sending them a request for business use. There are several local LLM tools available for Mac, Windows, and Linux. The following are the six best tools you can pick from.

1. LM Studio

LM Studio can run any model file with the format gguf. It supports gguf files from model providers such as Llama 3.1, Phi 3, Mistral, and Gemma. To use LM Studio, visit the link above and download the app for your machine. Once you launch LM Studio, the homepage presents top LLMs to download and test. There is also a search bar to filter and download specific models from different AI providers.

Searching for a model from a specific company presents several models, ranging from small to large quantization. Depending on your machine, LM Studio uses a compatibility guess to highlight the model that will work on that machine or platform.

Key Features of LM Studio

LM Studio provides similar functionalities and features as ChatGPT. It has several functions. The following highlights the key features of LM Studio.

Model Parameters Customization: This allows you to adjust temperature, maximum tokens, frequency penalty, and more.
Chat History: Allows you to save prompts for later use. Parameters and UI Hinting: You can hover on info buttons to lookup model parameters and terms.
Cross-platform: LM Studio is available on Linux, Mac, and Windows operating systems.
Machine Specification Check: LM studio checks computer specifications like GPU and memory and reports on compatible models. This prevents downloading a model that might not work on a specific machine.
AI Chat and Playground: Chat with a large language model in a multi-turn chat format and experiment with multiple LLMs by loading them concurrently.
Local Inference Server for Developers: Allows developers to set up a local HTTP server similar to OpenAI’s API.

The local server provides sample Curl and Python client requests. This feature helps to build an AI application using LM Studio to access a particular LLM.

# Example: reuse your existing OpenAI setup
from openai import OpenAI

# Point to the local server
client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

completion = client.chat.completions.create(
  model="TheBloke/Mistral-7B-Instruct-v0.1-GGUF",
  messages=[
    {"role": "system", "content": "Always answer in rhymes."},
    {"role": "user", "content": "Introduce yourself."}
  ],
  temperature=0.7,
)

print(completion.choices[0].message)

With the above sample Python code, you can reuse an existing OpenAI configuration and modify the base url to point to your localhost.

OpenAI’s Python Library Import: LM Studio allows developers to import the OpenAI Python library and point the base URL to a local server (localhost).
Multi-model Session: Use a single prompt and select multiple models to evaluate.

Benefits of Using LM Studio

This tool is free for personal use and it allows developers to run LLMs through an in-app chat UI and playground. It provides a gorgeous and easy to use interface with filters and supports connecting to OpenAI's Python library without the need for an API key. Companies and businesses can use LM studio on request. However it requires a M1/M2/M3 Mac or higher, or a Windows PC with a processor that supports AVX2. Intel and AMD users are limited to using the Vulkan inference engine in v0.2.31.

2. Jan

Think of Jan as an open-source version of ChatGPT designed to operate offline. It is built by a community of users with a user-owned philosophy. Jan allows you to run popular models like Mistral or Llama on your device without connecting it to the internet. With Jan, you can access remote APIs like OpenAI and Groq.

Key Features of Jan

Jan is an electron app with features similar to LM Studio. It makes AI open and accessible to all by turning consumer machines into AI computers. Since it is an open source project, developers can contribute to it and extend its functionalities. The following breaks down the major features of Jan.

Local: You can run your preferred AI models on devices without connecting them to the internet.
Ready to Use Models: After downloading Jan, you get a set of already installed models to start. There is also a possibility to search for specific models.
Model Import: It supports importing models from sources like Hugging Face.
Free, Cross-Platform and Open Source: Jan is 100% free, open source, and works on Mac, Windows, and Linux.
Customize Inference Parameters: Adjust model parameters such as Maximum token, temperature, stream, frequency penalty, and more. All preferences, model usage, and settings stay locally on your computer.
Extensions: Jan supports extensions like TensortRT and Inference Nitro for customizing and enhancing your AI models.

Benefits of Using Jan

Jan provides a clean and simple interface to interact with LLMs and it keeps all your data and processing information locally. It has over seventy large language models already installed for you to use. The availability of these ready-to-use models makes it easy to connect and interact with remote APIs like OpenAI and Mistral. Jan also has a great GitHub, Discord, and Hugging Face communities to follow and ask for help. However, like all the LLM tools, the models work faster on Apple Silicon Macs than on Intel ones.

3. Llamafile

Llamafile is backed by Mozilla whose aim is to support and make open source AI accessible to everyone using a fast CPU inference with no network access. It converts LLMs into multi-platform Executable Linkable Format (ELF). It provides one of the best options to integrate AI into applications by allowing you to run LLMs with just a single executable file.

How Llamafile Works

It is designed to convert weights into several executable programs that require no installation to run on architectures such as Windows, MacOS, Linux, Intel, ARM, FreeBSD, and more. Under the hood, Llamafile uses tinyBLAST to run on OSs like Windows without requiring an SDK.

Key Features of Llamafile

Executable File: Unlike other LLM tools like LM Studio and Jan, Llamafile requires only one executable file to run LLMs.
Use Existing Models: Llamafile supports using existing models tools like Ollama and LM Studio.
Access or Make Models: You can access popular LLMs from OpenAI, Mistral, Groq, and more. It also provides support for creating models from scratch.
Model File Conversion: You can convert the file format of many popular LLMs, for example, .gguf into .llamafile with a single command.

llamafile-convert mistral-7b.gguf

Get Started With Llamafile

To install Llamafile, head to the Huggingface website, select Models from the navigation, and search for Llamafile. You can also install your preferred quantized version from the URL below.

https://huggingface.co/Mozilla/Meta-Llama-3.1-8B-Instruct-llamafile/tree/main

Note: The larger the quantization number, the better the response. As highlighted in the image above, this article uses Meta-Llama-3.1-8B-Instruct.Q6_K.llamafile where Q6 represents the quantization number.

Step 1: Download Llamafile

From the link above, click any of the download buttons to get your preferred version. If you have the wget utility installed on your machine, you can download Llamafile with the command below.

wget https://huggingface.co/Mozilla/Meta-Llama-3.1-8B-Instruct-llamafile/blob/main/Meta-Llama-3.1-8B-Instruct.Q6_K.llamafile

You should replace the URL with the version you like.

Step 2: Make Llamafile Executable

After downloading a particular version of Llamafile, you should make it executable using the following command by navigating to the file’s location.

chmod +x Meta-Llama-3.1-8B-Instruct.Q6_K.llamafile

Step 3: Run Llamafile

Prepend a period and forward slash ./ to the file name to launch Llamafile.

./Meta-Llama-3.1-8B-Instruct.Q6_K.llamafile

The Llamafile app will now be available at http://127.0.0.1:8080 to run your various LLMs.

Benefits of Using Llamafile

Llamafile helps to democratize AI and ML by making LLMs easily reachable to consumer CPUs. As compared to other local LLM apps like Llama.cpp, Llamafile gives the fastest prompt processing experience and better performance on gaming computers. Since it has a faster performance, it is an excellent option for summarizing long text and large documents. It runs 100% offline and privately, so users do not share their data to any AI server or API. Machine Learning communities like Hugging Face supports the Llamafile format, making it easy to search for Llamafile related models. It also has a great open source community that develops and extends it further.

4. GPT4ALL

GPT4ALL is built upon privacy, security, and no internet-required principles. Users can install it on Mac, Windows, and Ubuntu. Compared to Jan or LM Studio, GPT4ALL has more monthly downloads, GitHub Stars, and active users.

Key Features of GPT4ALL

GPT4All can run LLMs on major consumer hardware such as Mac M-Series chips, AMD and NVIDIA GPUs. The following are its key features.

Privacy First: Keep private and sensitive chat information and prompts only on your machine.
No Internet Required: It works completely offline.
Models Exploration: This feature allows developers to browse and download different kinds of LLMs to experiment with. You can select about 1000 open-source language models from popular options like LLama, Mistral, and more.
Local Documents: You can let your local LLM access your sensitive data with local documents like .pdf and .txt without data leaving your device and without a network.
Customization options: It provides several chatbot adjustment options like temperature, batch size, context length, etc.
Enterprise Edition: GPT4ALL provides an enterprise package with security, support, and per-device licenses to bring local AI to businesses.

Get Started With GPT4All

To start using GPT4All to run LLMs locally, Download the required version for your operating system.

Benefits of Using GPT4ALL

With the exception of Ollama, GPT4ALL has the most significant number of GitHub contributors and about 250000 monthly active users (according to https://www.nomic.ai/gpt4all) and compared to its competitors. The app collects anonymous user data about usage analytics and chat sharing. However, users have the options to opt in or out. Using GPT4ALL, developers benefit from its large user base, GitHub, and Discord communities.

5. Ollama

Using Ollama, you can easily create local chatbots without connecting to an API like OpenAI. Since everything runs locally, you do not need to pay for any subscription or API calls.

Key Features of Ollama

Model Customization: Ollama allows you to convert .gguf model files and run them with ollama run modelname.
Model Library: Ollama has a large collection of models to try at ollama.com/library.
Import Models: Ollama supports importing models from PyTorch.
Community Integrations: Ollama integrates seamlessly into web and desktop applications like, Ollama-SwiftUI, HTML UI, Dify.ai, and more.
Database Connection: Ollama supports several data platforms.
Mobile Integration: A SwiftUI app like Enchanted brings Ollama to iOS, macOS, and visionOS. Maid is also a cross-platform Flutter app that interfaces with .ggufmodel files locally.

Get Started With Ollama

To use Ollama for the first time, visit https://ollama.com and download the version for your machine. You can install it on Mac, Linux, or Windows. Once you install Ollama, you can check its detailed information in Terminal with the following command.

ollama

To run a particular LLM, you should download it with:

ollama pull modelname, where modelname is the name of the model you want to install. Checkout Ollama on GitHub for some example models to download. The pull command is also used for updating a model. Once it is used, only the difference will be fetched.

After downloading for example, llama3.1, running ollama run llama3.1 in the command line launches the model.

A video showing Ollama responding to a prompt in the command line

In the above example, we prompt the llama3.1 model to solve a Physics work and energy question.

Benefits of Using Ollama

Ollama has over 200 contributors on GitHub with active updates. It has the largest number of contributors and is more extendable among the other open-source LLM tools discussed above.

6. LLaMa.cpp

LLaMa.cpp is the underlying backend technology (inference engine) that powers local LLM tools like Ollama and many others. Llama.cpp supports significant large language model inferences with minimal configuration and excellent local performance on various hardware. It can also run in the cloud.

Key Features of LLaMa.cpp

Setup: It has a minimal setup. You install it with a single command.
Performance: It performs very well on various hardware locally and in the cloud.
Supported Models: It supports popular and major LLMs like Mistral 7B, Mixtral MoE, DBRX, Falcon, and many others.
Frontend AI Tools: LLaMa.cpp supports open-source LLM UI tools like MindWorkAI/AI-Studio (FSL-1.1-MIT), iohub/collama, etc.

Get Started With LLaMa.cpp

To run your first local large language model with llama.cpp, you should install it with:

brew install llama.cpp

Next, download the model you want to run from Hugging Face or any other source. For example, download the model below from Hugging Face and save it somewhere on your machine.

https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf

Using your preferred command line tool like Terminal, cd into the location of the .gguf model file you just downloaded and run the following commands.

llama-cli --color \ 
-m Mistral-7B-Instruct-v0.3.Q4_K_M.ggufb \ 
-p "Write a short intro about SwiftUI"

In summary, you first invoke the LLaMa CLI tool and set color and other flags. The -m flag specifies the path of the model you want to use. The -p flag specifies the prompt you wish to use to instruct the model.

After running the above command, you will see the result in the following preview.

A video showing LLaMa.cpp responding to a prompt in the command line

Local LLMs Use Cases

Running LLMs locally can help developers who want to understand their performance and how they work in detail. Local LLMs can query private documents and technical papers so that information about these documents does not leave the devices used to query them to any cloud AI APIs. Local LLMs are useful in no-internet locations and places where network reception is poor.

In a telehealth setting, local LLMs can sort patient documents without having to upload them to any AI API provider due to privacy concerns.

Evaluating LLMs’ Performance To Run Locally

Knowing the performance of a large language model before using it locally is essential for getting the required responses. There are several ways you can determine the performance of a particular LLM. Here are a few ways.

Training: What dataset is the model trained on?
Fine-tuning: To what extent can the model be customized to perform a specialized task or can it be fine-tuned to for a specific domain?.
Academic Research: Does the LLM have an academic research paper?

To answer the above questions, you can check excellent resources like Hugging Face and Arxiv.org. Also, Open LLm Leaderboard and LMSYS Chatbot Arena provide detailed information and benchmarks for varieties of LLMs.

Local LLM Tools Conclusion

As discussed in this article, several motives exist for choosing and using large language models locally. You can fine-tune a model to perform a specialized task in a telemedicine app if you do not wish to send your dataset over the internet to an AI API provider. Many open-source Graphic User Interface (GUI-based) local LLM tools like LLm Studio and Jan provide intuitive front-end UIs for configuring and experimenting with LLMs without subscription-based services like OpenAI or Claude. You also discovered the various powerful command-line LLM applications like Ollama and LLaMa.cpp that help you run and test models locally and without an internet connection. Check out Stream's AI Chatbot solution to integrate an AI chat into your app and visit all the related links to learn more.

Secure Data and Safeguard Users an App With End-to-End Encryption

Amos Gyamfi — Wed, 07 Aug 2024 14:22:00 +0000

Have you ever wondered about the cost and wreaking havoc data breaches cause worldwide every year? Let’s look at how end-to-end encryption eliminates this concern.

In securing web and mobile applications, developers generally use traditional methods such as Transport Layer Security (TLS), Encryption at Rest, Encryption in Transit, etc. Although these approaches are acceptable, the methods stated here do not adequately secure data as end-to-end encryption. This article will explore everything you need to know about using end-to-end encryption (E2EE) to safeguard users and potential implementation solutions to enhance user security.

Before You Begin

In this article, you will learn an overview of E2EE, its challenges, and why you should implement it to provide world-class security for your app's users. To see end-to-end encryption implementation in action, keep an eye out for another tutorial that focuses on integrating E2EE into a web app using Stream Chat’s React SDK and Seald E2EE SDK. Although we cover general E2EE concepts in this article, we illustrate most of the E2EE examples for chat messaging.

Continue reading 📖 to unlock 🔓 how end-to-end encryption works under the hood.

What is App Encryption?

Encryption secures an app’s data to ensure user privacy and application safety. The users’ information can be text and voice messages, media attachments (photos, documents, audio, images), profiles, and more. Encryption keeps them secure by changing data at a source or when transmitting it from a user into a form that cannot be read. As the image above demonstrates, an encrypted message transmitted from a source will appear as a series of text characters without meaning when it gets into the hands of unauthorized users.

Encryption Tools

Encryption uses different methods to convert plain text communication into ciphertext. Below are some ways developers can encrypt web, mobile, and desktop applications.

Encryption in Transit: Encryption in transit works similarly to Transport Layer Security (TLS). It secures a "pipe" and protects everything inside it. Nothing is protected before and after the pipe.
Encryption at Rest: Encryption at rest (such as hard disk, database, or object storage encryption) secures data when unused. If someone breaks into the building housing, the hard disks containing the data, and steals them, they won't be able to do anything with them. On the other hand, if someone gains access to the backend (via an application vulnerability, for example), they'll have access to everything.
End-to-end Encryption: End-to-end encryption, on the other hand, protects data from the sender's device, the first "end" to the final recipient's device, and the second "end" without interruption. This ensures that data can only be accessed by the sender and the final recipient. Not even the data host, the application developer, any administrator or government, and any malicious person attacking one or more of these intermediaries will be able to access the data. End-to-end encryption is much stricter than encryption in transit combined with encryption at rest.

A simple web or mobile application can implement encryption at rest or in transit for data security and preservation of user integrity. However, a telemedicine app with complex structures and interactions must implement end-to-end encryption for fully-fledged privacy. End-to-end encryption is particularly useful in a telehealth app with features such as appointment scheduling, real-time chat messaging, and video calling between patients and doctors. Read the weakness of encryption in transit and at rest to learn more.

End-to-End Encryption: Overview

A video showing end-to-end encrypted chat

Fundamentally, end-to-end encryption uses a core concept called cryptography to secure data. In cryptography, a ciphertext is used to protect data by converting it into a non-human-readable format.

The video preview above demonstrates an end-to-end encrypted chat in action. Users need secret keys to cipher or decrypt the ciphered information into readable form. In end-to-end encryption, only intended users can access the data if the sender sends an encrypted information.

For example, when you power your messaging app with Stream’s Chat SDK and implement end-to-end encryption, nobody, including Stream’s backend servers, can see messages or listen to audio conversations users will send. All messages, including text, voice, files, video, and audio, are encrypted as they are shared among participants' devices.

How E2EE Works

E2EE uses public and private encryption keys for its operation. A public key encrypts information, while a private key decrypts it. For example, in chat messaging, only the chat participants will hold the encryption keys in one-on-one and group chats. In case third parties or hackers gain access to a message sent between participants, they cannot decrypt it since they don't hold encryption keys. The keys disappear when the sender and receiver delete the message from their devices.

As demonstrated in the image above, when A wants to send a message to B, they exchange their public keys to connect through a server. A uses B's public key to encrypt the message before sending it via the server. Since the server does not have a private key, the message becomes non-readable on the server. The server's responsibility is to ensure seamless communication between A and B. The users' private keys are kept on their devices. Therefore, when the server sends the ciphered message to B's device, the private key of B decrypts the message to make it readable. The exact process is used when B wants to send a message to A.

E2EE Challenges

Although E2EE aims to provide the highest data security level on the client side, compared to other alternatives like encryption at rest and in transit, it is prone to the following attacks.

Backdoor Attack: With a backdoor attack, hackers can exploit an application’s vulnerabilities to bypass the encryption mechanism and authentication to access users' data.
Endpoint Attack: In E2EE, data protection occurs between endpoints. These endpoints are also prone to malicious attacks and can be compromised.
Man-in-the-Middle Attack: In this attack, an attacker can intercept a sender's message by joining the conversation or data exchange between the endpoints. For instance, if A intends to send a message to B, an attacker can impersonate the communication, making B believe that the message is from A.

The above challenges generally arise from E2EE implementation from scratch or using an in-house team. To eliminate these challenges when integrating E2EE in your app, consider using an E2EE SDK like Seald. The Seald SDK handles all the threats mentioned above. The SDK is built on the Zero Trust security model and handles all cryptographic complexities, such as key management, recovery of private keys, group management, and breach of user credentials.

Why Should Your App Offer End-to-End Encryption?

Data privacy and safety are increasingly critical in enterprise and community-based applications. Implementing end-to-end encryption is becoming a must and a standard. The following points out why E2EE should be integrated as a security measure.

It can help minimize cyber attacks and data breaches since attackers may not have anything accessible on the systems they attack.
It ensures customer trust and prevents reputational damage.
Password protection: A service like 1Password uses end-to-end encryption to protect users' passwords using a key.
E2EE is much stricter than its other alternatives, such as encryption at rest and encryption in transit.

What Data Should Be End-to-End Encrypted?

Developers can use different methods to encrypt an app. However, only some apps need end-to-end encryption. E2EE is the best choice when user privacy is of the utmost concern, such as in the following application areas.

Healthcare data: Patients' information should be end-to-end encrypted to prevent access from unauthorized parties and to maximize patients privacy.
Unstructured data: Users’ chat history, media storage, file sharing and data about access to personal messages from multiple devices should be end-to-end encrypted. In WhatsApp, personal messages, - video calls, and voice messages are protected with E2EE to ensure only the sender and receiver can read sent text messages and listen to audio messages.
Team conversations: When end-to-end encrypted, it will help maintain confidentiality and prevent unauthorized interception of group communications.
Marketplace information: In a marketplace setting, E2EE can safeguard proprietary information and intellectual properties of buyers and sellers from attackers.
Financial data: End-to-end encryption can help to secure sensitive financial transactions and personal information from theft.
Educational information: End-to-end encryption of educational data can ensure the confidentiality of privileged communications and sensitive educational documents.

Identifying End-to-End Encryption in Communication Apps

When you implement end-to-end encryption in your app, there should be a way to make users aware of this feature. As shown in the image above, when a user initiates a chat in WhatsApp, there is a notification that indicates all messages will be encrypted end-to-end. Some popular messaging applications like WhatsApp turn on end-to-end encryption by default.

Others may provide in-app settings to turn E2EE on and off like the implementation in Telegram. Users should select the option Start Secret Chat to turn on the E2EE support.

In business messaging apps like Google Messages, a lock icon appears beside the send button when users compose messages. The lock icon indicates that the chat is end-to-end encrypted.

Some messaging applications may also use other forms to indicate and notify users if the messages they send to people will be end-to-end encrypted. For instance, iMessage uses a blue bubble for message delivery to indicate that the communication is end-to-end encrypted. Apple also encrypts FaceTime conversations end-to-end.

Note: The chat bubble appears blue when you send a message across Apple devices using the Messages app.

However, when you send a message with the Messages app for iOS to any Android device, the chat bubble turns green when delivered, which means it is a Short Message Service (SMS). An SMS sent with the Messages app does not have E2EE implemented.

Note: A message you send from an iPhone to an Android device is an SMS with a green chat bubble.

When initiating a voice call in WhatsApp, the UI shows a text indicating the call is end-to-end encrypted before establishing the connection to the other participant.

Conclusion

Congratulations! You now understand end-to-end encryption and why it is essential for ensuring the privacy and security of user data. It protects private and confidential information, such as text and voice messages, media attachments, user profiles, and more, from unauthorized access. By adopting end-to-end encryption, you demonstrate a commitment to safeguarding data privacy and ensuring users' trust.

We looked at E2EE for a single use case, providing maximum security for real-time chat messaging users. Keep an eye out for our step-by-step guide, which aims at encrypting Stream Chat end-to-end using Seald to implement the feature into your app.

Build Interactive In-App Polls With SwiftUI

Amos Gyamfi — Fri, 26 Jul 2024 08:44:29 +0000

Interactive polling in Slido, WhatsApp, Messenger, and Telegram make communicating via in-app chat or video conferencing more engaging by providing an interactive way to get participants’ opinions and make everyone in a virtual event, remote education, online gaming, or team collaboration setting feel more connected.

Follow the steps outlined in this guide to build the functionality required to allow users to create, display, vote, and provide feedback in your iOS apps using polls. You can also implement the functionality in other SDKs like React by checking out the Polls API.

Get Started

The polling support we implement in this tutorial will be something other than a stand-alone app. It is a new built-in feature of the SwiftUI SDK and is available in its 4.57.0 version. We will install the chat SDK shortly, configure it, and enable the ready-to-use polling feature.

If you plan to integrate polls for an actual app rather than the demo app in this tutorial, you should turn on polls on your Stream Dashboard to be available in your app. Create a free chat trial account and follow our Getting Started Guide to use your dashboard credentials to generate a token to follow this tutorial. You may also use the hard-coded user credentials we introduce later in this article to recreate it.

Grab the Sample Source Code

We made the demo app's source code available for download on GitHub iOS-SwiftUI -> PollsSwiftUI. The app's polls functionality is enabled by default. Get the app, run it with Xcode, and start creating, commenting, and voting on polls.

Step 1: Enable the Polls Support

Before users can use your Stream Chat-powered app to create and send polls to others, the feature should be enabled on your administration dashboard. Log in to your dashboard, create a new app, and navigate to the Chat Messaging category. Allow polls by selecting Channel Types -> messaging and toggling the corresponding button.

Step 2: Start With a New Stream Chat SwiftUI App

Create a new SwiftUI project in Xcode 15 or a later version and set the following privacies by selecting the project's root folder and navigating to the Info tab.

Privacy - Camera Usage Description
Privacy - Photo Library Usage Description

Install and Configure the Chat SDK

In the Xcode toolbar, select File -> Add Package Dependencies…. Then, copy and paste https://github.com/getstream/stream-chat-swiftui in the search bar to fetch the SDK from GitHub and complete the few steps to install it.

Configure the SwiftUI Chat SDK

In this section, we should set up the Chat SDK to present a list of channels when the app runs. This configuration should be made at the app’s entry point. Open the app entry file and replace the content with the following. Our app's root file is PollsSwiftUIApp.swift since the product name is PollsSwiftUI.

//
//  PollsSwiftUIApp.swift
//  PollsSwiftUI
//
//  How To Integrate Polls Into a SwiftUI App

import SwiftUI
import StreamChat
import StreamChatSwiftUI

@main
struct SwiftUIChatDemoApp: App {

    var chatClient: ChatClient = {
        //For the tutorial we use a hard coded api key and application group identifier
        var config = ChatClientConfig(apiKey: .init("8br4watad788"))
        config.isLocalStorageEnabled = true
        config.applicationGroupIdentifier = "group.io.getstream.iOS.ChatDemoAppSwiftUI"

        // The resulting config is passed into a new `ChatClient` instance.
        let client = ChatClient(config: config)
        return client
    }()

    @State var streamChat: StreamChat?

    init() {


        streamChat = StreamChat(chatClient: chatClient)

        connectUser()
    }

    var body: some Scene {
        WindowGroup {
            CustomChannelList()
        }
    }

    private func connectUser() {
        // This is a hardcoded token valid on Stream's tutorial environment.
        let token = try! Token(rawValue: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoibHVrZV9za3l3YWxrZXIifQ.kFSLHRB5X62t0Zlc7nwczWUfsQMwfkpylC6jCUZ6Mc0")

        // Call `connectUser` on our SDK to get started.
        chatClient.connectUser(
            userInfo: .init(
                id: "luke_skywalker",
                name: "Luke Skywalker",
                imageURL: URL(string: "https://picsum.photos/id/237/200/300")!
            ),
            token: token
        ) { error in
            if let error = error {
                // Some very basic error handling only logging the error.
                log.error("connecting the user failed \(error)")
                return
            }
        }
    }
}

In the sample code above, we set up the Chat SDK by connecting a valid user to the SDK's backend with a hard-coded apikey and token. The hard-coded credentials here are just for running the app to see the demo. In your production implementation, you should replace the apikey with the one in your dashboard and use it with your app id to generate a token.

Present a List of Channels

Let's display the list of chat channels you see in the image above by creating a new file CustomChannelList.swift and using this sample code to substitute its content.

import SwiftUI
import StreamChat
import StreamChatSwiftUI

struct CustomChannelList: View {

    @StateObject private var viewModel: ChatChannelListViewModel
    @StateObject private var channelHeaderLoader = ChannelHeaderLoader()

    public init(
        channelListController: ChatChannelListController? = nil
    ) {
        let channelListVM = ViewModelsFactory.makeChannelListViewModel(
            channelListController: channelListController,
            selectedChannelId: nil
        )
        _viewModel = StateObject(
            wrappedValue: channelListVM
        )
    }

    var body: some View {
        NavigationView {
            ChannelList(
                factory: DefaultViewFactory.shared,
                channels: viewModel.channels,
                selectedChannel: $viewModel.selectedChannel,
                swipedChannelId: $viewModel.swipedChannelId,
                onItemTap: { channel in
                    viewModel.selectedChannel = channel.channelSelectionInfo
                },
                onItemAppear: { index in
                    viewModel.checkForChannels(index: index)
                },
                channelDestination: DefaultViewFactory.shared.makeChannelDestination()
            )
            .toolbar {
                DefaultChatChannelListHeader(title: "Stream Tutorial")
            }
        }
    }
}

Refer to our documentation for more information about the chat channel list. For a step-by-step guide on setting up the chat SDK and displaying the channel list, check out the iOSChat App tutorial.

Step 3: How to Use Built-In Messaging Polls

When you perform the previous two steps to enable the polls support in your dashboard and install and configure the chat SDK in a new SwiftUI app, the polling feature will be ready in the chat messaging app. Like in WhatsApp, polls are available via media attachment in the SDK. Let's create our first poll in the following section.

Create a Poll

To initiate a new poll for people to vote, tap the paper clip 📎 button at the left side of the message composer and select the poll 📊 icon as illustrated below. Then, enter your question and options and configure the poll with the toggles.

Vote on a Poll

Users create and send polls to others that appear like outgoing messages. Polls can be reacted to, voted on, and configured to present a specific functionality. Tap and hold a poll to express feelings with emojis. Users can also tap the corresponding radio button 🔘 for each option in a poll to vote. The results of a poll are then shown on a detailed page.

Step 4: Customize the Poll’s Functionality and Presentation

You can tailor a poll's core features and visual appearance to provide unique functions, looks, and feels for online video meetings and group chats.

This section will modify the built-in poll structure to remove the Add a Comment field, allowing responders to send feedback. Head to the Polls Docs to go beyond simple customization of removing a comment field. Thanks 👏 to the flexibility of the SwiftUI Chat SDK. You can easily customize the experience to create multiple-choice, quiz-style, ranking, and survey-like polls.

Remove Poll Options

The SDK provides a PollsConfig object for configuring the default options available during a poll's creation. This object is useful in determining which poll options should be enabled for users. For instance, we can add an initialization with the code snippet below to the PollsSwiftUIApp.swift file we implemented in one of the earlier sections to:
Disable anonymous polls.
Disallow users to suggest poll options.
Hide the comments field.

    init() {
        //streamChat = StreamChat(chatClient: chatClient)
        // Custom configuration to hide comments and allow multiple votes
        let pollsConfig = PollsConfig(
            multipleAnswers: PollsEntryConfig(configurable: true, defaultValue: true),
            anonymousPoll: .init(configurable: false, defaultValue: false),
            suggestAnOption: .init(configurable: false, defaultValue: false),
            addComments: PollsEntryConfig(configurable: false, defaultValue: false),
            maxVotesPerPerson: .default
        )

        let utils = Utils(
            pollsConfig: pollsConfig
        )
        //streamChat = StreamChat(chatClient: chatClient)
        streamChat = StreamChat(chatClient: chatClient, utils: utils)

        connectUser()
    }

Let’s update the PollsSwiftUI.swift file with the code snippet above. The sample code below represents the modified file.

//
//  PollsSwiftUIApp.swift
//  PollsSwiftUI
//
//  How To Integrate Polls Into a SwiftUI App

import SwiftUI
import StreamChat
import StreamChatSwiftUI

@main
struct SwiftUIChatDemoApp: App {

    var chatClient: ChatClient = {
        //For the tutorial we use a hard coded api key and application group identifier
        var config = ChatClientConfig(apiKey: .init("8br4watad788"))
        config.isLocalStorageEnabled = true
        config.applicationGroupIdentifier = "group.io.getstream.iOS.ChatDemoAppSwiftUI"

        // The resulting config is passed into a new `ChatClient` instance.
        let client = ChatClient(config: config)
        return client
    }()

    @State var streamChat: StreamChat?

    init() {
        // Custom configuration to hide comments and allow multiple votes
        let pollsConfig = PollsConfig(
            multipleAnswers: PollsEntryConfig(configurable: true, defaultValue: true),
            anonymousPoll: .init(configurable: false, defaultValue: false),
            suggestAnOption: .init(configurable: false, defaultValue: false),
            addComments: PollsEntryConfig(configurable: false, defaultValue: false),
            maxVotesPerPerson: .default
        )

        let utils = Utils(
            pollsConfig: pollsConfig
        )
        //streamChat = StreamChat(chatClient: chatClient)
        streamChat = StreamChat(chatClient: chatClient, utils: utils)

        connectUser()
    }

    var body: some Scene {
        WindowGroup {
            CustomChannelList()
        }
    }

    private func connectUser() {
        // This is a hardcoded token valid on Stream's tutorial environment.
        let token = try! Token(rawValue: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoibHVrZV9za3l3YWxrZXIifQ.kFSLHRB5X62t0Zlc7nwczWUfsQMwfkpylC6jCUZ6Mc0")

        // Call `connectUser` on our SDK to get started.
        chatClient.connectUser(
            userInfo: .init(
                id: "luke_skywalker",
                name: "Luke Skywalker",
                imageURL: URL(string: "https://picsum.photos/id/237/200/300")!
            ),
            token: token
        ) { error in
            if let error = error {
                // Some very basic error handling only logging the error.
                log.error("connecting the user failed \(error)")
                return
            }
        }
    }
}

Previously, the init() function had this streamChat = StreamChat(chatClient: chatClient) initialization. In the sample code above, we define a Swift utility property and pass it to the streamChat initialization.

let utils = Utils(
            pollsConfig: pollsConfig
        )
            streamChat = StreamChat(chatClient: chatClient, utils: utils)

Swap the Poll's UI

If you want the poll's UI to have a completely different look and feel, you can implement a custom-made UI using the SDK's ViewFactory protocol method makeComposerPollView. The ViewFactory provides slots for replacing built-in UIs with one's own implementation.

func makeComposerPollView(
        channelController: ChatChannelController,
        messageController: ChatMessageController?
) -> some View {
    CustomComposerPollView(channelController: channelController, messageController: messageController)
}

After creating your custom poll view, it should be registered in the ViewFactory. Add a new file, CustomPollsFactory.swift, and replace its content with the following.

//
//  PollsUIFactory.swift
//  PollsSwiftUI

import Foundation
import SwiftUI
import StreamChat
import StreamChatSwiftUI

class CustomPollsFactory: ViewFactory {

    @Injected(\.chatClient) public var chatClient

    private init() {}

    public static let shared = CustomPollsFactory()

    func makeComposerPollView(
            channelController: ChatChannelController,
            messageController: ChatMessageController?
    ) -> some View {
        CustomComposerPollView(channelController: channelController, messageController: messageController)
    }

}

Checkout Customizing Components to learn more about the ViewFactory protocol.

Step 5: Test the Customized Poll

With the implementation of the previous customization, you can now run the app and create a new poll from the attachment view, as described in one of the sections above. However, the Add a Comment field is no longer visible. That is easy, huh? You now have a chat messaging app with fully working polling support to gather real-time responses and insights from people.

SwiftUI Poll Recap

In this tutorial, you learned how to integrate polls into a SwiftUI app for any use case and to capture real-time in-app feedback from participants. You discovered polls' configuration options, basic customization, how to create polls, and view them in chat messaging. You can dive into advanced customization possibilities for a poll's functionality and how it is presented to participants in our documentation.

The chat messaging part of the tutorial has a step-by-step guide to get up and running with the iOS SDK. Visit the SwiftUI Chat tutorial to learn more.

Build an Angular Video Chat App

Amos Gyamfi — Fri, 05 Jul 2024 05:21:02 +0000

Popular chat messaging apps like WhatsApp and Telegram offer real-time video calls, while video conferencing apps like Zoom and Google Meet provide group chat during meetings. Chat apps with video calling and video conferencing apps with chat support focus on two similar communication needs.

This article guides you in building an Angular chat app with integrated video conferencing support. You can customize the final project to prioritize video conferencing with chat as a secondary feature.

Prerequisites

The two main features of the sample project in this tutorial are chat messaging and video calling. For messaging functionality, we will use the Stream Chat Angular SDK. We will also power video calling support with the above SDK's companion JavaScript Video SDK. By integrating these two SDKs as in-apps, developers can build seamless communication experiences, allowing people to send and receive chat messages, livestream, video call, join, and leave live audio room conversations in their apps.

You can recreate this tutorial’s project with your favorite IDE, preferably VS Code.

Run the Source Code

To test the sample project, you can clone and have a local version or create a GitHub Codespace for the app in this repo.

Start With the Chat Feature

In this section, you will build the Angular chat UI with a look and feel similar to the image above. The leading chat features include instant messaging, media attachments, emojis, threads and replies, and message interactions. Let's initiate a new Angular project and get started with the following steps.

Create a New Angular Project

Creating a new Angular project requires the installation of Node.js. Open Terminal or your favorite command line tool and check the version of Node on your machine with the following:

node -v

If you don't have Node installed, click the link above to install the latest stable (LTS) version.

Execute the following command to install the Angular CLI globally on your machine.

npm install -g @angular/cli

Note: On macOS, you may be required to prepend sudo to the above command.

sudo npm install -g @angular/cli

With the Angular CLI installed, you can now create a new project with:

ng new video-chat-angular --routing false --style css --ssr false

where video-chat-angular is the project name. For styling, we use **CSS and also turn off routing. You can use whatever name you want for the project.

Open the app in your IDE and install the necessary dependencies as follows.

The example above shows the app opened in VS Code. Open the integrated terminal in VS Code and install Stream Chat Angular as a dependency with the following.

npm install stream-chat-angular stream-chat @ngx-translate/core

Stream-chat-angular: Stream Chat Angular consists of reusable Angular chat components.
stream-chat: The core Angular SDK without UI components. You can use this to build a completely custom chat messaging experience.
ngx-translate/core: An internationalization (i18n) library for Angular projects.

Configure the Angular Chat SDK

Before we setup the Angular SDK, we should add the required imports in the project's src/app/app.config.ts and src/app/app.component.ts

Open the file src/app/app.config.ts and import the TranslateModule in appConfig with the following.

providers: [importProvidersFrom(TranslateModule.forRoot())],

Next, open src/app/app.component.ts, and import the TranslateModule, StreamAutocompleteTextareaModule, StreamChatModule in the @Component decorator.

imports: [TranslateModule, StreamAutocompleteTextareaModule, StreamChatModule],

The modified content of this file should look like this:

import { Component, OnInit } from '@angular/core';
import { TranslateModule } from '@ngx-translate/core';
import {
  ChatClientService,
  ChannelService,
  StreamI18nService,
  StreamAutocompleteTextareaModule,
  StreamChatModule,
} from 'stream-chat-angular';

// Video imports
@Component({
  selector: 'app-root',
  standalone: true,
  imports: [TranslateModule, StreamAutocompleteTextareaModule, StreamChatModule],
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.scss'],
})
export class AppComponent implements OnInit {
  constructor(
    private chatService: ChatClientService,
    private channelService: ChannelService,
    private streamI18nService: StreamI18nService,
  ) {
    const apiKey = 'REPLACE_WITH_YOUR_STREAM_APIKEY';
    const userId = 'REPLACE_WITH_YOUR_STREAM_USERID';
    const userToken = 'REPLACE_WITH_TOKEN';
    this.chatService.init(apiKey, userId, userToken);
    this.streamI18nService.setTranslation();
  }

  async ngOnInit() {
    const channel = this.chatService.chatClient.channel('messaging', 'talking-about-angular', {
      // add as many custom fields as you'd like
      image: 'https://upload.wikimedia.org/wikipedia/commons/thumb/c/cf/Angular_full_color_logo.svg/2048px-Angular_full_color_logo.svg.png',
      name: 'Talking about Angular',
    });
    await channel.create();
    this.channelService.init({
      type: 'messaging',
      id: { $eq: 'talking-about-angular' },
    });
  }
}

In the code above, a valid user should be connected to the SDK's backend infrastructure to access the chat functionality. We are using a hard-coded user credential from our Angular Chat Tutorial. To implement a chat functionality for a production app using this SDK, you should sign up for a Dashboard account. On your Stream's Dashboard, you will find your API key to generate a token with our User JWT Generator. After filling out the user credentials, we create a new channel and define a condition to populate it when the app runs.

In addition to the above configurations, the Stream Chat SDK uses TypeScript's concept Allow Synthetic Default Imports to write default imports in a more efficient way. Add the following to tsconfig.json in the project's root folder to turn it on.

"allowSyntheticDefaultImports": true

Configure the Chat UI Structure and Style

In our generated Angular project, the UI structure is defined in src/app/app.component.html, while its styles are declared in src/app/styles.css. Replace the HTML content of src/app/app.component.html with the following to load the SDK's chat components such as Channel, Message List, Message Input, and Thread when the app launches.

<div id="root">
  <stream-channel-list></stream-channel-list>
  <stream-channel>
    <stream-channel-header></stream-channel-header>
    <stream-message-list></stream-message-list>
    <stream-notification-list></stream-notification-list>
    <stream-message-input></stream-message-input>
    <stream-thread name="thread">
      <stream-message-list mode="thread"></stream-message-list>
      <stream-message-input mode="thread"></stream-message-input>
    </stream-thread>
  </stream-channel>
</div>

Let's define the following CSS styles in src/app/styles.css. The styles we specify here do not include that of the video calling integration. We will modify this file again in a later section to cover that.

@import 'stream-chat-angular/src/assets/styles/v2/css/index.css';

html {
    height: 100%;
}

body {
    margin: 0;
    height: 100%;
}

#root {
    display: flex;
    height: 100%;

    stream-channel-list {
        width: 30%;
    }

    stream-channel {
        width: 100%;
    }

    stream-thread {
        width: 45%;
    }
}

Test the Chat Functionality

To start the development server in VS Code to run and test the chat version, launch the integrated Terminal and run:

ng serve or npm start

When the server runs successfully, you should see a screen similar to the one below.

Open the link to the localhost http://localhost:4200/ in your preferred browser and start testing the chat interface.

For the rest of this tutorial, let's implement the ability to make video calls by clicking a button at the top-right of the messages list in the chat window.

Integrate the Video Calling Functionality

To allow users to initiate video calls from the chat UI, we will use Stream Video's low-level JavaScript client, which developers can implement with any other SDK or platform.

You can test the app's video calling feature by creating a GitHub Codespace from the final project. A Codespace allows you to run and test the app in the browser without cloning and installing it from the GitHub repo.

Install and Configure the JavaScript Video SDK

To install the Video SDK, navigate to the project’s root folder in VS Code, open a new Terminal instance and run the following.

npm i @stream-io/video-client

Then start the development server with:

npm start

Create a Video Calling Service

Ensure you are still in the project's app folder /video-chat-angular/src/app. Then, use the Angular CLI command below to generate a calling service.

ng generate service calling

After running the above, you will find two additional generated files When you expand the app directory to see its content.

calling.service.spec.ts
calling.service.ts

Note: The links above do not contain the original generated content. They have been modified to provide our calling service.

Let's open calling.service.ts and substitute what is in the file with the following.


import { Injectable, computed, signal } from '@angular/core';
import { Call, StreamVideoClient, User } from '@stream-io/video-client';

@Injectable({
  providedIn: 'root',
})
export class CallingService {
  callId = signal<string | undefined>(undefined);

  call = computed<Call | undefined>(() => {
    const currentCallId = this.callId();
    if (currentCallId !== undefined) {
      const call = this.client.call('default', currentCallId);

      call.join({ create: true }).then(async () => {
        call.camera.enable();
        call.microphone.enable();
      });
      return call;
    } else {
      return undefined;
    }
  });

  client: StreamVideoClient;

  constructor() {
    const apiKey = '5cs7r9gv7sny';
    const token =
      'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoiZmxvcmFsLW1lYWRvdy01In0.uASF95J3UqfS8zMtza0opjhtGk7r7xW0SJACsHXF0Do';
    const user: User = { id: 'floral-meadow-5' };

    this.client = new StreamVideoClient({ apiKey, token, user });
  }

  setCallId(callId: string | undefined) {
    if (callId === undefined) {
      this.call()?.leave();
    }
    this.callId.set(callId);
  }
}

The service we created with the sample code above is designed to manage video calls, including joining and creating calls with video and audio enabled and leaving calls. We use a reactive programming pattern Angular Signals to manage state changes reactively. Check out our YouTube tutorial to learn more about creating a video calling app with Angular Signals.

First, we import the JavaScript video SDK, create a video client instance, and initialize it with an API key and token. Check out our documentation's Client and Authentication section to learn more about user types and how to generate a user token for your projects.

Finally, we create and join a call if it has not been created using the call.join method. The call.join method also allows real-time audio and video transmission.

Next, open calling.service.spec.ts and replace the original code with the sample code below.

import { TestBed } from '@angular/core/testing';

import { CallingService } from './calling.service';

describe('CallingService', () => {
  let service: CallingService;

  beforeEach(() => {
    TestBed.configureTestingModule({});
    service = TestBed.inject(CallingService);
  });

  it('should be created', () => {
    expect(service).toBeTruthy();
  });
});

The sample code above performs a basic unit test configuration for the CallingService. It checks to see if a calling service can be created or successfully initiated.

How to Manage Audio/Video Calls

In this section, we should define a component to manage audio and video call-related functions such as toggling the microphone and camera on and off, accepting and leaving calls, and identifying participants with session IDs. Navigate to the app's src/app directory and create a new folder call.

Add the following files inside the call folder and fill out each of their content by copying them from the GitHub project’s links below.

The files above provide the required UI and the logic for managing participants’ states and controlling microphone and camera states during audio/video calls.

Display and Manage a Call Participant

In this section, we should define a component that sets up the necessary bindings for displaying a participant's video and playing audio in a video call. Let's create the following directory src/app/participant.

Add the participant's files in the image above in the call directory and replace each piece of content by copying it from its link to the GitHub project.

Add a Start Call Button to the Chat UI

So far, the header of the chat UI's messages list consists of an avatar and the chat channel's title. Let's modify the message list’s header to add a start call button. We will also implement additional styles for the start button.

Open style.css located in the project's root folder and add this code snippet below the @import.

:root {
  --custom-blue: hsl(218, 100%, 50%);
  --custom-blue-hover: hsl(218, 100%, 70%);

  --custom-red: hsl(0, 80%, 60%);
  --custom-red-hover: hsl(0, 80%, 50%);
}

The modified content of style.css is shown below.

@import "stream-chat-angular/src/assets/styles/v2/css/index.css";

:root {
  --custom-blue: hsl(218, 100%, 50%);
  --custom-blue-hover: hsl(218, 100%, 70%);

  --custom-red: hsl(0, 80%, 60%);
  --custom-red-hover: hsl(0, 80%, 50%);
}

html {
  height: 100%;
}

body {
  margin: 0;
  height: 100%;
}

#root {
  display: flex;
  height: 100%;

  stream-channel-list {
    width: 30%;
  }

  stream-channel {
    width: 100%;
  }

  stream-thread {
    width: 45%;
  }
}

Locate app.component.css, an empty CSS file that comes with Angular project generation. Fill out its content with the following style snippet.

.header-container {
  display: flex;
  align-items: center;
  justify-content: space-between;
  padding: 0 1rem;
}

button {
  background: #005fff;
  color: white;
  padding: 0.5rem 1rem;
  border-radius: 0.25rem;
  border: none;
  cursor: pointer;
}

Next, let's add the call service to the HTML structure of the chat UI.

</ng-container>
  <ng-container *ngIf="callingService.callId()">
    <app-call [call]="callingService.call()!"></app-call>
</ng-container>

You must add the code snippet above to the app.component.html file in the app directory.

<div id="root">
 <stream-channel-list></stream-channel-list>
 <ng-container *ngIf="!callingService.callId()">
 <stream-channel>
 <div class="header-container">
 <stream-channel-header></stream-channel-header>
 <button (click)="startCall()">Start call</button>
 </div>
 <stream-message-list></stream-message-list>
 <stream-notification-list></stream-notification-list>
 <stream-message-input></stream-message-input>
 <stream-thread name="thread">
 <stream-message-list mode="thread"></stream-message-list>
 <stream-message-input mode="thread"></stream-message-input>
 </stream-thread>
 </stream-channel>
 </ng-container>
 <ng-container *ngIf="callingService.callId()">
 <app-call [call]="callingService.call()!"></app-call>
 </ng-container>
</div>

Finally, we should modify the app.component.ts file that was added in one of the chat sections to import the CallingService and CallComponent and add a startCall method. The modified sample code in app.component.ts should look like below.

import { Component, OnInit } from '@angular/core';
import { TranslateModule } from '@ngx-translate/core';
import {
  ChatClientService,
  ChannelService,
  StreamI18nService,
  StreamAutocompleteTextareaModule,
  StreamChatModule,
} from 'stream-chat-angular';
import { CallingService } from './calling.service';
import { CommonModule } from '@angular/common';

// Video imports
import { CallComponent } from './call/call.component';

@Component({
  selector: 'app-root',
  standalone: true,
  imports: [
    TranslateModule,
    StreamAutocompleteTextareaModule,
    StreamChatModule,
    CommonModule,
    CallComponent,
  ],
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.scss'],
})
export class AppComponent implements OnInit {
  callingService: CallingService;

  constructor(
    private chatService: ChatClientService,
    private channelService: ChannelService,
    private streamI18nService: StreamI18nService,
    callingService: CallingService
  ) {
    this.callingService = callingService;
    const apiKey = '5cs7r9gv7sny';
    const userId = 'floral-meadow-5';
    const userName = 'Floral Meadow';
    const userToken =
      'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoiZmxvcmFsLW1lYWRvdy01In0.uASF95J3UqfS8zMtza0opjhtGk7r7xW0SJACsHXF0Do';
    this.chatService.init(apiKey, userId, userToken);
    this.streamI18nService.setTranslation();
  }

  async ngOnInit() {
    const channel = this.chatService.chatClient.channel(
      'messaging',
      'talking-about-angular',
      {
        // add as many custom fields as you'd like
        image:
          'https://upload.wikimedia.org/wikipedia/commons/thumb/c/cf/Angular_full_color_logo.svg/2048px-Angular_full_color_logo.svg.png',
        name: 'Talking about Angular',
      }
    );
    await channel.create();
    this.channelService.init({
      type: 'messaging',
      id: { $eq: 'talking-about-angular' },
    });
  }

  startCall() {
    const channelId = this.channelService.activeChannel?.id;
    this.callingService.setCallId(channelId);
  }
}

Test the Integrated Chat and Video Calling Features

Start the development server again with:

npm start

The start call button you just implemented will appear at the top right of the message list's header. Clicking the button presents a live video of the local participant. On the live video or active call screen, there are buttons for performing standard call operations like switching the camera and microphone on and off and ending a call.

Conclusion

Congratulations! 👏 You have followed all the steps to build a real-time and fully functioning Angular video chat application. The app allows users to send and receive rich text chats, add media as attachments, and connect with others through face-to-face video calling. Explore the related links of this tutorial to learn more about the advanced features and capabilities of the Angular Chat and JavaScript Video SDKs.

How To Build a TikTok Clone With SwiftUI

Amos Gyamfi — Wed, 19 Jun 2024 14:54:29 +0000

Apps like TikTok, Instagram Reels, and YouTube Shorts help users engage with their audiences through concise, bite-sized videos. Let's create a TikTok clone, allowing users to browse through short videos of their favorite people, discover new ones, and also record their own videos for others to watch.

Project Requirements

To follow along with this tutorial, you will need an Xcode installation. It is recommended to download Xcode 15 or a later version. Also, to provide access to the user's camera feed in this app, we will use the Stream Video SDK. The Video SDK allows developers to build FaceTime-style video calling](https://getstream.io/blog/facetime-clone/), Twitch-like content streaming](https://getstream.io/blog/stream-video-twitch-clone/), Zoom-like video conferencing](https://getstream.io/blog/swiftui-video-conferencing-app/), and audio room platforms with reusable components. After creating a new SwiftUI project, you can fetch and install Stream Video using Swift Package Manager in Xcode.

Explore the Sample SwiftUI TikTok Clone App

Before you follow the steps in this tutorial, it will be great to have a look and feel how the finished project works. You can get the final sample app from GitHub, open it in Xcode, explore the code structure and discover how it works. The final project already has microphone and camera usage permissions configured. The following section explains how to set those configurations in Xcode.

Create and Configure the SwiftUI Project

Please create a new SwiftUI project in Xcode and name it as you like. The sample project in this tutorial uses TikTokCloneSwiftUI as the app name. The app will rely on audio/sound from the user's device and live video capture from the camera feed. Access to these device capabilities (sound and video) requires setting camera and microphone usage description privacies in Xcode. Select the app's root folder in the Xcode Project Navigator and click the Info tab. Then, click any of the + buttons you see when hovering over the items under Key. Search through the Privacy category and add:

Privacy - Microphone Usage Description
Privacy - Camera Usage Description

Build the TikTok-Like UIs

The app we built in this tutorial will allow short-form video browsing and self-recording using the iOS device's camera. The primary interaction styles include snap-flicking (paging) to watch bite-sized videos and tapping to record a live video as demonstrated in the image below.

When the app launches, it displays the video feeds TabView with a horizontal paging scroll style. This effect is opposite to that of the original TikTok app, which snap-scrolls vertically. The video feeds use standard SwiftUI views that implement AVKit. In a later article, we will integrate this feature with Stream's Activity Feeds.

Create the Looping Videos

This project will have five looping videos that users can cycle through for demonstration. The video feeds can be implemented using activity feeds in an actual application. Since each looping video uses the same Swift code, let's demonstrate it with the following sample code. When you download the Xcode project from GitHub, you will find all the files in it.

//
//  FirstVideoView.swift

import SwiftUI
import AVFoundation
import AVKit

struct FirstVideoView: View {
    @State var player = AVPlayer()
    let avPlayer = AVPlayer(url: Bundle.main.url(forResource: "oneDancing", withExtension: "mp4")!)

    var body: some View {
        ZStack {
            VideoPlayer(player: avPlayer)
                .scaledToFill()
                .ignoresSafeArea()
                .onAppear {
                    avPlayer.play()
                    avPlayer.actionAtItemEnd = .none
                    NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: avPlayer.currentItem, queue: .main) { (_) in
                        avPlayer.seek(to: .zero)
                        avPlayer.play()
                    }
            }
        }
    }
}

#Preview {
    FirstVideoView()
        .preferredColorScheme(.dark)
}

The sample code above creates a SwiftUI video player that loops forever using AVKit and AVFoundation.

Create the Video Feed Overlays

As indicated in the image above, each of the looping videos has two groups of views overlaid on it. Let's create ReactionButtons1View.swift to display the user's profile, likes, comments, and sharing ability in a vertical stack parent view. This sample code displays the icons on all the videos when you swipe through them.

//
//  ReactionButtonsView.swift
//  TikTokCloneSwiftUI

import SwiftUI

struct ReactionButtons1View: View {
    var body: some View {
        VStack(spacing: 24) {
            Button {

            } label: {
                Image(.profile1)
                    .resizable()
                    .scaledToFit()
                    .frame(width: 60, height: 60)
                    .clipShape(Circle())
                    .overlay(
                        Circle()
                            .stroke(LinearGradient(gradient: Gradient(colors: [Color.red, Color.blue]), startPoint: .topLeading, endPoint: .bottomTrailing), lineWidth: 2)
                    )
            }

            Button {

            } label: {
                VStack {
                    Image(systemName: "suit.heart.fill")
                        .font(.title)

                    Text("5K")
                }
                .foregroundStyle(.white)
            }

            Button {

            } label: {
                VStack {
                    Image(systemName: "message.fill")
                        .font(.title)

                    Text("56")
                }
                .foregroundStyle(.white)
            }

            Button {

            } label: {
                VStack {
                    Image(systemName: "square.and.arrow.up.fill")
                        .font(.title)

                    Text("Share")
                }
                .foregroundStyle(.white)
            }
        }
        .padding()
    }
}

#Preview {
    ReactionButtons1View()
        .preferredColorScheme(.dark)
}

Create the TikTok-Like Tab Bar

The tab bar contains five tab items. Only one of the tab items has interactivity, a plus button to launch a live video of users. To create the UI, add FeedsView.swift to the project and use the sample code below to fill out its content.

//
//  FeedsView.swift
//  TikTokCloneSwiftUI
//
//  Created by Amos Gyamfi on 31.5.2024.
//

import SwiftUI

struct FeedsView: View {
    @State var top = 0
    @State private var isLocalVideoShowing = false

    var body: some View {
        NavigationStack {
            ZStack {
                HTabView()
                    .padding(.top, -200)

                HStack {
                    Spacer()
                    //ReactionButtons1View()
                }
            }
            .toolbar {

                ToolbarItem(placement: .principal) {

                    HStack {
                        Button {
                            self.top = 0
                        } label: {
                            Text("Following")
                                .fontWeight(self.top == 0 ? .bold : .none)
                                .foregroundStyle(self.top == 0 ? .white : .white.opacity(0.5))
                                .padding(.vertical)
                        }
                        .buttonStyle(.plain)

                        Button {
                            self.top = 1
                        } label: {
                            Text("For You")
                                .fontWeight(self.top == 1 ? .bold : .none)
                                .foregroundStyle(self.top == 1 ? .white : .white.opacity(0.5))
                                .padding(.vertical)
                        }
                        .buttonStyle(.plain)
                    }
                }

                ToolbarItemGroup {
                    Button {
                        //
                    } label: {
                        Image(systemName: "magnifyingglass")
                    }
                    .buttonStyle(.plain)
                }

                ToolbarItemGroup(placement: .bottomBar) {
                    Button {

                    } label: {
                        VStack {
                            Image(systemName:  "house.fill")
                            Text("Home")
                                .font(.caption)
                        }
                    }
                    .buttonStyle(.plain)

                    Spacer()

                    Button {

                    } label: {
                        VStack {
                            Image(systemName:  "person.2")
                            Text("Friends")
                                .font(.caption)
                        }
                    }
                    .buttonStyle(.plain)

                    Spacer()

                    Button {
                        isLocalVideoShowing.toggle()
                    } label: {
                        Image(systemName: "plus.rectangle.fill")
                    }
                    .font(.title3)
                    .buttonStyle(.plain)
                    .foregroundStyle(.black)
                    .padding(EdgeInsets(top: 0, leading: 2, bottom: 0, trailing: 2))
                    .background(LinearGradient(gradient: Gradient(colors: [.teal, .red]), startPoint: .leading, endPoint: .trailing))
                    .cornerRadius(6)
                    .fullScreenCover(isPresented: $isLocalVideoShowing, content: CreateJoinLiveVideo.init)

                    Spacer()

                    Button {

                    } label: {
                        VStack {
                            Image(systemName:  "tray")
                            Text("Inbox")
                                .font(.caption)
                        }
                    }
                    .buttonStyle(.plain)

                    Spacer()

                    Button {

                    } label: {
                        VStack {
                            Image(systemName:  "person")
                            Text("Profile")
                                .font(.caption)
                        }
                    }
                    .buttonStyle(.plain)
                }
            }
        }

    }
}

#Preview {
    FeedsView()
        .preferredColorScheme(.dark)
}

Create the Live Video Overlays

The live video overlays consist of a video settings view (vertical container), duration options (horizontal container), and recording UI (pink button), as demonstrated in the image above. Create LiveVideoSettingsView.swift and substitute its content with the sample code below.

//
//  LiveVideoSettingsView.swift
//  TikTokCloneSwiftUI
//
//  Created by Amos Gyamfi on 1.6.2024.
//

import SwiftUI

struct LiveVideoSettingsView: View {
    var body: some View {
        VStack(spacing: 24) {
            Button {
                //
            } label: {
                Image(systemName: "bolt.slash.fill")
            }

            Button {
                //
            } label: {
                Image(systemName: "timer")
            }

            Button {
                //
            } label: {
                Image(systemName: "camera.filters")
            }

            Button {
                //
            } label: {
                Image(systemName: "camera.aperture")
            }

            Button {
                //
            } label: {
                Image(systemName: "wand.and.stars")
            }
        }
        .font(.title2)
        .bold()
        .buttonStyle(.plain)
        .padding()
        .background(.quaternary)
        .cornerRadius(32)
    }
}

#Preview {
    LiveVideoSettingsView()
        .preferredColorScheme(.dark)
}

Also, create LiveVideoOptionsView.swift and replace the template code with the following.

//
//  LiveVideoOptionsView.swift
//  TikTokCloneSwiftUI
//
//  Created by Amos Gyamfi on 1.6.2024.
//

import SwiftUI

struct LiveVideoOptionsView: View {
    var body: some View {
        HStack(spacing: 20) {
            Button {
                //
            } label: {
                Text("10m")
            }

            Button {
                //
            } label: {
                Text("60s")
            }

            Button {
                //
            } label: {
                Text("15s")
            }
            .buttonStyle(.plain)
            .padding(EdgeInsets(top: 4, leading: 8, bottom: 4, trailing: 8))
            .background(.tertiary)
            .cornerRadius(16)

            Button {
                //
            } label: {
                Text("Photo")
            }

            Button {
                //
            } label: {
                Text("Text")
            }
        }
        .buttonStyle(.plain)
        .padding()
        .background(.quaternary)
        .cornerRadius(32)
    }
}

#Preview {
    LiveVideoOptionsView()
        .preferredColorScheme(.dark)
}

The sample code below creates the recording view in RecordingView.swift.

//
//  RecordingView.swift
//  TikTokCloneSwiftUI
//
//  Created by Amos Gyamfi on 2.6.2024.
//

import SwiftUI

struct RecordingView: View {
    var body: some View {
        ZStack {
            Circle()
                .fill(.pink)
                .frame(width: 64, height: 64)
            Circle()
                .stroke(lineWidth: 4)
                .frame(width: 72, height: 72)
        }
    }
}

#Preview {
    RecordingView()
        .preferredColorScheme(.dark)
}

Check all the folders in the Xcode Project Navigator for other files related to the app's UI.

Install and Configure the Video SDK

Supposing we are building a production app with Stream's iOS Video SDK, we will need authenticated user credentials from a server. Since the demo app in this tutorial is purposely for development, we will use hard-coded user credentials from the SDK's video calling tutorial. You can use the API key of your Stream account and the companion token generator service to generate random users and tokens for development testing. Check out the get started guide to learn more.

To access and work with the SDK, we need to install it as a dependency in the Xcode project. Select File -> Add Package Dependencies… and paste this URL, https://github.com/GetStream/stream-video-swift, into the search box to install it.

Create a Live Video Participant View

For example, when building an iOS video conferencing app with the Video SDK, you need to render both the local and remote participants' videos and call controls for muting, flipping the camera from front to back, accepting and rejecting calls. However, the above is not a requirement for our TikTokClone. We need to show only the live video of the local participants. Add ParticipantsView.swift to the project and fill out its content with the code below.

//
//  ParticipantsView.swift
//  TikTokCloneSwiftUI
//
//  Created by Amos Gyamfi on 1.6.2024.
//

import SwiftUI
import StreamVideo
import StreamVideoSwiftUI

struct ParticipantsView: View {

    var call: Call
    var participants: [CallParticipant]
    var onChangeTrackVisibility: (CallParticipant?, Bool) -> Void

    var body: some View {
        GeometryReader { proxy in
            if !participants.isEmpty {
                ScrollView {
                    LazyVStack {
                        if participants.count == 1, let participant = participants.first {
                            makeCallParticipantView(participant, frame: proxy.frame(in: .global))
                                .frame(width: proxy.size.width, height: proxy.size.height)
                        } else {
                            ForEach(participants) { participant in
                                makeCallParticipantView(participant, frame: proxy.frame(in: .global))
                                    .frame(width: proxy.size.width, height: proxy.size.height / 2)
                            }
                        }
                    }
                }
            } else {
                Color.black
            }
        }
        .edgesIgnoringSafeArea(.all)
    }

    @ViewBuilder
    private func makeCallParticipantView(_ participant: CallParticipant, frame: CGRect) -> some View {
        VideoCallParticipantView(
            participant: participant,
            availableFrame: frame,
            contentMode: .scaleAspectFit,
            customData: [:],
            call: call
        )
        .onAppear { onChangeTrackVisibility(participant, true) }
        .onDisappear{ onChangeTrackVisibility(participant, false) }
    }
}

// Floating Participant
struct FloatingParticipantView: View {

    var participant: CallParticipant?
    //var size: CGSize = .init(width: 120, height: 120)
    var size: CGSize = .init(width: UIScreen.main.bounds.width, height: UIScreen.main.bounds.height)

    var body: some View {
        if let participant = participant {
            VStack {
                HStack {
                    Spacer()

                    VideoRendererView(id: participant.id, size: size) { videoRenderer in
                        videoRenderer.handleViewRendering(for: participant, onTrackSizeUpdate: { _, _ in })
                    }
                    .frame(width: size.width, height: size.height)
                    .clipShape(RoundedRectangle(cornerRadius: 8))
                }
                Spacer()
            }
            .padding()
        }
    }
}

In summary, the above sample code does the following:

Imports the required dependencies: The StreamVideo dependency is the core SDK. It does not contain any UI, so it is an excellent choice if you want to build a fully custom TikTok-like experience. Our demo app will use the SDK's reusable UI components by importing StreamVideoSwiftUI.
Manages the layout with Geometry Reader.
Renders and displays the local participants.
Watches for visibility changes of the participant.

How to Capture a Device’s Camera Feed

The video SDK allows developers to access and display local and remote participants' device camera feeds when [building a video calling app] like WhatsApp. In the context of our TikTok clone, we will use the SDK's' VideoRenderer to display only the local participant's video. We use the VideoRender because our app's use case does not require CallControls. To render audio/video calling experiences consisting of active, incoming, and outgoing call screens, you should use the SDK's CallContainer. Visit our documentation to learn more about the CallContainer.

To get a live video from the iOS device’s camera feed, add a new Swift file CreateJoinLiveVideo.swift. Then, replace the template code with the following.

import SwiftUI
import StreamVideo
import StreamVideoSwiftUI

struct CreateJoinLiveVideo: View {
    @State var call: Call
    @ObservedObject var state: CallState
    @State var callCreated: Bool = false
    @State private var isRecording = false

    private var client: StreamVideo

    private let apiKey: String = "mmhfdzb5evj2" // The API key can be found in the Credentials section
    private let token: String = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoiQm9iYV9GZXR0IiwiaXNzIjoiaHR0cHM6Ly9wcm9udG8uZ2V0c3RyZWFtLmlvIiwic3ViIjoidXNlci9Cb2JhX0ZldHQiLCJpYXQiOjE3MTcxNTM5NTEsImV4cCI6MTcxNzc1ODc1Nn0.wLgPJKgrruRC_4gYT7G0Od2MqPrR1KA8DV-cKi0Yd6k" // The Token can be found in the Credentials section
    private let userId: String = "Boba_Fett" // The User Id can be found in the Credentials section
    private let callId: String = "2dAfJGl7wThv" // The CallId can be found in the Credentials section
    @Environment(\.dismiss) private var dismiss

    init() {
        let user = User(
            id: userId,
            name: "Martin", // name and imageURL are used in the UI
            imageURL: .init(string: "https://getstream.io/static/2796a305dd07651fcceb4721a94f4505/a3911/martin-mitrevski.webp")
        )

        // Initialize Stream Video client
        self.client = StreamVideo(
            apiKey: apiKey,
            user: user,
            token: .init(stringLiteral: token)
        )

        // Initialize the call object
        let call = client.call(callType: "default", callId: callId)

        self.call = call
        self.state = call.state
    }

    var body: some View {
        NavigationStack {
            VStack {
                if callCreated {
                    ZStack {
                        ParticipantsView(
                            call: call,
                            participants: call.state.remoteParticipants,
                            onChangeTrackVisibility: changeTrackVisibility(_:isVisible:)
                        )

                        FloatingParticipantView(participant: call.state.localParticipant)

                        VStack {
                            HStack {
                                Spacer()
                                LiveVideoSettingsView()
                            }
                            .padding(.horizontal, 32)

                            HStack {
                                Spacer()
                                EffectsButtonView()

                                Spacer()

                                Button {
                                    isRecording.toggle()
                                    if isRecording {
                                        func startRecording() {
                                            Task {
                                                try await call.startRecording()
                                            }
                                        }
                                    } else {
                                        func stopRecording() {
                                            Task {
                                                try await call.stopRecording()
                                            }
                                        }
                                    }
                                } label: {
                                    RecordingView()
                                }
                                .buttonStyle(.plain)

                                Spacer()
                                UploadButtonView()
                                Spacer()
                            }
                            .padding(.top, 128)
                        }
                    }
                } else {
                    //Text("loading...")
                    ProgressView()
                }
            }
            .onAppear {
                Task {
                    guard callCreated == false else { return }
                    try await call.join(create: true)
                    callCreated = true
                }
            }
            .toolbar {
                ToolbarItem(placement: .topBarLeading) {
                    Button {
                        dismiss()
                    } label: {
                        Image(systemName: "xmark")
                    }
                    .buttonStyle(.plain)
                }

                ToolbarItem(placement: .principal) {
                    Button {

                    } label: {
                        HStack {
                            Image(systemName: "music.quarternote.3")
                            Text("Add sound")
                        }
                        .font(.caption)
                    }
                    .buttonStyle(.plain)
                    .padding(EdgeInsets(top: 8, leading: 10, bottom: 8, trailing: 10))
                    .background(.quaternary)
                    .cornerRadius(8)
                }

                ToolbarItemGroup(placement: .topBarTrailing) {
                    Button {

                    } label: {
                        Image(systemName: "arrow.triangle.2.circlepath")
                    }
                    .buttonStyle(.plain)
                }

                ToolbarItem(placement: .bottomBar) {
                    LiveVideoOptionsView()
                }
            }
        }
    }

    /// Changes the track visibility for a participant (not visible if they go off-screen).
    /// - Parameters:
    ///  - participant: the participant whose track visibility would be changed.
    ///  - isVisible: whether the track should be visible.
    private func changeTrackVisibility(_ participant: CallParticipant?, isVisible: Bool) {
        guard let participant else { return }
        Task {
            await call.changeTrackVisibility(for: participant, isVisible: isVisible)
        }
    }

}

To summarize the sample code above, we create an instance of the StreamVideo client and a user with a hard-coded token and API key. We check to see if a call has not been created, and then we create and join it to display a live video. The method, call.join(create: true) allows real-time sound and video transmission. With the above sample code, our TikTok clone app is ready. Let’s implement live video recording in the next step.

Add a Recording Functionality

When users start a live video, the SDK provides an easy way to integrate recording. This feature allows users to record ongoing audio and video activities. To use the SDK's recording functionality, create a state variable to toggle between the user's recording and non-recording states @State private var isRecording = false. Then, add a button to start and stop recording.

            Button {
                isRecording.toggle()
                if isRecording {
                    func startRecording() {
                        Task {
                            try await call.startRecording()
                        }
                    }
                } else {
                    func stopRecording() {
                        Task {
                            try await call.stopRecording()
                        }
                    }
                }
            } label: {
                RecordingView()
            }
            .buttonStyle(.plain)

You can see the recording implementation in CreateJoinLiveVideo.swift you added in the previous section.

Congratulations 👏. You deserve applause for following this step-by-step guide to building a fully functioning TikTok-like live video using SwiftUI and Stream's iOS Video SDK.

What’s Next?

This tutorial guided you in integrating short-form video support into a SwiftUI app to build a functioning TikTok clone. However, we did not cover the endless scrollable video feeds feature using a third-party activity feeds solution. That will come in a future tutorial. You can easily integrate activity feeds in your app with the [Stream Activity Feeds API](https://getstream.io/activity-feeds/.

Stream’s Video SDK has much more to offer you as a developer. Head to the iOS docs to learn more about integrating screen sharing, initiating calls from deep links, video/ audio filters, and more.

Create a Chat App With Node.js

Amos Gyamfi — Tue, 30 Apr 2024 13:34:18 +0000

Node.js is an open-source JavaScript runtime environment for building backend services and command line applications. This tutorial will guide you in creating an instant Node-based chat app that runs on a JavaScript server and outside a web browser.

First, grab your favorite command-line tool, Terminal or Warp, and a code editor, preferably VS Code and let’s begin.

Before You Start

Before proceeding with the tutorial, make sure you have the following.

Install Node: Download the latest prebuilt installer or use a package manager.
Install Yarn or NPM to add the required packages and modules.
Signup for a Stream account: To get an API key and app secret. You will use these credentials later in this tutorial.

A Preview of What You Will Build

The image above illustrates the chat application you will build using Node and Stream’s chat API. Download the completed project from GitHub, explore the source code, and test it using VS Code.

Create a New Node Project

Launch your favorite command line tool and make a new directory with the command below.

mkdir node-chat

cd into the node-chat folder and initialize a new Node project with the following command.

yarn init -y

The above command adds two files, package.json and README.md, to the folder node-chat.

Install Server Dependencies

In this tutorial, we must build a Node server to run the chat app. Creating the server requires the following dependencies.

Dotenv: A module for making environment variables from a .env file accessible to a process.env object.
Express: A lightweight framework for building web applications.
Stream Chat JS: A real-time chat SDK for JavaScript.
Cors: A package for providing middleware for Node.js.
Body-parser: A body passing middleware for Node.js.

Enter the command below to install all the packages. If you prefer, you can use yarn instead of npm to add the packages.

npm add dotenv express stream-chat cors body-parser

Add Required Files in VS Code

Still in the project's root folder, node-chat in Terminal or using your chosen command line tool, enter code . to launch the project in VS Code.

Note: Using code . to open VS Code only works if you have already enabled the feature in VS Code.

Add a .env file to store the following Stream Dashboard credential placeholders in the project's root folder. You will replace the placeholder credentials shortly.

PORT=5500
STREAM_API_KEY=<your api key>
STREAM_APP_SECRET=<your app secret>

Next, add a new file, server.js in the project's root folder and use the following sample code as its content.


// server.js
require('dotenv').config();

const express = require('express');
const cors = require('cors');
const bodyParser = require('body-parser');
const { StreamChat } = require('stream-chat');

const app = express();

app.use(cors());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

// initialize Stream Chat SDK
const serverSideClient = new StreamChat(
  process.env.STREAM_API_KEY,
  process.env.STREAM_APP_SECRET
);

app.post('/join', async (req, res) => {
  const { username } = req.body;
  const token = serverSideClient.createToken(username);
  try {
    await serverSideClient.updateUser(
      {
        id: username,
        name: username,
      },
      token
    );

    const admin = { id: 'admin' };
    const channel = serverSideClient.channel('team', 'general', {
      name: 'General',
      created_by: admin,
    });

    await channel.create();
    await channel.addMembers([username, 'admin']);

    res
      .status(200)
      .json({ user: { username }, token, api_key: process.env.STREAM_API_KEY });
  } catch (err) {
    console.error(err);
    res.status(500);
  }
});

const server = app.listen(process.env.PORT || 5500, () => {
  const { port } = server.address();
  console.log(`Server running on PORT ${port}`);
});

The dotenv package we added in the previous section will make the .env file available to the process.env object in the sample code above, server.js.

Obtain App Credentials From Your Stream’s Dashboard

In this section, you will create a new app in your Stream’s dashboard and use your API key and the app's secret to populate the .env file you added earlier. If you do not have an account yet, sign up for a free one. When you log in to your dashboard:

Select the Apps category in the left sidebar and click the Create App button.

Fill in the required information to create the new in app in your Dashboard.

When you finish the app creation, click the its name to access the key and secret. Copy them to fill their placeholders in the .env file.

Run the Server

At this point, we can run the server via the command line. Since we have already opened the app in VS Code, it is more convenient to use the built-in VS Code Terminal.

Launch the Terminal in VS Code by navigating the toolbar and selecting Terminal -> New Terminal.
Add the command node server.js to start the server. Run the command in the root folder, as illustrated in the image below.

You will now see the message Server running on PORT 5500. Congratulations 🎉 👏. You now have the server up and running. Let’s create a chat client in the next section.

Create a Chat Client

Our chat client will have the following dependencies:

Axios: An HTTP client for Node.js and the web.
Prompt: A command-line prompt for Node.js.
Ora: A dynamic loading spinner for Terminal.
Util: A utility module for Node.js.
Neo-blessed: A cursor-like library for Node applications.

Add the following command in the VS Code’s Terminal to install all the above dependencies.

npm add axios prompt ora util neo-blessed

Next, add a new file, app.js, to the project’s root folder in VS Code. Use the sample code below to fill in the file’s content.


// app.js
const axios = require('axios');
const prompt = require('prompt');
const ora = require('ora');
const { StreamChat } = require('stream-chat');
const util = require('util');
const blessed = require('neo-blessed');

function fetchToken(username) {
  return axios.post('http://localhost:5500/join', {
    username,
  });
}

async function main() {
  const spinner = ora();
  prompt.start();
  prompt.message = '';

  const get = util.promisify(prompt.get);

  const usernameSchema = [
    {
      description: 'Enter your username',
      name: 'username',
      type: 'string',
      pattern: /^[a-zA-Z0-9\-]+$/,
      message: 'Username must be only letters, numbers, or dashes',
      required: true,
    },
  ];

  const { username } = await get(usernameSchema);
  try {
    spinner.start('Fetching authentication token...');
    const response = await fetchToken(username);
    spinner.succeed(`Token fetched successfully!`);

    const { token } = response.data;
    const apiKey = response.data.api_key;

    const chatClient = new StreamChat(apiKey);

    spinner.start('Authenticating user...');
    await chatClient.setUser(
      {
        id: username,
        name: username,
      },
      token
    );
    spinner.succeed(`Authenticated successfully as ${username}!`);

    spinner.start('Connecting to the General channel...');
    const channel = chatClient.channel('team', 'general');
    await channel.watch();
    spinner.succeed('Connection successful!');
  } catch (err) {
    spinner.fail();
    console.log(err);
    process.exit(1);
  }
}
main();

The above sample code uses the Prompt package we installed earlier to request the user's username. The username request is then sent to the server to retrieve a token for authentication. To verify that the user authentication works successfully, keep the server running with the command node server.js.

While the server is still running, open another Terminal in VS Code and enter the command node app.js to receive a prompt to enter a user name. Press the return key after adding the user name. You will then get a similar result in the image below.

Note: If you get an ora package error after running node app.js in the Terminal, remove the package with npm r ora. Then, install an older version of ora with the command, npm i ora@5.4.1.

Create the Chat UI With Neo-blessed

Let's create a chat UI using the package called Neo-blessed. Neo-blessed is a cursor-like library that helps developers to build Node-based Terminal interfaces. To create the chat UI, you should update the sample code in app.js with the following.


// app.js: updated
const axios = require('axios');
const prompt = require('prompt');
const ora = require('ora');
const { StreamChat } = require('stream-chat');
const util = require('util');
const blessed = require('neo-blessed');

function fetchToken(username) {
  return axios.post('http://localhost:5500/join', {
    username,
  });
}

async function main() {
  const spinner = ora();
  prompt.start();
  prompt.message = '';

  const get = util.promisify(prompt.get);

  const usernameSchema = [
    {
      description: 'Enter your username',
      name: 'username',
      type: 'string',
      pattern: /^[a-zA-Z0-9\-]+$/,
      message: 'Username must be only letters, numbers, or dashes',
      required: true,
    },
  ];

  const { username } = await get(usernameSchema);
  try {
    spinner.start('Fetching authentication token...');
    const response = await fetchToken(username);
    spinner.succeed(`Token fetched successfully!`);

    const { token } = response.data;
    const apiKey = response.data.api_key;

    const chatClient = new StreamChat(apiKey);

    spinner.start('Authenticating user...');
    await chatClient.setUser(
      {
        id: username,
        name: username,
      },
      token
    );
    spinner.succeed(`Authenticated successfully as ${username}!`);

    spinner.start('Connecting to the General channel...');
    const channel = chatClient.channel('team', 'general');
    await channel.watch();
    spinner.succeed('Connection successful!');
    process.stdin.removeAllListeners('data');

    const screen = blessed.screen({
      smartCSR: true,
      title: 'Stream Chat Demo',
    });

    var messageList = blessed.list({
      align: 'left',
      mouse: true,
      keys: true,
      width: '100%',
      height: '90%',
      top: 0,
      left: 0,
      scrollbar: {
        ch: ' ',
        inverse: true,
      },
      items: [],
    });

    // Append our box to the screen.
    var input = blessed.textarea({
      bottom: 0,
      height: '10%',
      inputOnFocus: true,
      padding: {
        top: 1,
        left: 2,
      },
      style: {
        fg: '#787878',
        bg: '#454545',

        focus: {
          fg: '#f6f6f6',
          bg: '#353535',
        },
      },
    });

    input.key('enter', async function() {
      var message = this.getValue();
      try {
        await channel.sendMessage({
          text: message,
        });
      } catch (err) {
        // error handling
      } finally {
        this.clearValue();
        screen.render();
      }
    });

    // Append our box to the screen.
    screen.key(['escape', 'q', 'C-c'], function() {
      return process.exit(0);
    });

    screen.append(messageList);
    screen.append(input);
    input.focus();

    screen.render();

    channel.on('message.new', async event => {
      messageList.addItem(`${event.user.id}: ${event.message.text}`);
      messageList.scrollTo(100);
      screen.render();
    });
  } catch (err) {
    spinner.fail();
    console.log(err);
    process.exit(1);
  }
}
main();

The previous content of app.js authenticated the user and printed a message about the authentication on the screen. Let’s take a look at the following changes.

const screen = blessed.screen({
      smartCSR: true,
      title: 'Stream Chat Demo',
    });

    var messageList = blessed.list({
      align: 'left',
      mouse: true,
      keys: true,
      width: '100%',
      height: '90%',
      top: 0,
      left: 0,
      scrollbar: {
        ch: ' ',
        inverse: true,
      },
      items: [],
    });

    // Append our box to the screen.
    var input = blessed.textarea({
      bottom: 0,
      height: '10%',
      inputOnFocus: true,
      padding: {
        top: 1,
        left: 2,
      },
      style: {
        fg: '#787878',
        bg: '#454545',

        focus: {
          fg: '#005fff',
          bg: '#353535',
        },
      },
    });

In the updated version of app.js, we used the Neo-blessed package to create a simple chat interface consisting of a text area and a container for holding a list of messages as shown in the code snippet above.

Retrieve and Send a Message

    input.key('enter', async function() {
      var message = this.getValue();
      try {
        await channel.sendMessage({
          text: message,
        });
      } catch (err) {
        // error handling
      } finally {
        this.clearValue();
        screen.render();
      }
    });

In this code snippet, when a user types a new message and presses the return key, the message is retrieved and sent to the message channel using the Stream Chat's sendMessage method. Head to our documentation to learn more about sending simple messages using Stream.

Run the Node Server and the Chat App

Open a new Terminal window in VS Code and run the Node server with node server.js. Next, launch another Terminal window and run the updated chat app with node app.js.

Bravo 👏. You will now see a chat interface similar to the one in the image below, allowing you to enter and send instant text messages.

Chat Between Multiple Terminal Instances

When the app is running after performing the previous steps, you can open an additional terminal instance and chat between them. Here, you should authenticate yourself with a different user name and tile the Terminal windows vertically or horizontally in VS Code. You can now send and receive messages between the two authenticated users, similar to the demonstration in the video below.

Wrap Up

In this tutorial, you learned how to create a simple chat application using Node.js. You made a backend server that authenticates chat users with Stream's Dashboard credentials. Eventually, you discovered how to send and receive messages among multiple users in a channel. The Node.js chat application you built in this tutorial demonstrates a basic use case of what you can build with Node and Stream's JavaScript SDK for chat messaging. Check out the related links to learn more and discover where to go from here.

Build a Cross-Platform Messaging App with React Native Expo

Amos Gyamfi — Wed, 17 Apr 2024 07:21:11 +0000

Building a complete chat messaging app like WhatsApp can take time and effort. There are so many aspects developers need to consider to provide a full and feature-rich messaging experience, such as offline support, media attachment, and more. This tutorial uses Stream Chat from React Native Directory and the Expo framework to create a fully featured cross-platform mobile messaging experience.

Tools and Technologies To Get Started

After following the steps in this tutorial to build the sample project, we will test it on an iPhone and an actual Android device. Since we are using the Expo framework to initialize the React Native project, it removes the need for platform-specific configurations in Xcode and Android Studio. Let’s begin by:

Installing VS Code: For developing and running our cross-platform application.
Installing the Expo Go app from the App Store and Google Play Store: To test the messaging app on an iPhone and Android using a QR Code generated from VS Code after running it.

Test the Sample App

To run the sample project on your iOS and Android devices, you should first install the chat SDK and its dependencies. Install the Expo Go app for iOS and Android. Then, download and add the files in this GitHub repository to the root folder of the Expo project. For the files that come with the Expo project generation, replace their content with the ones in the GitHub repo.

iOS and Android Versions of the Sample App

The image above represents the iOS version of the React Native app. You have noticed a chat channel list (contacts), messages view, media attachment, and reactions.

Similarly, the Android version shown below has the same features as the iOS counterpart.

Choosing a React Native Chat SDK

There are several factors to consider when choosing a chat SDK for a React Native Project. Let's outline some key considerations when selecting a universal messaging SDK for your app.

Offline Support: Users should be able to browse contact lists and send and receive messages offline.
React Native CLI Support: The SDK should integrate seamlessly with apps developers create with React Native CLI.
Expo Support: If you prefer initializing your React Native projects with popular frameworks like Expo, you should be able to implement the chat SDK in an Expo app with minimal effort.
React Native Directory Score: The chat SDK you choose for your project should be one of the recommended chat libraries in the React Native directory.
Regular updates: On React Native Directory, you should check whether the chat SDK has regular updates and higher monthly downloads.

The points above make Stream's Chat SDK for React Native an excellent choice for our use case. Let's install it along with the required dependencies in the following sections to build our cross-platform messaging app.

Create a New Expo App and Install Core Dependencies

The Stream's Chat SDK for React Native provides two installation options.

Using React Native CLI: Requires complicated command line configurations, Android Studio, and Xcode settings.
Using Expo framework: Handles all the configurations mentioned above and provides seamless testing with its GO app on iPhone and Android devices.

Let's go with the Expo approach so that we do not have to worry about performing any additional settings required to run the app. Launch your favorite command line tool, Terminal or Warp, and run the below command to initialize a new React Native Expo app.

First, install Node and run npx create-expo-app NativeChat. Here, we use npx with your Node installation to create a new React Native Expo app called NativeChat.

Now, we have a blank Expo project. Next, we can install the Stream Chat Expo SDK using the command below.

expo install stream-chat-expo

The chat SDK also has the following dependencies.

Flat List: For maintaining visible content position of scrollable UIs.
Netinfo: Helps retrieve information about a network.
Expo File System: Provides access to the file system on Android and iOS. It will enable us to attach files and media to messages.
Expo Image Manipulator: Allows image manipulation on Android and iOS.
Expo Image Picker: Provides access to select and pick images and videos in a device's Photos Library.
Expo Media Library: Helps access a device's Media Library.
React Native Gesture Handler: A gesture and touch system for React Native.
React Native Animated: An animation library for React Native projects.
SVG support for React Native.

Open the project in VS Code and cd into the root folder NativeChat. Go to the Toolbar and click Terminal -> New Terminal and run the following multi-command to install the peer dependencies outlined above.

expo install @stream-io/flat-list-mvcp @react-native-community/netinfo expo-file-system expo-image-manipulator expo-image-picker expo-media-library react-native-gesture-handler react-native-reanimated react-native-svg

Configure the App’s babel.config.js and App.js

In the project's root folder, find babel.config.js and add the following code snippet to register the React Native Reanimated plugin we installed previously plugins: [ 'react-native-reanimated/plugin', ],. The content of babel.config.js will look like this:

module.exports = function(api) {
  api.cache(true);
  return {
    presets: ['babel-preset-expo'],
    plugins: [
      'react-native-reanimated/plugin', // The plugin must be the last item
    ],
  };
};

Also, previously, we installed the React Native Gesture Handler. When the app launches, the gesture recognizer should be immediately available. The best location to add it is at the app's entry point, thus in App.js. Open App.js from the project's root folder and add the following to the import section.

import { GestureHandlerRootView } from 'react-native-gesture-handler';

Add Optional Expo Libraries

To improve media upload and document attachment in our messaging app, let's install the following components for the Expo SDK. Note: You may skip the installations below because they are optional.

Expo Document Picker: A library that helps users to pick documents from their devices.
Expo AV: Provides audio and video playback.
Expo Haptics: Helps to access the sound and vibration engine on Android and iOS devices.
Expo Sharing: Provides support for file sharing.
Expo Clipboard: Provides the ability to copy text to the clipboard.

Use the following multi-command to install all the above Expo libraries.

npm add xpo-document-picker expo-av expo-haptics expo-sharing expo-clipboard

Note: You can also add them with yarn.

Setup the Chat SDK for React Native

In this section, we will create a JavaScript object chatClient, to connect the Expo app to a user from the Stream Chat SDK's backend. The chatClient should be initialized with an API key for a production app. Sign up for a free dashboard account to get an API key if you do not have one yet. In this tutorial, we will use hard-coded user credentials just for testing.

Follow the steps below to configure the chat SDK to work with our Expo app.

Step 1: Store the User Credentials

In the project's root folder, add a new JavaScript file, chatConfig.js, and add the following code snippet to store user credentials, such as API key, user ID, token, and name for the app.

// chatConfig.js

export const chatApiKey = 'dz5f4d5kzrue'
export const chatUserId = 'lively-wood-7';
export const chatUserToken = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoibGl2ZWx5LXdvb2QtNyIsImV4cCI6MTcxMDUzMjQ0Nn0.KMkjl68QocLTO_B5E4jIw2s5VBmRLOIvFnZkYNQepmc';
export const chatUserName = 'lively';

Step 2: Connect the User

Create a new JavaScript file, useChatClient.js, and add the sample code below to connect the user to the Stream's backend infrastructure.

// useChatClient.js

import { useEffect, useState } from 'react';
import { StreamChat } from 'stream-chat';
import { chatApiKey, chatUserId, chatUserName, chatUserToken } from './chatConfig';

const user = {
  id: chatUserId,
  name: chatUserName,
};

const chatClient = StreamChat.getInstance(chatApiKey);

export const useChatClient = () => {
  const [clientIsReady, setClientIsReady] = useState(false);

  useEffect(() => {
    const setupClient = async () => {
      try {
        chatClient.connectUser(user, chatUserToken);
        setClientIsReady(true);

      } catch (error) {
        if (error instanceof Error) {
          console.error(`An error occurred while connecting the user: ${error.message}`);
        }
      }
    };

    // If the chat client has a value in the field `userID`, the user gets connected
    if (!chatClient.userID) {
      setupClient();
    }
  }, []);

  return {
    clientIsReady,
  };
};

Step 3: Share Channel and Thread Data Between Screens


// AppContext.js

import React, { useState } from 'react';

export const AppContext = React.createContext({
  channel: null,
  setChannel: (channel) => {},
  thread: null,
  setThread: (thread) => {},
});

export const AppProvider = ({ children }) => {
  const [channel, setChannel] = useState();
  const [thread, setThread] = useState();

  return <AppContext.Provider value={{ channel, setChannel, thread, setThread }}>{children}</AppContext.Provider>;
};

export const useAppContext = () => React.useContext(AppContext);

Step 4: Configure the Chat Screen Components in App.js

In this section, we configure all the chat screen components at the app's entry point, App.js.
Let's use a step-by-step approach to configure the app screens' navigation, channels filtering and sorting, ChannelListScreen, ChannelScreen, UI overlays, and ThreadsScreen.

1. Import Components and Modules in App.js

import React from 'react';
import { NavigationContainer } from '@react-navigation/native';
import { createStackNavigator } from '@react-navigation/stack';
import { SafeAreaView } from 'react-native-safe-area-context';
import { Text } from 'react-native';
import { useChatClient } from './useChatClient';
import { GestureHandlerRootView } from 'react-native-gesture-handler';
import { AppProvider } from "./AppContext";
import {
  Chat,
  ChannelList,
  OverlayProvider,
  Channel,
  MessageList,
  MessageInput,
  Thread,
} from 'stream-chat-expo'; // stream-chat-react-native Or stream-chat-expo
import { StreamChat } from 'stream-chat';
import { chatApiKey, chatUserId } from './chatConfig';
import { useAppContext } from "./AppContext";

Here, we will use the useChatClient to connect the user to the backend chat client and integrate it into the app's NavigationStack. The GestureHandlerRootView will make gestures available soon after the app launches. Also, the AppProvider will make the current stored channel and threads available in the navigation stack.

2. Create a Stack Navigator, Filter, and Sort Channels

const Stack = createStackNavigator();
const chatClient = StreamChat.getInstance(chatApiKey);

const filters = {
  members: {
    '$in': [chatUserId]
  },
};

const sort = {
  last_message_at: -1,
};

The code snippet above creates a navigation Stack for the channel list, channel, and thread screens, a chatClient instance with an API key, filters the channel to show members only, and sorts the channel by a last message.

3. Configure the ChannelListScreen Component

const ChannelListScreen = () => {
  const { setChannel } = useAppContext();
  return (
    <ChannelList
      onSelect={(channel) => {
        const { navigation } = props;
        setChannel(channel);
        navigation.navigate('ChannelScreen');
      }}
      filters={filters}
      sort={sort}
    />
  );
}

To display the chat ChannelList, we wrap it in the ChannelListScreen component and use the onSelect function to show the channel screen when a user selects a channel.

4. Configure the ChannelScreen Component

const ChannelScreen = props => {
  const { navigation } = props;
  const { channel, setThread } = useAppContext();

  return (
    <Channel channel={channel}>
      <MessageList 
        onThreadSelect={(message) => {
          if (channel?.id) {
            setThread(message);
            navigation.navigate('ThreadScreen');
          }
        }}
      />
      <MessageInput />
    </Channel>
  );
}

The ChannelScreen is configured to display three components, chat Channel, list of messages MessageList, and message composer MessageInput.

5. Configure the ThreadsScreen

const ThreadScreen = props => {
  const { channel, thread } = useAppContext();
  return (
    <Channel channel={channel} thread={thread} threadList>
      <Thread />
    </Channel>
  );
}

In the above code snippet, we wrap the Thread component with the Channel component to display the thread messages in the app

6. Setup the NavigationStack

const NavigationStack = () => {
  const { clientIsReady } = useChatClient();

  if (!clientIsReady) {
    return <Text>Loading chat ...</Text>
}

The app's NavigationStack component wraps the Chat component and the OverlayProvider to communicate all components in the app to the chatClient and display the overlay components such as reactions view, fullscreen image preview, and attachment preview.

7. Display UI Overlays

  return (
    <OverlayProvider>
      <Chat client={chatClient}>
        <Stack.Navigator>
        <Stack.Screen name="ChannelList" component={ChannelListScreen} />
        <Stack.Screen name="ChannelScreen" component={ChannelScreen} />
        <Stack.Screen name="ThreadScreen" component={ThreadScreen} />
        </Stack.Navigator>
      </Chat>
    </OverlayProvider>
  );
};

The OverlayProvider helps to react to messages with a long press gesture. Configure it in the app's root and outside the navigation stack to show reactions, fullscreen images, and attachment views.

8. Display the default Component

export default () => {
  return (
    <AppProvider>
        <GestureHandlerRootView style={{ flex: 1 }}>
            <SafeAreaView style={{ flex: 1 }}>
                <NavigationContainer>
                    <NavigationStack />
                </NavigationContainer>
            </SafeAreaView>
        </GestureHandlerRootView>
    </AppProvider>
);
};

Finally, in this code snippet, we wrap the default component with the AppProvider and GestureHandlerRootView to use the AppContext and make gestures available immediately when the app launches.

Putting It All Together

Here are complete implementations of the above code snippets in the App.js file.


// App.js

import React from 'react';
import { NavigationContainer } from '@react-navigation/native';
import { createStackNavigator } from '@react-navigation/stack';
import { SafeAreaView } from 'react-native-safe-area-context';
import { Text } from 'react-native';
import { useChatClient } from './useChatClient';
import { GestureHandlerRootView } from 'react-native-gesture-handler';
import { AppProvider } from "./AppContext";
import {
  Chat,
  ChannelList,
  OverlayProvider,
  Channel,
  MessageList,
  MessageInput,
  Thread,
} from 'stream-chat-expo'; // stream-chat-react-native Or stream-chat-expo
import { StreamChat } from 'stream-chat';
import { chatApiKey, chatUserId } from './chatConfig';
import { useAppContext } from "./AppContext";

const Stack = createStackNavigator();
const chatClient = StreamChat.getInstance(chatApiKey);

const filters = {
  members: {
    '$in': [chatUserId]
  },
};

const sort = {
  last_message_at: -1,
};

const ChannelListScreen = () => {
  const { setChannel } = useAppContext();
  return (
    <ChannelList
      onSelect={(channel) => {
        const { navigation } = props;
        setChannel(channel);
        navigation.navigate('ChannelScreen');
      }}
      filters={filters}
      sort={sort}
    />
  );
}

const ChannelScreen = props => {
  const { navigation } = props;
  const { channel, setThread } = useAppContext();

  return (
    <Channel channel={channel}>
      <MessageList 
        onThreadSelect={(message) => {
          if (channel?.id) {
            setThread(message);
            navigation.navigate('ThreadScreen');
          }
        }}
      />
      <MessageInput />
    </Channel>
  );
}

const ThreadScreen = props => {
  const { channel, thread } = useAppContext();
  return (
    <Channel channel={channel} thread={thread} threadList>
      <Thread />
    </Channel>
  );
}

const NavigationStack = () => {
  const { clientIsReady } = useChatClient();

  if (!clientIsReady) {
    return <Text>Loading chat ...</Text>
}

  return (
    <OverlayProvider>
      <Chat client={chatClient}>
        <Stack.Navigator>
        <Stack.Screen name="ChannelList" component={ChannelListScreen} />
        <Stack.Screen name="ChannelScreen" component={ChannelScreen} />
        <Stack.Screen name="ThreadScreen" component={ThreadScreen} />
        </Stack.Navigator>
      </Chat>
    </OverlayProvider>
  );
};

export default () => {
  return (
    <AppProvider>
        <GestureHandlerRootView style={{ flex: 1 }}>
            <SafeAreaView style={{ flex: 1 }}>
                <NavigationContainer>
                    <NavigationStack />
                </NavigationContainer>
            </SafeAreaView>
        </GestureHandlerRootView>
    </AppProvider>
);
};

Run the App on iOS

We have configured the Stream Chat SDK to work with our React Native Expo app and set up the required navigation and chat screens. To run the app on an iOS device, you can use the Expo Go app. First, run the following command by navigating to the app's root folder and opening a new Terminal in VS Code.

npx expo start

This command starts the Metro Server and shows a QR code with instructions on how to run the app on iOS, Android, and the web.

Ensure you have installed the Expo Go app on your iPhone or iPad. Launch the default Camera app for iOS and scan the QR code you see after running npx expo start. You will see a button to open the project with the Expo Go app.
Following the steps above will launch the chat channel list screen. You can tap any channel list items to enter the messages screen to start chatting, add images and videos to chats, and show how you feel with reactions.

Run the App on Android

On Android, you should scan the QR code the Metro Server generates with the Expo Go app you have installed instead of the Camera app for Android. After the app launches, you can perform the functions explained in the iOS section.

Where To Go From Here

You have followed this tutorial to create a fully working cross-platform messaging app for React Native using the Expo framework and Stream. Congratulations 👏. You can take what we just built and go further by customizing the chat channel list, message list, message composer, and more. The app supports offline browsing. However, this feature is turned off by default in the SDK. Head to our documentation to learn more about enabling offline support with a single line of code. Also, you can refer to our GitHub sample apps to learn how to build iMessage, Slack, and WhatsApp clones.

DEV Community: Amos Gyamfi

Compare the 3 Best Agentic CLI Coding Tools

What Is a CLI Coding Tool?

CLI-Based Coding Platforms vs. AI-First Code Editors

CLI Coding Tools Use Cases Across Teams

The 3 Leading Command-Line Coding Platforms

What is Aider?

Key Features of Aider

Get Started With Aider

What is Claude Code?

Key Features of Claude Code

Get Started With Claude Code

Use Claude Code With Non-Anthropic Models

What is Codex CLI?

Key Features of Codex CLI

Get Started With Codex CLI

What to Consider When Using CLI Coding Platforms

The Future of CLI Coding Platforms

Top 7 Frameworks To Integrate MCP With LLMs

LLM Context Provision Approaches and Specifications

What is MCP?

How MCP Works

Why Adopt MCP for AI Agents and LLM-based Apps?

An AI Agent With A Tool Integration

An AI Agent With MCP Integration

Advantages of Using MCP Over Traditional Toolkits Integration

Kinds of MCP Servers

Access an Ecosystem of MCP Registries/Servers

Top 7 Client Frameworks to Add MCP to LLMs and Agents

1. Build a Git MCP Agent With OpenAI Agents SDK

2. Build MCP AI Agents With Praison AI

3. Using MCP for LangChain AI Apps

4. Using MCP for Chainlit AI Apps

Configure Your Chainlit App To Connect to an MCP Server

5. Integrate MCP for Agno AI Agents

Prerequisites

Configure the Agno MCP Team of Agents

6. Using MCP for Upsonic Agents

7. Using MCP for Mastra Agents

Challenges and What’s Next for MCP in LLM Apps and Agents

How To Run OpenAI Agents SDK Locally With 100+ LLMs and Custom Tracing

Prerequisites

Why is Tracing Important in Agentic Workflows?

What Problems Are We Solving?

Run locally

Not limited to only the OpenAI models

Tracing

Build a File System MCP Agent

Configure Your Local Environment

Ollama-Supported Models and Streamlit UI Example

Ollama + Gradio + LiteLLM: Access 100+ LLMs

Implement Your Agents’ Logging System With AgentOps

Where to Go From Here

The 3 Best Python Frameworks To Build UIs for AI Apps

AI Chat UIs: Overview

How To Build AI Apps in Python

Why Use Python Libraries For AI Chat UIs?

AI Chat UI Builders and Use Cases

Tools For Building AI Chat UIs in Python

1. Gradio: Build UIs for Testing and Deploying AI Apps

Key Features and Benefits of Gradio

Gradio: Build a Real-Time Voice UI

Gradio: Integration With Popular APIs and LLM Libraries

2. Streamlit: Build AI Apps and Share in the Cloud

Key Features and Benefits of Streamlit

Build a Streamlit UI To Run Models Locally

3. Chainlit: Build UIs for Conversational AI

Key Features and Benefits of Using Chainlit

Build a Chainlit UI to Access 100+ LLMs

Other Frameworks To Build AI App UIs in Python

Final Thoughts on Chat UIs for AI Apps

Using a Speech Language Model That Can Listen While Speaking

In this Article

Traditional Turn-Based Conversational Models

A Proposed SLM That Can Speak and Listen Simultaneously

Key Features of LSLMs

LSLM: How It Works

LSLM: Fusion Mechanisms

LSLM Interruption Handling

Full Duplex Speech Language Models