DEV Community: Lizzie Siegle

The Ralph Loop Is Not Enough!

Lizzie Siegle — Thu, 11 Jun 2026 00:48:08 +0000

Check out the video here: Watch on LinkedIn

"I don't prompt Claude anymore. My job is to write loops."

— Boris Cherny, Claude Code creator

Though I see where he's coming from, I'd put it differently.

A developer's job isn't to write loops. It's to design state machines.

Every major agent framework — Claude Code, Codex, Cursor, LangGraph — does the same thing under the hood. A while loop calls an LLM, checks if it wants to use a tool, runs the tool, repeats until done. The loop isn't just a solved problem. It's a boring problem.

The hard part is everything around it.

A loop has no idea what state the work is in. It just keeps going until something breaks or you run out of tokens. That's the Ralph Loop — named after the Simpsons kid who put a crayon in his nose. Agent, infinite loop, go. The Ralph Loop works, is famous, and has zero memory of where it is in the job. Like Ralph, it keeps going without knowing why.

The Fix: A Finite State Machine

Think about the NBA Finals. The Spurs and the Knicks aren't improvising — every possession has a state. Fast break. Inbound play. Half court set. Each one has specific reads and triggers for what happens next. Point guard De'Aaron Fox isn't making it up as he goes. The system tells him what situation he's in, and the situation tells him what to do.

Your agent works the same way. You define the stages — planning, implementing, reviewing, error handling — and you define what triggers each transition. The agent doesn't orchestrate. It executes. One focused job per state.

Why This Matters in Production

When agents break, it's almost always one of three things:

Infinite loops — one system repeated the same answer 58 times before anyone noticed.
Context overflow — the history gets so long the model starts quietly forgetting things.
Goal drift — 70 turns in, "don't touch auth" has completely evaporated.

State machines fix all three. The loop runs until the list is empty, not until you run out of tokens. The goal lives in the transition logic, not in the context getting squeezed. And failures don't spiral — they transition to an error state with explicit rules for what's next.

Observability

State machines also give you observability. You can see the exact sequence of states the agent passed through before it broke.

That's what Entire is built for — full transcript history across your agent sessions, so you can see not just what your agents did, but what state they were in when they did it.

A system you can't inspect isn't a system. It's a loop with better vibes.

Put your Coding Agents in Drive w/ Superpowers (aka How Superpowers is the Automatic Transmission of Agentic Coding)

Lizzie Siegle — Fri, 29 May 2026 00:37:53 +0000

Watch the video on LinkedIn here!

The most downloaded and widely used methodology in the Claude Code ecosystem is Superpowers, with over 208k stars⭐️ and 18.5k forks🍴 on GitHub (and counting!)

Going from raw Claude Code to using Superpowers is as revolutionary as going from a manual transmission to an automatic. Here's why this analogy holds:

Planning Mode vs. Coding Mode

When we build with Claude (or Codex, or Gemini...), we're usually in one of two states:

planning mode
coding mode

Like shifting gears in a car🚘, moving between these modes effectively is what makes it all work. Skip the shift, and you're just revving your engine (aka burning context and going nowhere.)

Superpowers lets Claude automatically detect and shift between these modes. It picks up on ambiguity in your prompt and ensures it fully understands the task before entering coding mode. No manual shifting required!

Slower, but for Good Reason

Superpowers can feel slower and less satisfying than raw Claude Code, like how driving an automatic can feel less engaging than a manual. But both serve the same critical purpose: they stop you from blowing up your engine!

That said, there's still a time and place for raw Claude. Sometimes you want direct, fast, unmediated output. But that minor gain in speed often isn't worth the added complexity, the ambiguity risk, and the tradeoffs that come with it.

The Layer Underneath

Entire is building the foundation beneath all of this — capturing every session and decision, and linking it to the commit it produced. It's the full record of how the code was written, not just the code itself.

So the question is this: how will you put your agent in drive🚘?

Claude Code Slash Commands You Should Know (I didn't either)

Lizzie Siegle — Wed, 27 May 2026 23:34:00 +0000

Watch the video on LinkedIn here

Most people (myself included) mainly use Claude Code like a chatbot — type a request, get some code, repeat. But there's a whole layer of slash commands that make it significantly more powerful. Here's a rundown of the ones worth knowing.

Session Management

/resume
Closed your terminal last night? This picks up exactly where you left off, letting you reopen and continue a previous coding session without losing your context.

/branch
This forks your current conversation into a new, separate session. Want to try a risky refactor without blowing up your current progress? Branch it. Think of it like git branch, but for your conversation.

/clear
Context getting bloated and Claude going in circles? Wipe it and start fresh. Sometimes a clean slate is the fastest fix.

Before You Write a Single Line of Code

/init
Run this first in any new project. It sets up your CLAUDE.md file — the place where Claude learns about your stack, your conventions, and your project structure. Think of it as onboarding Claude to your codebase.

/plan
Don't just say "build this." Ask Claude to plan first. Slowing down to get a written plan before implementation leads to noticeably better output — especially on complex features.

/memory
This is what Claude remembers about your project across sessions. Read it regularly, edit it when things change, and keep it accurate. Garbage in, garbage out.

During and After Development

/diff
Launches an interactive viewer showing all the changes made in the current session. You can toggle between a full git diff, view per-turn changes, and navigate between files. It's a quick sanity check: what exactly has changed since my last commit?

/compact
Conversation going back and forth? Session getting slow? This compresses your context while keeping the important parts — faster responses without losing the thread.

/security-review
Self-explanatory. Run this before you ship anything that touches auth, payments, APIs, or user data. Better to catch it here than in production.

Did you already know some of these? Drop a comment with how you're using Claude Code — always curious what workflows people have built around it.

Building a WNBA Analytics Dashboard with Streamlit, LangChain, and Cloudflare Workers AI

Lizzie Siegle — Fri, 18 Oct 2024 19:57:30 +0000

In this tutorial, we'll walk through the process of creating an interactive WNBA (Women's National Basketball Association) analytics dashboard. This powerful tool combines data visualization, AI-driven insights, and a chatbot interface to provide a comprehensive view of WNBA player statistics and team information.

Project Overview

This WNBA analytics dashboard offers the following features:

Player statistics visualization
Team comparisons
Interactive map showing WNBA team locations
AI-powered chatbot for WNBA-related queries
Data filtering and sorting capabilities
Responsive design for various screen sizes

We'll be using the following technologies:

Python
Streamlit for the web application framework
LangChain for natural language processing
Cloudflare Workers AI for machine learning capabilities
Pandas for data manipulation
Plotly for interactive charts
Folium for map visualizations

Step 1: Setting Up the Environment

First, let's set up our development environment:

Create a new Python virtual environment

python3 -m venv venv                                                            
source venv/bin/activate

Include the required packages at the top of your Python file. From the GitHub repo, you can download the requirements.txt file and run pip install -r requirements.txt to install them.

import base64
from dotenv import load_dotenv
import json
import os
import requests
import time
import webcolors

import streamlit as st
import folium
from streamlit_folium import st_folium

from geopy.geocoders import Nominatim
from geopy.exc import GeocoderTimedOut, GeocoderServiceError

from langchain.memory import ConversationBufferMemory
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.runnables import RunnablePassthrough

import numpy as np
import pandas as pd
from pathlib import Path

import plotly.graph_objects as go

Set up a Cloudflare account and obtain the necessary credentials for Workers AI. When you login to the dashboard, your account ID is the string of characters that follow https://dash.cloudflare.com/ To get a Workers AI auth token, click on AI on the lefthand sidebar followed by the blue Use Rest API button. Then, click Create a Workers AI API Token.

Add them to a .env file and reference them by adding these lines beneath the import statements.

load_dotenv()

# Cloudflare Workers AI setup
ACCOUNT_ID = os.getenv('CF_ACCOUNT_ID')
AUTH_TOKEN = os.getenv('CF_AUTH_TOKEN')

Configuring the Streamlit Page

Let's set up the basic Streamlit page configuration and add some custom CSS for styling:

st.set_page_config(page_title="WNBA Player Analytics Dashboard, AI Insights, && AI Assistant", page_icon="🏀", layout="wide")

# Custom CSS (truncated for brevity)
st.markdown("""
<style>
    .hover-link {
        color: #1E90FF;
        text-decoration: none;
        transition: color 0.3s ease;
    }
    .hover-link:hover {
        color: #FF4500;
        text-decoration: underline;
    }
    /* ... more custom CSS -> see https://github.com/elizabethsiegle/wnba-analytics-dash-ai-insights ... */
</style>
""", unsafe_allow_html=True)

Data Collection and Preprocessing

Create a function to fetch and preprocess WNBA player data:

def fetch_player_data(season):
    url = f"https://www.basketball-reference.com/wnba/years/{season}_per_game.html"
    dataframes = pd.read_html(url, header=0)
    df = dataframes[0]
    df = df[df.G != 'G'].fillna(0)  # Remove header rows and fill NaNs
    df = df.drop(['G'], axis=1)

    # Convert percentage columns to float
    percentage_columns = ['FG%', '3P%', '2P%', 'eFG%', 'FT%']
    for col in percentage_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col].astype(str).str.rstrip('%'), errors='coerce') / 100

    # Convert other numeric columns to float
    numeric_columns = ['Age', 'GS', 'MP', 'FG', 'FGA', '3P', '3PA', '2P', '2PA', 'FT', 'FTA', 'ORB', 'DRB', 'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS']
    for col in numeric_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col], errors='coerce')

    # Ensure 'Player' and 'Team' columns are strings
    df['Player'] = df['Player'].astype(str)
    df['Team'] = df['Team'].astype(str)

    # Handle any remaining problematic columns
    for col in df.columns:
        if df[col].dtype == object:
            df[col] = df[col].astype(str)

    return df

# Usage
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
player_data = fetch_player_data(selected_season)

This function fetches data from basketball-reference.comfor a given season and performs necessary data cleaning, processing it for our use.

Building the User Interface

Now, let's set up the basic structure of our Streamlit app and sidebar controls:

st.set_page_config(page_title="WNBA Analytics Dashboard", page_icon="🏀", layout="wide")
st.title("WNBA Player Analytics Dashboard")
# Sidebar for filters
st.sidebar.header('Filter Options')
st.sidebar.success("Filter players by season, team, and position to explore the data.")
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
# Team and position selection
teams = sorted([team for team in player_data.Team.unique() if team != 'TOT'])
selected_teams = st.sidebar.multiselect('Team', teams, teams)

positions = ['C', 'F', 'G', 'F-G', 'C-F']
selected_positions = st.sidebar.multiselect('Position', positions, positions)
# Fetch and display data
player_data = fetch_player_data(selected_season)
st.write(player_data)

This creates a simple dashboard with a season selector and displays the raw data.

Step 4: Adding Data Visualization

Here, we have a helper function to clean percentage data, converting string percentages to floats and handling various data types. This dictionary maps user-friendly names to column names in the dataset.

We create a 2x2 grid layout for organizing dashboard components.
The Chart Section (col1):

allows users to select a statistic and chart type (Pie or Bar).
filters data based on a minimum value slider.
creates and displays a chart of the top 5 players for the selected statistic. The stats Section (col2):
displays quick stats like average, highest player, and number of players shown.
shows a styled table of top players for the selected statistic.

Key Features include interactive elements (dropdowns, radio buttons, slider) for user customization and dynamic chart creation based on user selection.

AI Insights Section (col3) adds an AI-powered feature to the dashboard:

It creates a button that, when clicked, generates AI insights about the chart displayed above.

When the button is clicked, it checks if the necessary data is available in the session state.

If data is available, it creates a DataFrame of the top 5 players and their stats.

It then calls the generate_insights() function (which you'd need to implement using a language model like GPT-3 or GPT-4) to analyze this data.

The generated insights are displayed to the user.

A warning is shown to remind users that the insights are AI-generated and should be verified.

This feature demonstrates how to integrate AI capabilities into a data dashboard, providing users with quick, automated analysis of the visualized data.

Implementing Player Comparison

Add a section for comparing two players:

st.markdown('<div class="player-comparison-section">', unsafe_allow_html=True)
    st.subheader("🏀 Player Comparison (players must have played in the same season)")

    # Allow users to select players to compare
    players = player_data['Player'].unique()

    # Find the indices of Caitlin Clark and Angel Reese
    caitlin_index = players.tolist().index('Caitlin Clark') if 'Caitlin Clark' in players else 0
    angel_index = players.tolist().index('Angel Reese') if 'Angel Reese' in players else 0

    player1 = st.selectbox("Select first player", players,index=caitlin_index, key='player1')
    player2 = st.selectbox("Select second player", players, index=angel_index, key='player2')

    def normalize(value, min_value, max_value):
        try:
            value = float(value)
            return 100 * (value - min_value) / (max_value - min_value) if max_value > min_value else 50
        except (ValueError, TypeError):
            return 0  # or some default value for non-numeric entries

    if player1 and player2:
        # Get data for selected players
        stats1 = player_data[player_data['Player'] == player1].iloc[0]
        stats2 = player_data[player_data['Player'] == player2].iloc[0]

        # Select stats to compare
        stats_to_compare = ['PTS', 'AST', 'TRB', 'STL', 'BLK', 'FG%', '3P%', 'FT%']
        # Convert columns to numeric, replacing non-numeric values with NaN
        for stat in stats_to_compare:
            player_data[stat] = pd.to_numeric(player_data[stat], errors='coerce')

        normalized_stats = {}
        for stat in stats_to_compare:
            min_val = player_data[stat].min()
            max_val = player_data[stat].max()
            normalized_stats[stat] = [
                normalize(stats1[stat], min_val, max_val),
                normalize(stats2[stat], min_val, max_val)
            ]

        # Create a radar chart
        fig = go.Figure()

        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][0] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player1
        ))
        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][1] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player2
        ))

        fig.update_layout(
            polar=dict(radialaxis=dict(visible=True, range=[0, 100])),
            showlegend=True,
            legend=dict(
                font=dict(size=16),  # Increase font size
                itemsizing='constant',  # Make legend items a constant size
                itemwidth=30,  # Adjust item width
                yanchor="top",  # Anchor to the top
                y=0.99,  # Position at the top
                xanchor="right",  # Anchor to the right
                x=0.99,  # Position at the left
                bgcolor="rgba(255, 255, 255, 0.5)",  # Semi-transparent background
                bordercolor="Black",  # Border color
                borderwidth=2,  # Border width
            ),
            title=dict(
                text=f"{player1} vs {player2} Comparison",
                font=dict(size=24)  # Increase title font size
            ),
            width=700,  # Adjust as needed
            height=700  # Adjust as needed
        )

This creates a radar chart comparing two selected players across multiple statistics.

Adding an Interactive Map

Let's add a map showing WNBA team locations:

@st.cache_data
def create_wnba_map():
    # WNBA teams, their locations, and home page URLs
    wnba_teams = {
        'Atlanta Dream': ('Atlanta, GA', 'https://dream.wnba.com/'),
        'Chicago Sky': ('Chicago, IL', 'https://sky.wnba.com/'),
        'Connecticut Sun': ('Uncasville, CT', 'https://sun.wnba.com/'),
        'Dallas Wings': ('Arlington, TX', 'https://wings.wnba.com/'),
        'Indiana Fever': ('Indianapolis, IN', 'https://fever.wnba.com/'),
        'Las Vegas Aces': ('Las Vegas, NV', 'https://aces.wnba.com/'),
        'Los Angeles Sparks': ('Los Angeles, CA', 'https://sparks.wnba.com/'),
        'Minnesota Lynx': ('Minneapolis, MN', 'https://lynx.wnba.com/'),
        'New York Liberty': ('Brooklyn, NY', 'https://liberty.wnba.com/'),
        'Phoenix Mercury': ('Phoenix, AZ', 'https://mercury.wnba.com/'),
        'Seattle Storm': ('Seattle, WA', 'https://storm.wnba.com/'),
        'Washington Mystics': ('Washington, D.C.', 'https://mystics.wnba.com/')
    }

    # Create a map centered on the United States
    m = folium.Map(location=[39.8283, -98.5795], zoom_start=4)

    # Geocoding to get coordinates
    geolocator = Nominatim(user_agent="wnba_app")
    # Team name to abbreviation mapping
    team_abbr = {
        'Atlanta Dream': 'ATL', 'Chicago Sky': 'CHI', 'Connecticut Sun': 'CON',
        'Dallas Wings': 'DAL', 'Indiana Fever': 'IND', 'Las Vegas Aces': 'LVA',
        'Los Angeles Sparks': 'LAS', 'Minnesota Lynx': 'MIN', 'New York Liberty': 'NYL',
        'Phoenix Mercury': 'PHO', 'Seattle Storm': 'SEA', 'Washington Mystics': 'WAS'
    }

    # Add markers for each team
    for team, (city, url) in wnba_teams.items():
        try:
            location = geocode_with_retry(geolocator, city)
            if location is None:
                # Use fallback coordinates if geocoding fails
                lat, lon = fallback_coordinates[team]
            else:
                lat, lon = location.latitude, location.longitude
            # Get team abbreviation and color
            abbr = team_abbr.get(team, 'ATL')  # Default to ATL if not found
            hex_color = team_colors.get(abbr, '#000000')  # Default to black if color not found
            rgb = webcolors.hex_to_rgb(hex_color)
            closest_folium_color = closest_color(rgb)

            # Create popup HTML with team info and link
            popup_html = f"""
            <b>{team}</b><br>
            {city}<br>
            <a href="{url}" target="_blank">Visit Team Website</a>
            """

            folium.Marker(
                [lat, lon],
                popup=folium.Popup(popup_html, max_width=300),
                tooltip=team,
                icon=folium.Icon(color=closest_folium_color, icon='basketball', prefix='fa')
            ).add_to(m)
        except Exception as e:
            st.warning(f"Couldn't add marker for {team}: {str(e)}")

    return m

This creates an interactive map with markers for each WNBA team.

Step 8: Adding a Chatbot Interface

Finally, let's add a chatbot for answering WNBA-related questions:

    st.markdown('<div class="chatbot-section">', unsafe_allow_html=True)
    st.subheader("🏀 Chat💬 w/ WNBA AI Assistant powered by LangChain && Cloudflare Workers AI🤖")

    # Add a loading message
    chat_loading = st.empty()
    chat_loading.info("Chat is initializing... This may take a few moments.")

    # Initialize the LLM and conversation chain
    @st.cache_resource
    def initialize_chat(filtered_data: pd.DataFrame):
        llm = CloudflareWorkersAI(
            account_id=ACCOUNT_ID,
            api_token=AUTH_TOKEN,
            model="@cf/meta/llama-2-7b-chat-int8"
        )
        # Convert filtered_data to a string representation
        data_context = filtered_data.to_string()

        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a knowledgeable assistant specializing in WNBA statistics, players, and teams. 
            Provide accurate and helpful information about the WNBA.
            Here's the current WNBA data you have access to:
            {data_context}
            Use this data to answer questions, but don't mention the data directly unless asked."""),
            ("human", "{input}"),
            ("ai", "{agent_scratchpad}")
        ])

        memory = ConversationBufferMemory(return_messages=True, output_key="agent_scratchpad")
        def get_chat_history(inputs):
            return memory.chat_memory.messages

        chain = (
            RunnablePassthrough.assign(
                agent_scratchpad=get_chat_history,
                data_context=lambda _: data_context[:100] + "..." # Truncate for brevity 
            )
            | prompt
            | llm
        )

        return chain, memory, data_context

    # Initialize the chat
    chain, memory, data_context = initialize_chat(filtered_data)

    # Remove the loading message
    chat_loading.empty()

    # Initialize chat history
    if "messages" not in st.session_state:
        st.session_state.messages = []

    # Display chat messages from history on rerun
    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])

    # React to user input
    if user_input := st.chat_input("Ask me anything about WNBA stats, players, or teams!"):
        # Display user message in chat message container
        st.chat_message("user").markdown(user_input)
        # Add user message to chat history
        st.session_state.messages.append({"role": "user", "content": user_input})

        # Get AI response
        with st.spinner("Thinking..."):
            try:
                response = chain.invoke({
                    "input": user_input,
                    "data_context": data_context
                })

                if not response or response.strip() == "":
                    response = "I apologize, but I couldn't generate a response. This could be due to an issue with the AI model or the input. Please try asking your question in a different way or try again later."
            except Exception as e:
                response = f"An error occurred: {str(e)}"
                st.error(f"Debug: Error details: {e}")

        # After getting the response from the model
        if isinstance(response, list) and len(response) > 0 and hasattr(response[0], 'content'):
            response_text = response[0].content
        elif isinstance(response, dict) and 'content' in response:
            response_text = response['content']
        elif isinstance(response, str):
            response_text = response
        else:
            response_text = str(response)


        # Display assistant response in chat message container
        with st.chat_message("assistant"):
            st.markdown(response_text)
        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": response_text})

        # Update memory
        memory.chat_memory.add_user_message(user_input)
        memory.chat_memory.add_ai_message(response_text)


        # Add some styling to make the chat interface look better
        st.markdown("""
        <style>
        .stChatFloatingInputContainer {
            bottom: 20px;
            background-color: #f0f2f6;
            padding: 10px;
            border-radius: 10px;
            box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
        }
        </style>
        """, unsafe_allow_html=True)

This creates a chatbot interface that can answer questions about WNBA statistics using the provided data.

Conclusion

We've now built a comprehensive WNBA analytics dashboard with data visualization, AI insights, and a chatbot interface. This project demonstrates the power of combining data analysis with machine learning to create interactive and informative tools for sports analytics.

The complete code can be found here on GitHub.

Happy coding, and enjoy exploring WNBA statistics with your new analytics dashboard!

Building a WNBA Analytics Dashboard with Streamlit, LangChain, and Cloudflare Workers AI

Lizzie Siegle — Fri, 18 Oct 2024 19:57:30 +0000

Project Overview

This WNBA analytics dashboard offers the following features:

Player statistics visualization
Team comparisons
Interactive map showing WNBA team locations
AI-powered chatbot for WNBA-related queries
Data filtering and sorting capabilities
Responsive design for various screen sizes

We'll be using the following technologies:

Python
Streamlit for the web application framework
LangChain for natural language processing
Cloudflare Workers AI for machine learning capabilities
Pandas for data manipulation
Plotly for interactive charts
Folium for map visualizations

Step 1: Setting Up the Environment

First, let's set up our development environment:

Create a new Python virtual environment

python3 -m venv venv                                                            
source venv/bin/activate

Include the required packages at the top of your Python file. From the GitHub repo, you can download the requirements.txt file and run pip install -r requirements.txt to install them.

import base64
from dotenv import load_dotenv
import json
import os
import requests
import time
import webcolors

import streamlit as st
import folium
from streamlit_folium import st_folium

from geopy.geocoders import Nominatim
from geopy.exc import GeocoderTimedOut, GeocoderServiceError

from langchain.memory import ConversationBufferMemory
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.runnables import RunnablePassthrough

import numpy as np
import pandas as pd
from pathlib import Path

import plotly.graph_objects as go

Set up a Cloudflare account and obtain the necessary credentials for Workers AI. When you login to the dashboard, your account ID is the string of characters that follow https://dash.cloudflare.com/ To get a Workers AI auth token, click on AI on the lefthand sidebar followed by the blue Use Rest API button. Then, click Create a Workers AI API Token.

Add them to a .env file and reference them by adding these lines beneath the import statements.

load_dotenv()

# Cloudflare Workers AI setup
ACCOUNT_ID = os.getenv('CF_ACCOUNT_ID')
AUTH_TOKEN = os.getenv('CF_AUTH_TOKEN')

Configuring the Streamlit Page

Let's set up the basic Streamlit page configuration and add some custom CSS for styling:

st.set_page_config(page_title="WNBA Player Analytics Dashboard, AI Insights, && AI Assistant", page_icon="🏀", layout="wide")

# Custom CSS (truncated for brevity)
st.markdown("""
<style>
    .hover-link {
        color: #1E90FF;
        text-decoration: none;
        transition: color 0.3s ease;
    }
    .hover-link:hover {
        color: #FF4500;
        text-decoration: underline;
    }
    /* ... more custom CSS -> see https://github.com/elizabethsiegle/wnba-analytics-dash-ai-insights ... */
</style>
""", unsafe_allow_html=True)

Data Collection and Preprocessing

Create a function to fetch and preprocess WNBA player data:

def fetch_player_data(season):
    url = f"https://www.basketball-reference.com/wnba/years/{season}_per_game.html"
    dataframes = pd.read_html(url, header=0)
    df = dataframes[0]
    df = df[df.G != 'G'].fillna(0)  # Remove header rows and fill NaNs
    df = df.drop(['G'], axis=1)

    # Convert percentage columns to float
    percentage_columns = ['FG%', '3P%', '2P%', 'eFG%', 'FT%']
    for col in percentage_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col].astype(str).str.rstrip('%'), errors='coerce') / 100

    # Convert other numeric columns to float
    numeric_columns = ['Age', 'GS', 'MP', 'FG', 'FGA', '3P', '3PA', '2P', '2PA', 'FT', 'FTA', 'ORB', 'DRB', 'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS']
    for col in numeric_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col], errors='coerce')

    # Ensure 'Player' and 'Team' columns are strings
    df['Player'] = df['Player'].astype(str)
    df['Team'] = df['Team'].astype(str)

    # Handle any remaining problematic columns
    for col in df.columns:
        if df[col].dtype == object:
            df[col] = df[col].astype(str)

    return df

# Usage
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
player_data = fetch_player_data(selected_season)

This function fetches data from basketball-reference.comfor a given season and performs necessary data cleaning, processing it for our use.

Building the User Interface

Now, let's set up the basic structure of our Streamlit app and sidebar controls:

st.set_page_config(page_title="WNBA Analytics Dashboard", page_icon="🏀", layout="wide")
st.title("WNBA Player Analytics Dashboard")
# Sidebar for filters
st.sidebar.header('Filter Options')
st.sidebar.success("Filter players by season, team, and position to explore the data.")
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
# Team and position selection
teams = sorted([team for team in player_data.Team.unique() if team != 'TOT'])
selected_teams = st.sidebar.multiselect('Team', teams, teams)

positions = ['C', 'F', 'G', 'F-G', 'C-F']
selected_positions = st.sidebar.multiselect('Position', positions, positions)
# Fetch and display data
player_data = fetch_player_data(selected_season)
st.write(player_data)

This creates a simple dashboard with a season selector and displays the raw data.

Step 4: Adding Data Visualization

We create a 2x2 grid layout for organizing dashboard components.
The Chart Section (col1):

allows users to select a statistic and chart type (Pie or Bar).
filters data based on a minimum value slider.
creates and displays a chart of the top 5 players for the selected statistic. The stats Section (col2):
displays quick stats like average, highest player, and number of players shown.
shows a styled table of top players for the selected statistic.

Key Features include interactive elements (dropdowns, radio buttons, slider) for user customization and dynamic chart creation based on user selection.

AI Insights Section (col3) adds an AI-powered feature to the dashboard:

It creates a button that, when clicked, generates AI insights about the chart displayed above.

When the button is clicked, it checks if the necessary data is available in the session state.

If data is available, it creates a DataFrame of the top 5 players and their stats.

It then calls the generate_insights() function (which you'd need to implement using a language model like GPT-3 or GPT-4) to analyze this data.

The generated insights are displayed to the user.

A warning is shown to remind users that the insights are AI-generated and should be verified.

This feature demonstrates how to integrate AI capabilities into a data dashboard, providing users with quick, automated analysis of the visualized data.

Implementing Player Comparison

Add a section for comparing two players:

st.markdown('<div class="player-comparison-section">', unsafe_allow_html=True)
    st.subheader("🏀 Player Comparison (players must have played in the same season)")

    # Allow users to select players to compare
    players = player_data['Player'].unique()

    # Find the indices of Caitlin Clark and Angel Reese
    caitlin_index = players.tolist().index('Caitlin Clark') if 'Caitlin Clark' in players else 0
    angel_index = players.tolist().index('Angel Reese') if 'Angel Reese' in players else 0

    player1 = st.selectbox("Select first player", players,index=caitlin_index, key='player1')
    player2 = st.selectbox("Select second player", players, index=angel_index, key='player2')

    def normalize(value, min_value, max_value):
        try:
            value = float(value)
            return 100 * (value - min_value) / (max_value - min_value) if max_value > min_value else 50
        except (ValueError, TypeError):
            return 0  # or some default value for non-numeric entries

    if player1 and player2:
        # Get data for selected players
        stats1 = player_data[player_data['Player'] == player1].iloc[0]
        stats2 = player_data[player_data['Player'] == player2].iloc[0]

        # Select stats to compare
        stats_to_compare = ['PTS', 'AST', 'TRB', 'STL', 'BLK', 'FG%', '3P%', 'FT%']
        # Convert columns to numeric, replacing non-numeric values with NaN
        for stat in stats_to_compare:
            player_data[stat] = pd.to_numeric(player_data[stat], errors='coerce')

        normalized_stats = {}
        for stat in stats_to_compare:
            min_val = player_data[stat].min()
            max_val = player_data[stat].max()
            normalized_stats[stat] = [
                normalize(stats1[stat], min_val, max_val),
                normalize(stats2[stat], min_val, max_val)
            ]

        # Create a radar chart
        fig = go.Figure()

        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][0] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player1
        ))
        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][1] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player2
        ))

        fig.update_layout(
            polar=dict(radialaxis=dict(visible=True, range=[0, 100])),
            showlegend=True,
            legend=dict(
                font=dict(size=16),  # Increase font size
                itemsizing='constant',  # Make legend items a constant size
                itemwidth=30,  # Adjust item width
                yanchor="top",  # Anchor to the top
                y=0.99,  # Position at the top
                xanchor="right",  # Anchor to the right
                x=0.99,  # Position at the left
                bgcolor="rgba(255, 255, 255, 0.5)",  # Semi-transparent background
                bordercolor="Black",  # Border color
                borderwidth=2,  # Border width
            ),
            title=dict(
                text=f"{player1} vs {player2} Comparison",
                font=dict(size=24)  # Increase title font size
            ),
            width=700,  # Adjust as needed
            height=700  # Adjust as needed
        )

This creates a radar chart comparing two selected players across multiple statistics.

Adding an Interactive Map

Let's add a map showing WNBA team locations:

@st.cache_data
def create_wnba_map():
    # WNBA teams, their locations, and home page URLs
    wnba_teams = {
        'Atlanta Dream': ('Atlanta, GA', 'https://dream.wnba.com/'),
        'Chicago Sky': ('Chicago, IL', 'https://sky.wnba.com/'),
        'Connecticut Sun': ('Uncasville, CT', 'https://sun.wnba.com/'),
        'Dallas Wings': ('Arlington, TX', 'https://wings.wnba.com/'),
        'Indiana Fever': ('Indianapolis, IN', 'https://fever.wnba.com/'),
        'Las Vegas Aces': ('Las Vegas, NV', 'https://aces.wnba.com/'),
        'Los Angeles Sparks': ('Los Angeles, CA', 'https://sparks.wnba.com/'),
        'Minnesota Lynx': ('Minneapolis, MN', 'https://lynx.wnba.com/'),
        'New York Liberty': ('Brooklyn, NY', 'https://liberty.wnba.com/'),
        'Phoenix Mercury': ('Phoenix, AZ', 'https://mercury.wnba.com/'),
        'Seattle Storm': ('Seattle, WA', 'https://storm.wnba.com/'),
        'Washington Mystics': ('Washington, D.C.', 'https://mystics.wnba.com/')
    }

    # Create a map centered on the United States
    m = folium.Map(location=[39.8283, -98.5795], zoom_start=4)

    # Geocoding to get coordinates
    geolocator = Nominatim(user_agent="wnba_app")
    # Team name to abbreviation mapping
    team_abbr = {
        'Atlanta Dream': 'ATL', 'Chicago Sky': 'CHI', 'Connecticut Sun': 'CON',
        'Dallas Wings': 'DAL', 'Indiana Fever': 'IND', 'Las Vegas Aces': 'LVA',
        'Los Angeles Sparks': 'LAS', 'Minnesota Lynx': 'MIN', 'New York Liberty': 'NYL',
        'Phoenix Mercury': 'PHO', 'Seattle Storm': 'SEA', 'Washington Mystics': 'WAS'
    }

    # Add markers for each team
    for team, (city, url) in wnba_teams.items():
        try:
            location = geocode_with_retry(geolocator, city)
            if location is None:
                # Use fallback coordinates if geocoding fails
                lat, lon = fallback_coordinates[team]
            else:
                lat, lon = location.latitude, location.longitude
            # Get team abbreviation and color
            abbr = team_abbr.get(team, 'ATL')  # Default to ATL if not found
            hex_color = team_colors.get(abbr, '#000000')  # Default to black if color not found
            rgb = webcolors.hex_to_rgb(hex_color)
            closest_folium_color = closest_color(rgb)

            # Create popup HTML with team info and link
            popup_html = f"""
            <b>{team}</b><br>
            {city}<br>
            <a href="{url}" target="_blank">Visit Team Website</a>
            """

            folium.Marker(
                [lat, lon],
                popup=folium.Popup(popup_html, max_width=300),
                tooltip=team,
                icon=folium.Icon(color=closest_folium_color, icon='basketball', prefix='fa')
            ).add_to(m)
        except Exception as e:
            st.warning(f"Couldn't add marker for {team}: {str(e)}")

    return m

This creates an interactive map with markers for each WNBA team.

Step 8: Adding a Chatbot Interface

Finally, let's add a chatbot for answering WNBA-related questions:

    st.markdown('<div class="chatbot-section">', unsafe_allow_html=True)
    st.subheader("🏀 Chat💬 w/ WNBA AI Assistant powered by LangChain && Cloudflare Workers AI🤖")

    # Add a loading message
    chat_loading = st.empty()
    chat_loading.info("Chat is initializing... This may take a few moments.")

    # Initialize the LLM and conversation chain
    @st.cache_resource
    def initialize_chat(filtered_data: pd.DataFrame):
        llm = CloudflareWorkersAI(
            account_id=ACCOUNT_ID,
            api_token=AUTH_TOKEN,
            model="@cf/meta/llama-2-7b-chat-int8"
        )
        # Convert filtered_data to a string representation
        data_context = filtered_data.to_string()

        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a knowledgeable assistant specializing in WNBA statistics, players, and teams. 
            Provide accurate and helpful information about the WNBA.
            Here's the current WNBA data you have access to:
            {data_context}
            Use this data to answer questions, but don't mention the data directly unless asked."""),
            ("human", "{input}"),
            ("ai", "{agent_scratchpad}")
        ])

        memory = ConversationBufferMemory(return_messages=True, output_key="agent_scratchpad")
        def get_chat_history(inputs):
            return memory.chat_memory.messages

        chain = (
            RunnablePassthrough.assign(
                agent_scratchpad=get_chat_history,
                data_context=lambda _: data_context[:100] + "..." # Truncate for brevity 
            )
            | prompt
            | llm
        )

        return chain, memory, data_context

    # Initialize the chat
    chain, memory, data_context = initialize_chat(filtered_data)

    # Remove the loading message
    chat_loading.empty()

    # Initialize chat history
    if "messages" not in st.session_state:
        st.session_state.messages = []

    # Display chat messages from history on rerun
    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])

    # React to user input
    if user_input := st.chat_input("Ask me anything about WNBA stats, players, or teams!"):
        # Display user message in chat message container
        st.chat_message("user").markdown(user_input)
        # Add user message to chat history
        st.session_state.messages.append({"role": "user", "content": user_input})

        # Get AI response
        with st.spinner("Thinking..."):
            try:
                response = chain.invoke({
                    "input": user_input,
                    "data_context": data_context
                })

                if not response or response.strip() == "":
                    response = "I apologize, but I couldn't generate a response. This could be due to an issue with the AI model or the input. Please try asking your question in a different way or try again later."
            except Exception as e:
                response = f"An error occurred: {str(e)}"
                st.error(f"Debug: Error details: {e}")

        # After getting the response from the model
        if isinstance(response, list) and len(response) > 0 and hasattr(response[0], 'content'):
            response_text = response[0].content
        elif isinstance(response, dict) and 'content' in response:
            response_text = response['content']
        elif isinstance(response, str):
            response_text = response
        else:
            response_text = str(response)


        # Display assistant response in chat message container
        with st.chat_message("assistant"):
            st.markdown(response_text)
        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": response_text})

        # Update memory
        memory.chat_memory.add_user_message(user_input)
        memory.chat_memory.add_ai_message(response_text)


        # Add some styling to make the chat interface look better
        st.markdown("""
        <style>
        .stChatFloatingInputContainer {
            bottom: 20px;
            background-color: #f0f2f6;
            padding: 10px;
            border-radius: 10px;
            box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
        }
        </style>
        """, unsafe_allow_html=True)

This creates a chatbot interface that can answer questions about WNBA statistics using the provided data.

Conclusion

The complete code can be found here on GitHub.

Happy coding, and enjoy exploring WNBA statistics with your new analytics dashboard!

Building a WNBA Analytics Dashboard with Streamlit, LangChain, and Cloudflare Workers AI

Lizzie Siegle — Fri, 18 Oct 2024 19:57:30 +0000

Project Overview

This WNBA analytics dashboard offers the following features:

Player statistics visualization
Team comparisons
Interactive map showing WNBA team locations
AI-powered chatbot for WNBA-related queries
Data filtering and sorting capabilities
Responsive design for various screen sizes

We'll be using the following technologies:

Python
Streamlit for the web application framework
LangChain for natural language processing
Cloudflare Workers AI for machine learning capabilities
Pandas for data manipulation
Plotly for interactive charts
Folium for map visualizations

Step 1: Setting Up the Environment

First, let's set up our development environment:

Create a new Python virtual environment

python3 -m venv venv                                                            
source venv/bin/activate

Include the required packages at the top of your Python file. From the GitHub repo, you can download the requirements.txt file and run pip install -r requirements.txt to install them.

import base64
from dotenv import load_dotenv
import json
import os
import requests
import time
import webcolors

import streamlit as st
import folium
from streamlit_folium import st_folium

from geopy.geocoders import Nominatim
from geopy.exc import GeocoderTimedOut, GeocoderServiceError

from langchain.memory import ConversationBufferMemory
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.runnables import RunnablePassthrough

import numpy as np
import pandas as pd
from pathlib import Path

import plotly.graph_objects as go

Set up a Cloudflare account and obtain the necessary credentials for Workers AI. When you login to the dashboard, your account ID is the string of characters that follow https://dash.cloudflare.com/ To get a Workers AI auth token, click on AI on the lefthand sidebar followed by the blue Use Rest API button. Then, click Create a Workers AI API Token.

Add them to a .env file and reference them by adding these lines beneath the import statements.

load_dotenv()

# Cloudflare Workers AI setup
ACCOUNT_ID = os.getenv('CF_ACCOUNT_ID')
AUTH_TOKEN = os.getenv('CF_AUTH_TOKEN')

Configuring the Streamlit Page

Let's set up the basic Streamlit page configuration and add some custom CSS for styling:

st.set_page_config(page_title="WNBA Player Analytics Dashboard, AI Insights, && AI Assistant", page_icon="🏀", layout="wide")

# Custom CSS (truncated for brevity)
st.markdown("""
<style>
    .hover-link {
        color: #1E90FF;
        text-decoration: none;
        transition: color 0.3s ease;
    }
    .hover-link:hover {
        color: #FF4500;
        text-decoration: underline;
    }
    /* ... more custom CSS -> see https://github.com/elizabethsiegle/wnba-analytics-dash-ai-insights ... */
</style>
""", unsafe_allow_html=True)

Data Collection and Preprocessing

Create a function to fetch and preprocess WNBA player data:

def fetch_player_data(season):
    url = f"https://www.basketball-reference.com/wnba/years/{season}_per_game.html"
    dataframes = pd.read_html(url, header=0)
    df = dataframes[0]
    df = df[df.G != 'G'].fillna(0)  # Remove header rows and fill NaNs
    df = df.drop(['G'], axis=1)

    # Convert percentage columns to float
    percentage_columns = ['FG%', '3P%', '2P%', 'eFG%', 'FT%']
    for col in percentage_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col].astype(str).str.rstrip('%'), errors='coerce') / 100

    # Convert other numeric columns to float
    numeric_columns = ['Age', 'GS', 'MP', 'FG', 'FGA', '3P', '3PA', '2P', '2PA', 'FT', 'FTA', 'ORB', 'DRB', 'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS']
    for col in numeric_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col], errors='coerce')

    # Ensure 'Player' and 'Team' columns are strings
    df['Player'] = df['Player'].astype(str)
    df['Team'] = df['Team'].astype(str)

    # Handle any remaining problematic columns
    for col in df.columns:
        if df[col].dtype == object:
            df[col] = df[col].astype(str)

    return df

# Usage
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
player_data = fetch_player_data(selected_season)

This function fetches data from basketball-reference.comfor a given season and performs necessary data cleaning, processing it for our use.

Building the User Interface

Now, let's set up the basic structure of our Streamlit app and sidebar controls:

st.set_page_config(page_title="WNBA Analytics Dashboard", page_icon="🏀", layout="wide")
st.title("WNBA Player Analytics Dashboard")
# Sidebar for filters
st.sidebar.header('Filter Options')
st.sidebar.success("Filter players by season, team, and position to explore the data.")
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
# Team and position selection
teams = sorted([team for team in player_data.Team.unique() if team != 'TOT'])
selected_teams = st.sidebar.multiselect('Team', teams, teams)

positions = ['C', 'F', 'G', 'F-G', 'C-F']
selected_positions = st.sidebar.multiselect('Position', positions, positions)
# Fetch and display data
player_data = fetch_player_data(selected_season)
st.write(player_data)

This creates a simple dashboard with a season selector and displays the raw data.

Step 4: Adding Data Visualization

We create a 2x2 grid layout for organizing dashboard components.
The Chart Section (col1):

allows users to select a statistic and chart type (Pie or Bar).
filters data based on a minimum value slider.
creates and displays a chart of the top 5 players for the selected statistic. The stats Section (col2):
displays quick stats like average, highest player, and number of players shown.
shows a styled table of top players for the selected statistic.

Key Features include interactive elements (dropdowns, radio buttons, slider) for user customization and dynamic chart creation based on user selection.

AI Insights Section (col3) adds an AI-powered feature to the dashboard:

It creates a button that, when clicked, generates AI insights about the chart displayed above.

When the button is clicked, it checks if the necessary data is available in the session state.

If data is available, it creates a DataFrame of the top 5 players and their stats.

It then calls the generate_insights() function (which you'd need to implement using a language model like GPT-3 or GPT-4) to analyze this data.

The generated insights are displayed to the user.

A warning is shown to remind users that the insights are AI-generated and should be verified.

This feature demonstrates how to integrate AI capabilities into a data dashboard, providing users with quick, automated analysis of the visualized data.

Implementing Player Comparison

Add a section for comparing two players:

st.markdown('<div class="player-comparison-section">', unsafe_allow_html=True)
    st.subheader("🏀 Player Comparison (players must have played in the same season)")

    # Allow users to select players to compare
    players = player_data['Player'].unique()

    # Find the indices of Caitlin Clark and Angel Reese
    caitlin_index = players.tolist().index('Caitlin Clark') if 'Caitlin Clark' in players else 0
    angel_index = players.tolist().index('Angel Reese') if 'Angel Reese' in players else 0

    player1 = st.selectbox("Select first player", players,index=caitlin_index, key='player1')
    player2 = st.selectbox("Select second player", players, index=angel_index, key='player2')

    def normalize(value, min_value, max_value):
        try:
            value = float(value)
            return 100 * (value - min_value) / (max_value - min_value) if max_value > min_value else 50
        except (ValueError, TypeError):
            return 0  # or some default value for non-numeric entries

    if player1 and player2:
        # Get data for selected players
        stats1 = player_data[player_data['Player'] == player1].iloc[0]
        stats2 = player_data[player_data['Player'] == player2].iloc[0]

        # Select stats to compare
        stats_to_compare = ['PTS', 'AST', 'TRB', 'STL', 'BLK', 'FG%', '3P%', 'FT%']
        # Convert columns to numeric, replacing non-numeric values with NaN
        for stat in stats_to_compare:
            player_data[stat] = pd.to_numeric(player_data[stat], errors='coerce')

        normalized_stats = {}
        for stat in stats_to_compare:
            min_val = player_data[stat].min()
            max_val = player_data[stat].max()
            normalized_stats[stat] = [
                normalize(stats1[stat], min_val, max_val),
                normalize(stats2[stat], min_val, max_val)
            ]

        # Create a radar chart
        fig = go.Figure()

        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][0] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player1
        ))
        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][1] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player2
        ))

        fig.update_layout(
            polar=dict(radialaxis=dict(visible=True, range=[0, 100])),
            showlegend=True,
            legend=dict(
                font=dict(size=16),  # Increase font size
                itemsizing='constant',  # Make legend items a constant size
                itemwidth=30,  # Adjust item width
                yanchor="top",  # Anchor to the top
                y=0.99,  # Position at the top
                xanchor="right",  # Anchor to the right
                x=0.99,  # Position at the left
                bgcolor="rgba(255, 255, 255, 0.5)",  # Semi-transparent background
                bordercolor="Black",  # Border color
                borderwidth=2,  # Border width
            ),
            title=dict(
                text=f"{player1} vs {player2} Comparison",
                font=dict(size=24)  # Increase title font size
            ),
            width=700,  # Adjust as needed
            height=700  # Adjust as needed
        )

This creates a radar chart comparing two selected players across multiple statistics.

Adding an Interactive Map

Let's add a map showing WNBA team locations:

@st.cache_data
def create_wnba_map():
    # WNBA teams, their locations, and home page URLs
    wnba_teams = {
        'Atlanta Dream': ('Atlanta, GA', 'https://dream.wnba.com/'),
        'Chicago Sky': ('Chicago, IL', 'https://sky.wnba.com/'),
        'Connecticut Sun': ('Uncasville, CT', 'https://sun.wnba.com/'),
        'Dallas Wings': ('Arlington, TX', 'https://wings.wnba.com/'),
        'Indiana Fever': ('Indianapolis, IN', 'https://fever.wnba.com/'),
        'Las Vegas Aces': ('Las Vegas, NV', 'https://aces.wnba.com/'),
        'Los Angeles Sparks': ('Los Angeles, CA', 'https://sparks.wnba.com/'),
        'Minnesota Lynx': ('Minneapolis, MN', 'https://lynx.wnba.com/'),
        'New York Liberty': ('Brooklyn, NY', 'https://liberty.wnba.com/'),
        'Phoenix Mercury': ('Phoenix, AZ', 'https://mercury.wnba.com/'),
        'Seattle Storm': ('Seattle, WA', 'https://storm.wnba.com/'),
        'Washington Mystics': ('Washington, D.C.', 'https://mystics.wnba.com/')
    }

    # Create a map centered on the United States
    m = folium.Map(location=[39.8283, -98.5795], zoom_start=4)

    # Geocoding to get coordinates
    geolocator = Nominatim(user_agent="wnba_app")
    # Team name to abbreviation mapping
    team_abbr = {
        'Atlanta Dream': 'ATL', 'Chicago Sky': 'CHI', 'Connecticut Sun': 'CON',
        'Dallas Wings': 'DAL', 'Indiana Fever': 'IND', 'Las Vegas Aces': 'LVA',
        'Los Angeles Sparks': 'LAS', 'Minnesota Lynx': 'MIN', 'New York Liberty': 'NYL',
        'Phoenix Mercury': 'PHO', 'Seattle Storm': 'SEA', 'Washington Mystics': 'WAS'
    }

    # Add markers for each team
    for team, (city, url) in wnba_teams.items():
        try:
            location = geocode_with_retry(geolocator, city)
            if location is None:
                # Use fallback coordinates if geocoding fails
                lat, lon = fallback_coordinates[team]
            else:
                lat, lon = location.latitude, location.longitude
            # Get team abbreviation and color
            abbr = team_abbr.get(team, 'ATL')  # Default to ATL if not found
            hex_color = team_colors.get(abbr, '#000000')  # Default to black if color not found
            rgb = webcolors.hex_to_rgb(hex_color)
            closest_folium_color = closest_color(rgb)

            # Create popup HTML with team info and link
            popup_html = f"""
            <b>{team}</b><br>
            {city}<br>
            <a href="{url}" target="_blank">Visit Team Website</a>
            """

            folium.Marker(
                [lat, lon],
                popup=folium.Popup(popup_html, max_width=300),
                tooltip=team,
                icon=folium.Icon(color=closest_folium_color, icon='basketball', prefix='fa')
            ).add_to(m)
        except Exception as e:
            st.warning(f"Couldn't add marker for {team}: {str(e)}")

    return m

This creates an interactive map with markers for each WNBA team.

Step 8: Adding a Chatbot Interface

Finally, let's add a chatbot for answering WNBA-related questions:

    st.markdown('<div class="chatbot-section">', unsafe_allow_html=True)
    st.subheader("🏀 Chat💬 w/ WNBA AI Assistant powered by LangChain && Cloudflare Workers AI🤖")

    # Add a loading message
    chat_loading = st.empty()
    chat_loading.info("Chat is initializing... This may take a few moments.")

    # Initialize the LLM and conversation chain
    @st.cache_resource
    def initialize_chat(filtered_data: pd.DataFrame):
        llm = CloudflareWorkersAI(
            account_id=ACCOUNT_ID,
            api_token=AUTH_TOKEN,
            model="@cf/meta/llama-2-7b-chat-int8"
        )
        # Convert filtered_data to a string representation
        data_context = filtered_data.to_string()

        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a knowledgeable assistant specializing in WNBA statistics, players, and teams. 
            Provide accurate and helpful information about the WNBA.
            Here's the current WNBA data you have access to:
            {data_context}
            Use this data to answer questions, but don't mention the data directly unless asked."""),
            ("human", "{input}"),
            ("ai", "{agent_scratchpad}")
        ])

        memory = ConversationBufferMemory(return_messages=True, output_key="agent_scratchpad")
        def get_chat_history(inputs):
            return memory.chat_memory.messages

        chain = (
            RunnablePassthrough.assign(
                agent_scratchpad=get_chat_history,
                data_context=lambda _: data_context[:100] + "..." # Truncate for brevity 
            )
            | prompt
            | llm
        )

        return chain, memory, data_context

    # Initialize the chat
    chain, memory, data_context = initialize_chat(filtered_data)

    # Remove the loading message
    chat_loading.empty()

    # Initialize chat history
    if "messages" not in st.session_state:
        st.session_state.messages = []

    # Display chat messages from history on rerun
    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])

    # React to user input
    if user_input := st.chat_input("Ask me anything about WNBA stats, players, or teams!"):
        # Display user message in chat message container
        st.chat_message("user").markdown(user_input)
        # Add user message to chat history
        st.session_state.messages.append({"role": "user", "content": user_input})

        # Get AI response
        with st.spinner("Thinking..."):
            try:
                response = chain.invoke({
                    "input": user_input,
                    "data_context": data_context
                })

                if not response or response.strip() == "":
                    response = "I apologize, but I couldn't generate a response. This could be due to an issue with the AI model or the input. Please try asking your question in a different way or try again later."
            except Exception as e:
                response = f"An error occurred: {str(e)}"
                st.error(f"Debug: Error details: {e}")

        # After getting the response from the model
        if isinstance(response, list) and len(response) > 0 and hasattr(response[0], 'content'):
            response_text = response[0].content
        elif isinstance(response, dict) and 'content' in response:
            response_text = response['content']
        elif isinstance(response, str):
            response_text = response
        else:
            response_text = str(response)


        # Display assistant response in chat message container
        with st.chat_message("assistant"):
            st.markdown(response_text)
        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": response_text})

        # Update memory
        memory.chat_memory.add_user_message(user_input)
        memory.chat_memory.add_ai_message(response_text)


        # Add some styling to make the chat interface look better
        st.markdown("""
        <style>
        .stChatFloatingInputContainer {
            bottom: 20px;
            background-color: #f0f2f6;
            padding: 10px;
            border-radius: 10px;
            box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
        }
        </style>
        """, unsafe_allow_html=True)

This creates a chatbot interface that can answer questions about WNBA statistics using the provided data.

Conclusion

The complete code can be found here on GitHub.

Happy coding, and enjoy exploring WNBA statistics with your new analytics dashboard!

Build a Route Generator App with Cloudflare Workers AI, LangChain, Streamlit, and Mapbox

Lizzie Siegle — Tue, 10 Sep 2024 22:20:06 +0000

In this tutorial, you will learn how to create a web app that generates optimized tourist routes using Cloudflare Workers AI, Mapbox, LangChain, and Streamlit. The app allows users to select landmarks in a chosen city and get the shortest route between them, solving the Traveling Salesman Problem.

Set Up Your Dev Environment and Environment Variables

On the command line, make a new folder called tsp.py. Create a virtual environment by running

python3 -m venv venv
source venv/bin/activate

Next, install the required libraries with

pip install python-dotenv geopy langchain langchain-core langchain-community markdown pandas requests streamlit streamlit-searchbox folium streamlit-folium

You need a Mapbox API token--you can get one here and a Cloudflare Workers AI token, which you get by clicking AI on the lefthand side of your dashboard, followed by clicking the blue Use REST API button then the blue Create a Workers AI API Token button.

You can find your Cloudflare Account SID on that same page or in the dashboard URL--it's the string of characters following dash.cloudflare.com/.

Make a .env file and replace those values below:

MAPBOX_TOKEN=
CLOUDFLARE_API_TOKEN=
CLOUDFLARE_ACCOUNT_ID=

The app uses environment variables for secure access to APIs like Mapbox and Cloudflare Workers AI. They are accessed in Python code using dotenv and os (and would be replaced by lines like mapbox_token = st.secrets["MAPBOX_TOKEN"] if you were to deploy to Streamlit.

At the top, include the following import statements and variables to reference the environment variables:

from dotenv import load_dotenv
import folium
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
import os
import requests
import streamlit as st
from streamlit_folium import folium_static

load_dotenv()

mapbox_token = os.getenv('MAPBOX_TOKEN')
cf_account_id = os.getenv('CLOUDFLARE_ACCOUNT_ID')
cf_api_token = os.getenv('CLOUDFLARE_API_TOKEN')

Create User Interface with Streamlit

The UI is created using Streamlit. Add the title and description with the code below:

st.title('Route Me🚴‍♀️🚶‍♀️‍➡️🏃‍♀️')

st.markdown("""
This app uses Cloudflare Workers AI, LangChain, and Mapbox to solve the Traveling Salesman problem. 
1. Enter a city🏙️
2. Pick landmarks🌁🗽
3. Generate the shortest path.
4. Explore! 🗺️
""")

We also add a footer for aesthetic and branding purposes:

footer_html = """
<div class="footer">
    Made with ❤️ in SF🌉 with Cloudflare Workers AI ➡️ 👩🏻‍💻 
    <a href="https://github.com/your-repo">code here on GitHub</a>
</div>
"""
st.markdown(footer_html, unsafe_allow_html=True)

Fetch Cities and Landmarks

We use Mapbox's searchbox API to fetch information about the input city and corresponding landmark information. The app provides a list of cities and landmarks according to the user input.

The following code searches for cities:

def find_city(city_inp: str):
    url = "https://api.mapbox.com/search/searchbox/v1/suggest"
    params = {"q": city_inp, "access_token": mapbox_token, "types": "place"}
    res = requests.get(url, params=params)

    # Extract suggestions
    suggestions = res.json().get('suggestions', [])
    return [(f"{s['name']}, {s['place_formatted']}", s['mapbox_id']) for s in suggestions]

This function retrieves details about the input city:

def retrieve_city(city_id: str):
    url = f"https://api.mapbox.com/search/searchbox/v1/retrieve/{city_id}"
    params = {"access_token": mapbox_token}
    res = requests.get(url, params=params)
    return res.json().get('features', [])[0]  # Get first result

This function retrieves landmarks:

def retrieve_landmark(name: str, proximity: str):
    url = "https://api.mapbox.com/search/searchbox/v1/forward"
    params = {
        "q": name,
        "access_token": mapbox_token,
        "proximity": proximity,
        'types': 'poi'
    }
    res = requests.get(url, params=params)
    return res.json()['features'][0]

Use Cloudflare Workers AI and LangChain for Landmarks

We use Cloudflare Workers AI to generate a list of landmarks. This is done through LangChain’s prompt chain.

def lmchain():
    llm = CloudflareWorkersAI(account_id=cf_account_id, api_token=cf_api_token, model='@cf/meta/llama-3.1-8b-instruct')
    prompt = PromptTemplate(
        template="Return a list of 7 landmarks in {city}.",
        input_variables=["city"]
    )
    return LLMChain(llm=llm, prompt=prompt)

Traveling Salesman Problem and Optimized Routes

Ultimately, this app solves the Traveling Salesman problem by utilizing Mapbox's optimized-trips API. It calculates the shortest route between landmarks selected by the user.

def travelingsalesman(landmarks_df):
    coordinates = ";".join([f"{row['longitude']},{row['latitude']}" for _, row in landmarks_df.iterrows()])
    url = f"https://api.mapbox.com/optimized-trips/v1/mapbox/cycling/{coordinates}"
    params = {"access_token": mapbox_token, "geometries": "geojson"}
    res = requests.get(url, params=params)
    return res.json()

Display the Map

We use Folium to generate the map and display it in the Streamlit app with the streamlit-folium library.

def create_route_map(landmarks, optimized_coords):
    m = folium.Map(location=[landmarks['latitude'].mean(), landmarks['longitude'].mean()], zoom_start=12)
    folium.PolyLine(optimized_coords, color='red').add_to(m)
    return folium_static(m)

Generate Textual Route Description

The LLM generates a textual description of the optimized bike/walk/run route:

def make_route(city, landmarks):
    chain = lmchain()
    return chain.run({"city": city, "landmarks": "\n".join(landmarks['Name'])})

Download Map and Route Options

The app allows users to download the route map and description:

st.download_button(label='Download Route Map', data=map_to_html(st.session_state.route_map), file_name='route_map.html')
st.download_button(label='Download Route Description', data=route_html, file_name='route_description.html')

The complete Python file should look like this:

from dotenv import load_dotenv
from geopy import distance
import io
from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
from langchain.output_parsers import CommaSeparatedListOutputParser
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
import markdown
import os
import pandas as pd
import requests
import streamlit as st
from streamlit_searchbox import st_searchbox
from typing import List
import uuid
import folium
from folium.plugins import Draw
from streamlit_folium import folium_static

# Display the title and description
st.title('Route Me🚴‍♀️🚶‍♀️‍➡️🏃‍♀️')

st.markdown("""
    <style>
        .footer {
            position: fixed;
            bottom: 0;
            left: 0;
            width: 100%;
            background-color: #f1f1f1;
            text-align: center;
            padding: 10px 0;
            color: #000;
            font-size: 14px;
            box-shadow: 0 -2px 5px rgba(0,0,0,0.1);
        }
        .content {
            padding-bottom: 150px; /* Adjust this value to create space between content and footer */
        }
    </style>
""", unsafe_allow_html=True)
# Footer HTML
footer_html = """
    <div class="footer">
        Made with ❤️ in SF🌉 with Cloudflare Workers AI ➡️ 👩🏻‍💻 <a href="https://github.com/elizabethsiegle/bike_walk_route_map_generator">code here on GitHub</a>
    </div>
"""

# Inject the footer into the app
st.markdown(footer_html, unsafe_allow_html=True)

# Define markdown content directly
markdown_content = """
This app uses [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/), [LangChain](https://langchain.dev/),  landmark/city data from Mapbox, [Folium](https://python-visualization.github.io/folium/latest/) for visualizing maps and routes, and [Streamlit](https://streamlit.io/)/[Streamlit Folium](https://folium.streamlit.app/) to tackle the Traveling Salesman problem!

1. Enter a city🏙️ you wish to visit
-> get a few must-visit landmarks in your chosen city
2. Pick the landmarks🌁🗽 you want to visit.
3. Generate the shortest path between these landmarks.
4. Explore! 🗺️
"""
st.markdown(markdown_content)

# Load environment variables
load_dotenv()
mapbox_token = st.secrets["MAPBOX_TOKEN"] # os.environ.get('MAPBOX_TOKEN')
cf_account_id = st.secrets["CLOUDFLARE_ACCOUNT_ID"]# os.environ.get('CLOUDFLARE_ACCOUNT_ID')
cf_api_token = st.secrets["CLOUDFLARE_API_TOKEN"] # os.environ.get('CLOUDFLARE_API_TOKEN')

token = str(uuid.uuid4())

# find cities
def find_city(city_inp: str) -> List[tuple]:
    if len(city_inp) < 3:
        return []

    # searchbox api = return list of suggestions for city
    url = "https://api.mapbox.com/search/searchbox/v1/suggest"
    params = {"q": city_inp, "access_token": mapbox_token, "session_token": token, "types": "place"}

    res = requests.get(url, params=params)
    if res.status_code != 200:
        return []

    try:
        suggestions = res.json().get('suggestions', [])
        results = []
        for s in suggestions:
            print(f"s[name] {s['name']} s[place_formatted] {s['place_formatted']}")
            results.append((f"{s['name']}, {s['place_formatted']}", s['mapbox_id']))

        return results
    except Exception as e:
        st.error(f"Error fetching city suggestions: {e}")
        return []

# Function to retrieve city details
@st.cache_data
def retrieve_city(id):
    url = f"https://api.mapbox.com/search/searchbox/v1/retrieve/{id}"
    params = {"access_token": mapbox_token,"session_token": token}
    res = requests.get(url, params=params)
    if res.status_code != 200:
        return []
    try:
        features = res.json().get('features', [])
        if not features:
            st.warning("No features returned for the city.")
            return []
        return features[0]
    except Exception as e:
        st.error(f"An error occurred: {e}")
        return []

# Function to retrieve landmark details
@st.cache_data
def retrieve_landmark(name, proximity):
    mapbox_url = "https://api.mapbox.com/search/searchbox/v1/forward"
    params = {"access_token": mapbox_token, "q": name, "proximity": proximity, 'types': 'poi', 'poi_category': 'tourist_attraction,museum,monument,historic,park,church,place_of_worship'}

    res = requests.get(mapbox_url, params=params)
    print(f'res.json {res.json()}')
    if res.status_code != 200:
        return []

    try:
        return res.json()['features'][0]
    except Exception as e:
        print(f"Error retrieving landmark: {e}")
        return []

# find city w/ searchbox
city_id = st_searchbox(find_city, key="city")

# Function to make landmark chain
@st.cache_resource
def lmchain():
    outp_parser = CommaSeparatedListOutputParser()
    form_instructions = outp_parser.get_format_instructions()

    llm = CloudflareWorkersAI(account_id=cf_account_id, api_token=cf_api_token, model='@cf/meta/llama-3.1-8b-instruct',)
    prompt = PromptTemplate(
        template="""Return a comma-separated list of the 7 best landmarks in {city}. Only return the list. {form_instructions}""",
        input_variables=["city"],
        partial_variables={"form_instructions": form_instructions},
    )

    chain = LLMChain(llm=llm, prompt=prompt, output_parser=outp_parser)
    return chain

# Function to get landmark locations
@st.cache_data
def get_landmarks(landmarks, long_city, lat_city):
    data = []
    for lm in landmarks:
        features = retrieve_landmark(lm, f"{long_city},{lat_city}")
        if not features:
            continue

        coor = features['geometry']['coordinates']
        long, lat = coor
        dist = distance.distance((lat_city, long_city), (lat, long)).km

        if dist <= 7:
            data.append([lm, long, lat, True])

    return pd.DataFrame(data=data, columns=['Name', 'longitude', 'latitude', 'Include'])

@st.cache_data
def travelingsalesman(chosen_landmarks):
    profile = "mapbox/cycling"
    coordinates = ";".join([f"{row['longitude']},{row['latitude']}" for _, row in chosen_landmarks.iterrows()])

    # optimized trips API -> optimized route to hit all landmarks
    url = f"https://api.mapbox.com/optimized-trips/v1/{profile}/{coordinates}"
    params = {"access_token": mapbox_token, "geometries": "geojson"}  # Request GeoJSON format

    res = requests.get(url, params=params)
    if res.status_code != 200:
        st.error(f"Error: API request failed with status code {res.status_code}")
        return None, []

    try:
        json_response = res.json()

        if 'trips' not in json_response or not json_response['trips']:
            st.error("Error: No trips found in the API response")
            return None, []

        trip = json_response['trips'][0]

        if 'geometry' not in trip:
            st.error("Error: No geometry found in the trip data")
            return None, []

        geometry = trip['geometry']

        if isinstance(geometry, dict) and 'coordinates' in geometry:
            optimized_coords = [(coord[1], coord[0]) for coord in geometry['coordinates']]
            return json_response, optimized_coords
        else:
            st.error("Error: Unexpected geometry format in the API response")
            return None, []
    except Exception as e:
        st.error(f"Error in travelingsalesman: {str(e)}")
        st.write("JSON Response:", json_response)  # Debug: Print the JSON response
        return None, []

def create_route_map(landmarks, optimized_coords):
    # Create a map centered on the mean of all coordinates
    all_lats = landmarks['latitude'].tolist() + [coord[0] for coord in optimized_coords]
    all_lons = landmarks['longitude'].tolist() + [coord[1] for coord in optimized_coords]
    center_lat = sum(all_lats) / len(all_lats)
    center_lon = sum(all_lons) / len(all_lons)

    m = folium.Map(location=[center_lat, center_lon], zoom_start=12)
    Draw(export=True).add_to(m)

    # Add markers for each landmark
    for _, row in landmarks.iterrows():
        folium.Marker(
            [row['latitude'], row['longitude']],
            popup=row['Name']
        ).add_to(m)

    # Add the optimized route line if coordinates are available
    if optimized_coords:
        folium.PolyLine(
            optimized_coords,
            weight=6,
            color='red',
            opacity=0.8
        ).add_to(m)
    else:
        st.warning("No optimized route available. Displaying landmarks only.")

    # Fit the map to the bounds of all coordinates
    sw = min(all_lats), min(all_lons)
    ne = max(all_lats), max(all_lons)
    m.fit_bounds([sw, ne])

    return m

# Function to create a bike route
@st.cache_data
def make_route(city, landmarks, _llm):
    prompt = PromptTemplate(
        template="""You are an experienced tour guide in {city}. You love telling more about landmarks in a short way. Create a bike route for {city} in markdown, using headings with ##, passing by the following landmarks: {landmarks}. End with the introduction of the next landmark {end}, as if it was the next destination, but don't discuss it.""",
        input_variables=["landmarks", "city", "end"]
    )
    chain = LLMChain(llm=_llm, prompt=prompt)
    landmarks_string = "\n".join([f"{row['Name']}" for _, row in landmarks.iloc[:5, :].iterrows()])

    # Handle case where there are fewer than 6 landmarks
    if len(landmarks) < 6:
        part_one = chain.run({'city': city, 'landmarks': landmarks_string, 'end': ""})
        return part_one
    else:
        part_one = chain.run({'city': city, 'landmarks': landmarks_string, 'end': landmarks.iloc[5]['Name']})

    prompt = PromptTemplate(
        template="""You are an experienced tour guide in {city}. You love telling more about landmarks in a short way. Create a bike route for {city} in markdown, using headings with ##, passing by the following landmarks: {landmarks}. Start your explanation with 'Continuing from {previous}'.""",
        input_variables=["landmarks", "city", "previous"]
    )

    chain = LLMChain(llm=_llm, prompt=prompt)
    landmarks_string = "\n".join([f"{row['Name']}" for _, row in landmarks.iloc[5:, :].iterrows()])
    part_two = chain.run({'city': city, 'landmarks': landmarks_string, 'previous': landmarks.iloc[4]['Name']})

    return part_one + " " + part_two

# Function to convert markdown data to HTML
@st.cache_data
def to_html(data, filename='route'):
    return markdown.markdown(data)

# New function to convert Folium map to HTML string
def map_to_html(map_object):
    map_html = io.BytesIO()
    map_object.save(map_html, close_file=False)
    return map_html.getvalue().decode()

if 'route' not in st.session_state:
    st.session_state.route = None

# Function to generate a bike route
def gen_route(city, stops):
    route = make_route(city['properties']['full_address'], stops, CloudflareWorkersAI(account_id=cf_account_id, api_token=cf_api_token))
    st.session_state.route = route

def create_route_map(landmarks, optimized_coords):
    # Create a map centered on the first landmark
    m = folium.Map(location=[landmarks.iloc[0]['latitude'], landmarks.iloc[0]['longitude']], zoom_start=12)

    # Add markers for each landmark
    for _, row in landmarks.iterrows():
        folium.Marker(
            [row['latitude'], row['longitude']],
            popup=row['Name'],
            tooltip=row['Name']
        ).add_to(m)

    # Add the optimized route line if coordinates are available
    if optimized_coords:
        folium.PolyLine(
            optimized_coords,
            weight=2,
            color='red',
            opacity=0.8
        ).add_to(m)
    else:
        st.warning("No optimized route available. Displaying landmarks only.")

    return m

# Update the main app logic
if city_id:
    city = retrieve_city(city_id)
    if city:
        coords = city['geometry']['coordinates']
        long, lat = coords
        landmarks = lmchain().run({"city": city['properties']['full_address']})

        if 'landmark_locations' not in st.session_state:
            st.session_state.landmark_locations = get_landmarks(landmarks, long, lat)

        user_inp = st.data_editor(
            st.session_state.landmark_locations,
            hide_index=True,
            disabled=('Name', 'longitude', 'latitude'),
            column_config={'longitude': None, 'latitude': None},
            key='user_input',
            use_container_width=True
        )

        st.session_state.landmark_locations.update(user_inp)

        selected_landmarks = st.session_state.landmark_locations[st.session_state.landmark_locations['Include']]

        output, optimized_coords = travelingsalesman(selected_landmarks)
        if output is not None and optimized_coords:
            dist = output['trips'][0]['distance']
            conv_fac = 0.000621371
            miles = dist * conv_fac
            st.write(f"Total distance: {miles:.3f} mi")

            waypoints = [wp['waypoint_index'] for wp in output['waypoints']]
            stops = selected_landmarks.iloc[waypoints, :]

            # Store the route map in session state
            st.session_state.route_map = create_route_map(stops, optimized_coords)

            # Display the route map
            folium_static(st.session_state.route_map)

            st.button('Generate route!', on_click=lambda: gen_route(city, stops))
        else:
            st.error("Unable to generate the optimized route. Please try again or select different landmarks.")

# Show generated route and offer map download
if 'route' in st.session_state and st.session_state.route:
    route = st.session_state.route
    st.markdown(route)

    # Offer route map download if available
    if 'route_map' in st.session_state:
        map_html = map_to_html(st.session_state.route_map)
        st.download_button(
            label='Download Route Map',
            data=map_html,
            file_name='route_map.html',
            mime='text/html'
        )

    # Offer route description download
    route_html = to_html(route)
    st.download_button(
        label='Download the route description!',
        data=route_html,
        file_name='cf-workers-ai-tourist-route.html',
        mime='text/html'
    )
st.markdown('</div>', unsafe_allow_html=True)
# Footer HTML
footer_html = """
    <div class="footer">
        Made with ❤️ in SF🌉
    </div>
"""
The [complete code can be found here on GitHub](https://github.com/elizabethsiegle/bike_walk_route_map_generator).

# Inject the footer into the app
st.markdown(footer_html, unsafe_allow_html=True)

Lastly, run the file with streamlit run tsp.py!
app gif

So much fun is to be had for developers with Mapbox, LangChain, Cloudflare, and Streamlit. Let me know online what you're building!

Summarize articles with Cloudflare Workers AI LoRAs

Lizzie Siegle — Fri, 12 Jul 2024 18:14:09 +0000

This tutorial will go over how to make a Streamlit web app in Python that takes in a URL and, with Workers AI LoRA adapters, summarizes the article in the tone of the user's choosing, such as gen z or humorous.

It's free to use most AI models hosted on Cloudflare!

Test it out yourself here.

The complete code can be found here on GitHub.

LoRAs

Fine-tuning is a process that takes a pre-trained model and further trains it on a specific dataset to adapt it to a particular task or domain, enhancing its performance and relevance to that task. LoRAs are a more efficient approach to fine-tuning that adds a small number of low-rank matrices to the model's parameters during fine-tuning.

This means that LoRAs offer a more efficient and scalable way to adapt LLMs to specific tasks without the heavy computational and memory demands of traditional fine-tuning.

Setup

Since this project requires Python package installations, make a new project directory and virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir lora-summarize 
cd lora-summarize
python3 -m venv venv 
source venv/bin/activate 
pip install streamlit
pip install bs4
pip install requests
pip install load_dotenv

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir lora-summarize
cd lora-summarize
python -m venv venv 
venv\Scripts\activate 
pip install streamlit
pip install bs4
pip install requests
pip install load_dotenv

The last command uses pip, the Python package installer, to install the packages that you are going to use in this project, which are:

The load_dotenv library to load environment variables from a .env file
The streamlit library to make webapps in Python, easily connecting the frontend and backend
The bs4 library to extract text from a webpage
The requests library to hit the Cloudflare endpoint for a LLM hosted on Cloudflare

This project needs a Cloudflare API Token and your Cloudflare Account ID to use an LLM. After making a Cloudflare account, click Workers&Pages on the left. Then, you can find your Account ID on the right side.

Get a Cloudflare API Key by clicking AI on the left, followed by Use Rest API. Finally, click Create a Workers AI API Token. Grab it and in the root directory, make a .env file and put them in there as such:

CF_API_TOKEN=XXXX
CF_ACCOUNT_ID=XXXX

Now it's time to code the Streamlit app.

Build the Streamlit App

I love how the Streamlit app lets you build your web page from the top-down.

At the top of a Python file called app.py, add the following code to import the required libraries, load the .env secret credentials, and create the variable for the Cloudflare URL to hit in Python:

import streamlit as st
from bs4 import BeautifulSoup
import json
import os
import requests
# Load API secrets
from dotenv import load_dotenv
load_dotenv()
CLOUDFLARE_ACCOUNT_ID = os.environ.get("CF_ACCOUNT_ID")
CLOUDFLARE_API_TOKEN= os.environ.get("CF_API_TOKEN")
url = f'https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1'

Then comes the meat of the app: Make a main function, include some Streamlit modules to display markdown on the page, write text to the page, get text input asking for a news URL to summarize, and display a selectbox with options the user can select for the tone of the summary.

When the user clicks the enter button, a spinner is displayed while Beautifulsoup extracts paragraph tags from the input web page URL.

def main():
    st.markdown("""
        <style>
            .big-font {
                font-size:40px !important;
                color:green;
            }
        </style>
    """, unsafe_allow_html=True)
    st.markdown('<p class="big-font"<p>AI🤖 News🗞️ Summarizer</p>', unsafe_allow_html=True)
    st.write(":blue[This Python🐍 web🕸️ app is built👩🏻‍💻 w/ [Streamlit](https://streamlit.io/) && [Cloudflare Workers AI](https://ai.cloudflare.com/)]")

    news_link = st.text_input('Please enter a news link to summarize') # news_link = "https://www.npr.org/2024/07/08/g-s1-8731/emma-navarro-coco-gauff-wimbeldon"
    tone = st.selectbox(
        ':green[What tone do you want the news summary to take?]',
        ('humorous🤣', 'majestic🌊', 'academic📚', '✨inspirational✨', 'dramatic🎭', 'gen z👧🏻')
    )
    st.write("You selected: ", tone)
    if st.button('Enter') and tone is not None and news_link is not None:
        with st.spinner('Processing📈...'):
            resp1 = requests.get(news_link)
            soup = BeautifulSoup(resp1.text, 'html.parser')

            # Extract text data from website
            text_data = ''
            for tag in soup.find_all(['p']):
                text_data += tag.get_text()

            print('text_data' , text_data)

That text extraction of paragraphs represents the body of the input URL. It is included (with the selected tone of the summary) in the prompt passed to the LoRA model adapter that hits the LLM (Mistral) model hosted on Cloudflare via GET request. To run inference with public LoRAs, you just need to define the LoRA name in the request. The response returned is the summary of the selected tone. It is then parsed and displayed. Finally, a footer is displayed at the bottom of the webpage.

            # Define the headers
            headers = {
                'Authorization': f'Bearer {CLOUDFLARE_API_TOKEN}',
                'Content-Type': 'application/json'
            }

            # Define the data
            data = {
                "messages": [
                    {
                        "role": "user",
                        "content": f"Summarize the following content from a news article in a {tone} tone: {text_data}"
                    }
                ],
                "lora": "cf-public-cnn-summarization"
            }

            # Make the POST request
            response = requests.post(url, headers=headers, data=json.dumps(data))

            # Parse the response
            response_data = response.json()
            summary = response_data["result"]["response"]
            print("summary ", summary)
        html_str = f"""
        <p style="font-family:Comic Sans; color:Pink; font-size: 18px;">{summary}</p>
        """
        st.markdown(html_str, unsafe_allow_html=True)

    st.write("Made w/ ❤️ in SF 🌁 || ✅ out the [👩🏻‍💻GitHub repo](https://github.com/elizabethsiegle/cf-ai-lora-news-summarizer)")

The complete app.py code is below:

import streamlit as st
from bs4 import BeautifulSoup
import json
import os
import requests
# Load API secrets
from dotenv import load_dotenv
load_dotenv()
CLOUDFLARE_ACCOUNT_ID = os.environ.get("CF_ACCOUNT_ID")
CLOUDFLARE_API_TOKEN= os.environ.get("CF_API_TOKEN")
url = f'https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1'

def main():
    st.markdown("""
        <style>
            .big-font {
                font-size:40px !important;
                color:green;
            }
        </style>
    """, unsafe_allow_html=True)
    st.markdown('<p class="big-font"<p>AI🤖 News🗞️ Summarizer</p>', unsafe_allow_html=True)
    st.write(":blue[This Python🐍 web🕸️ app is built👩🏻‍💻 w/ [Streamlit](https://streamlit.io/) && [Cloudflare Workers AI](https://ai.cloudflare.com/)]")

    news_link = st.text_input('Please enter a news link to summarize') # news_link = "https://www.npr.org/2024/07/08/g-s1-8731/emma-navarro-coco-gauff-wimbeldon"
    tone = st.selectbox(
        ':green[What tone do you want the news summary to take?]',
        ('humorous🤣', 'majestic🌊', 'academic📚', '✨inspirational✨', 'dramatic🎭', 'gen z👧🏻')
    )
    st.write("You selected: ", tone)
    if st.button('Enter') and tone is not None and news_link is not None:
        with st.spinner('Processing📈...'):
            resp1 = requests.get(news_link)
            soup = BeautifulSoup(resp1.text, 'html.parser')

            # Extract text data from website
            text_data = ''
            for tag in soup.find_all(['p']):
                text_data += tag.get_text()

            print('text_data' , text_data)

            # Define the headers
            headers = {
                'Authorization': f'Bearer {CLOUDFLARE_API_TOKEN}',
                'Content-Type': 'application/json'
            }

            # Define the data
            data = {
                "messages": [
                    {
                        "role": "user",
                        "content": f"Summarize the following content from a news article in a {tone} tone: {text_data}"
                    }
                ],
                "lora": "cf-public-cnn-summarization"
            }

            # Make the POST request
            response = requests.post(url, headers=headers, data=json.dumps(data))

            # Parse the response
            response_data = response.json()
            summary = response_data["result"]["response"]
            print("summary ", summary)
        html_str = f"""
        <p style="font-family:Comic Sans; color:Pink; font-size: 18px;">{summary}</p>
        """
        st.markdown(html_str, unsafe_allow_html=True)

    st.write("Made w/ ❤️ in SF 🌁 || ✅ out the [👩🏻‍💻GitHub repo](https://github.com/elizabethsiegle/cf-ai-lora-news-summarizer)")

if __name__ == "__main__":
    main()

What's Next for LoRAs and Cloudflare Workers AI

It's free to use many open source LLMs hosted on Cloudflare. They offer so many possibilities to builders and developers of all backgrounds for building generative AI applications.

LoRAs are helpful because they provide a cost-effective, efficient, flexible, and scalable method for adapting LLMs to specific tasks, making advanced AI capabilities more accessible for a wide variety of applications.

Let me know online what you're building with AI!

Create AI-generated art via SMS with Replicate in Python

Lizzie Siegle — Thu, 21 Sep 2023 21:22:37 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

It's fun to get creative! Read on to learn how to create AI-generated images via SMS using Replicate and Twilio Programmable Messaging.

Do you prefer learning via video more? Check out this TikTok summarizing this tutorial.

Prerequisites

A Twilio account - sign up for a free Twilio account here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
Replicate account to use the Stable Diffusion text-to-image model – make a Replicate account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access.

⚠️ ngrok is needed for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. You can also choose to automate ngrok as shown in this article.

Replicate

Replicate offers a cloud API and tooling so you can more easily run machine learning models, abstracting away some lower-level AI concepts and handling infrastructure so you can focus more on your own applications. You can run open-source models that others have published, or package and publish your own, either publicly or privately.

Configuration

Since you will be installing some Python packages for this project, you need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir replicate-prompt-to-image-sms 
cd replicate-prompt-to-image-sms 
python3 -m venv venv 
source venv/bin/activate 
pip install replicate flask twilio

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir replicate-prompt-to-image-sms  
cd replicate-prompt-to-image-sms  
python -m venv venv 
venv\Scripts\activate 
pip install replicate flask twilio

Grab your default Replicate API Token or create a new one here. Run this command in the terminal in your current folder:

export REPLICATE_API_TOKEN={replace with your api token}

Now it's time to write some code to receive a prompt via SMS and return an AI-generated image!

Turn a Prompt into an AI-generated image via SMS with Replicate

Make a file called app.py and place the following import statements at the top.

import replicate
from flask import Flask, request
from twilio.twiml.messaging_response import MessagingResponse

Next, make a Flask app so your app can receive the inbound text message to your Twilio phone number. That string is then passed to Replicate 's stable diffusion model to generate photo-realistic images given that prompt.

app = Flask(__name__)
@app.route('/sms', methods=['GET', 'POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip()
    output = replicate.run(
        "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",
        input={"prompt": inb_msg}
        )
    res_prompt = "prompt was " + inb_msg
    msg = resp.message(res_prompt)
    img = output[0]
    msg.media(img)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

On the command line, run python app.py to start the Flask app. Now it's time to set up a Twilio phone number so you can text it a prompt and get an AI-generated image back!

Configure a Twilio Number for the SMS Chatbot

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL with the /sms path in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio phone number is configured so that it maps to your web application server running locally on your machine and your application can run. Text a prompt of a picture you'd like to generate to your Twilio number and you should get an AI-generated image of whatever you sent over SMS!

Instead of ngrok, you could alternatively use another tunneling service like Tailscale–my teammate Rishab reviewed this post, and kept getting a 403 error with ngrok. The solution was to switch port numbers with ngrok–it seems that Tailscale has more privileges to the network settings compared to ngrok. You can view the complete code on GitHub here.

What's Next for Replicate and Twilio Programmable Messaging?

There is so much fun for developers to have around building with LLMs! Replicate offers so many models to play with like sdxl, a text-to-image generative AI model that creates beautiful 1024x1024 images, or you can fine-tune your own. I can't wait to see what you build with AI–let me know online what you're working on!

Build an AI Personal Trainer with LangChain Agents and SendGrid

Lizzie Siegle — Tue, 19 Sep 2023 19:53:32 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

Can AI help me run a faster marathon?

Problem: I play a lot of tennis and run, bike, and walk as my forms of transit, so I have never trained for a half marathon or 10k. However, I KNOW I must train to run a marathon. As a developer, I turned to code, wondering how AI could help me run a faster marathon.

Solution: Read on to learn how to build a personal marathon training plan generator Streamlit-hosted app from personal workout data using the Strava API, LangChain agents, chains, few-shot prompt templating, PromptTemplates, OpenAI, and SendGrid. This can also be applied to personal training, nutrition, and more!

In the Streamlit page above, a user inputs their Strava API token so the app can access their workout data, the date they want to begin training, and the marathon (or race) day. When they click Submit, a custom training plan is generated.

You can test the app out here using a Strava API token generated by following the directions under Strava Activity Webhook Authentication in this tutorial on querying data with LangChain agents.

Do you prefer learning via video more? Check out this TikTok relating to this tutorial--massive thank you to Twilio Community Engagement Manager Sandra Mendez for spearheading this video.

Understanding LangChain Agents

This meaty app uses many features of LangChain, such as agents: agents use LLMs to decide what actions should be taken and they have access to tools like web search or calculators which can be packaged into a logical loop of operations.

For more background on LangChain, you can read this blog post here.

Prerequisites

A free SendGrid account
Python installed - download Python here
An email address to test out this project
OpenAI Account – make an OpenAI Account here
A Strava account - sign up for a Strava account if you don't have one already
A Strava app and API key - follow directions here under Setup the Strava API
A Strava Auth token - follow directions here under Strava Activity Webhook Authentication You need a Strava app and API key–following the directions above will provide you with the necessary Strava Client Secret in order to generate a Strava Access Token to make a request to access your Strava data. Different Strava API app values include:
Client ID: Your application ID
Client Secret: Your client secret (please keep this confidential)
Authorization/Access token: Your Strava API app's authorization token will need to be regenerated every six hours (should be confidential) Next, after making an OpenAI account if you don't have one already, you can get an OpenAI API Key here by clicking on + Create new secret key. Then, grab a SendGrid API key here.

The Python application will need to have access to these keys, so in your root directory (called strava-langchain) make a .env file to store the API keys safely. The application we create will be able to import this key as an environment variable soon.

In the .env file add the following lines of text, making sure to replace <YOUR-OPENAI-API-KEY> and <YOUR-SENDGRID-KEY> with your actual keys. STRAVA_CLIENT_SECRET should have been used above to generate a Strava Access Token:

OPENAI_API_KEY= <YOUR-OPENAI-KEY>
SENDGRID_API_KEY = <YOUR-SENDGRID-KEY>
STRAVA_CLIENT_SECRET= <YOUR-STRAVA-CLIENT-SECRET>

Your Strava Access Token will be placed into a Streamlit form. Streamlit is an open-source Python app framework that helps developers quickly create web apps for data science and machine learning.

In the root directory run the following commands to install the project's required libraries after making a virtual environment:

python3 -m venv venv 
source venv/bin/activate

Make a requirements.txt file containing the following:

chromadb==0.4.5
google-search-results==2.4.2
langchain==0.0.261
numpy==1.25.0
openai==0.27.2
pandas==2.0.3
python-dotenv==1.0.0
PyYAML==6.0
reportlab==4.0.4
Requests==2.31.0
sendgrid==6.10.0
streamlit==1.25.0
tabulate==0.9.0
urllib3==2.0.4

Install them on the command line by running pip install -r requirements.txt.

Then run mkdir data_files to make a folder called data_files to store the activity data.

We will import these in our code soon.

Few-shot prompting Examples

There are a few ways one could go about making an AI personal marathon trainer, workout trainer, or recommendation system. My college classmate-turned Redis AI engineer Sam Partee suggested using few-shot prompting, in which a few explicit examples (or "shots") guide the Large Language Model (LLM) to respond in a certain way.

Create a file named stapp.py and include some examples of marathon training plans. I used OpenAI to create these plans and then formatted them a bit because most marathon plans I found online were not copy-and-pasteable.

examples = [
  {
    "training_start_date" : "July 19",
    "marathon_date" : "November 8",
    "pd_run_output": "The average distance is 3.71 miles and the maximum distance is 9.52 miles. The average moving time is not available and the maximum moving time is not available. Other running statistics include total elevation gain, with an average of 47.6 and a maximum of 200.6.",
    "num_workouts": "112",
    "plan": 
"""
Week 1: 
Day 1, {training_start_date}: Cross-training - Swim
Day 2: Easy run - 4 miles at an easy pace
Day 3: Cross-training - Elliptical 
Day 4: Medium run - 5 miles at a medium pace 
...
{marathon_date}: Marathon Day!
Note: The above plan assumes that the student is currently able to comfortably run 4 miles at an easy pace. Adjustments may need
to be made if the student is not yet at this level. Additionally, the plan includes cross-training and rest days to
ensure proper recovery and prevent injury.
"""
  }
]

The complete code for that file is here on GitHub. Now let's make some custom tools to help LangChain suggest personalized workouts.

Make Custom Tools to assist LangChain Agents

Using agents allows LLMs access to tools with which LLMs can search the web, do math, run code, and more. Though LLMs are powerful, they still have trouble with certain tasks–that's why you may sometimes need to make custom tools.

LangChain offers a large selection of pre-built tools, but only so many problems can be solved using existing tools.

Create a file named request.py and include the following import statements.

import re
from datetime import date, datetime
import os
import base64
from typing import List, Union
import pandas as pd  # dataframe

from reportlab.platypus import SimpleDocTemplate, Paragraph
from reportlab.lib.pagesizes import letter
from reportlab.lib.styles import getSampleStyleSheet
from reportlab.lib.units import inch

import requests
import urllib3
from dotenv import dotenv_values
import streamlit as st

from langchain import LLMChain, LLMMathChain
from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser, create_pandas_dataframe_agent, initialize_agent, AgentType
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI  # , GPT4All
from langchain.prompts import BaseChatPromptTemplate
from langchain.prompts.few_shot import FewShotPromptTemplate
from langchain.prompts.prompt import PromptTemplate
from langchain.schema import AgentAction, AgentFinish, HumanMessage

from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import (
    Mail, Attachment, FileContent, FileName, FileType, Disposition)

import stapp  # file
urllib3.disable_warnings(
    urllib3.exceptions.InsecureRequestWarning
)
config = dotenv_values(".env")

Next, we set some variables such as the Strava URL to retrieve a user's Activity data, have the app access its API keys, and then chain LLMs together: the llm variable passes the desired gpt-4-32k model to use with ChatOpenAI and sets the temperature to 0.2–play around with this value. A higher temperature value means the LLM output will include more randomness. Lastly, pass LLM to a math chain to perform complex math problems.

activities_url = "https://www.strava.com/api/v3/athlete/activities"
os.environ["OPENAI_API_KEY"] = config.get('OPENAI_API_KEY')
os.environ["SENDGRID_API_KEY"] = config.get('SENDGRID_API_KEY')
llm = ChatOpenAI(model_name='gpt-4-32k', temperature=0.2)
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)

Now make some functions for the custom tools to help the LLM convert meters to miles, calculate the number of days between two dates, and get the number of days in a Pandas dataframe object.

def convert_to_miles(num):
    return ((num/m_conv_factor)*1000)/1000.0

def calc_days_btwn(training_start_date, marathon_date): 
    training_start_date = training_start_date.split(',')[1].strip()
    training_start_date = datetime.strptime(training_start_date, '%Y-%m-%d') 
    training_start_date = datetime.date(training_start_date)

    delta = marathon_date - training_start_date
    return delta.days-1

#number of workouts in Strava data
def num_rows_in_dataframe(df):
    return len(df.index)

Beneath that, add a helper function to help validate user emails.

email_regex = r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,7}\b'
def validate_email(email):
    # pass the regular expression and the string into the fullmatch() method
    if(re.fullmatch(email_regex, email)):
        return True
    else:
        return False

It's time to make a Custom Prompt Template in LangChain to make constructing prompts using dynamic inputs easier.

Make Custom Prompt Template

LangChain provides a set of default prompt templates to generate prompts for different tasks. However, for some projects like this one, developers might want to create a prompt template with specific dynamic instructions for the LLM–in instances like that, you can create a custom prompt template similar to the one below right inside request.py beneath validate_email:

#Set up a prompt template
class CustomPromptTemplate(BaseChatPromptTemplate):
    template: str
    # The list of tools available
    tools: List[Tool]

    def format_messages(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join([f"{tool.name}: {tool.description}" for tool in self.tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
        formatted = self.template.format(**kwargs)
        return [HumanMessage(content=formatted)] 

class CustomOutputParser(AgentOutputParser):
    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
        # Check if agent should finish
        if "Marathon Day" in llm_output:
            return AgentFinish(
                return_values={"output": llm_output.split("Marathon Day:")[-1].strip()},
                log=llm_output,
            )
        elif marathon_date in llm_output:
            return AgentFinish(
                # Return values is generally always a dictionary with a single `output` key
                # It is not recommended to try anything else at the moment :)
                return_values={"output": llm_output.split(marathon_date)[-1].strip()},
                log=llm_output,
            )
        # Parse out the action and action input
        regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
        match = re.search(regex, llm_output, re.DOTALL)
        if not match:
            try:
                action = match.group(1).strip()
                action_input = match.group(2)
                # Return the action and action input
                return AgentAction(tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output)
            except ValueError as e:
                response = str(e)
                if not response.startswith("Could not parse LLM output: `"):
                    raise e
                response = response.removeprefix("Could not parse LLM output: `").removesuffix("`")

That helps shape the output. Let's work on receiving the input.

Make a Streamlit web app in Python

To make a webpage to receive input about the user for their personal marathon training plan, we will use Streamlit.

In request.py, we set a title, subheader, the dates that are first displayed when the date input widget is rendered, and a Streamlit form containing two text_input fields for the user's Strava token and email, make the date input widget, and then write the input dates to the page and split up the input dates into two variables. This code goes directly beneath response = response.removeprefix("Could not parse LLM output: ").removesuffix("").

st.title('Personal Marathon Training plan generator')
st.subheader('enter details below')
today = datetime.now()
jul_30 = date(today.year, 7, 30)
dec_31 = date(today.year, 12, 31)

with st.form('my_form'):
    strava_token_input = st.text_input('Strava API token')
    email = st.text_input('Email to send plan to')
    dates = st.date_input(
        "Select your training dates for the year: it should be from the first training date to the marathon date",
        (jul_30, date(today.year, 12, 10)),
        jul_30,
        dec_31
    )
    st.write('Training date: ', dates[0], "Marathon date: ", dates[1])
    training_start_date = dates[0]
    marathon_date = dates[1]
    submitted = st.form_submit_button('Submit')

Run the command streamlit run request.py in your terminal. An application should open up on your localhost:8080:

Beneath the last line marathon_date = dates[1], add the following code to validate the user input:

if submitted:

        if not validate_email(email):
            st.error("invalid email", icon="🚨")
            st.cache_data()
            VALIDANSWERS = False
        elif training_start_date > marathon_date:
            st.error("Start date must be earlier than end date", icon="🚨")
            st.cache_data()
            VALIDANSWERS = False     
        try:
            r = requests.get('https://www.strava.com/api/v3/activities?access_token=' + strava_token_input)
            r.raise_for_status()
            VALIDANSWERS = True
        except requests.exceptions.HTTPError as e:
            print(e)
            st.error("Check Strava token")
            st.cache_data()
            VALIDANSWERS = False
        if VALIDANSWERS:

Get Strava data into Pandas and CSV form

The parameters and headers need to be set in order to call a Strava API request. my_dataset contains data from just the first page of user activities, so to retrieve more activity data, loop through four pages of Strava activities and add them to my_dataset before calling pd.json_normalize on it to turn the semi-structured JSON data into a flat table to work with it.

Directly beneath the if VALIDANSWERS: line, add the following code:

 header = {'Authorization': 'Bearer ' + strava_token_input} 
            params = {'per_page': 200, 'page': 1} #max 200 per page, can only do 1 page at a time
            my_dataset = requests.get(activities_url, headers=header, params=params).json() #activities 1st page
            page = 0
            for x in range(1,5): #loop through 4 pages of strava activities
                page +=1 
            params = {'per_page': 200, 'page': page}
            my_dataset += requests.get(activities_url, headers=header, params=params).json() 

            activities = pd.json_normalize(my_dataset)

Next, create a new Pandas Dataframe object with specific columns containing the categories you want to consider: this tutorial is only looking at the activity name, type (bike ride, run, swim, tennis, etc), distance, moving time, total elevation gain, and start date.

The following code goes beneath activities = pd.json_normalize(my_dataset) and creates a CSV file of the activities data after removing items from 2021 so it only includes workouts from 2022 and 2023. You can replace this with another year! It then makes a CSV file named data_files/runs.csv containing the runs and no other activity types.

cols = ['name', 'type', 'distance', 'moving_time', 'total_elevation_gain', 'start_date']
            activities = activities[cols]
            activities = activities[activities["start_date"].str.contains("YEAR-YOU-WANT-TO-IGNORE-WORKOUTS") == False] 
            activities.to_csv('data_files/activities.csv', index=False)

            runs = activities.loc[activities['type'] == 'Run']
            runs.to_csv('data_files/runs.csv', index=False)

Now, convert meters to miles and the moving time seconds to minutes and hours with the code below:

 data_run_df = pd.read_csv('data_files/runs.csv')
            data_activity_df = pd.read_csv('data_files/activities.csv')
            m_conv_factor = 1609

            data_run_df['distance'] = data_run_df['distance'].map(lambda x: convert_to_miles(x))
            data_activity_df['distance'] = data_activity_df['distance'].map(lambda x: convert_to_miles(x))
            #convert moving time secs to mins, hours
            data_run_df['moving_time'] = data_run_df['moving_time'].astype(str).map(lambda x: x[7:]) #slices off 0 days from moving_time
            data_run_df.to_csv('data_files/runs.csv')
            data_activity_df = pd.read_csv('data_files/activities.csv')

Next, this data will be passed to a LangChain agent.

Generate a Personal Marathon Plan with LangChain Agents

In the same request.py file, make an array called tools containing the tools the agent has access to. Each tool gets a name, function to run, and description. The description is optional, but recommended because it's used by an agent to determine tool use.

Beneath creating the data_activity_df variable, add the tools array:

tools = [
                Tool(
                    name = "rows in csv",
                    func = lambda df: num_rows_in_dataframe(df),
                    description="use to get the number of rows in csv file to calculate averages from running data"
                ),
                Tool(
                    name="Calculator",
                    func=llm_math_chain.run,
                    description="useful for when you need to answer questions about math"
                ),
                Tool(
                    name="days_between",
                    func=lambda day1, day2=marathon_date: calc_days_btwn(day1, day2),
                    description="useful for when you need to calculate the number of days between two dates of type date, like between the training start date and the marathon date"
                )
            ]

Now, create some Pandas dataframe agents to interact with the Strava data. These agents are similar to CSV agents which load data from CSV files instead of dataframes and perform queries. You could use both and see how the output differs.

Agents can be chained together to build more complex applications. Another agent used below is of type ZERO_SHOT_REACT_DESCRIPTION. Zero-shot means the agent functions only on the present action — it lacks memory. It uses the ReAct framework to decide which tool to use, solely based on the tool’s description.

The code runs the agents, telling them what to do and saving the output to variables for use later. Be specific with the prompts here and play around with different questions or commands.

Beneath the tools array, add:

pd_agent_run = create_pandas_dataframe_agent(OpenAI(temperature=0.3), data_run_df, verbose=True) 
pd_agent_activity = create_pandas_dataframe_agent(OpenAI(temperature=0.3), data_activity_df, verbose=True)
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
num_workouts = agent.run(f'Calculate the number of days between these two dates representing the number of workouts in the plan: {marathon_date} and {training_start_date}. You have access to tools')
pd_run_output = pd_agent_run.run("Calculate average distance and maximum distance in miles and average moving time and maximum moving time in minutes. Additionally, calculate any other running statistics from the data you think would be helpful to consider in marathon training.")

Next, make a prompt template, a "reproducible way to generate a prompt" containing a text string ("the template"). It accepts a set of parameters from the user and generates a prompt. This mirrors the items in stapp.py's examples array.

prefix contains directions and context to the LLM and suffix is the user input and output indicator. Finally, these, along with the input variables which are unique to the student the plan is being made for, are passed into FewShotPromptTemplate.

coach_template = """training_start_date:{training_start_date}, marathon_date:{marathon_date}, pd_run_output: {pd_run_output}, num_workouts: {num_workouts}, plan: {plan}
            """
example_prompt = PromptTemplate(input_variables=["training_start_date", "marathon_date", "pd_run_output", "num_workouts", "plan"], template=coach_template)

prefix = """
            You are a marathon trainer tasked with crafting one personalized marathon training plan containing  according to your student's previous runs and activities.
            For each week you should recommend 2 cross-training activities and no more than one run greater than 14 miles per week. The longest run you recommend should be 20 miles around 2 weeks before {marathon_date}.
            There should be {num_workouts} workouts for {num_workouts} days, and there should be easy workouts and rest days in the week leading up to {marathon_date}.
            Slightly modify the following example marathon training plans based on your student and the days beginning on {training_start_date} up until {marathon_date} to create one personalized training plan with each workout on a new line: 
            """
suffix = """
            training_start_date: {training_start_date}, marathon_date: {marathon_date}, pd_run_output: {pd_run_output}, num_workouts: {num_workouts}
            """
few_shot_prompt_template = FewShotPromptTemplate(
                examples=stapp.examples, 
                example_prompt=example_prompt, 
                prefix=prefix,
                suffix=suffix, 
                input_variables=["marathon_date", "training_start_date", "pd_run_output", "num_workouts"]
            )

Create a custom output parser object to help structure the response from the LLM and LLM chain, the most common type of chain. The LLM chain consists of the model and prompt. For this case it's an LLM but it could be a ChatModel. The LLM is passed to a LLMSingleActionAgent to complete a single action and structure the response from the LLM.

This agent is passed to an AgentExecutor. An executor differs from an agent in that it is a running instance of an agent used to execute tasks according to the agent’s decisions and configured tools. In short, agents decide which tools to use and the executors execute them. The output of the Executor (which is given the user input) is the personalized marathon training plan. Beneath the FewShotPromptTemplate() function, add the following code:

output_parser = CustomOutputParser()

#LLM chain consisting of the LLM and a prompt
llm_chain = LLMChain(llm=llm, prompt=few_shot_prompt_template)
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain, 
    output_parser=output_parser,
    stop=["\Marathon Day:"], 
    allowed_tools=tool_names
)
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
plan = agent_executor({"marathon_date":marathon_date, "training_start_date": training_start_date, "pd_run_output": pd_run_output, "num_workouts": num_workouts})
print('plan being print ', plan)

Email the Plan as a PDF using Twilio SendGrid

We access the part of the plan containing the dates, replace new lines with breaks because the ReportLab Python library to generate and format PDFs ignores \n, and send an outbound email using Twilio SendGrid–the email body says "good luck at your marathon on {marathon_date}" with the plan attached.

plan = plan['output']
plan = str(plan).replace('\n','<br />') #each workout on new line
message = Mail(
                from_email='langchain_sendgrid_marathon_trainer@sf.com',
                to_emails=email,
                subject='Your AI-generated marathon training plan',
                html_content='<strong>Good luck at your marathon on %s</strong>!\n\nYour plan is attached.'%(marathon_date))

The email attachment is formatted and styled using ReportLab. The following code returns a stylesheet with a few basic heading and text styles to format the PDF attachment which is in the form of a SimpleDocTemplate document. The document is built using the Paragraph class that lets us format the text with inline font style (and optional color changes) using an XML style markup.

The encoded file is passed as an attachment to the SendGrid email. If the email is sent successfully, the Streamlit app displays a success message; otherwise, it displays an error message.

styleN = getSampleStyleSheet()['Normal']
            story = []

            pdf_name = 'plan.pdf'
            doc = SimpleDocTemplate(
                pdf_name,
                pagesize=letter,
                bottomMargin=.4 * inch,
                topMargin=.6 * inch,
                rightMargin=.8 * inch,
                leftMargin=.8 * inch)
            P = Paragraph(plan, styleN)
            story.append(P)

            doc.build(
                story,
            )
            with open(pdf_name, 'rb') as f:
                data = f.read()
                f.close()
            encoded_file = base64.b64encode(data).decode()

            attachedFile = Attachment(
                FileContent(encoded_file),
                FileName('personal_ai_generated_marathon_training_plan.pdf'),
                FileType('application/pdf'),
                Disposition('attachment')
            )
            message.attachment = attachedFile
            sg = SendGridAPIClient()
            response = sg.send(message)
            if response.status_code == 202:
                st.success("Email sent! Check your email for your personal training plan")
                print(f"Response Code: {response.status_code} \n Message sent!")
            else:
                st.warning("Email not sent--check email")

Run streamlit run request.py in your terminal. Fill in your information (like Strava Access Token, the email where you want to receive the generated plan, the training start date, and marathon race date!) on the Streamlit app, click Submit, and wait for your personal marathon training plan in your email.

If the Submit button is clicked, we check the input (dates and email). If the inputs are either:

invalid: the Streamlit app displays an error message and caches the function so the user can try valid input.
valid: the Streamlit app displays a success message and it's time to use the user's Strava API token to get their Strava workout data.

Play around with different agents, prompting, and different parameters like temperature (which affects the risks the agent takes and creativity of the output).

The complete code above can be found here on GitHub.

What's Next for Agents and SendGrid

So will I use this to train for my marathon? I haven't coded this much since college (or had this much fun doing so) and this was a great way to learn many of LangChain's features, but time will tell –work travel, tennis practices, and AI hackathons make consistent training difficult. I do want to continue trying different ways of changing the output--this is a lot of code which I feel like could be unnecessary, but it yielded the best results.

There's so much developers can do with communication APIs and AI–this is just the tip of the iceberg. You can use a different LLM, look at different Strava data, try different agents (or use chains instead!), embeddings, use LLaMA 2 or other similar models on Replicate, and more. You could use SMS or Twilio Conversations to collect user input instead of Streamlit, or make the LLM recommend other workouts for different goals or meals based on nutrition data!

Let me know online what you're building (and send running tips!)

Build an AI SMS Chatbot with Replicate, LLaMA 2, and LangChain

Lizzie Siegle — Tue, 12 Sep 2023 22:16:09 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

Recently, Meta and Microsoft introduced the second generation of the LLaMA LLM (Large Language Model) to help developers and organizations to build generative AI-powered tools and experiences. Read on to learn how to build an AI SMS chatbot that answers questions like Ahsoka (from Star Wars) using LangChain templating, LLaMa 2, Replicate, and Twilio Programmable Messaging!

Do you prefer learning via video more? Check out this TikTok summarizing this tutorial in 1 minute!

Prerequisites

A Twilio account - sign up for a free Twilio account here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
Replicate account to host the LlaMA 2 model – make a Replicate account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access.

Replicate

Replicate offers a cloud API and tools so you can more easily run machine learning models, abstracting away some lower-level machine learning concepts and handling infrastructure so you can focus more on your own applications. You can run open-source models that others have published, or package and publish your own, either publicly or privately.

Configuration

Since you will be installing some Python packages for this project, you will need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir replicate-llama-ai-sms-chatbot  
cd replicate-llama-ai-sms-chatbot  
python3 -m venv venv 
source venv/bin/activate 
pip install langchain replicate flask twilio

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir replicate-llama-ai-sms-chatbot  
cd replicate-llama-ai-sms-chatbot   
python -m venv venv 
venv\Scripts\activate 
pip install langchain replicate flask twilio

Grab your default Replicate API Token or create a new one here.

On the command line run

export REPLICATE_API_TOKEN={replace with your api token}

Now it's time to write some code!

Code

Make a file called app.py and place the following import statements at the top.

from flask import Flask, request
from langchain import LLMChain, PromptTemplate
from langchain.llms import Replicate
from langchain.memory import ConversationBufferWindowMemory
from twilio.twiml.messaging_response import MessagingResponse

Though LLaMA 2 is tuned for chat, templates are still helpful so the LLM knows what behavior is expected of it. This starting prompt is similar to ChatGPT so it should behave similarly.

template = """Assistant is a large language model.

Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.

Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.

Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist. 

I want you to act as Ahsoka giving advice and answering questions. You will reply with what she would say.
SMS: {sms_input}
Assistant:"""

prompt = PromptTemplate(input_variables=["sms_input"], template=template)

Next, make a LLM Chain, one of the core components of LangChain. This allows us to chain together prompts and make a prompt history. The model is formatted as the model name followed by the version–in this case, the model is LlaMA 2, a 13-billion parameter language model from Meta fine-tuned for chat completions. max_length is 4096, the maximum number of tokens (called the context window) the LLM can accept as input when generating responses.

sms_chain = LLMChain(
    llm = Replicate(model="a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5"), 
    prompt=prompt,
    memory=ConversationBufferWindowMemory(k=2),
    llm_kwargs={"max_length": 4096}
)

Finally, make a Flask app to accept inbound text messages, pass that to the LLM Chain, and return the output as an outbound text message with Twilio Programmable Messaging.

app = Flask(__name__)


@app.route("/sms", methods=['GET', 'POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip()
    output = sms_chain.predict(sms_input=inb_msg)
    print(output)
    resp.message(output)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

On the command line, run python app.py to start the Flask app.

Configure a Twilio Number for the SMS Chatbot

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL with the /sms path in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio phone number is configured so that it maps to your web application server running locally on your computer and your application can run. Text your Twilio number a question relating to the text file and get an answer from that file over SMS!

You can view the complete code on GitHub here.

What's Next for Twilio, LangChain, Replicate, and LLaMA 2?

There is so much fun for developers to have around building with LLMs! You can modify existing LangChain and LLM projects to use LLaMA 2 instead of GPT, build a web interface using Streamlit instead of SMS, fine-tune LLaMA 2 with your own data, and more! I can't wait to see what you build–let me know online what you're working on!

Build a SMS Bot that Answers questions over Docs with LangChain in Python

Lizzie Siegle — Thu, 15 Jun 2023 20:33:43 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

With Natural Language Processing (NLP), you can chat with your own documents, such as a text file, a PDF, or a website. Read on to learn how to build a generative question-answering SMS chatbot that reads a document containing Lou Gehrig's Farewell Speech using LangChain, Hugging Face, and Twilio in Python.

LangChain Q&A

LangChain is an open-source tool that wraps around many large language models (LLMs) and tools. It is the easiest way (if not one of the easiest ways) to interact with LLMs and build applications around LLMs.

LangChain makes it easy to perform question-answering of those documents. Picture feeding a PDF or maybe multiple PDF files to a machine and then asking it questions about those files. This could be useful, for example, if you have to prepare for a test and wish to ask the machine about things you didn’t understand.

Prerequisites

A Twilio account - sign up for a free Twilio account here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
Hugging Face Account – make a Hugging Face Account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access. > ⚠️ ngrok is needed for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. You can also choose to automate ngrok as shown in this article.: ## Configuration Since you will be installing some Python packages for this project, you will need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir lc-qa-sms 
cd lc-qa-sms 
python3 -m venv venv 
source venv/bin/activate 
!pip install langchain
!pip install requests
pip install flask
pip install faiss-cpu
pip install sentence-transformers
pip install twilio
pip install load_dotenv

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir lc-qa-sms  
cd lc-qa-sms 
python -m venv venv 
venv\Scripts\activate 
pip install langchain
pip install requests
pip install flask
pip install faiss
pip install sentence-transformers
pip install twilio
pip install load_dotenv

Set up Hugging Face Hub

The Hugging Face Hub offers over 120k models, 20k datasets, and 50k demos people can easily collaborate in their ML workflows. As mentioned earlier, this project needs a Hugging Face Hub Access Token to use the LangChain endpoints to a Hugging Face Hub LLM. After making a Hugging Face account, you can get a Hugging Face Access Token here by clicking on New token. Give the token a name and select the Read Role.

Alternatively, you could use models from, say, OpenAI.

On the command line in your root directory, run the following command on a Mac to set the token as an environment variable.

export HUGGINGFACEHUB_API_TOKEN=replace-with-your-huggingfacehub-token

For the Windows command, check out this blog post on environment variables.

Now let's build that LangChain question-answering chatbot application.

Answer Questions from a Doc with LangChain via SMS

Inside your lc-qa-sms directory, make a new file called app.py.

At the top of the file, add the following lines to import the required libraries.

import requests
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains.question_answering import load_qa_chain
from langchain import HuggingFaceHub
from flask import Flask, request, redirect
from twilio.twiml.messaging_response import MessagingResponse

Beneath those import statements, make a helper function to write to a local file from a URL, and then load that file from the local file storage with LangChain's TextLoader library. Later the file will be passed over to the split method to create chunks.

def loadFileFromURL(text_file_url):
    output_file = "lougehrig.txt"
    resp = requests.get(text_file_url)
    with open(output_file, "w",  encoding='utf-8') as file:
      file.write(resp.text)

    # load text doc from URL w/ TextLoader
    loader = TextLoader('./'+output_file)
    txt_file_as_loaded_docs = loader.load()
    return txt_file_as_loaded_docs

In this tutorial, the file will be Lou Gehrig's famous speech, which is why the output_file is named "lougehrig.txt".

You could alternatively use a local file.

Next, add the following code for a helper function to split the document into chunks. This is important because LLMs can't process inputs that are too long. LangChain's CharacterTextSplitter function helps us do this–setting chunk_size to 1000 and chunk_overlap to 10 keeps the integrity of the file by avoiding splitting words in half.

def splitDoc(loaded_docs):
    # split docs into chunks
    splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=10)
    chunked_docs = splitter.split_documents(loaded_docs)
    return chunked_docs

Now convert the chunked document into embeddings (numerical representations of words) with Hugging Face and store them in a FAISS Vector Store. Faiss is a library for efficient similarity search and clustering of dense vectors.

def makeEmbeddings(chunked_docs):
    # Create embeddings and store them in a FAISS vector store
    embedder = HuggingFaceEmbeddings()
    vector_store = FAISS.from_documents(chunked_docs, embedder)
    return vector_store

That makeEmbeddings function will make it more efficient to retrieve and manipulate the stored embeddings to conduct a similarity search to get the most semantically-similar documents to a given input which the LLM needs to best answer questions in the following helper function.

def askQs(vector_store, chain, q):
    # Ask a question using the QA chain
    similar_docs = vector_store.similarity_search(q)
    resp = chain.run(input_documents=similar_docs, question=q)
    return resp

The final helper function defines and loads the Hugging Face Hub LLM to be used with your Access Token and starts the request on a similarity search embedded with some input question to the selected LLM, enabling a question-and-answer conversation.

def loadLLM():
    llm=HuggingFaceHub(repo_id="declare-lab/flan-alpaca-large", model_kwargs={"temperature":0, "max_length":512})
    chain = load_qa_chain(llm, chain_type="stuff")
    return chain

declare-lab/flan-alpaca-large is a LLM. You could use others from Hugging Face Hub such as google-flan-t5-xl. They are trained on different corpuses and this tutorial uses flan-alpaca-large because it's faster.

Lastly, make a Flask app using the Twilio REST API to call the helper functions and respond to inbound text messages with information pulled from the text file.

app = Flask(__name__)
@app.route('/sms', methods=['POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip() #get inbound text body
    chain = loadLLM()
    LOCAL_ldocs = loadFileFromURL('https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt')
    LOCAL_cdocs = splitDoc(LOCAL_ldocs) #chunked
    LOCAL_vector_store = makeEmbeddings(LOCAL_cdocs)
    LOCAL_resp = askQs(LOCAL_vector_store, chain, inb_msg)
    resp.message(LOCAL_resp)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

Your complete app.py file should look like this:

import requests
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains.question_answering import load_qa_chain
from langchain import HuggingFaceHub
from flask import Flask, request, redirect
from twilio.twiml.messaging_response import MessagingResponse

def loadFileFromURL(text_file_url): #param: https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt
    output_file = "lougehrig.txt"
    resp = requests.get(text_file_url)
    with open(output_file, "w",  encoding='utf-8') as file:
      file.write(resp.text)

    # load text doc from URL w/ TextLoader
    loader = TextLoader('./'+output_file)
    txt_file_as_loaded_docs = loader.load()
    return txt_file_as_loaded_docs

def splitDoc(loaded_docs):
    # split docs into chunks
    splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=10)
    chunked_docs = splitter.split_documents(loaded_docs)
    return chunked_docs

def makeEmbeddings(chunked_docs):
    # Create embeddings and store them in a FAISS vector store
    embedder = HuggingFaceEmbeddings()
    vector_store = FAISS.from_documents(chunked_docs, embedder)
    return vector_store

def askQs(vector_store, chain, q):
    # Ask a question using the QA chain
    similar_docs = vector_store.similarity_search(q)
    resp = chain.run(input_documents=similar_docs, question=q)
    return resp

def loadLLM():
    llm=HuggingFaceHub(repo_id="declare-lab/flan-alpaca-large", model_kwargs={"temperature":0, "max_length":512})
    chain = load_qa_chain(llm, chain_type="stuff")
    return chain


app = Flask(__name__)
@app.route('/sms', methods=['GET', 'POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip() #get inbound text body
    chain = loadLLM()
    LOCAL_ldocs = loadFileFromURL('https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt')
    LOCAL_cdocs = splitDoc(LOCAL_ldocs) #chunked
    LOCAL_vector_store = makeEmbeddings(LOCAL_cdocs)
    LOCAL_resp = askQs(LOCAL_vector_store, chain, inb_msg)
    resp.message(LOCAL_resp)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

On the command line, run python app.py to start the Flask app.

Configure a Twilio Number for the SMS Chatbot

Now, your Flask app will need to be visible from the web so Twilio can send requests to it. ngrok lets you do this. With ngrok installed, run ngrok http 5000 in a new terminal tab in the directory your code is in.

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL with the /sms path in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio phone number is configured so that it maps to your web application server running locally on your computer and your application can run. Text your Twilio number a question relating to the text file and get an answer from that file over SMS!

The complete code can be found here on GitHub.

What's Next for Twilio and LangChain

There's so much you can do with document-based question-answering. You can use a different LLM, use a longer document than a text file containing Lou Gehrig's famous speech, use other types of documents like a PDF or website (here's LangChain's docs on documents), store the embeddings elsewhere, and more. Other than a SMS chatbot, you could create an AI tutor, search engine, automated customer service agent, and more.

Let me know online what you're building!