DEV Community: Agenta AI

Create a Custom Playground to your LLM application

Mahmoud Mabrouk — Wed, 16 Apr 2025 15:59:43 +0000

This article is part of Agenta Launch Week (April 14-18, 2025), where we're announcing new features daily.

What is Agenta?

Agenta is an open-source platform built for AI engineers and teams working with large language models. Our playground, evaluation tools, and deployment solutions help streamline the entire LLM application development process. As an open-source tool, Agenta gives developers complete control over their AI infrastructure while providing enterprise-grade features for teams of all sizes.

Introducing Custom Workflows

LLM applications aren't just single prompts. They include complicated workflow logic with retrieved documents, multiple LLM calls, and various parameters:

RAG applications have embedding models, top-K values, and reranking settings
Agent workflows include reasoning steps and tool calls

When Subject Matter Experts (SMEs) like lawyers or doctors test prompts in isolation, the results often differ from what happens in the complete application or in production. The experts can't access the full application context or modify critical parameters. As a result:

SMEs working on prompts without full application context get unexpected results when those prompts run in the actual application
SMEs cannot change important parameters beyond the prompt (tool descriptions, embedding models, top-k values in RAG)
Time is wasted between developers and SMEs handing over prompt configurations only to find different results
Engineering bottlenecks form when only developers can run evaluations while SMEs can only work on single prompts

This creates unnecessary back-and-forth between developers and domain experts, slowing down development cycles.

What Custom Workflows does

Custom Workflows allows you to connect your application to Agenta with minimal changes. Once connected:

Your team will have access to a playground where they can modify any parameter you expose - prompts, embedding models, top-K values, etc.
You can version, compare, and deploy the entire configuration schema you specify to your application
SMEs can not only run the app with different configs, they can also run evaluations and annotations end-to-end for the app

How to implement it

To connect your application to Agenta, you need to expose an endpoint to your application using our Python SDK. Agenta uses the exposed endpoint to read your configuration schema and generate the appropriate playground to interact with your app, letting your team modify parameters without touching code.

Here's an example of our SDK in action:

from openai import OpenAI
from pydantic import BaseModel, Field
import agenta as ag
from agenta.sdk.types import PromptTemplate

ag.init()

client = OpenAI()
prompt1 = "Summarize the following blog post: {blog_post}"
prompt2 = "Write a tweet based on this: {output_1}"

# We define the configuration of the app using a Pydantic Class
class CoPConfig(BaseModel):
    prompt1: PromptTemplate = Field(default=PromptTemplate(user_prompt=prompt1))
    prompt2: PromptTemplate = Field(default=PromptTemplate(user_prompt=prompt2))

# Agenta will interact with your code using this route
@ag.route("/", config_schema=CoPConfig)
def generate(blog_post: str):
        # config contains the context fetched from the request
    config = ag.ConfigManager.get_from_route(schema=CoPConfig)
    formatted_prompt1 = config.prompt1.format(blog_post=blog_post)
    completion = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": formatted_prompt1}]
    )
    output_1 = completion.choices[0].message.content
    formatted_prompt2 = config.prompt2.format(output_1=output_1)
    completion = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": formatted_prompt2}]
    )
    return completion.choices[0].message.content

You can read the documentation for more details, get started, and watch the announcement for a short demo.

The Technical Solution

Behind the scenes, our SDK creates an endpoint authenticated with your Agenta credentials that Agenta uses to communicate with your app. The SDK uses FastAPI to create an openapi.json that exposes the Pydantic class you configure for your configuration. Agenta uses the openapi.jsonto discover the config schema and generate the UI playground. It then uses that endpoint to communicate with the application.

How to get started

Check out our documentation for step-by-step instructions and more examples.

Custom Workflows is the second of five major features we're releasing this week. Stay tuned for more tomorrow.

⭐ Star Agenta

Consider giving us a star! It helps us grow our community and gets Agenta in front of more developers.

Compare All LLMs in one Playground: Introducing Agenta's AI Model Hub

Mahmoud Mabrouk — Tue, 15 Apr 2025 16:13:17 +0000

This article is part of Agenta Launch Week (April 14-18, 2025), where we're announcing new features daily.

What is Agenta?

Introducing AI Model Hub

When working with LLMs, comparing different models is essential. Our users face several challenges:

You have fine-tuned models for your specific use cases but couldn't use them in Agenta's playground or evaluations.
Many of you run self-hosted models for privacy, security, or cost reasons – through Ollama or dedicated servers – but couldn't connect them to Agenta.
Enterprise users who rely on Azure OpenAI and AWS Bedrock had to switch between platforms when using Agenta.
And teams had no way to share model access without sharing raw API keys, creating security risks and complicating collaboration.

Introducing Model Hub: Your central connection point

Model Hub gives you a central place in Agenta to manage all your model connections:

Connect any model: Azure OpenAI, AWS Bedrock, self-hosted models, fine-tuned models – any model with an OpenAI-compatible API.
Configure once, use everywhere: Set up your models once and use them in playground experiments, evaluations, and deployments.
Secure team sharing: Give your team access to models without sharing API keys. Control permissions per model.
Consistent experience: Use the same Agenta interface for all your models.

How it works

Model Hub is under Configuration in your Agenta interface:

Select your provider (OpenAI, Azure, AWS, Ollama, etc.)
Enter your API keys
Connect to your self-hosted models
Share access with teammates

Once configured, your models appear throughout Agenta. Run comparisons in the playground, test performance in evaluations, and deploy with confidence – using models from any provider.

Security is built-in

When building Model Hub, we prioritized the security of your API keys and credentials:

Your model keys and credentials are encrypted at rest in our database and protected in transit with TLS encryption. We never log these sensitive details, and they're only held in memory for the minimum time needed to process requests.

Currently, access to models is managed at the project level, with all team members working on a project able to use the configured models. Every access to these credentials requires proper authentication with tokens tied to specific users and projects.

For Business and Enterprise customers, Role-Based Access Control (RBAC) provides additional security by letting you control exactly who on your team can view or modify model configurations.

We've designed this system from the ground up with security best practices, ensuring your valuable API keys and model access credentials remain protected.

What's next?

The AI Model Hub is just the first feature in our Launch Week. Four more announcements are coming in the next few days to improve your LLM development workflow.

Ready to try it? Check out Agenta on GitHub or log in to your Agenta account to set up your Model Hub.

Stay tuned for tomorrow's announcement!

⭐ Star Agenta

Consider giving us a star! It helps us grow our community and gets Agenta in front of more developers.

Build an AI code review assistant with v0.dev, litellm and Agenta

Mahmoud Mabrouk — Mon, 13 Jan 2025 19:49:47 +0000

The code for this tutorial is available here.

Ever wanted your own AI assistant to review pull requests? In this tutorial, we'll build one from scratch and take it to production. We'll create an AI assistant that can analyze PR diffs and provide meaningful code reviews—all while following LLMOps best practices.

You can try out the final product here. Just provide the URL to a public PR and receive a review from our AI assistant.

What we'll build

This tutorial walks through creating a production-ready AI assistant. Here's what we'll cover:

Writing the Code: Fetching the PR diff from GitHub and calling an LLM using LiteLLM.
Adding Observability: Instrumenting the code with Agenta to debug and monitor our app.
Prompt Engineering: Refining prompts and comparing different models using Agenta's playground.
LLM Evaluation: Using LLM-as-a-judge to evaluate prompts and select the best model.
Deployment: Deploying the app as an API and building a simple UI with v0.dev.

Let's get started!

Writing the core logic

Our AI assistant's workflow is straightforward: When given a PR URL, it fetches the diff from GitHub and passes it to an LLM for review. Let's break this down step by step.

First, we'll fetch the PR diff. GitHub provides this in an easily accessible format:

https://patch-diff.githubusercontent.com/raw/{owner}/{repo}/pull/{pr_number}.diff

Here's a Python function to retrieve the diff:

def get_pr_diff(pr_url):
    """
    Fetch the diff for a GitHub Pull Request given its URL.

    Args:
        pr_url (str): Full GitHub PR URL (e.g., https://github.com/owner/repo/pull/123)

    Returns:
        str: The PR diff text
    """
    pattern = r"github\.com/([^/]+)/([^/]+)/pull/(\d+)"
    match = re.search(pattern, pr_url)

    if not match:
        raise ValueError("Invalid GitHub PR URL format")

    owner, repo, pr_number = match.groups()

    api_url = f"https://patch-diff.githubusercontent.com/raw/{owner}/{repo}/pull/{pr_number}.diff"

    headers = {
        "Accept": "application/vnd.github.v3.diff",
        "User-Agent": "PR-Diff-Fetcher"
    }

    response = requests.get(api_url, headers=headers)
    response.raise_for_status()

    return response.text

Next, we'll use LiteLLM to handle our interactions with language models. LiteLLM provides a unified interface for working with various LLM providers—making it easy to experiment with different models later:

prompt_system = """
You are an expert Python developer performing a file-by-file review of a pull request. You have access to the full diff of the file to understand the overall context and structure. However, focus on reviewing only the specific hunk provided.
"""

prompt_user = """
Here is the diff for the file:
{diff}

Please provide a critique of the changes made in this file.
"""

def generate_critique(pr_url: str):
    diff = get_pr_diff(pr_url)
    response = litellm.completion(
        model=config.model,
        messages=[
            {"content": config.system_prompt, "role": "system"},
            {"content": config.user_prompt.format(diff=diff), "role": "user"},
        ],
    )
    return response.choices[0].message.content

Adding observability

Observability is crucial for understanding and improving LLM applications. It helps you track inputs, outputs, and the information flow, making debugging easier. We'll use Agenta for this purpose.

Agenta is an open-source LLMOps platform that provides you with all the tools needed to build production-ready LLM-powered applications. It offers a centralized environment to manage prompts, instrument applications for tracking inputs and outputs, and run evaluations to assess result quality. You can signup for free for the cloud platform or self-host our platform.

First, we initialize Agenta and set up LiteLLM callbacks. The callback automatically instruments all the LiteLLM calls:

import agenta as ag

ag.init()
litellm.callbacks = [ag.callbacks.litellm_handler()]

Then we add instrumentation decorators to both functions (generate_critique and get_pr_diff) to capture their inputs and outputs. Here's how it looks for the generate_critique function:

@ag.instrument()
def generate_critique(pr_url: str):
    diff = get_pr_diff(pr_url)
    config = ag.ConfigManager.get_from_route(schema=Config)
    response = litellm.completion(
        model=config.model,
        messages=[
            {"content": config.system_prompt, "role": "system"},
            {"content": config.user_prompt.format(diff=diff), "role": "user"},
        ],
    )
    return response.choices[0].message.content

To set up Agenta, we need to set the environment variable AGENTA_API_KEY (which you can find here) and optionally AGENTA_HOST if we're self-hosting.

We can now run the app and see the traces in Agenta.

Creating an LLM playground

Now that we have our POC, we need to iterate on it and make it production-ready. This means experimenting with different prompts and models, setting up evaluations, and versioning our configuration.

Agenta custom workflow feature lets us create an IDE-like playground for our AI-assistant workflow.

We'll add a few lines of code to create an LLM playground for our application. This will enable us to version the configuration, run end-to-end evaluations, and deploy the last versions in one click.

Here's the modified code:

import requests
import re
import sys
from urllib.parse import urlparse
from pydantic import BaseModel, Field
from typing import Annotated
import agenta as ag
import litellm
from agenta.sdk.assets import supported_llm_models

ag.init()

litellm.drop_params = True
litellm.callbacks = [ag.callbacks.litellm_handler()]

prompt_system = """
You are an expert Python developer performing a file-by-file review of a pull request. You have access to the full diff of the file to understand the overall context and structure. However, focus on reviewing only the specific hunk provided.
"""

prompt_user = """
Here is the diff for the file:
{diff}

Please provide a critique of the changes made in this file.
"""

# highlight-start
class Config(BaseModel):
    system_prompt: str = prompt_system
    user_prompt: str = prompt_user
    model: Annotated[str, ag.MultipleChoice(choices=supported_llm_models)] = Field(default="gpt-3.5-turbo")
# highlight-end

# highlight-next-line
@ag.route("/", config_schema=Config)
@ag.instrument()
def generate_critique(pr_url:str):
    diff = get_pr_diff(pr_url)
    # highlight-next-line
    config = ag.ConfigManager.get_from_route(schema=Config)
    response = litellm.completion(
        model=config.model,
        messages=[
            {"content": config.system_prompt, "role": "system"},
            {"content": config.user_prompt.format(diff=diff), "role": "user"},
        ],
    )
    return response.choices[0].message.content

Let's break it down:

Defining the configuration and the layout of the playground

from pydantic import BaseModel, Field
from typing import Annotated
from agenta.sdk.assets import supported_llm_models

class Config(BaseModel):
    system_prompt: str = prompt_system
    user_prompt: str = prompt_user
    model: Annotated[str, ag.MultipleChoice(choices=supported_llm_models)] = Field(default="gpt-3.5-turbo")

To integrate our code in Agenta, we first define the configuration schema. This helps Agenta understand the inputs and outputs of our function and create a playground for it. The configuration defines the playground's layout: the system and user prompts (str) appear as text areas, and the model appears as a multi-select dropdown (note that supported_llm_models is variable in the Agenta SDK that contains a dictionary of providers and their supported models).

Creating the entrypoint

We'll adjust our function to use the configuration. @ag.route creates an API endpoint for our function with the configuration schema we defined. This endpoint will be used by Agenta's playground, evaluation, and the deployed API to interact with our application.

ag.ConfigManager.get_from_route(schema=Config) fetches the configuration from the request sent to that API endpoint.

# highlight-next-line
@ag.route("/", config_schema=Config)
@ag.instrument()
def generate_critique(pr_url: str):
    diff = get_pr_diff(pr_url)
    # highlight-next-line
    config = ag.ConfigManager.get_from_route(schema=Config)
    response = litellm.completion(
        model=config.model,
        messages=[
            {"content": config.system_prompt, "role": "system"},
            {"content": config.user_prompt.format(diff=diff), "role": "user"},
        ],
    )
    return response.choices[0].message.content

Serving the application with Agenta

We can now add the application to Agenta. Here's what we need to do:

Run agenta init and specify our app name and API key
Run agenta variant serve app.py

The last command builds and serves your application, making it accessible through Agenta's playground. There, you can run the application end-to-end by giving it a PR URL and getting the review generated by our LLM.

Evaluating using LLM-as-a-judge

To evaluate the quality of our AI assistant's reviews and compare prompts and models, we need to set up evaluation.

First, we'll create a small test set with publicly available PRs.

Next, we'll set up an LLM-as-a-judge to evaluate the quality of the reviews.

To do this, navigate to the evaluation view, click on "Configure evaluators", then "Create new evaluator" and select "LLM-as-a-judge".

We'll get a playground where we can test different prompts and models for our human evaluator. We use the following system prompt:

You are an evaluator grading the quality of a PR review.
CRITERIA:
Technical Accuracy

The reviewer identifies and addresses technical issues, ensuring the PR meets the project's requirements and coding standards.
Code Quality

The review ensures the code is clean, readable, and adheres to established style guides and best practices.
Functionality and Performance

The reviewer provides clear, actionable, and constructive feedback, avoiding vague or unhelpful comments.
Timeliness and Thoroughness

The review is completed within a reasonable timeframe and demonstrates a thorough understanding of the code changes.

SCORE:
-The score should be between 0 and 10
-A score of 10 means that the answer is perfect. This is the highest (best) score.
A score of 0 means that the answer does not any of of the criteria. This is the lowest possible score you can give.

ANSWER ONLY THE SCORE. DO NOT USE MARKDOWN. DO NOT PROVIDE ANYTHING OTHER THAN THE NUMBER

For the user prompt, we'll use the following:

LLM APP OUTPUT: {prediction}

Note that the evaluator accesses the LLM app's output through the {prediction} variable. We can iterate on the prompt and test different models in the evaluator test view.

With our evaluator set up, we can run experiments and compare different prompts and models. In the playground, we can create multiple variants and run batch evaluations using the pr-review-quality LLM-as-a-judge.

After comparing models, we found similar performance across the board. Given this, we chose GPT-3.5-turbo for its optimal balance of speed and cost.

Deploying to production

Deployment is straightforward with Agenta:

Navigate to the overview page
Click the three dots next to your chosen variant
Select "Deploy to Production"

This gives you an API endpoint ready to use in your application.

Agenta works in both proxy mode and prompt management mode. You can either use Agenta's endpoint or deploy your own app and use the Agenta SDK to fetch the production configuration.

Building the frontend

For the frontend, we used v0.dev to quickly generate a UI. After providing our API endpoint and authentication requirements, we had a working UI in minutes. You can try it yourself: PR Review Assistant.

What's next?

With our AI assistant in production, Agenta continues to provide observability tools. We can continue enhancing it by:

Refine the Prompt: Improve the language to get more precise critiques.
Add More Context: Include the full code of changed files, not just the diffs.
Handle Large Diffs: Break down extensive changes and process them in parts.

Conclusion

In this tutorial, we've:

Built an AI assistant that reviews pull requests.
Implemented observability and prompt engineering using Agenta.
Evaluated our assistant with LLM-as-a-judge.
Deployed the assistant and connected it to a frontend.

One last thing

If you liked this tutorial, and you to learn more about AI engineering and how to build production-ready AI applications. Follow our page, and start our repository.

🔥🤖 Build an AI-Powered Discord Bot to Recommend HackerNews Posts using OpenAI, Novu and Agenta 🚀

Mahmoud Mabrouk — Wed, 06 Sep 2023 21:34:58 +0000

TL;DR

In this tutorial, you'll learn how to create an AI agent that alerts you about relevant Hacker News posts tailored to your interests. The agent sends Discord notifications whenever a post matches your criteria.

We'll write the code using Python, use Beautifulsoup for web scraping, use OpenAI with Agenta for building the AI, and Novu for Slack notifications.

Goal? No more endless scrolling on Hacker News. Let the AI bring worthy posts to you!

Agenta: The Open-source LLM app builder 🤖

A bit about us: Agenta is an end-to-end open-source LLM app builder. It enables you to quickly build, experiment, evaluate, and deploy LLM apps as APIs. You can use it by writing code in Langchain or any other framework, or directly from the UI.

Here's the plan 📝

We will write a script that does the following:

Scrape the first five Hacker News pages for post titles using Python and Beautifulsoup.
Use Agenta and GPT3.5 to categorize posts based on your interests.
Send compelling posts to your Slack channel.

Setting things up:

To get started, let's create a project folder and run Poetry. If you aren't familiar with Poetry, you should check it out as it provides an alternative to virtual environments that is much easier to use.

mkdir hnbot; cd hnbot/
poetry init

This command will guide you through creating your pyproject.toml config.


Would you like to define your main dependencies interactively? (yes/no) [yes] yes

Package to add or search for (leave blank to skip): novu
Add a package (leave blank to skip): beautifulsoup4


Do you confirm generation? (yes/no) [yes] yes

Follow the prompts to set up your project. Don't forget to install novu and beautifulsoup4.

Now let's create the folder for our package and initialize the poetry environement

% mkdir hn_bot; cd hn_bot
% cd hn_bot 
% poetry shell
(hn-bot-py3.9) (base) % poetry install

Now, we have a local environment where:

All our requirements are installed.
We have a Python package called hn_bot that is in our Python lib.

This means if we have multiple files in our library, we can import them using import hn_bot.module_name.

Scraping Hacker News Posts

Scraping the Hacker News page is straightforward since it does not use any complicated JavaScript. The pages are located at https://news.ycombinator.com/?p=pagenumber.

To find the titles and links on the page, we just need to open the web browser and access the dev console. Once there, we can check if there are any elements we can use to locate the titles and links. Luckily, it seems that every post is a span with the class "titleline."

We can use this to extract information from a single page. Let's write a function that extracts titles and links from Hacker News.

# hn_scraper.py
from typing import Dict, List

import requests
from bs4 import BeautifulSoup

def scrape_page(page_number: str) -> List[Dict[str, str]]:
    response = requests.get(f"https://news.ycombinator.com/news?p={page_number}")
    yc_web_page = response.text
    soup = BeautifulSoup(yc_web_page, 'html.parser')

    articles = []

    for article_tag in soup.find_all(name="span", class_="titleline"):
        title = article_tag.getText()
        link = article_tag.find("a")["href"]
        articles.append({"title": title, "link": link})

    return articles

We can test it by adding a print(scrape_page(1)) at the end of the script and running it on shell:

 % python hn_scraper.py
(['Linux Network Performance Parameters Explained (github.com/leandromoreira)', 'Double Commander – Changes in version 1.1.0 (github.com/doublecmd)', 'If You’ve Got a New Car, It’s a Data Privacy Nightmare (gizmodo.com)', 'Ask HN: I’m an FCC Commissioner proposing regulation of IoT security updates', 'Gcsfuse: A user-space file system for interacting with Google Cloud S

Congratulations!🎉 Now we have a script that scrapes post titles from HackerNews

Creating the AI Agent 🤖

Now that we have a list of posts, we need to use OpenAI gpt models to classify whether they are relevant based on the user's interests. For this, we are going to use Agenta.

Agenta allows you to create LLM apps from code or from the UI. Since our LLM app today is quite simple, we will create it from the UI.

Agenta can be self-hosted, however to get started quickly we'll use demo.agenta.ai.

Since our LLM app today is quite simple, we will just go ahead and create it from the UI.

You can self host agenta (Check out docs for that here (https://docs.agenta.ai/installation/local-installation/local-installation) or use the cloud-hosted demo. To get started quickly we'll do the later.

Let's go to demo.agenta.ai and login.

First, let's create a new app by clicking on "Create New App".

Then we select start from template

And use a single prompt template

Doing some Prompt Engineering In Agenta 🪄 ✨

Now we have a playground for creating the app.

First, let's add the inputs for our application. In this case, we will be using "title" for the Hacker News title and "interests" for the user's interests.

Next, we need to do a little prompt engineering. Since we are using gpt3.5 (the cheapest variant in OpenAI). It takes two messages: the system message and the user message. We can use the system message to guide the language model to answer in a certain way, while the prompt prompts the human to give the parameters of the task.

In this case, I tried a simple prompt for the system that ensures the answer is either "True" or "False." For the human prompt, I just asked the system to classify. Note that we used the fstring usual format to inject the inputs that we have added into the prompt.

Now we can then test the application with some examples of Hacker News titles:

Agenta provides tools to systematically evaluate applications and optimize prompts, parameters, and workflows (in case we are using something more complex with embeddings and retrieval augmented generations). However, in this case, such evaluation is unnecessary. The app itself is very simple, and gpt3.5 is able to solve the classification problem with minimal effort.

Let's save our changes

Then deploy the application as an API.

For this we jump to the endpoints menu and copy paste the code snippet to our code.

Wrapping it up 🌯

Now, we can create a function based on this code snippet.

# llm_classifier.py
import requests
import json

def classify_post(title: str, interests: str) -> bool:

    url = "https://demo.agenta.ai/64f1d1aefeebd024bbdb1ea4/hn_bot/v1/generate"
    params = {
        "inputs": {
            "title": title,
            "interests": interests
        },
        "temperature": 0,
        "model": "gpt-3.5-turbo",
        "maximum_length": 100,
        "prompt_system": "You are an expert in classification. You answer only with True or False.",
        "prompt_human": "Classify whether this hackernews post is interesting for someone with the following interests:\nHacker news post title: {title}\nInterests: {interests}",
        "stop_sequence": "\n",
        "top_p": 1,
        "frequence_penalty": 0,
        "presence_penalty": 0
    }

    response = requests.post(url, json=params)

    data = response.json()

    return bool(data)

Sending a Discord message 🎮

First we need to create a new channel in Discord

Next we need to create a webhook and copy the url

Now we need to setup the integration in Novu. For this we have to go to the Integration Store, click on “Add a provider”, select Discord, and don't forget to activate it!

Last, we need to create a workflow that triggers the message to be sent to our Discord. We will add the {{content}} variable to the message which we will later inject using the code.

Write the messaging function

Now it's time to write the message that will trigger the workflow

# novu_bot
from novu.config import NovuConfig
from novu.api import EventApi
from novu.api.subscriber import SubscriberApi
from novu.dto.subscriber import SubscriberDto

NovuConfig().configure("https://api.novu.co", "YOUR_API_KEY")
webhook_url = "..." # the webhook url we got from Discord

def send_message(msg):
    your_subscriber_id = "123"  # Replace this with a unique user ID.

    # Define a subscriber instance
    subscriber = SubscriberDto(
        subscriber_id=your_subscriber_id,
        email="abc@gmail.com",
        first_name="John",
        last_name="Doe"
    )

    SubscriberApi().create(subscriber)
    SubscriberApi().credentials(subscriber_id=your_subscriber_id,
                                provider_id="discord",

    EventApi().trigger(
        name="slackbot",  # The trigger ID of the workflow. It can be found on the workflow page.
        recipients=your_subscriber_id,
        payload={},  # Your Novu payload goes here
    )

Putting everything together

Now we're ready to assemble all the elements to get our AI assistant running.

Let's create an app.py file in which we first call the scraper, then the LLM classifier, and finally send a message with the interesting posts.

from hn_bot import hn_scraper, llm_classifier, novu_bot
import schedule
import time

interests = "LLMs, LLMOps, Python, Infrastructure, Tennis, MLOps, Data science, AI, startups, Computational Biology"

def main():
    novu_bot.send_message("Interesting posts at HackerNews:\n")

    posts = hn_scraper.scrape_page("1")
    for title, url in posts:
        if llm_classifier.classify_post(title, interests) == "True":
            novu_bot.send_message(f"{title}\n{url}")

if __name__ == "__main__":
    main()

Et voila, we have all the interesting posts coming up in our Discord.

Finally let's schedule this to run each hour ⏰

We would like to run the script to check new posts each hour. For this we need to add the python library schedule

from hn_bot import hn_scraper, llm_classifier, novu_bot
from time import sleep
interests = "LLM, LLMOps, MLOps, Data science, AI, startups"

done_post_titles = []


def main():
    novu_bot.send_message("Interesting posts at HackerNews:\n")

    posts = []
    for i in range(1, 5):
        posts += hn_scraper.scrape_page(i)
    for post in posts:
        title = post["title"]
        url = post["link"]
        if llm_classifier.classify_post(title, interests) and title not in done_post_titles:
            done_post_titles.append(title)
            novu_bot.send_message(f"{title}\n{url}")


if __name__ == "__main__":
    while True:
        main()
        sleep(3600)

Congratulations on making it thus far!🎉 You've now got an automated AI assistant keeping an eye on Hacker News for you.

Summary 📜

In this tutorial, we've built an AI-powered assistant to keep you in the loop with relevant Hacker News posts. You should have learned:

How to use Beautifulsoup for scraping hackernews
How to create an LLM app based on one prompt using Agenta and OpenAI gpt3.5
How to send notifications on Discord using Novu

You can check the code at this https://github.com/Agenta-AI/blog/tree/main/hackernews-bot

Thanks for reading!