DEV Community: Wilbert Misingo

How I Transformed How My Business Interacts with and Collects Data from Customers Using WhatsApp Forms-like Features

Wilbert Misingo — Thu, 12 Dec 2024 09:10:25 +0000

Introduction

With more that 2 Billion users online, WhatsApp have revolutionize the way businesses handle, manage and interact with their customers.

Despite the efforts, yet most of them fell short due to having a long and tiresome flow of conversation just in order to correct some information, thus limiting the user to His/her other needs.

With that in mind, I have developed a tool called "WhatsApp flows" which utilizes a feature called "WhatsApp flows" from Meta, which enable businesses to embed/add form like interactive elements/components into their conversation flow, enabling the business to collect a desired information by only sending a single message to the user.

Check the demo video below:

whatsapp flows.mp4 - Google Drive

drive.google.com

So, technically. . .

WhatsApp Flows is a Python library designed to streamline the creation, management, and deployment of automated workflows for WhatsApp. Whether you're building interactive chatbots, managing business messaging, or orchestrating complex message flows, this library provides a developer-friendly toolkit for efficient WhatsApp automation.

Why Use WhatsApp Flows?

Ease of Use: Simplifies WhatsApp Business API interactions with a high-level abstraction.
Flexibility: Supports creating both endpoint-based and non-endpoint flows for various use cases.
Scalability: Seamlessly handles complex workflows for businesses of all sizes.
Integration Ready: Works effortlessly with popular frameworks like FastAPI and Flask.

Installation

You can install the library using pip:

pip install whatsapp-flows

WhatsApp Flows Guide

This guide outlines the steps to create and manage WhatsApp flows using the Meta Developers platform. There are two types of WhatsApp flows:

Flows with Endpoints: These flows interact with external APIs to fetch or send dynamic data.
Flows without Endpoints: These flows operate independently and do not require external API interactions.

In this guide, we'll focus on creating a WhatsApp flow app without endpoints. Follow the steps below to set up your flow and deploy it successfully.

Steps to Create a WhatsApp Flow App Without Endpoints

1. Create an App on Meta Developers Account

To begin, create an app on the Meta Developers platform. This app will serve as the foundation for managing your WhatsApp flows.

2. Add a Phone Number

Add a phone number to your app. This number will be associated with your WhatsApp Business account and used for sending and receiving messages.

3. Enable Messaging Permissions

Ensure your app has the necessary messaging permissions enabled for interacting with WhatsApp messaging features.

4. Create a Business on Meta Business Account

Create a business account on Meta Business. This links your WhatsApp Business with your Meta Developers app.

5. Verify Your Business

Complete the verification process for your business to gain access to additional features and permissions.

6. Request Advanced Permissions

Request the following advanced permissions for your Meta Developers app:

whatsapp_business_management: Manage WhatsApp Business accounts, including creating flows.
whatsapp_business_messaging: Send and receive messages via the WhatsApp Business API.
whatsapp_business_phone_number: Access WhatsApp Business phone numbers.
business_management: Manage business assets like ad accounts and pages.
pages_messaging: Optional if flows interact with Facebook Pages for messaging.

7. Obtain Necessary Credentials

Gather the following credentials from your Meta Developers account. These will configure your WhatsApp flows:

WHATSAPP_BUSINESS_VERIFY_TOKEN
WHATSAPP_BUSINESS_PHONE_NUMBER_ID
WHATSAPP_BUSINESS_ACCESS_TOKEN
WHATSAPP_BUSINESS_ACCOUNT_ID

8. Create a Flow on Flow Development Playground

Design your WhatsApp flow using the Flow Development Playground.

To create a flow programmatically:

from whatsapp_flows import FlowsManager
import os
from dotenv import load_dotenv

load_dotenv()

flows_manager = FlowsManager(
    whatsapp_access_token=os.getenv("WHATSAPP_BUSINESS_ACCESS_TOKEN"),
    whatsapp_account_id=os.getenv("WHATSAPP_BUSINESS_ACCOUNT_ID"),
    whatsapp_phone_number_id=os.getenv("WHATSAPP_BUSINESS_PHONE_NUMBER_ID"),
)

try:
    response = flows_manager.create_flow(flow_name="TEST FLOW")
    print(response)
except Exception as e:
    print(e)

9. Deploy the Middleware/Webhook

Deploy the middleware or webhook to handle flow execution.

10. Configure the Webhook URL

Configure the webhook URL in your Meta Developers account. This links your flow to WhatsApp messaging.

11. Create and Manage Flows

Listing Flows:

try:
    response = flows_manager.list_flows()
    print(response)
except Exception as e:
    print(e)

Getting Flow Details:

try:
    response = flows_manager.get_flow_details(flow_id="1234567890")
    print(response)
except Exception as e:
    print(e)

12. Upload Your Flow JSON

Upload your flow JSON using the Flow Development Playground or programmatically:

SYSTEM_PATH = os.getcwd()
FLOW_JSON_FILE_PATH = os.path.join(SYSTEM_PATH, "data/flow.json")

try:
    response = flows_manager.upload_flow_json(
        flow_id="1234567890", flow_file_path=FLOW_JSON_FILE_PATH
    )
    print(response)
except Exception as e:
    print(e)

13. Test Your Flow

Test your flow programmatically:

try:
    response = flows_manager.simulate_flow(flow_id="1234567890")
    print(response)
except Exception as e:
    print(e)

14. Publish Your Flow

Publish your flow:

try:
    response = flows_manager.publish_flow(flow_id="1234567890")
    print(response)
except Exception as e:
    print(e)

15. Sending Published and Unpublished Flows

Send a Published Flow:

try:
    response = flows_manager.send_published_flow(
        flow_id="1234567890",
        flow_cta_header_text="Amazing Shop!!",
        flow_cta_body_text="Hello, welcome to our general shop!!",
        flow_cta_footer_text="Click the button to continue.",
        flow_cta_button_text="START SHOPPING",
        recipient_phone_number="255753456789"
    )
    print(response)
except Exception as e:
    print(e)

Send an Unpublished Flow:

try:
    response = flows_manager.send_unpublished_flow(
        flow_id="1234567890",
        flow_cta_header_text="Amazing Shop!!",
        flow_cta_body_text="Hello, welcome to our general shop!!",
        flow_cta_footer_text="Click the button to continue.",
        flow_cta_button_text="START SHOPPING",
        recipient_phone_number="255753456789"
    )
    print(response)
except Exception as e:
    print(e)

16. Update or Delete Flows

Update Flow JSON:

try:
    response = flows_manager.update_flow_json(
        flow_id="1234567890", flow_file_path=FLOW_JSON_FILE_PATH
    )
    print(response)
except Exception as e:
    print(e)

Delete a Flow:

try:
    response = flows_manager.delete_flow(flow_id="1234567890")
    print(response)
except Exception as e:
    print(e)

Conclusion

If you feel like you want to contribute, request a feature or reporting a bug, feel free to check me.

NB: This is the link to the project GitHub repository.

Create the fastest and precise invoice data extractor for structural output using AI

Wilbert Misingo — Tue, 06 Aug 2024 15:15:09 +0000

Using LlamaExtract with Pydantic Models for Shop Receipts Extraction

In this article, we'll explore how to use LlamaExtract incorporated with schemas from Pydantic models inorder to extract structured data from shop receipts. This approach helps in organizing receipt information systematically, making it easier to analyze and manage.

Setup

First, ensure you have the llama-extract client library installed. Use the following command:

pip install llama-extract pydantic

Note: If you see a notice about updating pip, you may update it using the command provided.

First, login and get an api-key for free from Llama Index Cloud

Set up the environment variable for your LlamaExtract API key:

import os

os.environ["LLAMA_CLOUD_API_KEY"] = "YOUR LLAMA INDEX CLOUD API HERE"

Load Data

For this example, let's assume we have a dataset of shop receipts in PDF format. Place these files in a directory named receipts.

DATA_DIR = "data/receipts"
fnames = os.listdir(DATA_DIR)
fnames = [fname for fname in fnames if fname.endswith(".pdf")]
fpaths = [os.path.join(DATA_DIR, fname) for fname in fnames]
fpaths

The output should list the file paths of the receipts:

['data/receipts/receipt.pdf']

Define a Pydantic Model

We'll define our data model using Pydantic, this would tell the API which fields/data we are expecting or want to extract from the PDF. For shop receipts, we might be interested in extracting the store name, date, total amount, and list of items purchased.

from pydantic import BaseModel
from typing import List

class Item(BaseModel):
    name: str
    quantity: int
    price: float

class Receipt(BaseModel):
    store_name: str
    date: str
    total_amount: float
    items: List[Item]

Create Schema

Now, we can use the Pydantic model to define an extraction schema in LlamaExtract.

from llama_extract import LlamaExtract

extractor = LlamaExtract(verbose=True)
schema_response = await extractor.acreate_schema("Receipt Schema", data_schema=Receipt)
schema_response.data_schema

The output schema should resemble the following:

{
    'type': 'object',
    '$defs': {
        'Item': {
            'type': 'object',
            'title': 'Item',
            'required': ['name', 'quantity', 'price'],
            'properties': {
                'name': {'type': 'string', 'title': 'Name'},
                'quantity': {'type': 'integer', 'title': 'Quantity'},
                'price': {'type': 'number', 'title': 'Price'}
            }
        }
    },
    'title': 'Receipt',
    'required': ['store_name', 'date', 'total_amount', 'items'],
    'properties': {
        'store_name': {'type': 'string', 'title': 'Store Name'},
        'date': {'type': 'string', 'title': 'Date'},
        'total_amount': {'type': 'number', 'title': 'Total Amount'},
        'items': {
            'type': 'array',
            'title': 'Items',
            'items': {'$ref': '#/$defs/Item'}
        }
    }
}

Run Extraction

With the schema defined, we can now extract structured data from our receipt files. By specifying Receipt as the response model, we ensure the extracted data is validated and structured.

responses = await extractor.aextract(
    schema_response.id, fpaths, response_model=Receipt
)

You can access the raw JSON output if needed:

data = responses[0].data
print(data)

Example JSON output:

{
    'store_name': 'ABC Electronics',
    'date': '2024-08-05',
    'total_amount': 123.45,
    'items': [
        {'name': 'Laptop', 'quantity': 1, 'price': 999.99},
        {'name': 'Mouse', 'quantity': 1, 'price': 25.00},
        {'name': 'Keyboard', 'quantity': 1, 'price': 50.00}
    ]
}

Conclusion

In this article, we demonstrated how to use LlamaExtract with Pydantic models for defining data schemas and extract structured data from shop receipts. This approach ensures that the extracted information is well-organized and validated, making it easier to handle and analyze.

This can also be use for many cases, invoices, receipts, reports etc.

Happy Coding!!

Do you have a project 🚀 that you want me to assist you email me🤝😊: wilbertmisingo@gmail.com

Have a question or wanna be the first to know about my posts:-
Follow ✅ me on GitHub
Follow ✅ me on LinkedIn 💼
Follow ✅ me on Twitter/X 𝕏

How you can create your own custom chatbot with your own custom data using Google Gemini API all for free

Wilbert Misingo — Mon, 12 Feb 2024 13:25:05 +0000

INTRODUCTION

Up until now, the conventional method for building multimodal models involves learning independent parts for various modalities and then piecing them together to approximate some of this functionality. Certain activities, like describing visuals, may be areas in which these models excel, but they have trouble with more conceptual and sophisticated reasoning.

As to this moment, Google's most adaptable model to date, Gemini can operate well on a wide range of platforms, including mobile phones and data centers. Its cutting-edge features will greatly improve how developers and business clients use AI to create and grow.

Google have optimized Gemini 1.0, their first version, for three different sizes:

Gemini Ultra, the largest and most capable model for highly complex tasks.
Gemini Pro, the best model for scaling across a wide range of tasks.
Gemini Nano, the most efficient model for on-device tasks.

This makes Gemini one of the most capable models in the world and thus creating an opportunity for people to explore it in many ways. With respect to that google have also released a huge free tier that could really be helpful to help people to create some cool stuff.

In this article, you are going to learn how to create your own custom chatbot using your own data using the Google Gemini API free tier.

IMPLEMENTATION

Step 01: Getting your API Key

To get started you first need to create an API key that would be used to reference the model, to create your API key you need to signup and create a new key at the Google AI Studio at Google Maker suite platform.

Step 02: Installing Libraries

To build a chatbot, we need to use some Python libraries that are specifically designed for natural language processing and machine learning. These would help to facilitate the implementation of the chatbot.

pip install -q llama_index google-generativeai chromadb pypdf transformers chromadb

Step 03: Importing Libraries

To begin, we need to import the necessary libraries and modules that will be used throughout the chatbot creation process. The code snippet below demonstrates the required imports.


from llama_index import SimpleDirectoryReader, VectorStoreIndex
from llama_index.llms import Gemini
from IPython.display import Markdown, display
from llama_index import ServiceContext
from llama_index.vector_stores import ChromaVectorStore
from llama_index.storage.storage_context import StorageContext
from llama_index.prompts import PromptTemplate
import chromadb
import os

Step 04: Loading data from the knowledge base

To make the chatbot knowledgeable about your anything, you need to load your desired documents into the chatbot's index. The documents can be in different formats, in this demo all the documents were in pdf. The code snippet below demonstrates loading the data from a specified directory.

documents = SimpleDirectoryReader("./data").load_data()

Step 05: Innitializing the data embedding database

ChromaDB, a versatile tool for storing vector representations and embeddings, serves as the backbone of our system. Initializing ChromaDB involves creating a client and establishing a collection for storing document embeddings. This sets the stage for efficient storage and retrieval of vector representations.


db = chromadb.PersistentClient(path="./embeddings/chroma_db")
chroma_collection = db.get_or_create_collection("quickstart")

Step 06: Innitializing the model

Initializing Gemini and creating a service context involves setting up the necessary environment and defining how the model is going to interact and process both user inputs and data source.


os.environ['GOOGLE_API_KEY'] = 'PUT YOUR GOOGLE API KEY HERE'
llm = Gemini()
service_context = ServiceContext.from_defaults(llm=llm, chunk_size=800, chunk_overlap=20, embed_model="local")

Step 7: Creating Vector Store Index

With the foundational components in place, the next step is to create the vector store index from the loaded documents. This process involves indexing the documents using the specified vector store and service context.


index = VectorStoreIndex.from_documents(documents, storage_context=storage_context, service_context=service_context
)

Step 08: Defining Prompt Template and Configuring Query Engine

To facilitate question-answering, a prompt template is defined. This helps the bot to understand how it should interact with the user according to how you set, giving it tone, role etc. .The query engine is then configured to leverage this template. This step lays the groundwork for engaging in meaningful interactions with the indexed data.


template = (
    "We have provided context information below. \n"
    "---------------------\n"
    "{context_str}"
    "\n---------------------\n"
    "Given this information, please answer the question: {query_str}\n"
)

qa_template = PromptTemplate(template)

query_engine = index.as_query_engine(text_qa_template=qa_template)

Step 09: Performing a query

The culmination of our journey involves performing a sample query and displaying the result.


response = query_engine.query("What its the shape of the earth?")
print(response)

CONCLUSION

The implementation of a text-to-vector indexing system using ChromaDB, Gemini, and VectorStore opens up a realm of possibilities for advanced NLP applications. This comprehensive guide serves as a foundation for building sophisticated text-based applications. As you continue your exploration, feel free to experiment with different document sets, query templates, and parameters to tailor the system to your specific requirements.

Happy coding!

Do you have a project 🚀 that you want me to assist you email me🤝😊: wilbertmisingo@gmail.com
Have a question or wanna be the first to know about my posts:-
Follow ✅ me on GitHub
Follow ✅ me on Twitter/X 𝕏
Follow ✅ me on LinkedIn 💼

How to create a perfectly YouTube video transcriber using AI

Wilbert Misingo — Tue, 26 Dec 2023 12:57:46 +0000

INTRODUCTION

YouTube has become an unparalleled resource for information, entertainment, and educational content. However, extracting the spoken words from videos programmatically can be a challenge.

IMPLEMENTATION

In this article, we'll explore how to harness the power of Python to read and transcribe YouTube video content using the YouTube Transcript API library and the machine learning models manager library called llama index.

Step 01: installing libraries and modules

Before diving into the code, make sure you have the required packages installed. You can do this by running the following command on the command line terminal.

pip install youtube_transcript_api llama_index

Step 02: importing libraries and modules

Now, let's import the crucial modules and components needed for
implementation. The code block below includes the necessary imports.

import re
from typing import Any, List, Optional
from llama_index.readers.base import BaseReader
from llama_index.readers.schema.base import Document
from importlib.util import find_spec

Step 03: Defining expected YouTube Videos URLs

The YOUTUBE_URL_PATTERNS list contains regular expressions to match various YouTube URL formats. These patterns are crucial for extracting the video ID.

YOUTUBE_URL_PATTERNS = [
    r"^https?://(?:www\.)?youtube\.com/watch\?v=([\w-]+)",
    r"^https?://(?:www\.)?youtube\.com/embed/([\w-]+)",
    r"^https?://youtu\.be/([\w-]+)",  # youtu.be does not use www
]

Step 04: Verifying YouTube Video

From a list of many YouTube videos links, the is_youtube_video function determines if a given URL is a valid YouTube video link by matching it against the defined patterns.

def is_youtube_video(url: str) -> bool:
    """
    Returns whether the passed in `url` matches the various YouTube URL formats
    """
    for pattern in YOUTUBE_URL_PATTERNS:
        if re.search(pattern, url):
            return True
    return False

Step 05: Initializing the transcriber

The YoutubeTranscriptReader class checks for the presence of the youtube_transcript_api package and raises an ImportError if not found.

class YoutubeTranscriptReader(BaseReader):
    """Youtube Transcript reader."""

    def __init__(self) -> None:
        if find_spec("youtube_transcript_api") is None:
            raise ImportError(
                "Missing package: youtube_transcript_api.\n"
                "Please `pip install youtube_transcript_api` to use this Reader"
            )
        super().__init__()

Step 06: Loading Videos Data/transcription

The load_data method takes a list of YouTube links (ytlinks) and optional language parameters. It uses the YouTubeTranscriptApi to fetch and compile transcripts for each video.

def load_data(
    self,
    ytlinks: List[str],
    languages: Optional[List[str]] = ["en"],
    **load_kwargs: Any,
) -> List[Document]:
    """Load data from the input directory.

    Args:
        pages (List[str]): List of youtube links \
            for which transcripts are to be read.
    """
    from youtube_transcript_api import YouTubeTranscriptApi

    results = []
    for link in ytlinks:
        video_id = self._extract_video_id(link)
        if not video_id:
            raise ValueError(
                f"Supplied url {link} is not a supported youtube URL."
                "Supported formats include:"
                "  youtube.com/watch?v={video_id} "
                "(with or without 'www.')\n"
                "  youtube.com/embed?v={video_id} "
                "(with or without 'www.')\n"
                "  youtu.be/{video_id} (never includes www subdomain)"
            )
        transcript_chunks = YouTubeTranscriptApi.get_transcript(
            video_id, languages=languages
        )
        chunk_text = [chunk["text"] for chunk in transcript_chunks]
        transcript = "\n".join(chunk_text)
        results.append(Document(text=transcript, extra_info={"video_id": video_id}))
    return results

Step 07: Extracting the video ID from its data

The _extract_video_id method extracts the video ID from a given YouTube link using the predefined URL patterns.

@staticmethod
def _extract_video_id(yt_link) -> Optional[str]:
    for pattern in YOUTUBE_URL_PATTERNS:
        match = re.search(pattern, yt_link)
        if match:
            return match.group(1)

    # return None if no match is found
    return None

CONCLUSION

By following these steps, you can implement a powerful YouTube transcript reader in Python. This opens the door to a wide range of applications, from content analysis to language processing. Experiment with different videos and languages to unlock the full potential of this simple yet effective tool.

Happy coding!

Do you have a project 🚀 that you want me to assist you email me🤝😊: wilbertmisingo@gmail.com
Have a question or wanna be the first to know about my posts:-
Follow ✅ me on Twitter/X 𝕏
Follow ✅ me on LinkedIn 💼

Create an Interactive AI chatbot that can analyze bulk resumes

Wilbert Misingo — Tue, 26 Dec 2023 12:18:14 +0000

INTRODUCTION

In the ever-evolving landscape of recruitment, leveraging artificial intelligence to streamline and enhance the hiring process has become increasingly essential. In this article, we will explore how to implement a resume screening process using advanced natural language processing techniques.

IMPLEMENTATION

Using artificial intelligence to your advantage is common in the fast-paced world of recruitment. One such program is ResumeScreenerPack, a flexible utility that automates and improves the resume screening process by leveraging OpenAI's state-of-the-art GPT-4 model. This article will walk you through the entire tool implementation process, from installation to use, providing a thorough explanation of each step.

Step 01: Installing libraries and modules

Begin by installing the required Python packages to set up your environment. Run the following command in your terminal or command prompt:

pip install llama_index pydantic

Step 02: Importing libraries and modules

Now, let's import the crucial modules and components needed for our resume screening implementation. The code block below includes the necessary imports.


from pathlib import Path
from typing import Any, Dict, List, Optional
from llama_index.llama_pack.base import BaseLlamaPack
from llama_index.readers import PDFReader
from llama_index.llms import OpenAI
from llama_index import ServiceContext
from llama_index.schema import NodeWithScore
from llama_index.response_synthesizers import TreeSummarize
from pydantic import BaseModel, Field

try:
    from llama_index.llms.llm import LLM
except ImportError:
    from llama_index.llms.base import LLM

Step 03: Setting environment and defining selection criterias

The ResumeScreenerPack takes a job description, a set of screening criteria, and a candidate's resume as inputs. It then utilizes OpenAI's GPT-4 model to analyze the resume against the specified criteria and provides a detailed decision-making process.


QUERY_TEMPLATE = """
You are an expert resume reviewer. 
Your job is to decide if the candidate passes the resume screen given the job description and a list of criteria:

{job_description}

{criteria_str}

"""

Step 04: Defining Selection Decision Models

Two Pydantic models, CriteriaDecision and ResumeScreenerDecision, encapsulate the decisions made based on individual criteria and the overall decision-making process, respectively.


class CriteriaDecision(BaseModel):
    decision: bool = Field(description="The decision made based on the criteria")
    reasoning: str = Field(description="The reasoning behind the decision")

class ResumeScreenerDecision(BaseModel):
    criteria_decisions: List[CriteriaDecision] = Field(
        description="The decisions made based on the criteria"
    )
    overall_reasoning: str = Field(
        description="The reasoning behind the overall decision"
    )
    overall_decision: bool = Field(
        description="The overall decision made based on the criteria"
    )

Step 05: Implementing Resume Analysis

The ResumeScreenerPack class orchestrates the entire process. It takes care of loading the resumes, initializing the OpenAI model, and synthesizing the decision.

The get_modules method provides an insight into the underlying modules used, and the run method executes the entire process, from loading the resume to synthesizing the decision.


class ResumeScreenerPack(BaseLlamaPack):
    def __init__(
        self, job_description: str, criteria: List[str], llm: Optional[LLM] = None
    ) -> None:
        # ... (omitted for brevity)
        self.query = QUERY_TEMPLATE.format(
            job_description=job_description, criteria_str=criteria_str
        )

    def get_modules(self) -> Dict[str, Any]:
        return {"reader": self.reader, "synthesizer": self.synthesizer}

    def run(self, resume_path: str, *args: Any, **kwargs: Any) -> Any:
        docs = self.reader.load_data(Path(resume_path))
        output = self.synthesizer.synthesize(
            query=self.query,
            nodes=[NodeWithScore(node=doc, score=1.0) for doc in docs],
        )
        return output.response

CONCLUSION

In this article, we've explored the implementation of a powerful resume screening tool using OpenAI's GPT-4 model. The modular structure of the ResumeScreenerPack makes it adaptable for various scenarios, providing a robust solution for automating the initial stages of the hiring process. As AI continues to shape the future of recruitment, tools like these become invaluable in efficiently identifying the most suitable candidates.

12 Super Cool Machine Learning Projects that you didn't know are Open Source and made of JavaScript

Wilbert Misingo — Mon, 26 Jun 2023 20:10:41 +0000

Introduction

Machine learning has revolutionized numerous industries and made significant contributions to various aspects of our lives. From image recognition and natural language processing to recommendation systems and autonomous vehicles, machine learning has become an essential technology in today's world.

One of the key factors that have fostered the growth and accessibility of machine learning is the vibrant open-source ecosystem. Open-source projects have played a pivotal role in promoting collaboration, knowledge sharing, and innovation in the field of machine learning.

With that, web platforms such as Vercel and Netlify have played a major role in supporting open source machine learning projects by simplifying the deployment process, offering scalability, enabling serverless architecture, facilitating integration with external APIs and services, promoting collaboration and community engagement, supporting CI/CD workflows, and providing analytics and monitoring capabilities.

These platforms have empowered developers to focus on building innovative machine learning applications while leveraging the power and convenience of cloud-based services, of which an example are those projects that you are going to learn about.

In this article, we will explore a curated list of open source machine learning projects built using JavaScript frameworks, showcasing their incredible capabilities and providing links to demos and source code repositories.

The following is the list of Curated Open Source Machine Learning Projects:-

1. AI Photo Restorer

An AI Face Photo Restorer built on Next.js and Replicate. This project uses cutting-edge algorithms to restore old or damaged photographs, bringing them back to life.

Demo: AI Photo Restorer
GitHub: AI Photo Restorer Repository

2. Alt Text Generator

Generate descriptions for your images using AI. This project helps improve web accessibility by automatically generating alt text for images, making them more accessible to visually impaired users.

Demo: Alt Text Generator
GitHub: Alt Text Generator Repository

3. Extrapolate – AI Aging App

Age transformation AI app powered by Next.js, Replicate, Upstash, and Cloudflare R2 + Workers. This project allows users to see how they might look as they age, using advanced AI algorithms.

Demo: Extrapolate – AI Aging App
GitHub: Extrapolate Repository

4. Paint by Text

Edit your photos by chatting with a generative AI model (InstructPix2Pix), powered by Replicate. This project leverages natural language processing and computer vision to enable photo editing through conversational interfaces.

Demo: Paint by Text
GitHub: Paint by Text Repository

5. Inpainter with Stable Diffusion

Next.js application for inpainting with Stable Diffusion using the Replicate API. This project allows users to remove unwanted objects or elements from images seamlessly.

Demo: Inpainter with Stable Diffusion
GitHub: Inpainter Repository

6. AI Film Recommendation Generator

Get curated show and movie recommendations with OpenAI GPT-3 API (text-davinci-003) and Vercel Edge Functions with streaming. This project utilizes state-of-the-art natural language processing to provide personalized movie and TV show recommendations.

Demo: AI Film Recommendation Generator
GitHub: AI Film Recommendation Generator Repository

7. AI Twitter Bio Generator

Generate your Twitter bio with OpenAI GPT-3 API (text-davinci-003) and Vercel Edge Functions with streaming. This project helps users create catchy and creative Twitter bios using AI-powered language generation.

Demo: AI Twitter Bio Generator
GitHub: AI Twitter Bio Generator Repository

8. Scribble Diffusion

Turn your rough sketch into a refined image using AI. Powered by Replicate and ControlNet, this project transforms simple sketches into detailed images by leveraging the power of deep learning.

Demo: Scribble Diffusion
GitHub: Scribble Diffusion Repository

9. AI GPT-3 Chatbot Example

Simple chatbot implemented with Next.js, API Routes, and OpenAI SDK (GPT-3). This project demonstrates the capabilities of AI-powered conversational agents, enabling interactive and engaging interactions with users.

GitHub: AI GPT-3 Chatbot Example Repository

10. Flask Alt Text Generator

A Flask API that generates a description for any image. This project provides an alternative implementation of the Alt Text Generator project, showcasing the versatility and flexibility of the underlying technology.

Demo: Flask Alt Text Generator
GitHub: Flask Alt Text Generator Repository

11. AI Code Translator

Use AI to translate code from one language to another, with out worrying about syntaxes or code bugs.

Demo: AI Code Translator
GitHub: AI Code Translator Repository

12. Dall-E 2 AI Art Generator

This project will generate images from text using AI, allowing you to produce seeming less and extraordinary arts masterpieces.

GitHub: Dall-E 2 AI Art Generator Repository

Conclusion

The open-source ecosystem has played a vital role in the advancement and accessibility of machine learning. The curated list of open source machine learning projects presented here demonstrates the incredible capabilities of JavaScript frameworks when combined with AI technologies.

These projects cover a wide range of applications, from photo restoration and image generation to recommendation systems and chatbots.

By exploring and contributing to these open-source projects, developers can not only learn and apply machine learning techniques but also contribute to the larger community, fostering innovation and progress in the field of artificial intelligence.

Happy Coding!!

🚀 Boost Your Career as a Developer: Mastering the Power of Effective Communication! 🗣️✨

Wilbert Misingo — Mon, 12 Jun 2023 11:52:49 +0000

Introduction

I have came to realize that in today's fast-paced and competitive world, being a skilled developer is no longer enough to guarantee a thriving career. Technical expertise alone won't set you apart from the sea of talented professionals vying for the same opportunities. To truly stand out and accelerate your trajectory to success, you need to master the often-overlooked superpower: effective communication.

Imagine being able to present your ideas with unwavering confidence, captivating your audience and leaving a lasting impression. Envision effortlessly collaborating with your peers, seamlessly exchanging thoughts and ideas, and achieving remarkable outcomes as a cohesive team. Picture yourself conveying complex technical concepts in a way that resonates with both tech-savvy individuals and non-technical stakeholders, building strong professional relationships that transcend language barriers.

The good news is that enhancing your communication skills is not an innate talent reserved for a select few. It's an art that can be learned, practiced, and mastered by anyone willing to invest the time and effort. In this article, we will explore five powerful strategies that will elevate your career as a developer by unlocking the full potential of effective communication.

Strategy 01:Embrace Public Speaking Opportunities

Step out of your comfort zone and seek opportunities to present your work or ideas to a wider audience. Whether it's a meetup, conference, or internal team meeting, speaking in front of others will boost your confidence and help you refine your ability to articulate complex technical concepts in a clear and concise manner.

Strategy 02:Practice Active Listening

Communication is a two-way street, and listening is a skill worth honing. Actively engage in conversations, attentively listen to your colleagues, clients, or end-users, and ask insightful questions. By genuinely understanding others' perspectives, you can adapt your communication style and deliver information in a way that resonates with your audience.

Strategy 03:Develop Your Writing Skills

Effective written communication is essential, especially in today's remote work environment. Take time to improve your writing skills by practicing clear and concise documentation, writing technical blog posts, or contributing to open-source projects. Remember, the ability to convey complex ideas in a coherent and understandable manner will set you apart from the rest.

Strategy 04:Participate in Collaborative Projects

Collaboration is at the heart of software development. Joining collaborative projects not only enhances your technical skills but also allows you to work in teams, fostering effective communication. Embrace tools like version control systems, project management platforms, and communication channels to collaborate seamlessly with your peers.

Strategy 05:Seek Feedback and Learn from Others

Constructive feedback is invaluable for growth. Actively seek feedback from your teammates, mentors, or even fellow developers in online communities. Embrace their insights and suggestions to refine your communication skills further. Additionally, observe and learn from excellent communicators within your field, analyzing their techniques and adopting effective strategies into your own repertoire.

Conclusion

💥 Emphasizing technical skills alone is no longer enough in today's highly competitive job market. By investing in your communication skills, you become a well-rounded developer, capable of effectively conveying your ideas, collaborating seamlessly, and building strong professional relationships. 🌐🤝 So, step out of your comfort zone, embrace new challenges, and watch your career soar to new heights! 🚀💼

By mastering the power of effective communication, you become a well-rounded developer equipped with the tools to convey your ideas, collaborate seamlessly, and build strong professional relationships. Step out of your comfort zone, embrace new challenges, and watch your career soar to new heights!

Are you ready to embark on this transformative journey? Let's dive into the world of effective communication for developers and unlock the full potential of your career!

Well!! Good Luck on your journey on becoming the better version of yourself.

ChatGPT knowledge is out of date all this time, this is how to keep it to date

Wilbert Misingo — Fri, 02 Jun 2023 14:16:48 +0000

Introduction

I was surprised that most of people didn't knew it, even I didn't knew it from the first place. Despite using the OpenAI's ChatGPT models API for a long time now, but I later came to realize that there are some cases neither ChatGPT nor its API are usefully aware when it came to asking or talking about events that from the year 2021 up to this time, this is due to the models being trained only limited to the data up to the year 2021.

For instance when you ask ChatGPT a question about a current event, e.g. "Who is the English Premier League Champion for 2023?", and here its the answer:-

But when I went to google and asked the same question, here its the answer that was given:-

Now you can see that that's a problem indeed, now this tutorial, we will explore how to create a chatbot using the OpenAI API, even though the knowledge cutoff of the API is limited to the year 2021. We will leverage other tools like Google Search, Wikipedia, and more to ensure that our chatbot remains up-to-date with the latest information. Let's get started!

Step 1: Installing Required Packages

To begin, we need to install the necessary packages. In this example, we will be using the langchain library, which provides convenient wrappers for interacting with various language processing tools. You can install it by running the following command:

pip install langchain

Additionally, make sure you have an OpenAI API key as we will be using the OpenAI API to power our chatbot.

Step 02: Importing Libraries and Modules

To begin, we need to import the necessary libraries and modules that will be used throughout the chatbot creation process. The code snippet below demonstrates the required imports:

from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.memory import ConversationBufferMemory
from langchain.chat_models import ChatOpenAl

Step 03: Defining and Initializing the chatbot's memory

Since we a creating a chatbot, its required to have a memory of its own by remembering and managing conversations to provide a realistic experience. By creating an instance of ConversationBufferMemory to store the conversation history.

memory = ConversationBufferMemory()

Step 4: Loading Tools and Models

First, we need to load the required tools and models. In our case, we will load tools like Wikipedia, Google Search, Python REPL, Wolfram Alpha, and more. These tools will enable our chatbot to retrieve up-to-date information from various sources. Here's an example code snippet to load the tools:


llm = ChatOpenAl()

tools = load_tools([
    'wikipedia',
    'llm—math',
    'google—search',
    'python_repl',
    'wolfram—alpha',
    'ternubak',
    'news—api',
    'podcast—api',
    'openweathermap—api'
], llm=llm)

agent = initialize_agent(
    tools,
    llm,
    AgentType.CONVERSATIONAL_REACT_DESCRIPTION,
    verbose=True,
    memory=memory
)

Here, we create an instance of ChatOpenAl to power our chatbot. We then load various tools using the load_tools function, passing a list of tool names. Finally, we initialize the agent by providing the loaded tools, model, agent type, verbosity, and memory.

Step 5: Interacting with the Chatbot

Once we have initialized the chatbot agent, we can start interacting with it. We can provide a prompt or a question to the chatbot and receive a response. Here's an example of how to interact with the chatbot:

response = agent.run(""Who is the English Premier League Champion for 2023?"")
print(response)

In this example, we pass the prompt "Who is the English Premier League Champion for 2023?" to the agent.run function, which triggers the chatbot to generate a response. The response is then printed to the User Interface.

Conclusion

In this tutorial, we have learned how to create a chatbot using the OpenAI API while keeping its knowledge up-to-date using other tools like Wikipedia, Google Search, and more. By leveraging these tools, we can ensure that our chatbot remains relevant and provides accurate information. Feel free to explore further and customize the chatbot based on your specific requirements.

Remember to refer to the official documentation of the langchain library and the respective tools for more details and customization options.

Happy building!

How to create your own custom ChatGPT like chatbot in less than 5 minutes using your own data and no OpenAI API

Wilbert Misingo — Fri, 19 May 2023 08:06:23 +0000

INTRODUCTION

In today's world, businesses are constantly looking for new ways to improve their customer service and engagement. One way to do this is by creating a chatbot that can quickly and accurately answer customer questions. In this article, we will show you how to create a chatbot that is based on your own company documents using Python and some powerful AI tools.

Although the use of chatbot have become increasingly popular as a way to provide instant and personalized customer support. Building a chatbot that can understand and respond to user queries based on your company's own documents can greatly enhance the efficiency and effectiveness of your customer service.

In my previous article which can be found here I described how you can create a chatbot using your custom data and OpenAI API, and then later wrote another one where you can integrate this chatbot to your WhatsApp business number, the article can be found here.

Thus in this article, we will guide you through the process of creating a chatbot using the code provided below.

To achieve the goal, I initially considered modifying the GPT model using my own data. But, fine-tuning is highly expensive and necessitates a sizable dataset with examples. Also, every time the document is altered, it is impossible to make final adjustments. Perhaps more importantly, fine-tuning teaches the model a new ability rather than merely letting it "know" all the information included in the documents. Consequently, fine-tuning is not the best approach for (multi-)document QA.

Prompt engineering, which includes context in the prompts, is the second strategy that springs to mind. For instance, I could insert the original document's text before the question itself instead of asking it directly. Nevertheless, the GPT model has a short attention span and can only process a small number of the prompt's 2,000 words (about 4000 tokens or 3000 words). Given that we have tens of thousands of emails from customers providing feedback and hundreds of product documentation, it is impossible to convey all the information in the prompt. Because the pricing is based on the number of tokens you use, it is also expensive if you give in a lengthy context to the API.

I thought of the notion of first using an algorithm to search the documents and select the pertinent extracts and then providing only these relevant contexts to the GPT model with my questions because the prompt has restrictions on the number of input tokens. I found a library called llama-index (formerly known as gpt-index) while doing research for my idea that accomplishes exactly what I wanted it to do and is easy to use.

And since the use of OpenAI API is a bit expensive, so I thought of a way to create a similar chatbot by the aid of OpenAI API alternative, which are free Open Source Models like GPT4All, OpenAssistant e.t.c

IMPLEMENTATION

Step 01: Preparing your training data

The first step is to gather all the documents that you want to use to create the chatbot. These documents can include product manuals, FAQs, and other helpful resources that your customers may need to reference. Once you have gathered your documents, you need to organize them into a folder called 'data' and save them in a format that can be easily read by Python.

Step 02: Installing all required libraries

To build a chatbot, we need to use some Python libraries that are specifically designed for natural language processing and machine learning. In this code snippet, we are using the llama_index , transformers and langchain libraries. You can install these libraries using pip:

$ pip install llama_index
$ pip install transformers
$ pip install langchain

Step 03: Importing Libraries and Modules

To begin, we need to import the necessary libraries and modules that will be used throughout the chatbot creation process. The code snippet below demonstrates the required imports:

import torch
from langchain.llms.base import LLM
from llama_index import SimpleDirectoryReader, GPTListIndex, PromptHelper
from llama_index import LLMPredictor, ServiceContext, QuestionAnswerPrompt
from transformers import pipeline
from typing import Optional, List, Mapping, Any

Step 04: Defining Prompt Variables

Next, we define some variables that will be used as prompt variables for the chatbot. These variables determine the maximum input size, the number of desired output tokens, and the maximum overlap between chunks. Here is the code segment that defines the prompt variables:

max_input_size = 2048
num_output = 256
max_chunk_overlap = 20

Step 05: Defining and Using the Prompt Helper

The PromptHelper class helps in handling prompts and chunking long documents. We initialize the prompt helper by passing the previously defined prompt variables. The code snippet below demonstrates the creation of the prompt helper:

prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)

Step 06: Creating a Custom Language Model (LLM)

In order to generate responses, we need to download and load a pre-trained language model. The code snippet below defines a custom LLM class that uses the facebook/opt-iml-max-30b model from Hugging Face:

class CustomLLM(LLM):
    model_name = "facebook/opt-iml-max-30b"
    pipeline = pipeline("text-generation", model=model_name, device="cuda:0", model_kwargs={"torch_dtype":torch.bfloat16})

    def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:
        prompt_length = len(prompt)
        response = self.pipeline(prompt, max_new_tokens=num_output)[0]["generated_text"]
        return response[prompt_length:]

    @property
    def _identifying_params(self) -> Mapping[str, Any]:
        return {"name_of_model": self.model_name}

    @property
    def _llm_type(self) -> str:
        return "custom"

NB:

Before using this you may consider finding the appropriate model that you find it suitable for you by considering:-

The license of the model
The size of the model

Step 07: Initializing the Language Model and Service Context

Once we have defined our custom LLM, we can initialize it and create a service context. The service context encapsulates the necessary components for our chatbot, including the LLM and prompt helper. Here's the code to initialize the LLM and service context:

llm_predictor = LLMPredictor(llm=CustomLLM())
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)

Step 08: Defining the Question-Answer Prompt Template

To structure the interaction with the chatbot, we define a template for the question-answer prompt. This template includes placeholders for the context information and the user's question. The code snippet below shows the template definition:

QA_PROMPT_TMPL = (
    "We have provided context information below. \n"
    "---------------------\n"
    "{context_str}"
    "\n---------------------\n"
    "Given this

 information, please answer the question: {query_str}\n"
)

QA_PROMPT = QuestionAnswerPrompt(QA_PROMPT_TMPL)

Step 09: Loading training data

To make the chatbot knowledgeable about your company, you need to load your company documents into the chatbot's index. The code snippet below demonstrates loading the data from a specified directory:

documents = SimpleDirectoryReader('./data').load_data()

Step 10: Generating the Index

Once the documents are loaded, we generate an index using the GPTListIndex class. The index is responsible for efficiently retrieving relevant information based on user queries. Here's the code to generate the index:

index = GPTListIndex.from_documents(documents, service_context=service_context)

Step 11: Saving and Loading the Index

To avoid re-indexing the documents every time the chatbot is restarted, we can save the index to disk and load it later. Here's how you can save and load the index:

index.save_to_disk('index.json')
index = GPTListIndex.load_from_disk('index.json')

Step 12: Querying the Chatbot and Getting a Response

Finally, we can interact with the chatbot by querying it with user input. The chatbot will process the query and provide a response based on the indexed company documents. The code snippet below demonstrates querying the chatbot and printing the response:

query_engine = index.as_query_engine()
response = query_engine.query("Hello, what is your function?", text_qa_template=QA_PROMPT)
print(response)

CONCLUSION

Congratulations! You have successfully created a chatbot based on your own company documents. This chatbot can provide accurate and relevant responses to user queries, leveraging the power of machine learning and natural language processing.

Remember, the provided code is just a starting point, and you can customize and extend it according to your specific requirements. Building a chatbot is an iterative process, so feel free to experiment and enhance the functionality based on user feedback and additional data.

Happy coding!

How to design and develop AI systems so cool that anyone would love them

Wilbert Misingo — Mon, 15 May 2023 19:48:35 +0000

Introduction

Have you ever wondered why everyone likes ChatGPT or why it have gained over 2 million of users in just a few days, which it’s totally a record break for any platform in the world.

Well, I have worked on a bit many artificial intelligence projects, especially on computer vision and natural language processing, but in the end they didn’t make it up as I expected, especially when I comes to the number of people who used them frequently.

I had come to realize that I wasn’t the only victim of such experience, many startups and companies have built many artificial intelligence projects, also especially on computer vision and natural language processing, but in the end they didn’t make it up as they expected, also especially when I comes to the number of people who used them frequently.

I then later came to realize that having an artificial intelligence project is more than having fancy GPUs or even powerful servers or computers, also on the other hand it’s not even about having powerful and highly optimized artificial intelligence models.

Since the systems were designed by humans in order to be used by humans, we forgot that and that’s why many of our systems didn’t work out.

When I applied these principles, the results were astonishing and left me to realize why I didn’t knew this from the first place.

And also I even came to realize that companies like Google, Microsoft and OpenAI use the same principle of which we take for granted that’s why even you can testify that it feels good to use their Artificial Intelligence systems.

The following are some of the guidelines to be followed when designing and developing Artificial Intelligence systems that are human centric.

The guidelines are divided according to the situations at which humans interact with Artificial Intelligence systems, this includes:-

Initially when a person interacts with the system for the first time.
During the interaction with the system after a while.
When something goes wrong during the interaction.
Overtime when the user gets highly familiar with the system.

Guidelines for Human-AI Interaction

Now let’s dive in to get a more understanding of the concepts:-

1. Initially when a person interacts with the system for the first time

a) Make clear what the system can do by making users understand what the system can do.

b) Make clear how well the system can do what it can do by making the user understand how often the system succeed or make mistakes.

2. During the interaction with the system after a while

a) Time all services based on their context by identifying the appropriate time when to act or interrupt based on the user’s current task and environment.

b) Show contextually relevant information by displaying the information relevant to the user’s current task and environment.

c) Match relevant social norms by ensuring that experienced is delivered in a way the users would expect, given their social and cultural context.

d) Mitigate social biases by ensuring that the system’s language and behaviors doesn’t reinforce undesirable and unfair stereotypes or biases.

3. When something goes wrong during the interaction

a) Support efficient invocation by making it easy for a user to invoke or request the system’s services when needed.

b) Support efficient dismissal by making it easy for the user to dismiss or to ignore undesired system services.

c) Support efficient correction by making it easy for the user to edit, refine or recover when the system is wrong.

d) Scope services when in doubt by sometimes engaging in disambiguation or graceful degrading the systems services when uncertain about a user’s goals.

e) Make it clear what the system did that by enabling the user to access the explanation of why the system behaved as it did.

4. Overtime when the user gets highly familiar with the system

a) Remember recent interactions with the user by maintaining short term memory and allowing the user to make efficient references to that memory.

b) Learn from the user’s behavior by personalizing the user’s experience by learning from their actions overtime.

c) Update and adapt cautiously by limiting disruptive changes when updating and adapting the system’s behaviors.

d) Encourage granular feedback by enabling the user to provide feedback indicating their preferences during regular interaction with the system.

e) Convey the consequences of user actions by immediately updating or conveying how user actions will impact future behaviors of the system.

f) Provide global controls by allowing the user to globally customize what the system monitors and how it behaves.

g) Notify users about changes by informing the user when the system adds or updates its capabilities.

I hope that you can use these Guidelines for Human-AI Interaction throughout your design process as you evaluate existing ideas, brainstorm new ones, and collaborate with the multiple disciplines involved in creating AI systems.

References:

_https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/

https://human-centered.ai/intelligent-user-interfaces-2019/_

How I created a chatbot that transformed my company using 68 documents and chatgpt .

Wilbert Misingo — Thu, 30 Mar 2023 16:53:28 +0000

INTRODUCTION

THE PROCESSS

Step 1: Prepare Your Company Documents

Step 2: Installing and importing the Required Libraries

To build a chatbot, we need to use some Python libraries that are specifically designed for natural language processing and machine learning. In this code snippet, we are using the llama_index and langchain libraries. You can install these libraries using pip:

$pip install llama_index
$pip install langchain

After a successful installation, the libraries are then imported:

import os
from llama_index import GPTSimpleVectorIndex, SimpleDirectoryReader, LLMPredictor, ServiceContext, QuestionAnswerPrompt
from langchain import OpenAI

Step 3: Set Up OpenAI API Key

To use the OpenAI language model, we need to set up our API key. You can get your own API key by signing up on the OpenAI website. Once you have your API key, replace 'YOUR_OPENAI_API_KEY' with your actual key in the following code line:

os.environ["OPENAI_API_KEY"] = 'YOUR_OPENAI_API_KEY'

Step 4: Load Your Company Documents

In this code snippet, we are using the SimpleDirectoryReader class from the llama_index library to load our company documents from the 'documents' folder:

documents = SimpleDirectoryReader('documents').load_data()

Step 5: Set Up the Language Model

Next, we need to set up the language model that we will use to analyze the text in our company documents. In this code snippet, we are using the text-davinci-003 model from OpenAI and setting the temperature to 0 to ensure that the responses are always the same:

llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-davinci-003"))

Step 6: Set Up the Service Context and Index

We are using the ServiceContext class to define the parameters for the chatbot service. Then, we create the GPTSimpleVectorIndex class from our company documents and the service context to build an index of our documents:

service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)
index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)

Step 7: Save and Load the Index

We save our index to a file called 'index.json' so that we can reuse it later:

index.save_to_disk('index.json')

We can then load the index from the file using the load_from_disk method:

index = GPTSimpleVectorIndex.load_from_disk('index.json', llm_predictor=llm_predictor)

Step 8: Set Up the Question-Answer Prompt

In this code snippet, we create a QuestionAnswerPrompt class and define a template for the chatbot's responses. The template includes the context information and a query string that the user can input to ask a question, this may be optional but it helps the chatbot with further instructions on how to answer some questions:

QA_PROMPT_TMPL = (

        "Context information is below. \n"
        "---------------------\n"
                "{context_str}"
        "\n---------------------\n" 
        "Given this information, please answer the question: {query_str}\n"
        )

QA_PROMPT = QuestionAnswerPrompt(QA_PROMPT_TMPL)

Step 9: Test the Chatbot

Finally, we test our chatbot by inputting a sample query string and printing the response:

query_str = "What is the mission of the company?"
response = index.query(query_str, text_qa_template=QA_PROMPT)
print(response)

The chatbot will analyze the query string and search through our company documents to find the most relevant answer. It will then provide a response based on the template that we defined in Step 8.

Conclusion

In this article, we have shown you how to create a chatbot that is based on your own company documents using Python and some powerful AI tools. With this chatbot, you can provide your customers with quick and accurate answers to their questions, improving your customer service and engagement. By customizing the chatbot's responses based on your company's unique resources, you can create a chatbot that is tailored to your specific needs.

Unsupervised Machine Learning: Non Text Clustering with DBSCAN

Wilbert Misingo — Thu, 12 Jan 2023 11:00:36 +0000

Introduction

Unsupervised machine learning is a type of machine learning where the model is not provided with labeled training data. Instead, it must find patterns or relationships in the data on its own. There are different types of unsupervised learning, such as clustering and dimensionality reduction. Clustering, in particular, is the task of grouping similar examples together, without being provided with a specific target variable.

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a density-based clustering algorithm. Unlike other clustering algorithms, such as k-means, DBSCAN does not require the number of clusters to be specified in advance. Instead, it automatically discovers the number of clusters based on the density of the data. DBSCAN works by defining a dense region as one where there are at least a specified number of examples within a certain distance (epsilon) of each other. These dense regions are then used as clusters.

The DBSCAN algorithm is implemented using the DBSCAN class from the sklearn.cluster module. It takes two parameters epsilon, and min_samples. The epsilon defines the radius of the neighborhood around a point, and min_samples defines the number of points required to form a dense region. The fit method is applied on X to get the cluster labels. The variable labels contains the cluster labels for each data point.

In this article we would cover how to create cluster for non text data using DBSCAN, during the process the following libraries would be used:-

Sci-kit learn
Numpy
Matplotlib

The process

Importing Libraries and modules

from sklearn.datasets import make_blobs
from sklearn.cluster import DBSCAN
import numpy as np
import matplotlib.pyplot as plt

Defining model and data configurations

num_samples_total = 1000
cluster_centers = [(3,3), (7,7)]
num_classes = len(cluster_centers)
epsilon = 1.0
min_samples = 13

Generating training data

X, y = make_blobs(n_samples = num_samples_total, centers = cluster_centers, n_features = num_classes, center_box=(0, 1), cluster_std = 0.5)

Saving the data for future uses and loading

np.save('./clusters.npy', X)
X = np.load('./clusters.npy')

Training the model

db = DBSCAN(eps=epsilon, min_samples=min_samples).fit(X)
labels = db.labels_

Getting information about the clusters

no_clusters = len(np.unique(labels) )
no_noise = np.sum(np.array(labels) == -1, axis=0)

print('Estimated no. of clusters: %d' % no_clusters)
print('Estimated no. of noise points: %d' % no_noise)

Visualizing the clusters

colors = list(map(lambda x: '#3b4cc0' if x == 1 else '#b40426', labels))
plt.scatter(X[:,0], X[:,1], c=colors, marker="o", picker=True)
plt.title('Two clusters with data')
plt.xlabel('Axis X[0]')
plt.ylabel('Axis X[1]')
plt.show()

That's all, I hope this helps!!